Right, so the 1.5" runners will not be able to move the same maximum cfm, but that's not the point. What you are wanting is the pressure that is seen at the exhaust valve head to be the same across all of the cylinders. Because the cross sectional area is smaller, the same amount of mass flowing through is will have to move faster, which equates to less pressure at the valve head (PV=nRT) because the mass isn't stacking up. The larger runners allow the mass to slow down and stack up, causing a higher net pressure and a slower flow. So yes, by mucking with the volumes of the runners you should be able to effectively produce similar pressures at the valve heads as an equal length tubular header that all the runners were the same cross sectional diameter.
As for how the mass moves through the volute and turbine, honestly I really want to do more in depth research here, but the only reason that the exhaust would have to compress again in the housing is if the housing/wheel combo simply couldn't flow the amount of exhaust being passed through it (boost creep). There's a bunch of physics going on all at the same time. The turbine works off of pressure and temperature (to a smaller extent) differential. The volume of the volute does decrease, but it's done so that the decreasing volume of exhaust keeps the same pressure differential around the turbine wheel. I guess you could argue this is "compressing", but the actual pressure theoretically should be the same all the way around.