You can pick and choose the population, but you have to use the "Transfer Cargo" command instead of "Load Cargo". This means if you're playing a simultaneous game you might lose some move points...
wish SE4 had a system like Stars! where you could give all those complex orders about picking up and dropping stuff off...