If you colonize the big one with the Spaceyard, you will be busy building colonizers there and cannot build up the planet at the same time. Building up the planet would then mean canibalizing your spaceyard.
I usually do the same - small planet spaceyard, then colonizer, then big planet and build there w/o spaceyard - because I like to fill big planets completely with mines/research/whatever to make optimum use of the computer facility to be built there later. Of course, a lot depends on the settings, the starting system, and the neighbours (religious, anyone?
)