Do what you think is right.
I usually build one cultural facility (Colonial for small colonies, Minor City for big ones), giving me added production for everything else on the planet, and then I usually fill the planet up with basic extraction/research facilities. Once that is over, I will upgrade that Minor City (or build one) to a better level, or I build a Central Computer for research planets.
But the best way of seeing what is best is to do some calculus, depending on what you want, how much time you are willing to wait for it, etc. Don't forget remote-mining, though: while it has been watered down, it is still very nice to supplement your income, particularly for radioactives, and everything that's remote-mined give your planets some more room for research facilities.