You just have to balance things.
In a recent MP game, Ctis was mad castling Hill Forts (!), and put just too much cash into Forts, and had no units to actually defend them. I assume he was planning on stocking them with lots of units when he was done with the building phase, but thanks to the units I had built, he never got to that point
Of course, I've seen the opposite and watched people have a single fort for a 21-province empire. I prefer to pick high income provinces to progessively castle as the game progresses. It booosts income significantly from those provinces (using standard castle), and protects the province from raiders.