[Taqwus]

Fortresses come in handy for a variety of reasons, such as keeping a short supply line -- supplies available get a bonus based on the shortest available path (not through hostile provines) to a friendly fort and it's admin rating, for instance. Handy unless you've got so much nature magic that you're rolling in wineskins, summer swords and cauldrons, or are using need-not-eat troops.

They're also useful for protecting a temple, obviously, but with a rapid-response team you can use a watchtower for that.
More importantly, if there's a unit type you like in that province -- be it from the independent type or a magic site -- a fortress will help you produce more if it's resource-limited. And it'll let you produce more of your national troops there and worry less about streaming reinforcements up, or it can serve to recruit your cheap researchers while your capital goes for some capital-only commanders. Or so forth.
If you're rolling in money because you can't recruit good troops fast enough, you probably need another castle. *g*