When you plan to have a strong army in a province for a while, not having a fort may be the better choice. In a no-fort province, everyone who isn't hidden will fight against an invasion. In a province with a fort, only patrolling units and PD defend against an initial invasion.