Yes "real" mountains are very helpful
if you want to build that strong chain of Castles (Peg-Castles, 700 defense 20 admin) without any draining from adjacent provinces; provinces where you in the future might want to build additional fortresses.
Forests are also good but not nearly as good as mountains because you only get to build Forest Fortresses (300 defense, 15 admin) and they cost the same amount of gold (1000).
The main strategy is that you want to cram in as may fortresses as possible, as resource effective as possible, within a small area so that they can help each other in with defense. You also want them to have maximum defense for Ulm, thus Peg-Castles.
If you later want more gold you should build a fortress on a high population farmland (they tend to have high pop), so that you get the 25% income bonus. But this is a much later worry, so don't spend those 1200 gold on that Fortified City in the beginning. Mainly because it won't give you much resources if it hasn't mountains and forests around it (that you have to clear out of course, taking time and resources in form of losses).
Building a Fortified City or Citadel (the latter is the standard type, meaning it gets built on border mountains) makes those provinces around them quite useless for further fortress building. Because if you do build additional fortresses there you cut of the (main) resource income to the Fortified City or Citadel. You still have some of the money though, but in the beginning it isn't worth it.
Another thing, as you get the 25% resource bonus in every Ulmish castle it is wise to try to have as MANY castles as possible instead of focusing on a small amount of high admin ones. With some races you should do the opposite but not with Ulm.
To summarize: Mountains, and to some extent forests, are the thing you want to look for when playing Ulm; or any resource heavy nation for that matter.