[General Tacticus]

The default order for units in a province with a castle is "defend castle", meaning they stay inside and man the walls, and won't challenge an invading force. If you change it to "patrol province", then and only then will they meet the invaders.

Once the invasion force is in command of the province, it starts sieging the castle. This takes time and men, and there is a repair factor too (I have had both "you can't damage the wall" and "the enemy is repairing the walls faster thatn you damage it" reports). I don't know what the repair factor is, it probably has to do with defense value and/or number of units inside. It is done automatically.

During the siege, the besieged party can send their armies to attack the besiegers ("break siege" order). Or they can stay put and wait "defend castle".

Eventually, the walls will be breached. At that point the invaders can storm the castle. This result in a battle in the castle, complete with walls (if the castle has any), open gate (since the walls are breached), towers that fire arrows, and so forth. And both armies of course.

Some units are better at sieging, as you noticed (try ulm's engineer). Otherwise a militia is about as good as a Ulmian plate with shield, as far as I know. The bastle inside the castle is basically just another battle, although with a few twists, and mostly the usual considerations as to which armies are better apply. It depends on who besieges whom, what armies are defending, and so forth... Of course the walls add unique considerations...

Defeating a castle is like any other province in terms of sites : whatever the enemy knew about, you know. What he didn't know about is still hidden and need to be searched for. With the added twist that you don't know what the enemy searched for (well, you can guess). In the scenarios so far, everybody knows about every site at his capital, so if you capture a capital you don't have to search, but that may not always be true.