Everyone but the patrolling units will be inside the castle, to the best of my knowledge, and although I ain't too certain about the pillaging and searching for sites, consider it an educated guess that they're safe in the castle as well.
The leaders on the move conduct a "friendly move" (into your own province), which happens before hostile movement, which means that they're safe and sound inside the castle walls before the enemy approaches.
If you move a unit into a friendly province, you can then select the order to "move & patrol" to have a supporting army defend the province, not go hiding in the castle.