AND you can have that scout carry the hammer (gatecleaver?) and the Horn when not in use. But I was also going to suggest having a scout or some high stealth unit sit on the castle and eyeball what defenders are present.
I've noticed that if I sit on a castle for a couple of turns the AI tends to sally forth and wear himself down. Which I promptly take care of.
