You did check the morale of your enemies, right? I'd quess even things like blessed Knights of the Chalice would have their morale at 4 or 5, and normal infantry might have -3 or so...
I wonder what would happen if i combined him with a few scorpion men (is it possible to reduce berserked/other extremely high morale units down to a routable level?)