Oh, wait. I may be horribly wrong (so I should better not talk probably
) but I may try a guess. The communion brings your communion master to F3 and he has just 2 gems... to get to F4 he needs a gem, but possibly other 2 are required to cast the spell as required by the 200 fatigue and he has just 1 now? So maybe it works when you add 4 more slaves because now he has F4 and he uses the 2 gems to cast the spell normally. See if adding the 4 more slaves when he has cast Flaming Arrows he has gems left or not.