[Taqwus]

If it's an MP game, there's also the slight possibility that somebody is playing around with a stealthy disease-causing unit (ex. Black Servant w/ Bane Venom charm, or a Harvester of Sorrows). A disease-causing site is more likely, however.

If you see an independent force slowly dwindling with no apparent reason, a disease site is a pretty likely explanation. Be cautious if you rely on living troops without an easy way of curing disease.