Jotunheim - Realm (heim can mean kingdom, home and world aswell tough) of the Giants. The place were evil dwelled. The great and terrible wolf Fenrir was enfettered there until the coming of Ragnarok. In Jotunheim there was a mighty fortress called Utgard.
Vanheim - Realm of the Vanir. The Vanir was a kind of rivaling family of gods mentioned here and there in the old Aesir mythology. Frey, Freya and Njord were three vanir gods that joined the Aesir pantheon, through a dubious trade.
Midgard - Middle Earth, but after some recent movies everyone knows that.
Nifelheim - The mistshrouded realm. The land of darkness and unbearable cold. Within it lies Helheim (or simply Hel, so named after the godess who rules it), the realm of the dead.
Muspelheim - The worldpyre. The opposite of Neifelheim. Not featured in the game but a possibility for a mod perhaps. Would love to see Surt and other giants of fire.