Mods are usually archived in a .zip file or similar. Extract the files from inside the archive, and put them in \dominions 3\mods folder. When you go to \mods, you should see modsname.dm file, and usually you'll also see a folder with the mod's name and the mod's graphics inside.