Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Someone has already publicly mentioned the unlisted switches of --allai (create game with all AIs) and --superhost (host the game over and over really really fast) so Johan has given permission for me to display some of my test data from the beta group. Not the cumulative data (yet) but the tests as they run which some might find interesting. Only one test of each era stays visible online, then its zipped up and the data is overwritten by the next test.

www.Dom3Minions.com/~mmdom3/AI_tests/
Its not ALL scoreboards. At the bottom is other things.

If I run
dom3 -Td --era 1 --allai --superhost --scoredump Early_AIs
(and lots of other switches) then I get an Early Era game called Early_AIs which has all of the era 1 nations in it as computer players. It will run until one of the AIs wins (or I kill it). Running such tests in many different settings (different sized maps, VP points, game settings) gives us an idea of which nations fail quickly every time and which ones tend to win too often.

The -T is text mode. I dont want gui screens, cant watch the game anyway altho this is one of the reasons why Im working on a watcher nation. :)

The -d is debug dump which creates the game log. People who have been curious about what the AI does (including your own commanders) can take a look at this. No we do NOT know what its telling us but some things you can figure out. The log is ONE turn and its huge. Much more info is available if I run it -dd or -ddddd but for these tests I dont need that. Feel free to do that to your computer if you want to. :)
Ive crashed my machines trying to get complete logs for a whole game.

--era 1 is of course which era. I have run the SingleAge mod this way. In fact, its why I creates the SingleAge mod. But thats just for fun. Other than occassionally testing the games maximums. the information isnt all that helpful to the devs.

--allai fill the era with all AIs

--superhost skips all countdowns and hosts, hosts, hosts. It also creates a superhost.txt which has information such as which nations AI was Standard (1), Defensive (2), or Aggressive (3). The scores are mentioned at turns 50, 100, 150, and 200.

--scoresdump creates the scores.html which I rename to another number in order to keep them all. Ignore the turn count after 250 because even though the game continues the counter stops. I think thats because the devs figured out long before I did that its insane to keep going. The game is usually fairly well decided by turn 250 such as getting down to Vanheim and Oceania so running it for another thousand turns to take the every single province is moot for our purposes.


Gandalf Parker

Re: Curious about Dominions?
 

When I use a command called html2text I can convert the scores.html to something like this..
**** Dominions 3 Scores, Early_AIs turn 12 ****
*** Provinces ***

|Nation |Provinces|
|Arcoscephale |4 |
|Ermor |10 |
|Ulm |4 |
|Marverni |7 |
|Sauromatia |4 |
|T'ien Ch'i |5 |
|Mictlan |7 |
|Abysia |7 |
|Caelum |10 |
|C'tis |8 |
|Pangaea |5 |
|Agartha |6 |
|Tir na n'Og |10 |
|Fomoria |13 |
|Vanheim |5 |
|Helheim |13 |
|Niefelheim |5 |
|Kailasa |6 |
|Yomi |6 |
|Hinnom |10 |

Im messing around with gathering all the score data into one file and generating a scoreboard from it like is inside the game. Maybe an animated one (gif frames?). Not that Johan needs it but I thought it would be interesting for the rest of us.

Eventually a Watcher.dm would allow me to watch everything in a game from inside the game. That would be really fun. However, there is apparently no way to accomplish that without affecting the outcome of the game (even hidden units affect how the AI acts) so such a game would not be of any use to the developers.

Re: Curious about Dominions?
 

I wish for a Watcher.dm too. :) I like the fact that Dom3 is a game of limited information, but at the same time it would be easier to tolerate long hosting times if I knew I would be getting more out of it than mysterious battles seen only by the AI. I want to be able to watch every battle that happens anywhere.

Actually, now that I think of it... maybe there's a way to mod Astral Projection so that it works over the whole map as a global. I'll probably never look into it but I do wonder.

-Max

Re: Curious about Dominions?
 

If you are interested in what can be gleaned from the debug log (and made use of for interesting games) some points were made in post #19 of this thread...
http://forum.shrapnelgames.com/showthread.php?t=35083

Re: Curious about Dominions?
 

There is a thread on a Watcher Nation in the mods forum. We have gotten quite a ways.

What I have so far is useful. It can last the game and can check out much of whats going on. But I would like to figure out how to get an "Eyes of God" that doesnt take a global slot.

Re: Curious about Dominions?
 

The game uses only three debug log levels: 0, 1 and 2. It makes no difference if you use -dd or -ddd or -dddddddddddddddddddddddddddddddddd. The only difference is in using -d or -dd or no debug switch at all.

That shouldn't happen. I create full debug logs for all the games that I am hosting and keep them, and none have ever crashed because of debug logs. The logging itself shouldn't hamper hosting in any way, just slowing it down a little.

Look at my sig, I already came up with something that creates charts for the current turn and later to generate graphs for the whole progression of the game, too (see the second page of the thread for the link to the whole-game graphs). Examples can be viewed from my directory on your server, too, so you should have access to all the files that are needed.

As I wrote in a reply in that thread, the best you can achieve without dissecting the turn files that are being generated to get the same as the Eyes of God is something that has Scrying on the whole map. There is no arcane hidden way in the game to make it easier than that.

Some months ago, I've started something that should display the province ownership on a map. I've gotten to the point where people need a GUI to divide the map into polygons, because the borders that you get just by computational geometry aren't always that consistent with those borders that have been drawn on the map. One of the test images that I generated last is at http://dom3.servegame.com/test.svg -- you need a SVG-capable browser or browser plugin to open this, like Mozilla or Opera should be. I got sidetracked and put my time into other projects lately, but if there is honest interest in something like this, which should enable after-game replays like Civilization has them, too, I could pick it up again later or somebody could continue with what I have got so far.

Re: Curious about Dominions?
 

I think this would be amazingly cool! I always enjoyed watching that replay in Civ, and was sad when they dropped it.

It would be extremely cool if there was one animation for province ownership, and another for dominion spread. But I'd be super-happy even with just a province ownership replay.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Another Early_AI test complete. The time-stamp on those scores rolled. The first one is actually 1440. (Ive fixed that for future runs)

Hinnom the winner with Oceania holding out till second place again. Yomi was first out followed closely by Kailasa. Wow since Yomi was the winner on the last one. I should check if it was the same AI (defense, standard, agressive) both times

It occurred to me that it would be helpful if someone wanted to provide data for mp to go with this. Rather than impressions and opinions maybe someone administratively skilled could break down the winners list from the sticky in the mp forum? Just totals for each nation. It wouldnt provide the 1st out info but it would provide winner stats

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Every time I scan a game log I see something new.
Ive often seen it say that a commander was given equipment but Ive never before noticed
"Com 650 returning Water Bracelet"

I wonder what the AI thinking is which causes that. As much as people say that it hands out equipment stupidly I didnt think there was anything that would cause it to redistribute eq.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

That is just amazing to me. I mean, I've seen AI commanders with boosters for paths they didn't have, or going out into combat with SDR, or research boosters - so it's rather shocking to think that the AI actually in some cases makes the judgement call to replace things to the lab.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Actually, many of those victory posts contain links to the original game threads - so many of those, will contain posts stating that one nation or another was defeated, and thanking someone for playing. So theoretically, with a few long hours of effort, some data could be pulled. The problem being how do you quantify it, if out of say 10 nations, 1 wins, 5 are defeated outright, and 4 others are alive at the end of the game?

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

The equip changing thing may be a fairly new change. I dont remember seeing it before either. Its been a couple patches since I last looked at a game log much.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Quite right. Its definitely not going to be as clinical a result as the AI tests are.

Ive gathered some data from the games that happen on my server. In fact its one of the reasons I started doing it.

But it doesnt have to be great data. Im HOPING that it tells us nothing at all. That would be best. But if the same nations seem to fall out of the game in the first wave, and the same ones appear to top the winner list, all with different people playing them... then that would be data worth noting.

People have often said "everyone knows" facts about weak and strong nations. When Ive glanced thru things I was never able to spot a correlation.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Its kindof fun watching the nations rise and fall.
Im thinking I could make a killing by giving odds and taking bets as each game progresses. :)

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

GP, I'm currently taking a course this semester that will require me to learn how to do all kinds of variance analysis (basically, correlation and covariance).

If you can get me the data, I can perform this analysis and comment usefully on the outcome.

In addition to simple placement, I (personally) am interested in the strength of a nation related to the distance it starts from every other nation. This might be a good measure to determine the shift in strength of a nation from early/mid/late.

As an example, many people claim that certain nations are vulnerable to rushes. Since not all nations can perform such rushes, an analysis comparing the 'rank' of a vulnerable nation to the distance it starts away from the rush nations might reveal whether there is, indeed, something to examine.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

The options for finding that out is:
peek into the game (which the methods corrupt the data)
Pre-set their locations on a map (which limits the data)
Or set the map size and collect lots of data to even out the numbers.

Im working with the last one though it takes the longest. At the moment all games are "medium" games by Dom3 standards. Im using a --randmap 15 on the game creation which generates a random map of 15 provinces per player. Later I will try the same with --randmap 10 (small game), and --randmap 20 (large game). And then I would be really curious to try it with something like --randmap 50 just to see the difference.

There is also another factor which has to be figured in. The AI uses type 1, type 2, or type 3 AI. Standard, Defensive, and Aggressive. I have no way to set those so every game, every nation, will receive a random AI.

So the end ratings would be 9? for each nation?
Early Ulm:
Small: std, def, agg
Medium: std, def, agg
Large: std, def, agg

I guess that each one could be just a number reflecting its avg rating. In Early Age there are 24 nations. Lets say 24 means usually first out of the game and 1 means usually winner. so if it was
Early Ulm:
....s.d.a
Sm:3:5:1
Md:6:9:3
Lg:10:13:8

then we would be able to show (at last as far as the AI handles Ea Ulm) that its game balance tends to be that it does well on quick-game/small-maps. Hopefully another nation shows exactly the opposite. (my hopes for that was on Arcos but Im beginning to show that Arcos has a problem which I will bringing to Johans attention soon)

BTW for those who prefer more than numbers. Im working on charts. And Im working on a "Watcher Nation" which will allow a Gods Eyes view of the game with many super stealthy scouts (flyers, maybe teleporters). It will be a non-playing nations which will allow "tourists" into the game. Not THESE games since it would violate the data but I will make some mega-huge AI game (possibly with players) available for spectating.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

GP: Yes, something like that can be done. My question is if it's possible to inspect various data to find patterns; e.g., is it possible to determine how far apart all the nations start?

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Not easily Im afraid. All we can go by is the outside info of small, medium, large map.

If I could view the map I would be more interested in who the neighbor is than how close it is. If Undead Ermor starts near Marignon then its weaker than if it starts near Ulm.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Wow. The latest Mid_AI test has run all night. Its too bad that the SuperStats stops recording at turn 200. That was fine when there were few nations but now thats definetly not the "game as good as over" point.

At turn 200 it was:
And I might have bet on Marignon or Rlyeh. Maybe Agartha.


When I went to bed the provinces were at
Agartha: 116
Machaka: 81
R'lyeh: 54
Man: 50
I would have bet that in a couple of hours Agartha was going to win with Rlyeh coming in second (just because the water nation tends to be the last taken out)

I wake up and its
Man: 106
Machaka: 97
Agartha: 95
Rlyeh: 0

Man in the lead.

Now 20 minutes later its Machaka in the lead 112 to Mans 99 and Agarthas 88.

Another interesting point is that the SuperStats shows
Man (ai 2 type 3)
Machaka (ai 2 type 2)
Agartha (ai 2 type 1)
Man is playing with aggressive AI, Machaka with defensive AI, and Agartha is with standard (middle of the road) AI.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Mm, that's a shame. The closeness of the neighbors will show the 'who' as well as 'what'. For example, if you're examining the probability that Ermor will win in the MA or LA, you can run a correlation analysis on the winning percentage (or perhaps 'xth to die') on their original distance from Marignon. If the analysis returns that there is a high positive correlation with distance, that would mean the the closer Ermor starts to Marignon, the worse they do.

Following that logic, if, say, MA Ulm consistently has a positive correlation with initial distance from most/all 'rush' nations, one inference you could draw would be that MA Ulm is highly vulnerable to rushes.

Now, I don't know much about the various switches involved, but... is it possible to manually set up a game, record the initial positions, and then set all the players to AIs, record what AI they receive, and use this utility to get the information?

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Hmmmmmm....
thats abit of a mix between the games I do, and pbem games that Im not well versed in. But it might be possible. Do a pbem start (sets up then quits), followed by superhosting after that.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

This Mid test is going to take 2 days to finish it looks like.
760 turns in and its:
Man: 137 playing aggressive AI
Machaka: 102 playing defensive AI
Agartha: 60 playing standard AI

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Wow this Mid test is still running.
Turn 883
Indepts 6
Man 155
Machaka 101
Agartha 40

Man and Machaka keep back and forth in first place position.
Man has more forts, Mahaka has more gems, Agartha has more research.
Interesting that Indepts are holding onto 6 provinces. They have an army size of 896 and climbing.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Turn 883 and the independents still have 6 provinces? The AI really doesn't know how to play this game. ;)

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Even worse, they haven't maxxed research yet. :shock:

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Indepts got to having 1006 units in 5 provinces. It might have been a smart AI to not tackle it if it wasnt a province it had to go thru. I see one of the Special Monsters got 8 units on the board and kept them till the end of the game.
The game ended at 921 turns with Man the winner. Indepts still had 2 provinces and 462 units. But since they werent needed for a win I guess its no big deal that Man left them.

I would love to have seen the turn before that. It went from
Man: 185 provinces and 16836 units
Machaka: 114 provinces and 7461 units
in one turn to Man being the winner. It might have been a taking of Machaka's capital and killing its god but since Machaka was playing defensive AI Im guessing it had a lot of army there.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Sounds like dom death, non?

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Your right.
Too bad, it would have been a glorious battle.

Well I am getting farther in my Watcher mod. Its usable now but I have to clean it up and come up with some filler before Id want to make it publicly available. Then I can watch, maybe record, some interesting battles.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Thats odd.
I just started a Late Era test. Atlantis (playing aggressive) was the first nation out (0 provinces). That isnt odd. But then they came back. They went up and down for abit between 1 and 3 provinces then died off again.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Atlantis is a nation which starts outside of the water in LA. If their cap is the last province under their control and being sieged, it shows as 0 provinces, until they successfully break the siege.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

As you were able to see how the AI allocates items, can you similarly see how the ai uses its gems?
Globals, how much they summon, if they just hoard gems in overgrowing piles and that kind of information would be interesting

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

That makes sense.:doh:
I could also have read the log. Thats why its there :)

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Probably random events.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

There's only one way that could happen--dominion kill. You can never take a castle in less than two turns.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Im putting the game logs in the same browser viewable directory so people can see some of what the game records. You can see spell castings, and I dont think construction shows up but it gives it to someone so you get an idea of what they like to make.

I could go ahead and turn that logging up a couple of notches to get more info.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

I have to go do some things now but I will try and look later to see what the last couple of turns were like. They went to 0 then up to 3 so they broke out at least once.

Maybe I should also "roll" the stats.txt to sequential copies also with the scoreboards. Its not much use since it only records who is AI (or human player) each turn. But it would record the actual turn that the AI "killed" a nation from the game.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Arrgghh!!
I think I might have to kill this test. This latest Late_AI test has been running for 3 days, and the latest turn in that game took 2.5 hours to host. And it still looks far from done. Im not sure I would even want to predict the winner. I hate to kill it but I could have done 3 other tests in this same time frame.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

By the way, for those who wish to "see" more of what the AI is doing:

Start a game with the --comptrn switch and add AIs into it.

This will create turn files for the computer players, and allow you to look at them. You can see what the computer is buying, making, researching, assigning, etc.

Hopefully this will allow some movement forward in creating mods which make use of the AIs actions, and avoid the worse ones.

Thank you Johan for another release of some of the NDA stuff (non-disclosure agreement). Im very sorry thru all of the arguments here that I was unable to discuss beta-group things such as the hidden switches. --allai, --superhost, and --comptrn being some of the more interesting and informative ones.

Re: Curious about Dominions?
 

You are correct. Unfortunately until recently I could not say that you were. For the longest time the -d switch was not to be discussed in forum at all. When it did get mentioned it came out as -d thru -ddddd. And as long as I could not find anywhere that Johan or Kristoffer had explained it further, I could not either.

But along with other non-documented switch info, Johan as verified that this game only suppors -d and -dd. You are correct.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Awesome! I am going to try this... soon. Next time I play Dom3, which will probably be within a week.

-Max

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Of course, they weren't really hidden. They reside in the same part of the executables as the other long opts.

where /path/to/your/dom3 should be your Dom3 install directory. I won't post the list here, because it is version-dependent. Also don't want anyone accusing me of reverse engineering in violation of license agreement, though I don't really see this as such....

Still very useful to have descriptions of what they do. Thanks.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

I know.
I often provide string-lists to people who request it. All of the units by name, all of the spells, all of the magic sites, all the forts, all the poptypes, all of the wish variables, all of the nation description texts, all of the tips file, all of the god titles, all the mod commands, all the spell effects, all the path variables, and of course all the switches. It saved alot of typing for people who wanted to create user-provided documentation on things.
You can find many in the docs directory of my server dom3minions.com

But I had to ask Johan before releasing any.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

OK now that --comptrn has also been revealed, Im considering adding it to these test runs. I would include into the web directory a zip file containing the rgb, fthrland, and all turn files for all the AIs.

A) it does allow seeing capital placements as analytic_kernel requested.
B) it would add to the processing time
C) it would be view only. You cannot play the game. Hitting "host" just gives you a "Game Over" message.
D) it might be of interest to new lurkers who would like to see a large end-game even if it was just AIs

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

If you are curious about the AI builds (pretender, scales, etc) then you can create an icon which will only create a game and allow you to view those.

Short Answer: add --allai and --comptrn to a dom3 command

Long Answer:
If you are using Windows, then right-click your Dom3 icon and choose "copy".
Then right-click a blank spot on your desktop and choose "paste".
Now you have a copy of your Dom3 you can change.
Right-Click on the new copy, select "Properties".
In the "Target" window is the command line calling Dominions 3. There are tons of fun switches you can add there. For this make it..
"C:\Program Files\dominions3\dom3.exe" --allai --comptrn

That will allow you to create a game with all of the nations in as AIs, pick up their first turn, look at the gods and scales. You wont be able to play but let me know what you want and I will see if I can work it out for you. :) You can usually find me in the chat room. (the chat button at the top of this forum)

Personally I love to add switches. I also like to add
-fwacd (windowed, no fades, no credit screen, keep a game log, no animated backgrounds)

--clickvol 10 to turn down the keyclick thing

--scoredump which creates a seperate scoreboard page each turn like you see on some of the multiplayer servers with numbers for how many provinces, castles, etc each player has

--magicsites 65 --renaming --indepstr 7 --hofsize 15 --eventrarity 1 still gives me a chance to change them when I start a game but it sets them as the defaults that the switches are on so I dont have to change them every time. So adding --renaming to my icon changes renaming from defaulting to off and me having to turn it on for every game, to defaulting to on with an option for me to turn it off if I want.

--seapart 20 --mountpart 50 --mapgcol 50 100 50 250 some of my preferences when it creates a map. I like less water, more mountains which creates many chokepoints, and I change the color of the plains from brown to green

MY icon for looking at the pretenders includes all of those. Also it has
--randmap 20 to create a large random map
--era 1 because thats what Im checking out right now
viewgame (no -- on this one) which creates the game name and overwrites the game viewgame each time I click the icon.

here is a list of all the switches...
http://www.dom3minions.com/docs/CommandLine.txt

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

'Twasn't me. Might have been VedalkenBear.
http://forum.shrapnelgames.com/showp...1&postcount=15

Still good to know. Thanks.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Any chance of a --setcombatvol XX switch ever? :D

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Afraid not anymore. :)
We are just stuck with the modified sound file collections which do make it easier to live with.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

Yes, GP, that was me. While being able to get these data is the first step, a proper analysis should be the most important step. If I have time in a couple of days, I will look into how to analyze this.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

The tests as they are done now were setup by Illwinter. They test everything. Not only balance but whether there is too much or too little random in map generating, god selection, scale selection, ai type, purchasing, research, etc etc. The problem being that with so much variation it takes a LOT of tests to average out any one thing such as the balance of nation strength. To test one thing, the other randoms should be limited as much as possible.

I have started looking at the maps generated on my site each day. If a set of statistics were to be done in a more controlled form then many controls could be added. The same map each time, close all the provinces so they cannot be starting provinces except specific ones so you know the distances already, give matching gods and scales (such as giving every race a monolith which is kindof hard for an AI to screw up).

Im thinking about using a medium sized map. But maps can affect the game SO much. A map with a central ocean surrounded by thin land strips, or one with a central land surrounded by thin ocean? One with multiple smaller oceans so each water nation can expand on its own but is in danger of dominion-death? Many smaller oceans and ponds which play toward the advantages of the sailing nations for having places they can strike by crossing one water province? How much of each terrain and how much should it be bunched.

Re: Curious about Dominions? how the AI thinks? which AI is best/worst?
 

I just have to say that --comptrn is indeed awesomely fun. I've always suspected, and now I know, that the AIs run out of money and stagnate because of troop upkeep for their massive armies; and now I know that they usually only research one school of magic up to very high levels before investing at all in other schools. Extremely interesting stuff.

-Max