Drawing some pretty graphs from scores.html files
 

Heya, I've been writing some script which parses the scores.html files that the game generates if it is run with the --scoredump option. That project was on hiatus for some time, but I've finally invested some time to clean it up and make it accessible for others.

I'll add more info at a later timer. For now, just look at the graphs for my Dom3 multiplayer games here:
http://dom3.servegame.com/scores/

You can get your own graphs that look like this! To make things easier for the non-techie folks, I have moved most of the code out of the PHP script and to Javascript which is handled by the browser. This allows others to use my system like I do and configure it to some greater extend, too.

Long story short: go to the URL above. You'll see a list of my MP games. When you click on one, you see the current graph for the ingame scores. You can get a page like that for your own game if you use the Upload button with your own scores.html files. You can then save the generated HTML file and put it on your own website or send it via email.

Source code is available on request (needs PHP 4/5).

README
 

Space reserved by me.

ToDo:
- make the page not look like ***.
- write documentation about the configuration options
- configuration of the display options via the upload form

Re: README
 

More space reserved by me.

Feedback
 

I'm greedy.

Re: Feedback
 

I'm probably missing a plugin, but all I see (firefox 2.0, opera 9.1, with flash player 7.0, javascript enabled in both browsers) is:

Provinces
Loading Adobe Flash graph, please wait...
Forts
Loading Adobe Flash graph, please wait...
Income
Loading Adobe Flash graph, please wait...
Gem Income
Loading Adobe Flash graph, please wait...
Research
Loading Adobe Flash graph, please wait...
Dominion
Loading Adobe Flash graph, please wait...
Army Size
Loading Adobe Flash graph, please wait...
Victory Points
Loading Adobe Flash graph, please wait...

Any idea what could be wrong?

Re: Feedback
 

I think you need at least Flash player version 8 (current is 9). Try this page, which should tell you what you need to do to get it working: http://blog.deconcept.com/swfobject/expressinstall.html

Re: Feedback
 

Great, thanks for the help!

Re: Drawing some pretty graphs from scores.html fi
 

Very, very impressive. Well done.



Now if there were only a way to track total stale time for each nation......

Re: Drawing some pretty graphs from scores.html fi
 

Beautiful!

Have a cake [img]/threads/images/Graemlins/Cake.gif[/img]

Updates
 

Some updates:
- graphs now scale correctly, the Newbies game doesn't look THAT bad anymore for R'lyeh http://forum.shrapnelgames.com/images/smilies/happy.gif
- upload mechanism allows fine-tuning of some options now, for example you can set the colors for a specific game era or turn off some graphs if you want
- source code can be inspected, but beware, it's ugly. I put everything into one big file instead of separating it into different PHP files. The core consists of only a couple dozen lines of code since the Javascript code does the real work, the rest is just HTML for the overview/upload pages.

Real graphs demo
 

A little preview of the future: http://valhalla.techno-mage.com/webstats.php

Not sure yet if I like Flash or Javascript graphs more...

P.S.: Zooming is done by selecting an area inside the graphs.

Re: Real graphs demo
 

Is your valhalla.techno-mage.com site hosted by Gandalf? I can't seem to get to it at the moment.

Re: Real graphs demo
 

Yes Im afraid that it is.
Estimates are that it should be back in a week. http://forum.shrapnelgames.com/images/smilies/frown.gif

If lch has copies we could upload them to the other server for now

Re: Real graphs demo
 

Nope, I don't have backups of those scripts.
Wouldn't make any sense without the game backups, anyway, since I'd have nothing to plot graphs for. http://forum.shrapnelgames.com/image...ies/tongue.gif

Re: Real graphs demo
 

Is Gandalf's server back up yet by any chance? Do you have access to those lovely scripts lch? I'd love to use them for the LlamaServer if you were happy to let me.

Re: Real graphs demo
 

I'm sorry, but I don't have anything that I could pass on to you right now. I can tell you how I implemented this, though. I wrote a simple parser for the scores.html files that the game can generate. For the initial per-turn charts, I used the column/bar Flash graph libraries from amCharts and for the later MySQL-backed graphs I used the JavaScript library Flotr, I think, or its predecessor Plotr. The flash graphs needed the input formatted as XML, I don't remember what the input format for the other graphs was, maybe CSV. Both could be supplied inline in the dynamically generated HTML - rendering was done on the client side. I picked both because they only require a small number of files on the webserver, so it's very manageable.

Here's a list of national colors, which you might be interested in when plotting: http://www.shrapnelcommunity.com/thr...?Number=561877

Re: Real graphs demo
 

Good timing. Yes its back and the links should all be working. http://forum.shrapnelgames.com/images/smilies/happy.gif Thanks to all who donated

Re: Real graphs demo
 

Ooh, awesome! I just looked at the graphs again, they are really sweet.

lch, now that Gandalf's server is back up, is it possible I could use your scripts? If you'd prefer me to make my own I'd understand. Yours are really cool though, and of course I'd give credit.

Re: Real graphs demo
 

As long as you don't ask compensation from me for having to deal with my hacked together code, you can do what you want with it. http://forum.shrapnelgames.com/images/smilies/happy.gif

I already linked to the source code which generates the per-turn graphs on the first page. What you'll have to add is the files from amCharts. If you have some time, then I'll package the stuff together and release it, I am planning to put all my code into proper Mercurial repositories anyway. If you want the goods NOW, then I could just zip up my web folder and remove unrelated content and let you figure out the rest.

Re: Real graphs demo
 

Thanks lch! Hmm, well I'm in no great rush really. Xietor wants me to sort out score graphs for KingMaker, but I think the main reason is to see how many capitals each nation controls, which I think this actually won't really help with (I can't remember if we've got VPs on or not, but if we have then we can see the scores in game, and if we haven't I think these graphs won't tell us capital counts - is that right?).

Re: Real graphs demo
 

That's right. You could generate graphs for graph-less games with this and e.g. display them after the game, but that requires that you enabled scores.html generation while hosting.

I was thinking of reverse-engineering the fthrlnd file format to be able to display who holds which provinces, but I haven't really begun on this yet. Another one of those plans that you "want to do, definitely, at some later time".

Re: Real graphs demo
 

There's no need to do that. If you just host using the -d switch I think you can parse the output to see provinces changing hands, and you can keep track like that.

How were you thinking of displaying that information? It would be super lovely to be able to colour in the map with who holds which provinces, then view it as a movie at the end of the game (a bit like Civ I, if you remember that). Seems pretty tricky to me though. The only thing I can think of using is scripting in GIMP, but I'm currently clueless on that.

Re: Real graphs demo
 

That's exactly what I wanted to do, yes. An easier intermediate result to achieve would be just to place national standards on the province centers just like the game does.

That's very interesting. I had a look at the debug output, but I didn't really see how to accurately tell who conquered which province. Maybe I should look again. I'm always hosting with -ddd, so maybe the information just got buried.

Re: Real graphs demo
 

Aha, that's an excellent thought! It would indeed be much easier, I'd think.

Come to think of it, I think that bit of info from me actually came from a comment Gandalf made some time ago, and not from my own testing. Hopefully it's true though.

Re: Real graphs demo
 

If I remember right, grep the log for " vs " and "winner".
I believe that lnr is "land number"

Edited:
Yes its
2 vs 25 in lnr 8 (castlemode 0) laststand_att0 laststand_def0
(lots of combat lines)
_____The winner is 2_______

that was Ulm vs Independents in land 8

Re: Real graphs demo
 

I do remember a conversation thread about this awhile back. We started collecting flag images, nation colors. And I started asking everyone who had a map generator project to include a way to save the initial random seed so that you could rerun the map with no colors and graphics the way DomMap did for Dom2. Of course that was back before we could all get free graphic software that handled layers easily.

Re: Real graphs demo
 

When I tested this there was some sort of problem, I don't really remember which one. It might have been one of those:
- problems with battles from the magic phase
- problems with assassinations
- problems with random events ("the people have revolted!")
It could just be that I actually looked and tested for something different, though, that might be possible as well.

I might look into this a little closer later.

Re: Real graphs demo
 

Hallo lch,

As it turns out I have a fair bit of time for sorting the graphs over the next couple of days. If it'd be possible to just zip up your folder and upload it so I could try to find the graph-making bits, that would be awesome. Thanks!

Re: Real graphs demo
 

Man, I just took another look at this to show a friend. It is _ridiculously_ nice.

Re: Real graphs demo
 

Lch could be ridiculed for making such awesome graphs? Man, life is tough. http://forum.shrapnelgames.com/images/smilies/wink.gif