Automatic backup of turn files
 

This is my first post and I hope I don't get a bad rep as 'the gal who propagates cheating tools', but 1) it was useful during my 'what? only 20 devils! Pretender, let's go!" phase, and 2) I want to contribute something (semi) useful, before I start bothering you with my 20000 questions.

Okay first the windows version for the mortals:

1) create a link of your dom3.exe (right drag to a location of your choice and select 'create link here')

2) right click on your link, select 'properties' and find the line containing 'dom3.exe'

3) add ' --preexec backup.bat'

4) open notepad

5) copy & paste the following lines:

<font class="small">Code:</font><hr /><pre>set Game=&lt;name&gt;
cd savedgames
del /S /Q "%Game%.backup"
mkdir "%Game%.backup"
copy "%Game%\" "%Game%.backup\"
</pre><hr />

save in your dom3 install directory as 'backup.bat'

You have to substitute "&lt;name&gt;" with the name of the game you're currently playing. It's clumsy but it's windows. So if your last game was called 'trash' and you start a new game called 'MoreTrash', you have to edit 'backup.bat' and set the appropriate name.

Now whenever something goes horribly wrong (power failure, your husband mentioning that the kitchen looks like a battle field or simply walking your mages while sneaking your meatshield), go to the 'savedgames' directory, delete the &lt;name&gt; directory and rename &lt;name&gt;.backup to &lt;name&gt;.

A much nicer version which finds the last played game, but requires python (though if you have Civ IV, you probably have python installed):


<font class="small">Code:</font><hr /><pre>#!/usr/bin/python
from os import *
from os.path import *
import shutil

def checkDate(name):
turnFile = join(name,"ftherlnd")
if access (turnFile,F_OK):
return getmtime(turnFile)
return 0

def findLastGame():
lastTime = 0
lastName = ""
chdir("savedgames")
for file in listdir("."):
if isdir(file):
time = checkDate(file)
if time &gt; lastTime:
lastTime = time
lastName = file
return lastName

def doCopy(name):
backup = name + ".backup"
shutil.rmtree(backup,True)
shutil.copytree(name,backup)

last = findLastGame()
if len(last) &gt; 0:
doCopy(last)
</pre><hr />

Use sparingly.

Hopefully someone find this useful..

kind regards
Pia Kraft

Re: Automatic backup of turn files
 

Very nice. And excellent explanation.

One note. If your icon has a Target line like
"C:\Program Files\dominions3\dom3.exe"
with the quotes (windows puts the quotes because of the space in that line) then put the --preexec backup.bat outside of the second quote.

It also answers a question I had about windows variables that I was going to look up so that I could convert some of my linux scripts to windows batch files.

Re: Automatic backup of turn files
 

Another thing I found useful is to pass the game name on the --preexec line.

--preexec "backup.bat GameName"

This way you can use the same script for every game you play and just change the line in properties.

If you also generate the score dump, you can parse that for the turn number and actually save each turn.

I had a fairly simple bash script version, but gave up on converting it to Windows, since the current version behaves badly when running the postexec, at least if it's full screen.

Re: Automatic backup of turn files
 

The passed name is a good thing. Thanks.
I can see it being useful since I tend to create a seperate icon for every server game Im in because it saves me changing the server and port when Im playing multiple games on multiple servers. With your hint I can have one bat file making backups of all of my MP games, and just have each icon telling it which game to save. Very handy since multiple games makes it hard to remember what I did last turn.

Re: Automatic backup of turn files
 

Thanks, that was exactly what I was looking for.

I attached a python script. Instead of long winded explanations:

<font class="small">Code:</font><hr /><pre>pia@Lollipop:~/dominions3$ ./dom3backup.py --help
usage: dom3backup.py [options]

options:
-h, --help show this help message and exit
-n NAME, --name=NAME Process file &lt;name&gt;. Otherwise latest file
-f FREQ, --freq=FREQ create a Zipfile every &lt;n&gt; turns. Default: 0 (no
zipfile)
-m MAXFILES, --maxfiles=MAXFILES
keep max. &lt;n&gt; Zipfiles. Delete oldest. Default: 0
(keep all)
-d ARCHDIR, --archdir=ARCHDIR
create Zipfiles in archdir. archdir must exists.
Default: &lt;dom3&gt;/archivedgames
--nocopy don't copy files to backup-dir. Default: False
--clean remove all zipfiles and the backup-dir
</pre><hr />

Re: Automatic backup of turn files
 

I posted a batch file for playing with the map making commands. It makes it easy to generate map after map with the same settings. It also lets you play with things like the color settings. I posted it into the MOD/MAP subforum since I think it is more useful to that crowd.

Re: Automatic backup of turn files
 

Many thanks Lolly (can I call you that ? http://forum.shrapnelgames.com/images/smilies/wink.gif ) !
I've installed Python and copied the script, it works quite well as advertised. Only small problem is that if Dom3 runs in fullscreen mode it's automatically minimized when executing the script... I suppose it's a Windowze thingie and can live in windowed mode.
Would it be easy to not only backup the last turn but also make a file history, I mean having subdirs with each turn's saves ? I know the game doesn't pass the turn numbers, so the utility should assume the player uses it by turn 1..
Just an idea, but it'll be cool http://forum.shrapnelgames.com/images/smilies/happy.gif

Re: Automatic backup of turn files
 

By way of an exception, cause you was brave enough to try my scripthttp://forum.shrapnelgames.com/image...es/biggrin.gif

Yes, it is. I doubt there's a way around


The script already does it, kind of. I was in a hurry when posting cause my boss was approaching and I wasn't supposed to work on a dom3 script:).

1) create a 'archivedgames' directory in your dom3 directory.

2) modify the dom3 link to:
dom3.exe --scoredump --preexec 'python dom3backup.py -f 1'

The script parses scores.html to get the turn number (assumes turn 1, if not found) and creates a zipfile (name-Turn&lt;turnnumber&gt;.zip) every '-f &lt;n&gt;' turns (in archivedgames, optionally use the -d arg), with '-f 1' every turn.

You can limit the number of zip-files kept with '-m &lt;n&gt;', default is to keep all.

I hope this isn't too confusing.

Caveat: I haven't tested it under xp. The pyhton-doc says, that you need zlib (prob. not present) only, when deflating, so I used a simple store mechanism. But who knows..

Thanks for testing

Pia

Edit: Gnu zlib for XP is here. Change the line:

zipFile = ZipFile(zipName,'w')

to:
zipFile = ZipFile(zipName,'w',ZIP_DEFLATED)

Compression ratio is 1:10, not bad.

Re: Automatic backup of turn files
 

Wunderbar !! That's just what we needed to have a real "save" feature in Dom3 http://forum.shrapnelgames.com/images/smilies/happy.gif
Can I kiss you Lolly ? http://forum.shrapnelgames.com/images/smilies/laugh.gif

In fact in my previous post I had just tested the first small script that you gave in your first post and didn't notice there was a second post with attachment !! I'm distracted...

About XP I had everything working but had to "spell" the shortcut command like this :
&lt;dom3path&gt;\dom3.exe --scoredump --preexec "&lt;Dom3 path&gt;\dom3backup.py -f 1"
With &lt;dom3path&gt; my Dom3 directory where I've also copied the .py file. Double apostrophs are required ("), simple ones (')don't work.

This tool give me ideas but I don't charm Pythons like you do so well http://forum.shrapnelgames.com/images/smilies/wink.gif , could you please continue this fine work ? http://forum.shrapnelgames.com/images/smilies/wink.gif
- I don't like much having the .backup reps directly under Dom3 rep, I'd prefer to have them underneath each game's rep, ie Dominions3/MyGame/MyGame.backup
I suppose it's rather easy to change ? Just tell me what lines/command to edit, I'm able to do that.
- For pbems, now that we got automatic numbered zipped turn files we just lack an automatic email sending to a list of addresses (the players) ! http://forum.shrapnelgames.com/images/smilies/cool.gif Do you think you could handle that ? (I'm sure you can http://forum.shrapnelgames.com/image...es/biggrin.gif ! )
For Dom2 I had a buddy who make scripts to use a small free command-line email tool called "postie", perhaps we could use it also ? http://forum.shrapnelgames.com/images/smilies/smile.gif

Re: Automatic backup of turn files
 

I guess I have to ask my husband first. http://forum.shrapnelgames.com/images/smilies/happy.gif

Very well, Master, your wish is my commandhttp://forum.shrapnelgames.com/image...es/biggrin.gif. I'm working on it.

I lack any dom pbem experience, but I think, --pre/postexec only works when hosting. So this would be a stand-alone program. Like reading the addresses from &lt;gamename&gt;.pbem and sending the .2h file? Something like that?

However, I only used python cause it was available at work (which says something about my working morale). Python isn't my favorite language &amp; if that thing gets bigger, I'll use ruby. No worries, it's a 1.x mb download iirc and called 'ruby-one-click-installer.exe'.

And could you do me a favor and test, if the zip file creation works under XP. TIA.

Pia

Re: Automatic backup of turn files
 

Lolly,
Sure I did test it and it works fine under XP ! http://forum.shrapnelgames.com/images/smilies/happy.gif
There's even a nice log file to knwo what happened http://forum.shrapnelgames.com/images/smilies/wink.gif.

About my request to change the path I've found where the script should be changed, in the Backup variable definition, but ain't sure about how to type those "\" or "/" according to this language conventions and with possible OSes incompatibilities... so I let you do it ! http://forum.shrapnelgames.com/images/smilies/wink.gif

As for pbem there are in fact two different needs :
A simple player has to send its 2h file but doesn't "host" as such, and there's no means to detect if he wants to really end the turn and send it or if he would like to send it later... Plus there's only 1 simple mail with the .2h file to send to the hosting player. Maybe a small tool can help, but not much,as anyway the player has to make an action.
I was rather thinking of something for the hoster, who has to send all .trn files (+ ftherland , some don't but it makes a backup indeed !) to all of the players.
Some even make separate emails for each player with each turn file attached !
Here a tool would be great, and it could be easily automated to follow the hosting.
Make it in Python, Ruby, or whatever snake species or gem type you want, it'll be great anyway ! http://forum.shrapnelgames.com/images/smilies/happy.gif

And what did reply your husband for the kiss btw ? .. http://forum.shrapnelgames.com/image...s/confused.gif http://forum.shrapnelgames.com/image...es/biggrin.gif

Re: Automatic backup of turn files
 

Incredible, but true. I did write some documentation, completely voluntary, without someone threatening me with a shotgun. http://forum.shrapnelgames.com/image...es/biggrin.gif

Here's the readme of the new version:
<font class="small">Code:</font><hr /><pre>*Synopsis*

A python script to auto-backup your dom3 games

*Requirements*
python 2.4 or better. Get it from http://www.python.org/download/. If you have Civ 4,you have it
(I hope :)

zlib. Get it from http://gnuwin32.sourceforge.net/packages/zlib.htm. Linux has it (usually)

*Installation*

- extract dom3backup.py to your dominions3 directory

- Linux:
create a shell script like:
#!/bin/bash
./dom3 --scoredump --preexec './dom3backup.py -q' &amp;

chmod +x &lt;your name goes here&gt; business as usual

-Windows:
-create a link of your dom3.exe (right drag to a location of your choice and select 'create link here')

-right click on your link, select 'properties' and find the line containing 'dom3.exe'

-add ' --scoredump --preexec "&lt;Dom3 path&gt;\dom3backup.py -q"'
the ' are only there to show blanks, don't enter. The " however are required.

-enter further arguments. Explained below

-Mac:
sorry, no idea

*Usage*
dom3backup always backups the game, you've last played. If you want to backup another game, use the
-g option.
dom3backup uses a simple substitution. Whenever you specify a directory, #name is replaced with the
current game name (either last game played or specified with -g), #turn is replaced with the
current turn-number.
Relative paths are relative to &lt;dom3&gt;/savedgames

-a ARCHIVE, --archive=ARCHIVE
create archive &lt;name&gt; now. Usually used from the commandline
-b BACKUPDIR, --backupdir=BACKUPDIR
create backup in backupdir. Default:
&lt;dom3&gt;/savedgames/#name.backup.
Example:
last game you played was 'ermor1'
no arguments:
backup all your files to &lt;dom3&gt;/savedgames/ermor1.backup/
-b ./#name/backup
backup all your files to &lt;dom3&gt;/savedgames/ermor1/backup/
-b c:/d3archive/#name
backup all your files to c:/d3archive/ermor1/
-b ../saved/#name.#turn
all your files to '&lt;dom3&gt;/saved/ermor.1',then &lt;dom3&gt;/saved/ermor.2' etc.
-b ../saved/#name/#turn
all your files to '&lt;dom3&gt;/saved/ermor/1',then &lt;dom3&gt;/saved/ermor/2' etc.
-d ARCHDIR, --archdir=ARCHDIR
create Zipfiles in archdir. archdir must exists.
Default: &lt;dom3&gt;/archivedgames. Absolute or relative paths as the -b option
-f FREQ, --freq=FREQ create a Zipfile every &lt;n&gt; turn(s). Default: 0 (no
zipfile)
-g GAME, --game=GAME backup game &lt;name&gt;. Otherwise backup latest game
-m MAXFILES, --maxfiles=MAXFILES
keep max. &lt;n&gt; Zipfiles. Delete oldest. Default: 0
(keep all)
-q, --quiet output messages are written to &lt;dom3&gt;/dom3backup.log
-r, --restore restore files form backup-dir. Use only from the command line
-t TURN, --turn=TURN assume turn &lt;n&gt;. Only useful with -r. Use from
commandline only
--nocopy don't copy files to backup-dir. Default: False

*Putting it all together*
Assuming windows...
Ex.1) dom3.exe --scoredump --preexec "&lt;Dom3 path&gt;\dom3backup.py -q"
once you hit 'e', your files are copied to &lt;Dom3 path&gt;\savedgames\&lt;gamename&gt;.backup
Ex.2) dom3.exe --scoredump --preexec "&lt;Dom3 path&gt;\dom3backup.py -q -f 1"
same as above, but every turn a zip file is created in &lt;Dom3 path&gt;/archivedgames
The zip-file is named &lt;game-name&gt;-Turn&lt;turnnumber&gt;.zip
Ex.3) dom3.exe --scoredump --preexec "&lt;Dom3 path&gt;\dom3backup.py -q -f 4"
same as above, but zip files are only created on Turn1, Turn 5, Turn9 etc.
Ex.4) dom3.exe --scoredump --preexec "&lt;Dom3 path&gt;\dom3backup.py -q -f 4 -m 2"
same as above, but only 2 zip-files are kept at any time. So if &lt;name&gt;-Turn9.zip is created, &lt;name&gt;-Turn1.zip is deleted
Ex.5) dom3.exe --scoredump --preexec "&lt;Dom3 path&gt;\dom3backup.py -q -b ../saved/#name"
your files are copied to '&lt;dom3&gt;/saved/&lt;gamename&gt;. Of course you're free to also create
zip files, see above
Ex.6) dom3.exe --scoredump --preexec "&lt;Dom3 path&gt;\dom3backup.py -q -b ../saved/#name/#turn"
copied to &lt;dom3&gt;/saved/&lt;gamename&gt;/1,&lt;dom3&gt;/saved/&lt;gamename&gt;/2 etc

Restoring backups:
If you don't used -b, the *last* turn from &lt;Dom3 path&gt;\savedgames\&lt;gamename&gt;.backup is
restored.
Same as above, if you used -b *and* no #turn substitution
If you used #turn substitution, you *must* use the -t argument

So start a shell and:
Ex.1) python dom3backup.py -r
= restore last turn
Ex.5) python dom3backup.py -r -b ../saved/#name
same as above
Ex.6) now you must pass a turn number
python dom3backup.py -r -b ../saved/#name/#turn --turn 1
start from scratch
*Caveats*
Windows: you must use -q
Windows: use / in pathnames, not \
All: if you mess around with the savedgames directory, the auto-detection might no longer work.
Files containing a '.' and newlords are ignored, but if you copy the 'MyFavoritePorns' folder
to savedgames, your pbem-partner might be up for a surprise :)

GPL applies......

Comments to ZarahNeander at t-online dot de

04/11/06 Pia Kraft

</pre><hr />

Re: Automatic backup of turn files
 

Thanks much, good to know

Done. See the new attachment. The substitute mechanism is quite flexible and should satisfy even you http://forum.shrapnelgames.com/images/smilies/happy.gif

Regarding / and \. Internally I only used /, so it seems, that python is smart enough to convert / to \ under XP. It *might* even necessary to use / in arguments passed to the script.

I see. Thanks for elaborating. Shouldn't be too hard. The only problem, while I don't run unit tests and such, I like to test my software at least rudimentary, which means, I have to host a pbem game. And a pbem with only a host is somewhat pointless. And I don't want to incur the eternal wrath of players by using a real pbem as my testbed. So I'm looking for some kind of sacrificial lamb...


Let me put it this way: I won't give away my age, suffice to say, I'm seasoned, quite seasoned http://forum.shrapnelgames.com/image...es/biggrin.gif

Re: Automatic backup of turn files
 

Hi again Lolly,
Well, the new version is even better, thanks !
I just have have a problem, the -d (archive dir) option doesn't work for me, whatever I specify all archive zips end up in savedgames directly !

Could I just ask (again) for a direct "restore" command to which one would just have to say (type rather)
"restore &lt;game&gt; &lt;turn&gt;" ?

I'm volunteer to test a pbem tool with you also http://forum.shrapnelgames.com/images/smilies/happy.gif

And I kiss you whatever your age may be, I'm not that young either !

Re: Automatic backup of turn files
 

Hi PDF,

The -d thingie is weird, it works for me. The zip file is created in savedgames and then moved to &lt;archdir&gt;, so it's safe to say, that the 'move' command failed.

- the archdir *must* exist. This is an anachronism from the early development, and it also means, that the #turn substitution doesn't work (except you're busy creating directorys). I'll fix that soon.

- It might be a /\ problem. Unfortunately I don't have XP at home.

You might fire up a shell and start dom3backup manually. Most likely it throws an exception &amp; most likely 'path not found' (shame on me, I didn't catch exceptions)

With regard to a 'restore' command, I image that dom3backup on XP with it's stone age shell is really hard to use. However the public interest isn't exactly overwhelming. I admit that the thing is hard to setup (only partly my fault) and that writing user documentation isn't my strongest side. But atm I'm not sure if I continue public development. Programming is fun for me, writing documentation not.

I will however try my hands on the pbem tool, if only cause I'm curious how python handles mime (which seems quite reasonable, another chance lost to bash python on #python-lang http://forum.shrapnelgames.com/image...es/biggrin.gif). So thanks for volunteering.

Pia

Re: Automatic backup of turn files
 

Hi again !
I'd a look at the code, maybe there's a problem with the way default archdir is defined as ../archivedgames ... but I can't get any path to work with archdir ! OTOH the backup path system works perfectly.
Additionaly I can't see the messages because the window closes too fast, and not everything is traced in the log...

I've tried to run the script in debug mode with IDLE to see the variables but I don't know how to "trace variable" in the debugger.

Well I think I'll don't bother anymore and let my archives accumulate in savedgames folder ! http://forum.shrapnelgames.com/images/smilies/wink.gif

Re: Automatic backup of turn files
 

send you a PM, PDF

Re: Automatic backup of turn files
 

Nice python scripts, thanks.

I have been using some crude bash scripting myself, you could drop this in your ~/.bashrc if you dare:

<font class="small">Code:</font><hr /><pre>function archive {
if test $# -lt 1; then return; fi
tar czvf $(basename $1)-$(date +%s).tgz $@
}

function archive-title {
if test $# -lt 2; then return; fi
argv=($@)
tar czvf $(date +%s)-$1.tgz ${argv[@]:1}
}</pre><hr />
The first "archive" command takes as parameter the path to your game in your savegame directory (you could chmod into the savegame directory first to save typing) and adds a timestamp to the archive, the second "archive-title" command expects some title as first parameter for the savegame and prefixes it with a timestamp (in case you want to have a better view over what the savegame is about, like "SpammedMindHunt").

Yes, that code is ugly and I never went around to improve it, so you better use the python script instead. http://forum.shrapnelgames.com/images/smilies/happy.gif

Re: Automatic backup of turn files
 

I set every game to run a --preexec and a --postexec which usually start out as blanks. But that way if the game gets interesting I can easily add commands to those. Such as having the pre exec do a zip of the game before each hosting.

Thats one of the many MANY reasons that I pushed to get those added to the command line. It handles so many things