![]() |
2.08 and Incompatible Battle Reports
Any news on this issue? I couldn't try it myself yet.
It seems like the mentioned compiler patches might include a fix for the bug described in http://www.shrapnelgames.com/cgi-bin...=000872#000014 [ February 17, 2004, 13:43: Message edited by: BugRoger ] |
Re: 2.08 and Incompatible Battle Reports
Not sure, except I looked at my battles run in 2.06 after I upgraded to 2.08 (on Windows Version), and I didn't notice any changes in what I remembered happening. Not very scientific, but it's not obviously messed up.
PvK |
Re: 2.08 and Incompatible Battle Reports
We havn't found the bug, so unless it was hidden in some other fixed feature it is not solved.
The good news is that we have added a new checksum instrument that hopefully will give us some clues regarding the bug if we come across a faulty battle replay. |
Re: 2.08 and Incompatible Battle Reports
If we are lucky bug rogers is right here and the microsoft compiler bugs caused most of battle replay bugs. Even if there's something more wrong it should at least be better now.
|
Re: 2.08 and Incompatible Battle Reports
Quote:
And if you use MSVC6 (or 5), complier patches [surprisingly] don't include STL patches which you can get from http://www.dinkumware.com/vc_fixes.html |
Re: 2.08 and Incompatible Battle Reports
Some good new for those who don't play Dominions on Windows exclusively. The problem with incompatible battle replays has been found. There was a disagrement on how to compile an expression between microsoft's and the others' compilers.
Magic resistance check: if (penetration+2d6 < MR+2d6) Other compiler solved it from left to right, but microsoft's solved it from right to left. This results in the same roll of dice appearing on different sides of the '<' for windows and linux. Bloody annoying! http://forum.shrapnelgames.com/images/icons/icon8.gif Alexti: Thanks for the STL info and I use MSVC6. [ February 25, 2004, 17:35: Message edited by: Johan K ] |
Re: 2.08 and Incompatible Battle Reports
Yay! Not only was the problem solved, but also, Microsoft gets another black mark http://forum.shrapnelgames.com/images/icons/icon10.gif
|
Re: 2.08 and Incompatible Battle Reports
Quote:
Don't use two function calls in the same expression, is the lesson. (And congrats on the Illwinter crew! When are we getting a patch so we can celebrate?) http://forum.shrapnelgames.com/images/icons/icon7.gif |
Re: 2.08 and Incompatible Battle Reports
Quote:
...unfortunately, it's calling from a moose-laden Smouldercone right now. http://forum.shrapnelgames.com/images/icons/icon10.gif |
Re: 2.08 and Incompatible Battle Reports
Quote:
1: a=2d6+penetration 2: b=2d6+mrst 3: if(a>b){do c} Lines 1 & 2 could be rearranged by a compiler if, for example, mrst is more local (already in a register) and penetration has to be fetched. That would have the same effect. I think C has some commands that let you force the complier to not rearrange things, though. So the moral is actually not to trust Microsoft products, since they take away your control http://forum.shrapnelgames.com/images/icons/icon10.gif |
Re: 2.08 and Incompatible Battle Reports
There may be #pragmas or compiler options to change reordering behavior, but I'm not too familiar with MSVC.
Wrapping the randomizer in an object and making sure that the "get a random number" method (perhaps as a static method, even) might serve as a hint to the compiler; could be more work than it's worth. [edit] Making sure that it's not flagged as constant, that is. IOW to encourage the compiler to consider side-effects of function invocation. [ February 25, 2004, 18:51: Message edited by: Taqwus ] |
Re: 2.08 and Incompatible Battle Reports
I have often battle inconstancies message now, with solo play on windows platform.
|
Re: 2.08 and Incompatible Battle Reports
Way outside of my area but I was thinking that the Devs might have to write their own randomizing routine, and have the host create a seed. Then pass the seed its using inside each of the player files so that the players copy of Dom would definetly use the same routine on the same seed. I never figured on different OSs doing standard math in different orders. Thats insane.
[ February 25, 2004, 18:58: Message edited by: Gandalf Parker ] |
Re: 2.08 and Incompatible Battle Reports
Quote:
|
Re: 2.08 and Incompatible Battle Reports
Quote:
|
Re: 2.08 and Incompatible Battle Reports
I have seen this bug about 3 times for all of the games I have played. It is very rare... and definitely a bizarre bug.
It has worked both ways for me... One battle it showed me losing... yet on the map I won. Another battle it showed me winnning... yet on the map I lost. 2 of these were seen with patch 2.08 installed. [ February 25, 2004, 19:17: Message edited by: NTJedi ] |
Re: 2.08 and Incompatible Battle Reports
*scratches head*
'volatile' may also help. The MSDN C++ Language Reference states, "Objects declared as volatile are not used in optimizations because their value can change at any time". Perhaps creating an 'int volatile do_not_reorder_me = 0;' declaration, explicitly separating the computations into multiple lines e.g. int lhs = lhs_base + invoke_2d6() + do_not_reorder_me; int rhs = rhs_base + invoke_2d6() + do_not_reorder_me; would work. I haven't played around with that keyword much however -- been awhile since I wrote MT code. |
Re: 2.08 and Incompatible Battle Reports
Quote:
1: a=2d6+penetration 2: b=2d6+mrst 3: if(a>b){do c} Lines 1 & 2 could be rearranged by a compiler if, for example, mrst is more local (already in a register) and penetration has to be fetched. That would have the same effect. I think C has some commands that let you force the complier to not rearrange things, though. So the moral is actually not to trust Microsoft products, since they take away your control http://forum.shrapnelgames.com/images/icons/icon10.gif </font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Having done quite a bit of compiler beta-testing in my younger days, what this basically amounts to is an issue of (no offense to IW intended) somewhat sloppy coding practice by programmers. If you want to force the compiler to evaluate things a certain way, then make liberal use of parenthesis and local-scope intermediate variables. But what I see all too often is programmers that try to cram as much code as possible into a single line. That's just begging for trouble. And it's an absolute no-no when writing cross-platform code. BTW, the MSVC compiler is actually very good. A bit odd at times, but good nonetheless. You just have to be very well acquainted with all it's various switches, directives, etc. The compiler doesn't really take control away from you. It's settings default to newbie usage. Which can be overridden. And should be, by professional coders. My $0.02 worth on the subject. |
Re: 2.08 and Incompatible Battle Reports
Quote:
Microsoft strives to prevent inter-os compatibility; maybe using an Intel compiler would be best=) |
Re: 2.08 and Incompatible Battle Reports
Quote:
The problem here is that the lines are NOT LONG ENOUGH! The Microsoft compiler sees a simple rearrangement it can do to break inter-OS compatibility, and does it. If the lines are so long and confusing that the compiler can't figure out how to mutate them without breaking the program, it will just process them in order like it is supposed to http://forum.shrapnelgames.com/images/icons/icon10.gif Really! I promise! http://forum.shrapnelgames.com/images/icons/tongue.gif |
Re: 2.08 and Incompatible Battle Reports
Eh... why bother with all the complicated methods of ensuring the calls happen in the right order. Instead, change:
if penetration+2d6 < MR+2d6 To: if penetration < MR+2d6-2d6 Since both of the random calls are on the same side of the equation, the defined precendence order will ensure they are called in a consistent manner. |
Re: 2.08 and Incompatible Battle Reports
Quote:
|
Re: 2.08 and Incompatible Battle Reports
Quote:
As I understood it, the real problem was that in the expression "a < b" the order of evaluation of "a" and "b" is undefined [Ansi-C, and its derivatives, with a few exceptions only specifies the order of evaluation for operators, not for sub-expressions]. Either side "a" or "b" can be evaluated first. Given that the seed is the same, assume the RNG will return R1 and R2 over the next two calls in that order. Then the inequality: (penetration+2d6 < MR+2d6) can legally be compiled such that the evaluation is either of the following: penetration+R1 < MR+R2 (left hand side evaluated first) penetration+R2 < MR+R1 (right hand side evaluated first) - Which may or may not give different return values. Changing the inequality as AStott suggested, to penetration < MR+2d6-2d6 would not necessarily solve the problem either, as the order of evaluation of the two 2d6 function calls is not defined either. (The order of the addition and subtraction is, but not the order of evaluation of the sub-expressions) If you are in doubt, split such expressions over multiple commands independently evaluated. A bad optimizer may still hurt you, but at least you won't be bitten by the "undefined evaluation order", which is nobody's fault but your own. Or, to quote Kernigham & Ritchie: Quote:
[ February 25, 2004, 23:23: Message edited by: Peter Ebbesen ] |
Re: 2.08 and Incompatible Battle Reports
A cogent, well-stated reply, Peter. The quote by K&R was what I tried to allude to in an earlier posting.
BTW, it's not so much that the opitimizer may be "bad" than that it may be using "overly aggressive" choices. I remember the Borland compiler dev team in the early 90s having many headaches over just how far they should go. In those days, Borland and MS kept trying to out-do each other via how powerful their optimizations were. After a few cycles of this we started seeing cases of too much optimization. A careful review of what all the default optimization switches do should be undertaken by anyone that's serious about code-writing, especially cross-platform code, to avoid potential pitfalls. Hell, one should also look at the linker's switches too. |
Re: 2.08 and Incompatible Battle Reports
Quote:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> void print_blank_line { printf("\n"); } void print_2blank_lines() { print_blank_line(); print_blank_line(); }</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Do you think compiler/optimizer will call print_blank_line just once? The similar situation will happen in the following: </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> std::vector<int> x; void foo() { x.push_back(1); x.push_back(1); }</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Will push_back be called only once (parameters are the same)? If any compiler does make one call instead of 2 in these cases, you're not likely to build anything usable with it. So it's safe to assume that any common compiler doesn't have this problem. |
Re: 2.08 and Incompatible Battle Reports
Quote:
|
Re: 2.08 and Incompatible Battle Reports
Alexti, your examples are not the same. In the first example: 1) they have void returns and invoke I/O, which the optimizer treats differently. 2) the statements are on separate lines, and IIRC there are various scoping rules to what the compiler will attempt to "consolidate" when it goes to generate machine code.
For the second example: it depends on the switches used. I have seen (though not recently) compilers that would, indeed, consolidate those two statements when you viewed generated assembler code. However, the most obvious thing you said that bears careful review is the statement "... it's safe to assume ...". It's never safe to assume anything. That's the first step towards making mistakes ... We've flown the Space Shuttle in cold weather before, and it's only a few degrees colder now, the Challenger should be okay. It's only a 2-pound chunk of foam. How can that damage reinforced carbon fiber? It probably shattered into dust on impact with the Columbia's wing. The list of assumptions people make that lead to bad results has a long and inglorious history. |
Re: 2.08 and Incompatible Battle Reports
Quote:
I guess you have few more places when the comparison looks like </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> if (x + 2d6 < y + 2d6)</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">To eliminate this problem you could make the function: </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int random_compare(int x, int y, int nx, int ny) { int x1 = x + nx*illwinter_dice(); int y1 = y + ny*illwinter_dice(); return (x1 - y1); }</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">and use comparison </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> if (0 > random_compare(x,y,2,2))</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Finding all the places where the dice function is used and replacing it with the new comparison is not difficult, but the problem is that if you make a small cut-and-paste mistake somewhere, how do you find it? I can think only about one method to ensure that everything is changed correctly. If you can save battle progress status on disk after every round, you can make "etalon" saves on the current Version and rerun the test on a modified Version, comparing battle status after every turn. If there's a difference you'd be able to go through debugger to find out where it comes from. (In the example I've given you'd have to run on Linux, so that the original evaluation order matches the one implemented in a new function). The drawback is that some obscure conditions may not get tested at all. Alternative method is to write a program which will automatically do conVersion http://forum.shrapnelgames.com/images/icons/icon7.gif Either way it doesn't look like an easy fix :-( But I still hope that you don't have that many places where dice function is called more than once in the expression, so that those can be examined manually http://forum.shrapnelgames.com/images/icons/icon7.gif |
Re: 2.08 and Incompatible Battle Reports
Quote:
Quote:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> f(); f();</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">and </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> f(); f();</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">are the same thing. Quote:
Consider the following: </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int a = 1; foo(a);</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">What if the compiler generate wrong code for assignement? Ok, here is an improvement: </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int a = 1; if (a != 1) ERROR("!!!"); foo(a);</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">But what if the code for comparison is wrong too? No problems: </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int a = 1; if (a != 1 || (a-1)) ERROR("!!!"); foo(a);</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">But what if operator or (||) produces wrong code too? Well, it's clear where it's going... |
Re: 2.08 and Incompatible Battle Reports
Quote:
Easy enough to modify, of course, though it makes for ugly code and only deals with inequalities. The order of evaluation problem can occur anywhere where two random calls are used within the same expression. |
Re: 2.08 and Incompatible Battle Reports
How about :
a = 2d6; b = 2d6; if ( x + a < y + b ) ... Don't tell me there's a compiler around daft enough to evaluate b before a in this case ! |
Re: 2.08 and Incompatible Battle Reports
Quote:
EDIT: caveat: it actually is possible to force the compilers I've used into screwing this up. It involves tweaking settings so the compiler re-orders the statements so that it reads ... b = a = 2d6 Which, of course, looks the same as your example to anyone who's not a gamer or mathematician. We know that 2 calls to "d6" do not necessarily return the same answer. http://forum.shrapnelgames.com/images/icons/icon12.gif [ February 26, 2004, 09:01: Message edited by: Arryn ] |
Re: 2.08 and Incompatible Battle Reports
Quote:
|
Re: 2.08 and Incompatible Battle Reports
Quote:
|
Re: 2.08 and Incompatible Battle Reports
Quote:
|
Re: 2.08 and Incompatible Battle Reports
Quote:
Warning: Tech speak coming up. If you're only into computer games and not programming, maybe you should skip this. Well, maybe you should skip the whole thread, apart from the "we found the bug and it will be corrected in the next patch" bit. If your function calls are guaranteed to not have any side effects, the rearrangement Arryn "suggested" is actually a good move; it makes the compiled program faster by saving a (potentially costly) function call. This is a case of the compiler "helping" a sloppy programmer (and all programmers are sloppy). Of course, if your function call has a side effect, it's a very bad move because one call will not have the same side effect as two calls. In this case, calling the dice-rolling function has a side effect, since it modifies the state of the random generator, so it's pretty important. I don't know enough of the C specification to say whether there's a keyword to let the compiler know that a given function call is guaranteed to not have any side effects, but I suppose some compilers can be tweaked to assume that they are... [And I don't even know whether "side effect" is the correct English translation for the French "effet de bord"; all my teaching is done in French, I admit...] |
Re: 2.08 and Incompatible Battle Reports
Philippe, "side effect" is the correct translation, AFAIK.
And I'm glad that at least one person who's been reading this thread "gets it" with regards to compiler optimization, and possible pitfalls. Thanks for posting. BTW, AFAIK, there aren't any C/C++ keywords that affect the sort of optimizations that can cause these troubles. The best way to avoid them is understanding how functions pass back values, most importantly, what all the various optimization switches do (and which ones you have turned "on"), and how the compiler actually optimizes the code (which is done at the assembly-code level, not source-code level). Breaking complex expressions into separate and simple statements will go a long ways to avoiding possible problems too. It takes a little longer to write the code, but the end result is easier to read and maintain. |
Re: 2.08 and Incompatible Battle Reports
Perfect. Thank you guys for hunting this bug down. I got a freshly installed Debian 24/7 server waiting for the next patch... http://forum.shrapnelgames.com/images/icons/icon12.gif
|
Re: 2.08 and Incompatible Battle Reports
Quote:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> time_t t1 = time(0); time_t t2 = time(0);</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">should not be optimized because function time has a "side effect" of returning the current time. I understand that you're talking about the functions which are function in mathematic sense, meaning that for a given set of arguments the result will always be the same, and the return value will be the only data that will be changed. I don't know if there's a term for such functions. The problem comes because optimizer can not really figure out if the function has any "side effects" (let's keep calling them this way). Compiling/optimizing are performed per compilation unit and if the function in question is not residing in the same unit, optimizer can't even try to analyse the function, so it has no choice but make 2 separate calls. Are there any optimizer which would try to do this kind of optimization? Usually, they just optimize common subexpressions, for </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> a1 = 4 + (x + y)*n; b1 = 8 + (x + y)*n;</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">they would evaluate (x+y)*n once, leave it in the register and then do additions and assignments. This method isn't safe either, consider: </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> void foo(int& a1, int& b1, int& x, int y, int n) { a1 = 4 + (x + y)*n; b1 = 8 + (x + y)*n; } void ouch() { int z, x = 1, y = 1, n = 2; foo(x,z,x,y,n); }</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Though for this case the optimizer often has some kind of pragmas that allow to tell that the function does not have any aliased variables. |
Re: 2.08 and Incompatible Battle Reports
Quote:
If one had to examing the code and switches for each particular optimizer, he'd better just optimize the source code. Of course, if one needs to compile something with particular compiler and that compiler is broken, that's bad luck http://forum.shrapnelgames.com/images/icons/icon9.gif And if using MSVC5/6 one can optimize for size, it often produces faster code then optimization for speed (even if speed-optimized executable is sort of working). |
Re: 2.08 and Incompatible Battle Reports
alexti --
If memory serves, there's no standard way to specify to a compiler that a given C function has no side-effects. Function calls should not be automatically collated unless the code is inlined, in which case it may be possible. There might be a way to search the transitive closure of a function, so long as the closure resides within a given source file, for unsafe operations (assignments to non-local variables, assorted pieces of assembler, function calls outside file, I/O...) but that would seem like an awful headache for a compiler designer to consider in light of the fact that the original programmer could easily handle the job himself by calling the function once and assigning the result to a variable, then assigning a second variable the same value as the first. |
Re: 2.08 and Incompatible Battle Reports
Quote:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;">p = f(); q = f(); r = p + q; </pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">so your example could be deemed irrelevant. However, JK subsequently revealed the actual structure of the code, and thus demonstrated that your example was actually more analogous to it than the previously suggested Version. So my pedantic objection to your example is at least as irrelevant, if not more so... http://forum.shrapnelgames.com/image...s/rolleyes.gif |
Re: 2.08 and Incompatible Battle Reports
Quote:
[Edit: re-read what I'd misread. Deleted half of my post.] [ February 26, 2004, 18:58: Message edited by: E. Albright ] |
Re: 2.08 and Incompatible Battle Reports
Quote:
Quote:
|
Re: 2.08 and Incompatible Battle Reports
Quote:
|
Re: 2.08 and Incompatible Battle Reports
DISCLAIMER Im just kidding.
There are no mismatched battle reports. You all just have cowardly messengers. "Ummm you won. Yeah thats it, YOU WON oh Mighty and Frightening Deity." I dont have that problem. But then I usually take wimply little mages. I dont take big scarey two-headed snakes, or giant floating heads. [ February 26, 2004, 20:47: Message edited by: Gandalf Parker ] |
Re: 2.08 and Incompatible Battle Reports
Quote:
In the following example you can't rely on whether left call or right call will be evaluated first, but you can rely that both of them will be evaluated. </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int print_empty_line() { printf("\n"); return 1; } void foo(int x) { if (x + print_empty_line() < 2 + print_empty_line()) ... }</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana"> Quote:
</font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Which case you considering? Concerning evaluation order it's standard-compliant, but if it replaces 2 calls with one, it's not. |
Re: 2.08 and Incompatible Battle Reports
Quote:
</font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Here's where those examples came from: Quote:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> f(); f(); f(), f(); f()+f()</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Edit: Fixed quotations. Again. P.S. I want real forum/client http://forum.shrapnelgames.com/images/icons/icon12.gif [ February 27, 2004, 00:23: Message edited by: alexti ] |
Re: 2.08 and Incompatible Battle Reports
OK master password is in now, but has anyone tried using it? I take it that it is only to override the player passwords? No AI access? I was hoping it would let me jump in midgame to check on what an AI was doing.
|
Re: 2.08 and Incompatible Battle Reports
Quote:
Of course, it helps that I never lose a battle, and that there are never bad news to report. And it helps that I have a first-class majordomo. (Tacticus Sanguinus, God of Mictlan) |
All times are GMT -4. The time now is 10:53 PM. |
Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©1999 - 2025, Shrapnel Games, Inc. - All Rights Reserved.