.com.unity Forums

.com.unity Forums (http://forum.shrapnelgames.com/index.php)
-   Dominions 2: The Ascension Wars (http://forum.shrapnelgames.com/forumdisplay.php?f=55)
-   -   2.08 and Incompatible Battle Reports (http://forum.shrapnelgames.com/showthread.php?t=17878)

BugRoger February 17th, 2004 03:42 PM

2.08 and Incompatible Battle Reports
 
Any news on this issue? I couldn't try it myself yet.

It seems like the mentioned compiler patches might include a fix for the bug described in http://www.shrapnelgames.com/cgi-bin...=000872#000014

[ February 17, 2004, 13:43: Message edited by: BugRoger ]

PvK February 17th, 2004 09:22 PM

Re: 2.08 and Incompatible Battle Reports
 
Not sure, except I looked at my battles run in 2.06 after I upgraded to 2.08 (on Windows Version), and I didn't notice any changes in what I remembered happening. Not very scientific, but it's not obviously messed up.

PvK

Kristoffer O February 17th, 2004 09:45 PM

Re: 2.08 and Incompatible Battle Reports
 
We havn't found the bug, so unless it was hidden in some other fixed feature it is not solved.

The good news is that we have added a new checksum instrument that hopefully will give us some clues regarding the bug if we come across a faulty battle replay.

Johan K February 17th, 2004 11:45 PM

Re: 2.08 and Incompatible Battle Reports
 
If we are lucky bug rogers is right here and the microsoft compiler bugs caused most of battle replay bugs. Even if there's something more wrong it should at least be better now.

alexti February 18th, 2004 01:11 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Johan K:
If we are lucky bug rogers is right here and the microsoft compiler bugs caused most of battle replay bugs. Even if there's something more wrong it should at least be better now.
<font size="2" face="sans-serif, arial, verdana">In particular, if you were optimizing for speed.
And if you use MSVC6 (or 5), complier patches [surprisingly] don't include STL patches which you can get from http://www.dinkumware.com/vc_fixes.html

Johan K February 25th, 2004 07:33 PM

Re: 2.08 and Incompatible Battle Reports
 
Some good new for those who don't play Dominions on Windows exclusively. The problem with incompatible battle replays has been found. There was a disagrement on how to compile an expression between microsoft's and the others' compilers.

Magic resistance check:
if (penetration+2d6 < MR+2d6)

Other compiler solved it from left to right, but microsoft's solved it from right to left. This results in the same roll of dice appearing on different sides of the '<' for windows and linux.
Bloody annoying! http://forum.shrapnelgames.com/images/icons/icon8.gif


Alexti: Thanks for the STL info and I use MSVC6.

[ February 25, 2004, 17:35: Message edited by: Johan K ]

Saber Cherry February 25th, 2004 07:45 PM

Re: 2.08 and Incompatible Battle Reports
 
Yay! Not only was the problem solved, but also, Microsoft gets another black mark http://forum.shrapnelgames.com/images/icons/icon10.gif

PhilD February 25th, 2004 07:47 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Johan K:
Some good new for those who don't play Dominions on Windows exclusively. The problem with incompatible battle replays has been found. There was a disagrement on how to compile an expression between microsoft's and the others' compilers.

Magic resistance check:
if (penetration+2d6 < MR+2d6)

Other compiler solved it from left to right, but microsoft's solved it from right to left. This results in the same roll of dice appearing on different sides of the '<' for windows and linux.
Bloody annoying! http://forum.shrapnelgames.com/images/icons/icon8.gif


Alexti: Thanks for the STL info and I use MSVC6.

<font size="2" face="sans-serif, arial, verdana">Hmm... it's been some time since I had a look at books where the norm is described, but I'm not sure it says anything for this. I believe the result is "unspecified"...

Don't use two function calls in the same expression, is the lesson.

(And congrats on the Illwinter crew! When are we getting a patch so we can celebrate?) http://forum.shrapnelgames.com/images/icons/icon7.gif

fahdiz February 25th, 2004 07:49 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by PhilD:
(And congrats on the Illwinter crew! When are we getting a patch so we can celebrate?) http://forum.shrapnelgames.com/images/icons/icon7.gif
<font size="2" face="sans-serif, arial, verdana">I second on both counts. http://forum.shrapnelgames.com/images/icons/icon7.gif After reading Arryn's AAR, Utgard is calling to me...

...unfortunately, it's calling from a moose-laden Smouldercone right now. http://forum.shrapnelgames.com/images/icons/icon10.gif

Saber Cherry February 25th, 2004 07:54 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by PhilD:
Don't use two function calls in the same expression, is the lesson.

<font size="2" face="sans-serif, arial, verdana">Compliers do strange things. A poorly-written compiler could rearrange consecutive lines if there is no obvious dependancy.

1: a=2d6+penetration
2: b=2d6+mrst
3: if(a>b){do c}

Lines 1 & 2 could be rearranged by a compiler if, for example, mrst is more local (already in a register) and penetration has to be fetched. That would have the same effect. I think C has some commands that let you force the complier to not rearrange things, though.

So the moral is actually not to trust Microsoft products, since they take away your control http://forum.shrapnelgames.com/images/icons/icon10.gif

Taqwus February 25th, 2004 08:26 PM

Re: 2.08 and Incompatible Battle Reports
 
There may be #pragmas or compiler options to change reordering behavior, but I'm not too familiar with MSVC.
Wrapping the randomizer in an object and making sure that the "get a random number" method (perhaps as a static method, even) might serve as a hint to the compiler; could be more work than it's worth.

[edit] Making sure that it's not flagged as constant, that is. IOW to encourage the compiler to consider side-effects of function invocation.

[ February 25, 2004, 18:51: Message edited by: Taqwus ]

Pocus February 25th, 2004 08:46 PM

Re: 2.08 and Incompatible Battle Reports
 
I have often battle inconstancies message now, with solo play on windows platform.

Gandalf Parker February 25th, 2004 08:57 PM

Re: 2.08 and Incompatible Battle Reports
 
Way outside of my area but I was thinking that the Devs might have to write their own randomizing routine, and have the host create a seed. Then pass the seed its using inside each of the player files so that the players copy of Dom would definetly use the same routine on the same seed. I never figured on different OSs doing standard math in different orders. Thats insane.

[ February 25, 2004, 18:58: Message edited by: Gandalf Parker ]

Johan K February 25th, 2004 09:06 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Pocus:
I have often battle inconstancies message now, with solo play on windows platform.
<font size="2" face="sans-serif, arial, verdana">Bah, can't you let me be happy for more than 1 hour and 13 minutes. http://forum.shrapnelgames.com/images/icons/tongue.gif

Peter Ebbesen February 25th, 2004 09:15 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Saber Cherry:
So the moral is actually not to trust Microsoft products, since they take away your control http://forum.shrapnelgames.com/images/icons/icon10.gif [/QB]
<font size="2" face="sans-serif, arial, verdana">No, the moral is to be damn careful when using random numbers in sync. http://forum.shrapnelgames.com/images/icons/icon10.gif

NTJedi February 25th, 2004 09:16 PM

Re: 2.08 and Incompatible Battle Reports
 
I have seen this bug about 3 times for all of the games I have played. It is very rare... and definitely a bizarre bug.

It has worked both ways for me... One battle it showed me losing... yet on the map I won. Another battle it showed me winnning... yet on the map I lost.

2 of these were seen with patch 2.08 installed.

[ February 25, 2004, 19:17: Message edited by: NTJedi ]

Taqwus February 25th, 2004 09:29 PM

Re: 2.08 and Incompatible Battle Reports
 
*scratches head*
'volatile' may also help. The MSDN C++ Language Reference states, "Objects declared as volatile are not used in optimizations because their value can change at any time". Perhaps creating an 'int volatile do_not_reorder_me = 0;' declaration, explicitly separating the computations into multiple lines e.g.
int lhs = lhs_base + invoke_2d6() + do_not_reorder_me;
int rhs = rhs_base + invoke_2d6() + do_not_reorder_me;
would work. I haven't played around with that keyword much however -- been awhile since I wrote MT code.

Arryn February 25th, 2004 11:12 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Saber Cherry:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">quote:</font><hr /><font size="2" face="sans-serif, arial, verdana">Originally posted by PhilD:
Don't use two function calls in the same expression, is the lesson.

<font size="2" face="sans-serif, arial, verdana">Compliers do strange things. A poorly-written compiler could rearrange consecutive lines if there is no obvious dependancy.

1: a=2d6+penetration
2: b=2d6+mrst
3: if(a>b){do c}

Lines 1 & 2 could be rearranged by a compiler if, for example, mrst is more local (already in a register) and penetration has to be fetched. That would have the same effect. I think C has some commands that let you force the complier to not rearrange things, though.

So the moral is actually not to trust Microsoft products, since they take away your control http://forum.shrapnelgames.com/images/icons/icon10.gif
</font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Having done quite a bit of compiler beta-testing in my younger days, what this basically amounts to is an issue of (no offense to IW intended) somewhat sloppy coding practice by programmers. If you want to force the compiler to evaluate things a certain way, then make liberal use of parenthesis and local-scope intermediate variables. But what I see all too often is programmers that try to cram as much code as possible into a single line. That's just begging for trouble. And it's an absolute no-no when writing cross-platform code.

BTW, the MSVC compiler is actually very good. A bit odd at times, but good nonetheless. You just have to be very well acquainted with all it's various switches, directives, etc. The compiler doesn't really take control away from you. It's settings default to newbie usage. Which can be overridden. And should be, by professional coders.

My $0.02 worth on the subject.

Saber Cherry February 25th, 2004 11:14 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Taqwus:
*scratches head*
'volatile' may also help.

<font size="2" face="sans-serif, arial, verdana">Yeah, volatile was what I was thinking about. I suspect there are similar commands as well. Volatile, IIRC, is mainly used in multithreading and i/o. It can prevent a variable from being replaced by a constant, or ignored, but I'm not sure if it can prevent reordering.

Microsoft strives to prevent inter-os compatibility; maybe using an Intel compiler would be best=)

Saber Cherry February 25th, 2004 11:23 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Arryn:
what this basically amounts to is an issue of (no offense to IW intended) somewhat sloppy coding practice by programmers... all too often (programmers) try to cram as much code as possible into a single line. That's just begging for trouble.
<font size="2" face="sans-serif, arial, verdana">Hahaha... My lines are rarely less than 5X that long, and would be longer with a bigger monitor. Parenthesis do not help you if a compiler ignores your instruction ordering.

The problem here is that the lines are NOT LONG ENOUGH! The Microsoft compiler sees a simple rearrangement it can do to break inter-OS compatibility, and does it. If the lines are so long and confusing that the compiler can't figure out how to mutate them without breaking the program, it will just process them in order like it is supposed to http://forum.shrapnelgames.com/images/icons/icon10.gif

Really! I promise! http://forum.shrapnelgames.com/images/icons/tongue.gif

AStott February 26th, 2004 12:34 AM

Re: 2.08 and Incompatible Battle Reports
 
Eh... why bother with all the complicated methods of ensuring the calls happen in the right order. Instead, change:
if penetration+2d6 &lt; MR+2d6

To:
if penetration &lt; MR+2d6-2d6

Since both of the random calls are on the same side of the equation, the defined precendence order will ensure they are called in a consistent manner.

Arryn February 26th, 2004 12:47 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by AStott:
Eh... why bother with all the complicated methods of ensuring the calls happen in the right order. Instead, change:
if penetration+2d6 &lt; MR+2d6

To:
if penetration &lt; MR+2d6-2d6

Since both of the random calls are on the same side of the equation, the defined precendence order will ensure they are called in a consistent manner.

<font size="2" face="sans-serif, arial, verdana">The problem comes in that the compiler's optimization will substitute the same call to the random number function for both die rolls. It won't make the two rand() calls the coders intend. What it does is make one call and plug the same value into both places. The optimizer does not know that in this circumstance, two calls to the same function do not return the same value.

Peter Ebbesen February 26th, 2004 01:17 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Arryn:
The problem comes in that the compiler's optimization will substitute the same call to the random number function for both die rolls. It won't make the two rand() calls the coders intend. What it does is make one call and plug the same value into both places. The optimizer does not know that in this circumstance, two calls to the same function do not return the same value.
<font size="2" face="sans-serif, arial, verdana">That would be an exceedingly poorly written optimizer. I certainly did not understand that as the original problem. I am certain that any decent compiler would make the two rand calls.

As I understood it, the real problem was that in the expression "a < b" the order of evaluation of "a" and "b" is undefined [Ansi-C, and its derivatives, with a few exceptions only specifies the order of evaluation for operators, not for sub-expressions]. Either side "a" or "b" can be evaluated first.

Given that the seed is the same, assume the RNG will return R1 and R2 over the next two calls in that order. Then the inequality:

(penetration+2d6 < MR+2d6)

can legally be compiled such that the evaluation is either of the following:
penetration+R1 < MR+R2 (left hand side evaluated first)
penetration+R2 < MR+R1 (right hand side evaluated first)

- Which may or may not give different return values.

Changing the inequality as AStott suggested, to
penetration < MR+2d6-2d6
would not necessarily solve the problem either, as the order of evaluation of the two 2d6 function calls is not defined either. (The order of the addition and subtraction is, but not the order of evaluation of the sub-expressions)

If you are in doubt, split such expressions over multiple commands independently evaluated. A bad optimizer may still hurt you, but at least you won't be bitten by the "undefined evaluation order", which is nobody's fault but your own. Or, to quote Kernigham & Ritchie:

Quote:

The moral is that writing code that depends on order of evaluation is a bad programming practice in any language. Naturally, it is necessary to know what things to avoid, but if you don't know how they are done on various machines, you won't be tempted to take advantage of a particular implementation.
<font size="2" face="sans-serif, arial, verdana">

[ February 25, 2004, 23:23: Message edited by: Peter Ebbesen ]

Arryn February 26th, 2004 01:36 AM

Re: 2.08 and Incompatible Battle Reports
 
A cogent, well-stated reply, Peter. The quote by K&R was what I tried to allude to in an earlier posting.

BTW, it's not so much that the opitimizer may be "bad" than that it may be using "overly aggressive" choices. I remember the Borland compiler dev team in the early 90s having many headaches over just how far they should go. In those days, Borland and MS kept trying to out-do each other via how powerful their optimizations were. After a few cycles of this we started seeing cases of too much optimization.

A careful review of what all the default optimization switches do should be undertaken by anyone that's serious about code-writing, especially cross-platform code, to avoid potential pitfalls. Hell, one should also look at the linker's switches too.

alexti February 26th, 2004 01:43 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Arryn:
The problem comes in that the compiler's optimization will substitute the same call to the random number function for both die rolls. It won't make the two rand() calls the coders intend. What it does is make one call and plug the same value into both places. The optimizer does not know that in this circumstance, two calls to the same function do not return the same value.
<font size="2" face="sans-serif, arial, verdana">Consider the following:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> void print_blank_line { printf(&quot;\n&quot;); }

void print_2blank_lines()
{
print_blank_line();
print_blank_line();
}</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Do you think compiler/optimizer will call print_blank_line just once? The similar situation will happen in the following:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> std::vector&lt;int&gt; x;

void foo()
{
x.push_back(1);
x.push_back(1);
}</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Will push_back be called only once (parameters are the same)?

If any compiler does make one call instead of 2 in these cases, you're not likely to build anything usable with it. So it's safe to assume that any common compiler doesn't have this problem.

alexti February 26th, 2004 01:47 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Taqwus:
*scratches head*
'volatile' may also help. The MSDN C++ Language Reference states, "Objects declared as volatile are not used in optimizations because their value can change at any time"

<font size="2" face="sans-serif, arial, verdana">Volatile won't help here, the idea behind volatile is that if you need to read/write into the some variables of one process from another process (after passing address by some IPC means), compiler/optimizer won't know that you plan to use the variable in such a way and may put it into the register. So volatile is just used to prevent it from happening.

Arryn February 26th, 2004 02:01 AM

Re: 2.08 and Incompatible Battle Reports
 
Alexti, your examples are not the same. In the first example: 1) they have void returns and invoke I/O, which the optimizer treats differently. 2) the statements are on separate lines, and IIRC there are various scoping rules to what the compiler will attempt to "consolidate" when it goes to generate machine code.

For the second example: it depends on the switches used. I have seen (though not recently) compilers that would, indeed, consolidate those two statements when you viewed generated assembler code.

However, the most obvious thing you said that bears careful review is the statement "... it's safe to assume ...". It's never safe to assume anything. That's the first step towards making mistakes ...

We've flown the Space Shuttle in cold weather before, and it's only a few degrees colder now, the Challenger should be okay.

It's only a 2-pound chunk of foam. How can that damage reinforced carbon fiber? It probably shattered into dust on impact with the Columbia's wing.

The list of assumptions people make that lead to bad results has a long and inglorious history.

alexti February 26th, 2004 02:02 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Johan K:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">quote:</font><hr /><font size="2" face="sans-serif, arial, verdana">Originally posted by Pocus:
I have often battle inconstancies message now, with solo play on windows platform.

<font size="2" face="sans-serif, arial, verdana">Bah, can't you let me be happy for more than 1 hour and 13 minutes. http://forum.shrapnelgames.com/images/icons/tongue.gif </font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">To Johan K:

I guess you have few more places when the comparison looks like
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> if (x + 2d6 &lt; y + 2d6)</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">To eliminate this problem you could make the function:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int random_compare(int x, int y, int nx, int ny)
{
int x1 = x + nx*illwinter_dice();
int y1 = y + ny*illwinter_dice();
return (x1 - y1);
}</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">and use comparison
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> if (0 &gt; random_compare(x,y,2,2))</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Finding all the places where the dice function is used and replacing it with the new comparison is not difficult, but the problem is that if you make a small cut-and-paste mistake somewhere, how do you find it?

I can think only about one method to ensure that everything is changed correctly. If you can save battle progress status on disk after every round, you can make "etalon" saves on the current Version and rerun the test on a modified Version, comparing battle status after every turn. If there's a difference you'd be able to go through debugger to find out where it comes from. (In the example I've given you'd have to run on Linux, so that the original evaluation order matches the one implemented in a new function). The drawback is that some obscure conditions may not get tested at all.

Alternative method is to write a program which will automatically do conVersion http://forum.shrapnelgames.com/images/icons/icon7.gif

Either way it doesn't look like an easy fix :-(
But I still hope that you don't have that many places where dice function is called more than once in the expression, so that those can be examined manually http://forum.shrapnelgames.com/images/icons/icon7.gif

alexti February 26th, 2004 02:14 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Arryn:
Alexti, your examples are not the same. In the first example: 1) they have void returns and invoke I/O, which the optimizer treats differently.

<font size="2" face="sans-serif, arial, verdana">How the optimizer would know about IO? printf is just some function residing in some library to which optimizer had no access. It only can see the header, where there's nothing that says that the function contains IO.

Quote:


2) the statements are on separate lines, and IIRC there are various scoping rules to what the compiler will attempt to "consolidate" when it goes to generate machine code.

<font size="2" face="sans-serif, arial, verdana">Some Languages consider line feeds as a language element, but not C/C++.
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> f(); f();</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">and
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> f();
f();</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">are the same thing.


Quote:

However, the most obvious thing you said that bears careful review is the statement "... it's safe to assume ...". It's never safe to assume anything. That's the first step towards making mistakes ...

<font size="2" face="sans-serif, arial, verdana">You have to assume that the compiler works correctly (according to the standard) until proven otherwise, how are going to write any code otherwise?

Consider the following:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int a = 1;
foo(a);</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">What if the compiler generate wrong code for assignement? Ok, here is an improvement:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int a = 1;
if (a != 1)
ERROR(&quot;!!!&quot;);
foo(a);</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">But what if the code for comparison is wrong too? No problems:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int a = 1;
if (a != 1 || (a-1))
ERROR(&quot;!!!&quot;);
foo(a);</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">But what if operator or (||) produces wrong code too?

Well, it's clear where it's going...

Peter Ebbesen February 26th, 2004 09:16 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by alexti:

</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> if (x + 2d6 &lt; y + 2d6)</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">To eliminate this problem you could make the function:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int random_compare(int x, int y, int nx, int ny)
{
int x1 = x + nx*illwinter_dice();
int y1 = y + ny*illwinter_dice();
return (x1 - y1);
}</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">and use comparison
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> if (0 &gt; random_compare(x,y,2,2))</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">

<font size="2" face="sans-serif, arial, verdana">A nice idea, though it fails in the execution. 2d6 is not shorthand for two times the result of a d6, but for the openended result of rolling 2 d6'es (openended: sixes are rerolled and added potentially ad infinitum:)

Easy enough to modify, of course, though it makes for ugly code and only deals with inequalities. The order of evaluation problem can occur anywhere where two random calls are used within the same expression.

General Tacticus February 26th, 2004 10:52 AM

Re: 2.08 and Incompatible Battle Reports
 
How about :

a = 2d6;
b = 2d6;
if ( x + a < y + b ) ...

Don't tell me there's a compiler around daft enough to evaluate b before a in this case !

Arryn February 26th, 2004 10:55 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by General Tacticus:
How about :

a = 2d6;
b = 2d6;
if ( x + a < y + b ) ...

Don't tell me there's a compiler around daft enough to evaluate b before a in this case !

<font size="2" face="sans-serif, arial, verdana">This should work just fine, under normal conditions.

EDIT: caveat: it actually is possible to force the compilers I've used into screwing this up. It involves tweaking settings so the compiler re-orders the statements so that it reads ...

b = a = 2d6

Which, of course, looks the same as your example to anyone who's not a gamer or mathematician. We know that 2 calls to "d6" do not necessarily return the same answer. http://forum.shrapnelgames.com/images/icons/icon12.gif

[ February 26, 2004, 09:01: Message edited by: Arryn ]

General Tacticus February 26th, 2004 11:37 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Arryn:


EDIT: caveat: it actually is possible to force the compilers I've used into screwing this up. It involves tweaking settings so the compiler re-orders the statements so that it reads ...

b = a = 2d6

Which, of course, looks the same as your example to anyone who's not a gamer or mathematician. We know that 2 calls to "d6" do not necessarily return the same answer. http://forum.shrapnelgames.com/images/icons/icon12.gif

<font size="2" face="sans-serif, arial, verdana">And you can also make them believe that '<' means '>', or for that matter that "=" in the code means 'print("I'm a genius")' http://forum.shrapnelgames.com/images/icons/icon12.gif . But we are here to help compilers do the right thing, not to screw their settings and help them be total idiots http://forum.shrapnelgames.com/images/icons/icon7.gif

Arryn February 26th, 2004 11:41 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by General Tacticus:
... and help them be total idiots http://forum.shrapnelgames.com/images/icons/icon7.gif
<font size="2" face="sans-serif, arial, verdana">This is just too good to resist: as opposed to what they would otherwise be? (less-than-total idiots) http://forum.shrapnelgames.com/images/icons/tongue.gif http://forum.shrapnelgames.com/images/icons/tongue.gif http://forum.shrapnelgames.com/images/icons/icon12.gif

Johan K February 26th, 2004 12:03 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by General Tacticus:
How about :

a = 2d6;
b = 2d6;
if ( x + a < y + b ) ...

<font size="2" face="sans-serif, arial, verdana">That's the one that is used now. Easy to read and it should be foolproof.

PhilD February 26th, 2004 12:14 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by General Tacticus:
And you can also make them believe that '<' means '>', or for that matter that "=" in the code means 'print("I'm a genius")' http://forum.shrapnelgames.com/images/icons/icon12.gif . But we are here to help compilers do the right thing, not to screw their settings and help them be total idiots http://forum.shrapnelgames.com/images/icons/icon7.gif
<font size="2" face="sans-serif, arial, verdana">You've obviously never had to work on optimizing a compiler... (neither have I, but I do occasionally teach some programming)

Warning: Tech speak coming up. If you're only into computer games and not programming, maybe you should skip this. Well, maybe you should skip the whole thread, apart from the "we found the bug and it will be corrected in the next patch" bit.

If your function calls are guaranteed to not have any side effects, the rearrangement Arryn "suggested" is actually a good move; it makes the compiled program faster by saving a (potentially costly) function call. This is a case of the compiler "helping" a sloppy programmer (and all programmers are sloppy).

Of course, if your function call has a side effect, it's a very bad move because one call will not have the same side effect as two calls. In this case, calling the dice-rolling function has a side effect, since it modifies the state of the random generator, so it's pretty important.

I don't know enough of the C specification to say whether there's a keyword to let the compiler know that a given function call is guaranteed to not have any side effects, but I suppose some compilers can be tweaked to assume that they are...

[And I don't even know whether "side effect" is the correct English translation for the French "effet de bord"; all my teaching is done in French, I admit...]

Arryn February 26th, 2004 12:32 PM

Re: 2.08 and Incompatible Battle Reports
 
Philippe, "side effect" is the correct translation, AFAIK.

And I'm glad that at least one person who's been reading this thread "gets it" with regards to compiler optimization, and possible pitfalls. Thanks for posting.

BTW, AFAIK, there aren't any C/C++ keywords that affect the sort of optimizations that can cause these troubles. The best way to avoid them is understanding how functions pass back values, most importantly, what all the various optimization switches do (and which ones you have turned "on"), and how the compiler actually optimizes the code (which is done at the assembly-code level, not source-code level).

Breaking complex expressions into separate and simple statements will go a long ways to avoiding possible problems too. It takes a little longer to write the code, but the end result is easier to read and maintain.

BugRoger February 26th, 2004 01:17 PM

Re: 2.08 and Incompatible Battle Reports
 
Perfect. Thank you guys for hunting this bug down. I got a freshly installed Debian 24/7 server waiting for the next patch... http://forum.shrapnelgames.com/images/icons/icon12.gif

alexti February 26th, 2004 03:12 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by PhilD:
If your function calls are guaranteed to not have any side effects, the rearrangement Arryn "suggested" is actually a good move; it makes the compiled program faster by saving a (potentially costly) function call. This is a case of the compiler "helping" a sloppy programmer (and all programmers are sloppy).

Of course, if your function call has a side effect, it's a very bad move because one call will not have the same side effect as two calls. In this case, calling the dice-rolling function has a side effect, since it modifies the state of the random generator, so it's pretty important.

I don't know enough of the C specification to say whether there's a keyword to let the compiler know that a given function call is guaranteed to not have any side effects, but I suppose some compilers can be tweaked to assume that they are...

[And I don't even know whether "side effect" is the correct English translation for the French "effet de bord"; all my teaching is done in French, I admit...]

<font size="2" face="sans-serif, arial, verdana">I'd say you translated "side effect" correctly, but in the context it sounds like the calls
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> time_t t1 = time(0);
time_t t2 = time(0);</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">should not be optimized because function time has a "side effect" of returning the current time.

I understand that you're talking about the functions which are function in mathematic sense, meaning that for a given set of arguments the result will always be the same, and the return value will be the only data that will be changed.
I don't know if there's a term for such functions.

The problem comes because optimizer can not really figure out if the function has any "side effects" (let's keep calling them this way). Compiling/optimizing are performed per compilation unit and if the function in question is not residing in the same unit, optimizer can't even try to analyse the function, so it has no choice but make 2 separate calls.

Are there any optimizer which would try to do this kind of optimization? Usually, they just optimize common subexpressions, for
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> a1 = 4 + (x + y)*n;
b1 = 8 + (x + y)*n;</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">they would evaluate (x+y)*n once, leave it in the register and then do additions and assignments.

This method isn't safe either, consider:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> void foo(int&amp; a1, int&amp; b1, int&amp; x, int y, int n)
{
a1 = 4 + (x + y)*n;
b1 = 8 + (x + y)*n;
}

void ouch()
{
int z, x = 1, y = 1, n = 2;
foo(x,z,x,y,n);
}</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Though for this case the optimizer often has some kind of pragmas that allow to tell that the function does not have any aliased variables.

alexti February 26th, 2004 03:25 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Arryn:
BTW, AFAIK, there aren't any C/C++ keywords that affect the sort of optimizations that can cause these troubles. The best way to avoid them is understanding how functions pass back values, most importantly, what all the various optimization switches do (and which ones you have turned "on"), and how the compiler actually optimizes the code (which is done at the assembly-code level, not source-code level).
<font size="2" face="sans-serif, arial, verdana">The idea of the standards is to be able to write code which will work correctly (meaning that as programmer specified in the source code - as opposite to what the programmer wants http://forum.shrapnelgames.com/images/icons/icon12.gif ) if compiled by any standard-compliant compiler. Optimizer should meet all criteria defined in the language standard concerning the produced binary code, so the only case when it is allowed to alter the results is when the programmer is using constructs which behaviour is undefined by the standard. And the programmer should not be using such constructs.

If one had to examing the code and switches for each particular optimizer, he'd better just optimize the source code.

Of course, if one needs to compile something with particular compiler and that compiler is broken, that's bad luck http://forum.shrapnelgames.com/images/icons/icon9.gif

And if using MSVC5/6 one can optimize for size, it often produces faster code then optimization for speed (even if speed-optimized executable is sort of working).

Taqwus February 26th, 2004 05:45 PM

Re: 2.08 and Incompatible Battle Reports
 
alexti --
If memory serves, there's no standard way to specify to a compiler that a given C function has no side-effects. Function calls should not be automatically collated unless the code is inlined, in which case it may be possible. There might be a way to search the transitive closure of a function, so long as the closure resides within a given source file, for unsafe operations (assignments to non-local variables, assorted pieces of assembler, function calls outside file, I/O...) but that would seem like an awful headache for a compiler designer to consider in light of the fact that the original programmer could easily handle the job himself by calling the function once and assigning the result to a variable, then assigning a second variable the same value as the first.

E. Albright February 26th, 2004 08:33 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by alexti:
Some Languages consider line feeds as a language element, but not C/C++.
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;">f(); f();</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">and
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;">f();
f();</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">are the same thing.

<font size="2" face="sans-serif, arial, verdana">Fair enough. However, Arryn was suggesting that your example was not analogous because the calls were in seperate statements (tho' to be fair, she used the term "lines", so the confusion is understandable). Based on presented pseudocode, JK's code at the time was supposed to resemble </font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> f() + f(); </pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">whereas your example was more akin to
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;">p = f();
q = f();
r = p + q; </pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">so your example could be deemed irrelevant. However, JK subsequently revealed the actual structure of the code, and thus demonstrated that your example was actually more analogous to it than the previously suggested Version. So my pedantic objection to your example is at least as irrelevant, if not more so... http://forum.shrapnelgames.com/image...s/rolleyes.gif

E. Albright February 26th, 2004 08:51 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by alexti:
I understand that you're talking about the functions which are function in mathematic sense, meaning that for a given set of arguments the result will always be the same, and the return value will be the only data that will be changed.
I don't know if there's a term for such functions.

<font size="2" face="sans-serif, arial, verdana">A function without side effects would be a referentially transparent function.

[Edit: re-read what I'd misread. Deleted half of my post.]

[ February 26, 2004, 18:58: Message edited by: E. Albright ]

Arryn February 26th, 2004 08:54 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by alexti:
The idea of the standards is to be able to write code which will work correctly (meaning that as programmer specified in the source code - as opposite to what the programmer wants http://forum.shrapnelgames.com/images/icons/icon12.gif ) if compiled by any standard-compliant compiler.
<font size="2" face="sans-serif, arial, verdana">The C/C++ standards do not address the issue that's been the focus of this interminable discussion. So your argument is sort of irrelevant.
Quote:

Optimizer should meet all criteria defined in the language standard concerning the produced binary code, so the only case when it is allowed to alter the results is when the programmer is using constructs which behaviour is undefined by the standard. And the programmer should not be using such constructs.
<font size="2" face="sans-serif, arial, verdana">1) The compiler does comply with the standard. 2) The side effect is not anticipated by the standards. 3) The construct is perfectly legal. But it assumes things. Bad things. (An analogy is writing code that depends on the bit-order of the hardware it's running on. A real bad no-no.) It's the responsibility of programmers to understand what their code is intended to do, and how the compiler will do its task, and whether there's a conflict between his/her intentions and its results. It's not the job of standards to protect programmers from themselves. (That's what good teachers, lots of reading, and not getting in the habit of sloppy coding is for. http://forum.shrapnelgames.com/images/icons/icon12.gif )

Arryn February 26th, 2004 08:57 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by E. Albright:
Oh, and a function without side effects would be a referentially transparent function.
<font size="2" face="sans-serif, arial, verdana">Thank you. That's the term I'd been wracking my brain to remember. Optimizers assume functions are referentially transparent. It's the responsibility of programmers to compensate ...

Gandalf Parker February 26th, 2004 10:43 PM

Re: 2.08 and Incompatible Battle Reports
 
DISCLAIMER Im just kidding.

There are no mismatched battle reports. You all just have cowardly messengers. "Ummm you won. Yeah thats it, YOU WON oh Mighty and Frightening Deity."

I dont have that problem. But then I usually take wimply little mages. I dont take big scarey two-headed snakes, or giant floating heads.

[ February 26, 2004, 20:47: Message edited by: Gandalf Parker ]

alexti February 27th, 2004 02:13 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Arryn:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">quote:</font><hr /><font size="2" face="sans-serif, arial, verdana">Originally posted by alexti:
The idea of the standards is to be able to write code which will work correctly (meaning that as programmer specified in the source code - as opposite to what the programmer wants http://forum.shrapnelgames.com/images/icons/icon12.gif ) if compiled by any standard-compliant compiler.

<font size="2" face="sans-serif, arial, verdana">The C/C++ standards do not address the issue that's been the focus of this interminable discussion. So your argument is sort of irrelevant.</font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">That's exactly the point. If C/C++ standard does not specify the order of evaluation (I haven't seen it there, but it doesn't mean that it's not there though http://forum.shrapnelgames.com/images/icons/icon7.gif ) you can not rely on any particular order.
In the following example you can't rely on whether left call or right call will be evaluated first, but you can rely that both of them will be evaluated.
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> int print_empty_line()
{
printf(&quot;\n&quot;);
return 1;
}

void foo(int x)
{
if (x + print_empty_line() &lt; 2 + print_empty_line())
...
}</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">
Quote:

</font><blockquote><font size="1" face="sans-serif, arial, verdana">quote:</font><hr /><font size="2" face="sans-serif, arial, verdana">Optimizer should meet all criteria defined in the language standard concerning the produced binary code, so the only case when it is allowed to alter the results is when the programmer is using constructs which behaviour is undefined by the standard. And the programmer should not be using such constructs.
<font size="2" face="sans-serif, arial, verdana">1) The compiler does comply with the standard.
</font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Which case you considering? Concerning evaluation order it's standard-compliant, but if it replaces 2 calls with one, it's not.

alexti February 27th, 2004 02:21 AM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by E. Albright:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">quote:</font><hr /><font size="2" face="sans-serif, arial, verdana">Originally posted by alexti:
Some Languages consider line feeds as a language element, but not C/C++.
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;">f(); f();</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">and
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;">f();
f();</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">are the same thing.

<font size="2" face="sans-serif, arial, verdana">Fair enough. However, Arryn was suggesting that your example was not analogous because the calls were in seperate statements (tho' to be fair, she used the term "lines", so the confusion is understandable).
</font><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Here's where those examples came from:
Quote:

Originally posted by Arryn:
[qb]The problem comes in that the compiler's optimization will substitute the same call to the random number function for both die rolls. It won't make the two rand() calls the coders intend. What it does is make one call and plug the same value into both places. The optimizer does not know that in this circumstance, two calls to the same function do not return the same value.
<font size="2" face="sans-serif, arial, verdana">And in all 3 cases f shall be called twice:
</font><blockquote><font size="1" face="sans-serif, arial, verdana">code:</font><hr /><pre style="font-size:x-small; font-family: monospace;"> f(); f();

f(), f();

f()+f()</pre><hr /></blockquote><font size="2" face="sans-serif, arial, verdana">Edit: Fixed quotations. Again.
P.S. I want real forum/client http://forum.shrapnelgames.com/images/icons/icon12.gif

[ February 27, 2004, 00:23: Message edited by: alexti ]

Gandalf Parker February 27th, 2004 04:06 PM

Re: 2.08 and Incompatible Battle Reports
 
OK master password is in now, but has anyone tried using it? I take it that it is only to override the player passwords? No AI access? I was hoping it would let me jump in midgame to check on what an AI was doing.

General Tacticus February 27th, 2004 05:26 PM

Re: 2.08 and Incompatible Battle Reports
 
Quote:

Originally posted by Gandalf Parker:
DISCLAIMER Im just kidding.

There are no mismatched battle reports. You all just have cowardly messengers. "Ummm you won. Yeah thats it, YOU WON oh Mighty and Frightening Deity."

I dont have that problem. But then I usually take wimply little mages. I dont take big scarey two-headed snakes, or giant floating heads.

<font size="2" face="sans-serif, arial, verdana">My messengers wouldn't dare report anything but the Truth ! First, it is official dogma that I, as a Pretender, am All-Knowing, therefore any effort at lying will be found out. And all my citizens are true believers of official dogma. Second, it is equally well known that bad news only result in a quick and painful death, while lying result in an eternal and even more painful afterlife. And finally, my majordomo knows that I don't like to be bothered by unsignificant events...

Of course, it helps that I never lose a battle, and that there are never bad news to report. And it helps that I have a first-class majordomo.

(Tacticus Sanguinus, God of Mictlan)


All times are GMT -4. The time now is 10:53 PM.

Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
Copyright ©1999 - 2025, Shrapnel Games, Inc. - All Rights Reserved.