Shrapnel to Take on Wargamer.com?

[Tim Brooks]

Jankee said:
Do I get any brownie points for correctly guessing the two titles?

Sure, let's hear 'em?

[Jankee]

A - War at World
B - Raging Tiger

[Annette]

Jankee said:
A - War at World
B - Raging Tiger

You've done your homework...Shall I send a t-shirt?

But really, I don't think the particular games matter for this discussion. The point I was trying to make is that review sites owned by game publishers should be very careful. I'll never know if the discrepency between saying a 120 page manual

helps enormously

and a 130 page manual will

put the Fear of God into You

was intentional. It's very possible it was not. Do I think it was intentional?

What this says to me is that entities where ownership may be perceived as being biased should exercise extreme caution by holding each review to a static list of criteria. We are very aware of this as we plan the launch of TGN and will expect our readers to hold us to our claims of objectivity. We don't claim that we will always be perfect; no one is. But we promise the relationship between the site and Shrapnel Games will be visible so that you, the reader, will be able to draw informed conclusions. We will expect to hear from you when we're not.

[Jim_Zabek]

Hi Annette,

I'm Jim Zabek, the Editor-in-Chief for The Wargamer.

You've got some excellent advice: you have to be very careful when you run a review site, though I wouldn't restrict that statement beyond that point. There seem to always be a few folks who can possibly misapprehend a review or statement made on the site. Perhaps I can help clarify a couple of things for you regarding your concern around review times and titles.

First, The Wargamer has never been known to have the fastest turnaround times on reviews. When a title is hot, we do try to get something out on it ASAP. In the case of Gary Grigsby’s World at War we had gold copy several weeks before the game was released commercially. In the case of Raging Tiger, we did not. Before I go further, I'd like to make a formal request: please send any exclusive screen shots, preview code, and review code to us as early as possible, especially for 82nd Airborne. There is a tremendous level of excitement around it and our readers and staff are really excited to see it. If you can get us Gold code three or four weeks before the game comes out I will do my best to get it reviewed before or coinciding with the game's commercial release.

As for timing, and since your concern centers around Matrix products, let me share a couple of examples. I don't keep a calendar as to when a game is commercially released and compare that to when we publish a review. However, I believe it's safe to say that Matrix Games' Starshatter was released sometime early last summer. A quick check of our site (where we do list the date our review was published) shows that review went up on...(drumroll please)

February 3, 2005.

Another "big" Matrix title, War in the Pacific, came out around the middle of last summer. We published its review on....

January 12, 2005.

There's no conspiracy on the part of The Wargamer to slight Shrapnel's games or cause Shrapnel any trouble, and I have to apologize to you if you received that impression. It's simply not true.

Something that is true, and you have rightly pointed that out, is that reviewers have opinions and those opinions can vary from person to person. Let’s talk about game manuals for a moment, since the subject is on many people’s minds:

I haven't seen the manual to Raging Tiger (we only received a single copy and it went to our reviewer) so I can't comment on it. I have seen the manual for Gary Grigsby's World at War and can say that it is lengthy but well written. Criticism of a manual usually isn't limited to its length but rather its quality and, more importantly, the need for it. A strong tutorial will help mitigate the need to flip through page after page of a paper manual. Without having seen Raging Tiger's manual, I can only speak in general terms: that care should be taken to make a manual as user friendly as possible, and ideally almost superfluous.

To illustrate a final point about manuals, perceptions, and game quality, let's take a look at Dominions II. I'll have to ask that you forgive my lack of savvy in quoting; this is my first post in the forums and I'm still learning the interface.

Let's take a look at what our reviewer (a third person who neither reviewed World at War nor Raging Tiger):

QUOTE
Documentation

This is one area that Dominions II could have been stronger. Don't get me wrong - the manual is well written, thorough, and has a complete listing of all the specialty items and spells available in the game. However, it does not go into enough depth to be truly satisfying. I would have appreciated an example of play or some tips about how to get started. By this I mean, what types of magic to specialize in depending on race, battle tactics and use of magic. To a certain extent, some of these shortcomings are addressed in the 'Tip of the Turn' that comes up between turns the orders of the previous turn are processed, especially when tactics are considered. Consequently, I found the learning curve to be steep and at times frustrating as I first delved into the game. A walk-through of the set-up and first few turns of a game is available on Shrapnel's website. I highly recommend it for any newbies.

/QUOTE

Care to guess what he thought of that title? Let's see what his conclusion was:

QUOTE
Summary

Dominions II is a great game. The more I played, the more I liked it not only because I am a strategy game fan, but also because of the challenge it presented. Rarely have I come across a game has made me work as hard as I did to improve my performance. Except for an improved diplomacy screen or at least a system to allow a player to see who they are and aren't at war with, I believe Dominions II has it all. This game won't be for everyone but if you like strategy games and can do without glitz and high tech graphics, Dominions II will be a great choice. I highly recommend it!

/QUOTE

Not only that, but Dominions II received our prestigious Award for Excellence. I think it’s safe to say that the game’s manual doesn’t reflect the overall quality of a game. It’s also safe to say that when we see a great game, we let the world know – irrespective of who the publisher is. It’s also safe to say that when we find a game we don’t like, we won’t hide that fact from our readers, either.

I can understand that some Shrapnel staffers may not have liked our review of Raging Tiger. However, The Wargamer's review isn't the only one on the net. I haven't seen any other reviews which refute it. That doesn't mean that some gamers won't like it - it's ideal for some, but not for others. That point was made in our review, though. And when Dr. Jim Cobb offered to take another look at it and I gladly accepted because I know he’ll be able to speak to the audience who might enjoy the game.

I can't control how my reviewers feel about the games they review. Forcing one to give a positive review to a title he doesn't like would be as bad as forcing one to write something negative about something they loved. I can't allow either.

What I can do is give you all the free press you can get if you send us developer diaries, screen shots, and previews. And when you publish another game as excellent as Dominions II, you'll hear our staff cheering for it, too.

Oh, and good luck launching your independent gaming news site. I look forward to reading it.

[Tim Brooks]

Hi Jim:

I am sure Annette will be along with her reply to your post, but I have a couple of questions / concerns from what you said.

First, The Wargamer has never been known to have the fastest turnaround times on reviews. When a title is hot, we do try to get something out on it ASAP.

How do you know a title is hot before it is released? I ask because Raging Tiger had a short pre-order period, yet was one of our top five pre-order successes. I would have considered that hot.

In the case of Gary Grigsby’s World at War we had gold copy several weeks before the game was released commercially. In the case of Raging Tiger, we did not.

So, if we had given you the review copy several weeks earlier it wouldn't have taken 7 months? Confused about how that works?

You seemd to have missed the point from what I can tell. I don't think a comparison with Dominions II is relevant, since this is about the heading of the Raging Tiger manual section in the review. We looked at several of the reviews of Wargames on your site and all the headings we saw said something like Installation, Documentation, and Tutorials or Documentation, etc. The only one that we found that headed the documentation section with a snide comment was Raging Tiger (Nothing like a Wargame with a Big Manual to put the Fear of God into You). This seems to be not so much the fault of the reviewer but the fault of the editors for allowing that kind of comment in a sub-heading. Care to comment?

Best regards,

[Jim_Zabek]

We don't know what titles are absolute winners, but I do know which titles my staff jumps for and which ones they don't.

There's no "way things work". Get me 82nd early, I'll ensure it's reviewed promptly.

Dominions II isn't missing the point for three reasons: one, Dominions II was the last Shrapnel game we reviewed prior to Raging Tiger; two, the review was made well after David Heath purchased The Wargamer; and three, it is a respectful reminder folks that Dominions II was given a Award well after David Heath acquired The Wargamer and Shrapnel's good efforts were recognized.

As for "snideness" of the subheader, well, it clearly it was an indication of what the reviewer thought of the documentation. Our writers have made other "witty" remarks about games as the author sees fit; it isn't restricted to the wargaming genre or a particular publisher. I'm sorry you took offense.

[Tim Brooks]

Jim:

We don't know what titles are absolute winners, but I do know which titles my staff jumps for and which ones they don't.

Well then, we just need to get some Shrapnel people on your staff.

Get me 82nd early, I'll ensure it's reviewed promptly.

Thanks, but no thanks.

As for "snideness" of the subheader, well, it clearly it was an indication of what the reviewer thought of the documentation. Our writers have made other "witty" remarks about games as the author sees fit; it isn't restricted to the wargaming genre or a particular publisher. I'm sorry you took offense.

If I had the time or inclination, I would go through the Matrix Games reviews on your site and see if this is true. However, I am pretty sure I already know the answer to this one. But do know that I don't think it is limited to just Shrapnel reviews.

And thank you for the Awards for both Dominions II and Dragoon!

Regards,

[Annette]

Hi, Jim,

I sincerely welcome you to our forums and am pleased that you have taken the time to voice your opinions, particularly on such an unpleasant topic which seems to be turning more unpleasant by the moment. Please allow me to openly address some of the points you’ve made.

I appreciate your invitation to send early review code for hot titles. Our policy is to ship review copies only after shipment of all pre-orders have been made.. We feel strongly about this commitment to our customers who have demonstrated faith in our games and made a purchase having never read a review.

I’m not sure I understand your conclusion that my comments here demonstrate a belief that all reviews published by The Wargamer are biased or there’s a “conspiracy” on the part of The Wargamer against Shrapnel Games. Surely if I were to make such claims for the general public to read, I would back them up with more documentation than excerpts from two game reviews. My intent was to demonstrate, as War_Oberst says, a review is ultimately the writer’s opinion. It is up to the editors to ensure the review is helpful to the reader. I think he has every right to expect a high set of standards from all sources of news and reviews and that sites such as The Wargamer and TGN should be held to even higher scrutiny as they both could easily be perceived as biased. I think The Wargamer’s editorial policy and peer review process fell apart on the Raging Tiger review.

I disagree with Tim on the relevance of your reference to Dominions II. I’d like to thank you for quoting the Documentation portion of that review. I believe it further illustrates my point. With all due respect, however, it was not the last of our titles reviewed by The Wargamer prior to Raging Tiger. Mario Kroll reviewed StarFury on Feb 19, 2004, and Jim Cobb reviewed Dragoon: The Prussian War Machine in December ’04 (which was also given The Wargamer Award for Excellence). In these three reviews (under general sub-headings such as “Documentation and Installation”), the reader is given an in-depth explanation of why the writer draws his overall conclusion of the documentation. This explanation is clearly lacking in the Raging Tiger review. You say yourself that “criticism of a manual usually isn’t limited to its length but rather its quality and, more importantly, the need for it.” In the Raging Tiger review, the writer admits the manual is “crammed full of essential detail” but later concludes the game,"...is incredibly detailed, yes, and the potential for play is vast, once the player gets past the flunky controls, poor graphics, and big manual.” I think your readers are left to wonder if the detail in the manual is essential, why is it something they must “get past”?

I’m not asking that you alter your writers’ opinions or only publish favorable reviews. I’m asking that you hold your writers’ to the standards The Wargamer has set for itself. Mr. McKenna quite clearly did not like the game. That’s okay. Cheap shots like saying in his footnote bio he needed new eye glasses after playing the game are not. In your most recent post you indicate that such comments are intended to be “witty.” It seems War_Oberst’s advise to me, “…a sense of humor like that may just make you seem less objectiv,” may be helpful to us both.

I’m sure I would be able to search The Wargamer site and find many, many examples of objectivity, balance and fairness. I’m sure I would also find other examples of bias. What I know I would not find is any disclosure that The Wargamer is owned by David Heath who also owns Matrix Games. Only long-time readers would understand the implication of the merger between MilitaryGamer and The Wargamer mentioned it's "Site History" section. For the rest of us, it’s pretty much a secret.

[Dave Erickson]

Although I DO believe that the timing of reviews does reveal a bias, I DON'T believe the divergent tones taken by these reviewers is due to a lack of objectivity. Rather it is due to what I have already described in the "Kicking the Retail Habit" thread. It is simply a matter of name recognition -- Gary Grigsby vs. some guy.

Not much you can do about it, I guess, except promote the heck out of your stuff and do what you can to get around this attitude -- start your own review site, find where these attitudes are prevalent and avoid them, find reviewers/consumers who aren't so enamored of mainstream stamps of approval and get them all together in an undiluted bunch, whatever.

But as far as Wargamer itself goes, aside from occasional weirdness (such as not mentioning "Prussia's Glory" in their new upcoming releases article and some other stuff I'd rather not go into), it is actually a lot MORE objective than they were under the previous management. A LOT MORE.

I think Annette laid out the problem indies face as well and as clearly as is possible. There IS a problem and I'm happy to see Shrapnel lay it out so clearly. I hope some kind of action will follow, for the sake of the hobby.