An editor-in-chief's conundrum thanks to Sim City's rampant issues.
Reviewing games isn’t the easiest thing in the world to do. Boiling down exactly how you feel about an experience while providing an objective assessment of the game’s components is a delicate balancing act. With the introduction of always-online games, specifically Sim City, this fragile process has been put to the test. As editor-in-chief of this website, I have to consider how this new component of gaming is factored into our review policies and make a call about how the game’s score is affected.
If there’s one thing we know for sure at this point, it’s that always-online games can have disastrous launches. Diablo 3 was the first time PixlBit has really had to consider this complication in a review and at the time, Julian Titus and I decided that it was best to not let launch day hiccups tarnish the score of an otherwise phenomenal game. But it’s hard to say if we truly handled this perfectly.
If you take a look around the internet today, you’ll see that a variety of sites have handled these Sim City issues in a variety of different ways – but one site in particular caught my attention. Polygon originally released a review of the game with glowing marks (9.5/10), but since the game released have continually lowered the score due to the server and connection issues; first to an 8/10, and finally to 4/10.
Now, I’m not trying to pick on Polygon, I think they are a fine editorial establishment; however, this seems like a completely unacceptable way to critique the game and actually touches ever so slightly on the fact that there are key differences between critical evaluations and recommendations. In this case, it seems as if Polygon has modified their score as a means of warning their readers against buying the game, while subsequently invalidating the entire body of text that once justified their original score.
Of course, all of this is on open display, so the reader can very clearly see the progression of the score from its original number to its final resting spot. Because they claim the review is indicative of the experience on launch day, it’s not likely that they will raise the score back to its original number when the server problems subside. Personally, I take issue with this approach.
Realistically, reviews have a shelf life well beyond the week they are released. There’s no question that they are more likely consumed in that time span by readers, but if they were out and out irrelevant beyond that time, most webmasters would likely remove the content to preserve bandwidth for more current stories. Bearing this in mind, it seems that the appropriate action for Polygon would be to raise the score when the problems subside.
… Or would it?
Part of the dialog of this argument stretches beyond the confines of the purpose of a review. As a gaming society, it’s pretty clear that always-online DRM isn’t an acceptable design choice. Even beyond the widespread launch issues these games tend to face, there’s a stated desire for players operating in a single player capacity to avoid online interaction at all. In that case, do you use the review as a soapbox to affect change? Perhaps in this regard, Polygon’s longstanding low score will serve as a historical marker and reminder of the negative effects of a poor launch due to always-online DRM.
As you can see, there’s a certain predicament here. Do you maintain the score and ignore the fact that the game is crippled due to this component or modify it and mark it down for including something that’s clearly a bad design? I’m not really sure there’s a “right” answer.
I’ve already revealed how we’ve chosen to handle this situation in the past, but again, I’m not sure that’s entirely fair either. I do think that the critical evaluation of the game and the score that’s assigned to the game as a result shouldn’t be lowered due to a temporal issue. However, I do think it’s completely fair game to caution your readers against buying the game, despite releasing a review that awards the game high marks.
Public Service Announcements go a long way and if you’re making it clear to your audience that a good game is hidden behind some draconian DRM, perhaps that’ll affect that change, without compromising the integrity of the review. An alternative I’ve considered, is simply withholding the review (still issuing a PSA) until the dust settles and your evaluation of the product is completely accurate.
However, I’m still not sure it’s quite that simple. As Mike Wall pointed out to me, it’s almost impossible to properly evaluate these online titles because it’s unclear at the time of release how the meta-game will evolve. In the case of something like Modern Warfare 2, while sound upon its release, legal troubles stalled the fix of some serious multiplayer bugs rendering the game effectively useless for series stalwarts.
We live in an interesting time where the product released to store shelves isn’t necessarily the final form. Games are viscous in this day and age; constantly evolving based on both customer feedback and reported bugs resultant from development cycles that may just be too short to release a properly polished product (at some point I’d love to do some more editorializing on the ethics of this practice). Seemingly vigilance and adept handling of these sensitive situations is the only means by which a site can properly represent games in an editorial fashion, quite unlike the once rock solid launch day review that covered the product as it would exist forever.
Comments