What's Broken: Reviews

 On a regular cycle, someone complains about the death of the OSR (whether it was 2013 or 2017 or yesterday or tomorrow), and inevitably somebody chimes in talking about how things got “too commercial” and everybody in the “scene” wanted to make a buck and crank out a product.  That’s worth talking about, but there are some other issues that dovetail with it that I think I need to discuss first so we get a complete picture of where I’m headed with all this.

One core issues I see - and I’ve occasionally complained about it on social media, I suppose - is our lack of a review culture.  We don’t have meaningful reviews to consult and compare.  We have shilling and we have tear-down rants, and very little in-between.

There are two large classes of OSR “reviews”:

• THIS IS BAD AND I HATE IT

• “This is not written the way I would write it” (fair enough, but diminishing returns as to usefulness of this sort of review)

• “This classic product sucks/can’t-be-run” (often with a side dish of “I’m much cleverer than these 70s-80s people”, which is distasteful even when true)

• THIS IS AWESOME AND I LOVE IT

• “Everything this guy/company does is equally great”  (usually not true, and how could it be?)

• “This looks really high-quality doesn’t it”  (so?)

• “This is super-innovative, subversive, artsy and I love it.  Also, my friend made it.  No, I haven’t run it LOL who actually plays games?”

• “I used parts of this, twisted to meet my needs under a different ruleset, and I changed most of it” (interesting reading, and a standard byproduct of the way we hobby, but perhaps less useful to some readers)

With most of both of these written without ever having run the product.  Imagine how different our discourse might be if we had the expectation that once you ran an adventure, you actually talked about how it went.  Instead, we’re stuck in a rut where we have a “balance” between rants and stroking.  Bryce Lynch writes reviews which are sometimes amusing to read, but have become so repetitive as to be much less useful than they used to seem (whether this is Bryce’s fault or that of the endless stream of mediocre adventures he dares to read is a separate question).  Fear of a Black Dragon produces interesting reviews much of the time, although often they take a larger work and bang it into a rules-light shape.  Questing Beast usually does “reviews” that are more about the paper quality and the art than about using the material at the table.  Some old-school bloggers go in-depth into “what’s wrong” with a TSR module - and maybe even attempt to provide solutions - but never bother to run the adventure, even WITH the proposed fixes.  

Can’t we have reviews by people who ran the product?  Answer: perhaps not if we want reviews to appear during the main sales push cycle for a product.  Yet if everything’s available forever, I’d happily take a robust long-tail review culture over what we have now.

Here’s a call to action: if you haven’t actually used the product, don’t call it a review.  Have some dignity.  Call it a “first look”, a “preview”, an “impressions”, whatever.  Let’s make words have some meaning?  And if you do a preview and later run the adventure or the game, come back and do an actual review!  Compare your first thoughts to your eventual conclusions!

So here we are without a review culture to guide consumers (ooh, he said the “c” word!).  We have tastemakers and influencers but not reviewers.  Consumers - potential buyers of an adventure or a game - will see some guidance.  They are hungry for information to help them make a good decision about how to spend their money and time.  Without any reliable source of guidance to distinguish between products, we’re going to fall back on a) known producers of good reputation (be it writers or companies), b) impressive visuals (be it artwork or production values).  See where this leaves the independent producer?  In the mud.  If you’re a newish writer with a small budget, you will very quickly feel like you cannot compete.  And you can’t. 

Why can’t we talk like adults about products?  Why can’t we talk like adults about art?  There’s a gulf between “being courteous in criticism” and “avoiding all negativity”.  It’s nonsense.  We’ve gone beyond courtesy to cultishness.  You can’t criticize modern (postmodern?) works without being dogpiled.  (You can, of course, criticize things from before 2000 with impunity, and are encouraged to do so, because obviously they’re all Neanderthal crap.)  In OSR/indie circles, it’s perfectly acceptable to say “80% of what’s on DM’s Guild is hastily-produced shovelware that nobody will actually use at the table”. Dare I say it about the arthouse/indie OSR?  I suppose I do: “80% of what the OSR produces is farty shovelware that nobody will actually use at the table.”  Does that make it valueless?  Maybe, maybe not.  But we’re at the point where we’re afraid to express opinions and to disagree civilly.  It’s silly.  (Leave aside for the moment the idea that “proper” OSR mindset doesn’t really need anybody else’s product, as that’s a can of purple worms for a different time.)  Meanwhile, we have also somehow decided that we should only “review” products we like (because our time is valuable, and criticism is mean?), so we’re not learning anything from our (collective) mistakes because we aren’t talking about them.

While we’re at it: social media folks and YouTube types, please stop pretending that vagueposting about a problem you had with a product you refuse to name is helpful to the hobby.  It makes me want to call you and tell you there’s a small leak in your basement; no, I won’t tell you where.

We gave up criticizing “scene” products because it felt dirty.  It’s fine to yell at WotC, or TSR, or whoever.  Publicly criticizing another scenester, however, that’s backstabbing for triple damage, and completely verboten.  Unless you have a full-on beef, of course, in which case we’ll all gladly drink in the drama.  But actually talking about someone else’s work in a civil way?  Nope, can’t have it.  Not only do we have a layer of “you’re fine just as you are and shouldn’t be goaded into self-improvement”, but we have the scenester layer of “we’re all in this together and we don’t talk crap about each other”.  It’s nonsense, because it doesn’t improve the products, the people, the hobby, or the games.  It encourages everybody to try to be an unassailable auteur (“artiste”) and then get depressed when their macaroni-art high-concept adventure doesn’t sell five hundred copies.  You think rpgs are a community?  What functioning community sets its own members up for failure like that?  That isn’t a guild of artisans who teach one another, that’s a giant crab bucket.

I’ll come back to OSR marketing in another post and try to keep this one to the topic at hand - reviews are broken.

How do we fix it?  We reverse course.  We establish that it’s okay to talk about products and what you didn’t like about them.  What didn’t work for you, what didn’t work at the table.  We place the focus of the hobby back on the DOING - the actual playing of the games, and the discussion about what worked and didn’t.  We normalize accepting civil criticism civilly.

Play the games.  Talk about playing the games.

(Side-note: watch the interaction numbers on the post climb, since it's "Drama". And that's precisely the problem...)