How relevant are reviews?


How relevant are reviews?

Book reviews are of critical importance for writers as they provide an independent assessment of a novel for potential readers, the problem is… they’re often bipolar.

Here are two reviews for my novel Mars Endeavour—one star and five stars.

  • The high rating for this book on amazon is incomprehensible. The writing feels like it was done by a fifteen-year old in a creative writing class
  • I rarely write reviews… You know a good story when it holds you and gives you an emotional reaction and maybe even a physical one, a slight increase in the heart rate, tension in the stomach as you turn the pages

So who’s right? Serious question. Which review should you believe? And why?

You see, the problem is most reviews are polarised—they represent the extremes rather than the norm.

When less than 1% of readers leave a review online, the result invariably represents the outer edges of a distribution curve rather than the sentiment of the majority. It seems, only those that either love or hate a book will bother to comment on it.

Looking at a classic distribution curve, it’s clear reviews catch only those on the fringes.

normal_curve

With 99% of readers not providing any rating, we never get to see what the majority of people think about a particular book.

The problem is two-fold.

  1. Not enough ratings/reviews are left by readers.
  2. There’s no way to know who to believe. The naysayers or the enthusiasts?

I’d like to propose a solution, and I dearly hope someone from Amazon considers this as I think it would work—personalize ratings.

At the moment, reviews on Amazon appear something like this.

screen-shot-2016-12-31-at-9-59-34-am

But what if Amazon also included a personal rating? Where a comparison is made between books you’ve rated in the past, and what those that agree with you back then think about the book you’re currently considering.

screen-shot-2016-12-31-at-10-12-01-am

Your personalized rating would be the intersection between these groups.

In other words, predicting whether I’ll enjoy a novel by matching my past reads with other readers that share similar likes/dislikes.

It really doesn’t matter how the other readers have rated other books, so long as we roughly agree. If we all rate the (hypothetical) novels…

  • Cars on Mars with three stars,
  • Loons on Moons five stars, and
  • Guns on Suns one star.

The question as to whether I’ll enjoy the fourth book in the series, Who goes to Pluto? is highly likely to be similar to those that rated Cars, Loons and Guns in a similar manner to me. It could potentially look something like this…

screen-shot-2016-12-31-at-10-30-40-am

Or conversely…

screen-shot-2016-12-31-at-10-30-52-am

With hyperlinks taking me directly to those reviews of this book by readers that rated other novels in the same way I did.

In both circumstances, the reviews are now tailored to be more applicable to my previous likes and dislikes, still giving me the choice to consider or reject reviews as I see fit, but ensuring I have a more accurate assessment of whether I’m likely to enjoy a particular novel.

This approach encourages readers to rate lots of books as the more books they rate the more accurate the predictions about future reads will become.

This would also be an effective means of dealing with both troll reviews and fake reviews, as they’re taken out of the equation.

Some other points to consider are “liked reviews” should count toward the personalized review rating. Also, it might be impractical to get a 100% match on “books other readers have rated the same as me,” so there may need to be a tolerance of 1-2 stars applied, but I suspect this would ensure reviews are relevant to readers and provide them with an accurate assessment of whether they’d enjoy a particular novel. There may need to be a minimum threshold of 10 comparative reviewers to ensure accuracy.

In essence, this would shift the focus from trusting random reviews to trusting in similar, like-minded reviewers. To my thinking, this approach would ensure reviews were relevant and remove confusion/uncertainty over whether someone is likely to enjoy a particular book. It also increases the level of difficulty for those gaming the system unfairly.

Do you agree?

Do you have any other ideas?

Feel free to comment below.

 

 

Advertisements

4 thoughts on “How relevant are reviews?

  1. Hey Peter,
    The issue of extremes has always been a problem with reviews, and makes it difficult for those seeking information about the book, both the author, and other readers looking to see if it is worth reading. Whilst constructive feedback on points someone might not like in a book is always useful, flat out negativity, or just mean/rude commentary is pointless. Reviews should be meaningful at all times anyway – regardless if the person liked or disliked the book.
    The concept of a system though that uses past reviews and other reviewers to give predictions is an interesting concept. A significantly different review based on this would need to be justified appropriately.
    It would need a decent amount of reviewers though to provide the initial input for consistency and accuracy.
    Interesting concept though and one that should be looked at.
    The current question system that has been implemented by Amazon US is not particularly good (checkboxes with a scale of questions that don’t necessarily match the book sometimes).
    Their system does need a decent overhaul to be more accurate and provide a better indication to both the reader and other stakeholders at a quick glance without having to read through hours of commentary.
    Cheers
    Jas

    • Yeah, they’re trying to improve things… the challenge is dealing with troll reviews (of which there are plenty) and fake reviews (which also abound). I got a review for Galactic Exploration that complained about the main character running around the ship killing people in the name of Allah(!)… smh

  2. Peter

    Thanks for writing this post. I’ve been meaning to do something like this for a while, but…

    I propose that Amazon publish completion rates.

    I have a novel – Cerelia’s Choice – that has been out for nearly two years and has a total of four reviews because I refuse to play any of the games authors and publishers use to get lots of reviews quickly, partly because I’m a lazy introvert, partly because I think it’s unethical, and partly because, like you, I have concluded reviews are almost useless.

    All four reviews are five star, which looks suspicious, though they are all absolutely genuine. But if I look at the Kindle Unlimited data, I can see that almost every copy downloaded under that program is read to the end (I sell few enough copies that I can see the bumps in the pages read data, bumps which are almost always equal to the page total). KDP reports don’t give me the same data for purchased titles, but there’s no reason to believe people who pay for the book would be less likely to finish it than people who download it for free.

    Amazon has this data for every Kindle title. “97% of all the people who download this book complete it” is surely far more useful to a prospective reader than “87% of the 1% of people who leave a review give it five stars.”

    P.S. I think there’s another fundamental flaw with reviews. I don’t have hard data, but common sense tells me people who start the book then give up are far less likely to leave a review than those that read to the end. The latter are certain to give it a higher rating than the others would. If you look at the distribution of average ratings, a book has to be a total stinker to get an average lower than four stars.

    Dan

    • Oh, yes… Amazon has a goldmine of data to work with… I like your idea of showing the percentage of people that complete reading the book (although the blurbs at the end might skew things — afterword, other books by, etc, but stopping at The End should at least be 97%+)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s