Follow Slashdot stories on Twitter


Forgot your password?
Communications Media Movies Network Networking Television The Internet Entertainment

Netflix Replacing Star Ratings With Thumbs Up and Thumbs Down ( 97

An anonymous reader quotes a report from Variety: Get ready to say goodbye to star ratings on Netflix: The company is getting ready to replace stars with Pandora-like thumbs ups and thumbs downs in the coming weeks. Previously-given star rating will still be used to personalize the profiles of Netflix users, but the stars are disappearing from the interface altogether. Netflix VP of Product Todd Yellin told journalists on Thursday during a press briefing at the company's headquarters in Los Gatos, Calif., that the company had tested the new thumbs up and down ratings with hundred of thousands of members in 2016. "We are addicted to the methodology of A/B testing," Yellin said. The result was that thumbs got 200% more ratings than the traditional star-rating feature. Netflix is also introducing a new percent-match feature that shows how good of a match any given show or movie is for an individual subscriber. For example, a show that should close to perfectly fit a user's taste may get a 98% match. Shows that have less than a 50% match won't display a match-rating, however.
This discussion has been archived. No new comments can be posted.

Netflix Replacing Star Ratings With Thumbs Up and Thumbs Down

Comments Filter:
  • by 93 Escort Wagon ( 326346 ) on Friday March 17, 2017 @06:10PM (#54062339)

    My experience with Netflix's star rankings is they matched my inclinations pretty well... except when it came to Netflix-produced content. With that stuff, Netflix invariably told me their "best guess" was between 4.7 and 5.0 stars, every time - but, after watching it, I don't think I gave any of it even 4 stars.

    So perhaps they're trying to hide the way they're gaming the system to favor their own products.

    • by PRMan ( 959735 )
      Now that you mention it, this seems very likely. All of the most disappointing ratings I've received lately were from Netflix produced shows.
    • I think you hit it on the head. I've rated Netflix produced stuff 1 star 90% of the time. So are the majority or ratings when you look through them.

    • Strangely this change is suddenly coming after Amy Schumer's latest comedy flop. And the claims of "review bombing" of course, every person that I've heard talk about watching it said it was a steaming pile of shit and worthy of it's 1 star review. A few people I know said it was so bad they wished they could have gotten back those precious minutes of their life.

      • by rtb61 ( 674572 )

        What is really stupid about this, it's like computers, duhh, they can only do one thing at a time, in long slow mud monkey time. They could simply do both and people who prefer stars focus on stars and people who prefer thumbs, go with twiddling their thumbs.

      • Quite clearly a case of "review bombing" to me. It is not her best work, but if you like her other stuff, chances are you will like this one too.
        Of course her style is vulgar and provocative, and this is not for everyone, but it in no way deserves 1 Star.

        It really looks like most reviews came from people that knew beforehand they would never ever like it, but have an axe to grind.
        Having these people reviewing the show clearly distorts the intent of the review system.

        You can say everyone are entitled t
        • by Mashiki ( 184564 ) < minus caffeine> on Saturday March 18, 2017 @05:07AM (#54064379) Homepage

          Should be very easy for you and netflix to prove then. After all, youtube does this already. You have to watch(aka play) a percentage of the video in a steady stream for your up/downvote to be counted outside of what's actually displayed. The fact that netflix isn't showing this to be the case, and that the media has simply run with the "review bombing" and giving Schumer a platform to cry over while screaming "harassment/sexism/alt-right/etc" says that it's not a case of that. But people believe that it was simply terrible and were more vocal than normal, especially how much she and netflix tried to hype it up.

          This isn't any different then the ghostbusters trailer flop, followed by the movie flop. She along with the other actors and director then went flailing their arms screaming "sexism/misogyny/etc" and the media started falling all over itself promoting that narrative. Despite the fact that it was simply a shit movie, the trailer was shit as well, and people in general hated it.

          • How the hell could I prove something like that? I have no access to netflix internal statistics.
            But, if there was a coordinated review bombing, I assume they are clever enough to leave the show streaming in the background so it looks like they actually watched it.
            The only indicator I see that could detect this is if it gets unusually many reviews quickly after launch. But since netflix don't disclose such numbers I guess we will never know.

            But I guess it doesn't really matter. I just find it fishy that
            • by Junta ( 36770 )

              I'm not that invested either way in her, but Netflix has thrown her in my face a lot. I have better things to do than do whatever Netflix tells me to, so I haven't bothered, but the fact that I know about her netflix at all says something...

              If someone is polarizing, *and* something like netflix promotes it and throws it in everyone's face, they shouldn't be surprised that overwhelming negative reviews come at them. Doesn't need to be a conspiracy. This is the problem with a lot of netflix original conte

            • by Mashiki ( 184564 )

              How the hell could I prove something like that? I have no access to netflix internal statistics. ...

              And you made the assumption just like her that it's review bombing. But we're supposed to take her word on it right? Seems that this is more along the lines of the trend in media and with elites that women are a super-protected class and you should believe everything they say, especially when there's multiple articles all saying exactly the same thing. You really have to believe her. After all, if you're critical of a women it's sexism, misogyny, conservatives or the alt-right. Just like ghostbusters la

        • This is why I miss the "Not Interested" button.
          For example, I don't watch horror movies. It may be the best horror movie ever so it may be worth 5 stars but I am still not interested.
  • Great shades of Tivo! Man, I miss that awesome interface...

    • by hawk ( 1151 )

      Tivo is still around.

      Well, sort of.

      I switched from Directv to the hated cable company to get back to a tivo (a romio). Turns out that the interface just isn't, well, what we liked tivo for.

      Rather than clicking on record in the listings, it's something like three. And for a season pass, rather than clicking record twice, it's several. Because, gosh, they've got to make buying it to watch the default first choice, don't they?

      Can't screen for series premiere any more either.

      Now, it's just a slightly better

    • by quenda ( 644621 )

      OK then, next stop the Facebook model: Just a thumbs up button, no down.

      Who cares? The better way to choose Netflix shows is to use your PC/tablet browser with Nenhancer plugin. []
      It will add IMDB and Rotton Tomatoes ratings and links for all shows. Add the interesting-looking ones to "my list" and go watch.

  • A/B Testing is an amazing tool. How many A/B tests have to be stacked with a 99/1 split to result in a 10% assurance that someone is going to be confused and disgusted with the change.

    This one was a 66/33 split. Recalculate. How many ...

    The closer we get to 50/50 the faster we make the user experience worse by listening to A/B tests without reason.

    How many A/B Tests have been run by our favorite applications to hate.

  • by LarryRiedel ( 141315 ) <> on Friday March 17, 2017 @06:17PM (#54062381)
    Netflix will control the narrative, with minimal input from pesky users. How long before there's only a heart symbol? How long before they take away comments like IMDB because you don't want them.
  • What Netflix thinks I want to see and what I actually want to see lines up less than half the time. I want to be challenged and I want the interface to give me an opportunity to step outside of my comfort zone.

    I don't see how reducing this to thumbs up/down is going to help that in any way.

    • I don't see how reducing this to thumbs up/down is going to help that in any way.

      More engagement -- "thumbs got 200% more ratings than the traditional star-rating feature." Anecdotally, I often finish a show and don't rate it, as I find myself wondering if I "just" liked (disliked) it, or if I REALLY liked (disliked) it -- but a binary choice is often pretty obvious for me.

      That said, I think it's often best to use some third-party tools/blog posts/friends/etc. for determining what to watch.

      • Rating is easy...and you could always change your mind.

        1 Star - A Sack of Still Steaming Shit.
        2 Stars - Stinks on a Good Day
        3 Stars - Meh
        4 Stars - Not Bad
        5 Stars - Pretty Damned Good

        Thumb up/Down only is freaking useless unless it also allowed for 2 Thumbs up, 2 Thumbs down, and Meh (1/2 thumb).

        Just like Google Music, there is no distinction between "heard before and it was meh". Like or Dislike is just not anywhere near distinct enough to account for taste.

        • It should be a two tier system. Click thumbs-up / thumbs-down, then pick a star rating. So you can mark it as "remember this, also remember a general bad/good" then say how much you actually liked it.

          What makes me mad about Pandora is I can't mark "remember this" without changing the recommended songs given to me. I don't want to change my channel, I just want to be able to listen to it later.
        • That's exactly the issue. My interpretation of 5 stars goes like:

          1 Star - Shit
          2 Stars - Bad, but with some minor redeeming features
          3 Stars - Mediocre, nothing special, OK if in a certain mood
          4 Stars - Good, solid, not groundbreaking, but good
          5 Stars - Amazing, great, top of the heap

          You see that our interpretations vary. We don't assign the same value to the ratings, which is a problem is a shared rating system. One persons "eh, it's OK" is another person's "would never watch again".

          With a binary system, it

          • How is it any different?

            1 - Sucks a lot
            2 - Sucks Less
            3 - Good but has suckitude
            4 - No Suckitude
            5 - Great

            4,5 is Thumbs in the general Up direction.
            1,2 is Thumbs in the general Down direction.
            3 is MEH

            So you lose any distinction between just how much its liked/disliked or whether it was merely average.
            • Why is it important to you to classify a difference between how much you like/dislike it? Isn't it enough to see your rating and then remember for yourself why you gave it that rating?

              I tried rating my music library, first with a 5 star scale, then a 4 star and a 3 star scale. In all cases, I spent more time fussing about ratings, than I did about the actual music.

              So I actually ended up going with the simplest solution of just marking my favorites and leaving the rest well enough alone. Now I'm on Spotify,

      • by Anonymous Coward
        I too am sometimes unsure whether to give something a 3, 4, or 5 star, but the real problem for me was the lack of a neutral option. There are many shows that I didn't particularly like, but didn't dislike either. Yes a 3 is the middle of a 5-star system and so should fit that, but they specifically label 3 stars as "like", and 2 stars is specifically labeled as "dislike". Without an actual neutral option, I either don't bother, or I just pick 3 anyway since it seems less incorrect to say I liked it than
  • All the ratings from every single viewer watching every single show and the best algorithm to process that data in the world won't help if you don't have a wide enough variety of content to recommend to your users.

    At one point in Netflix's history, the number of 'Action movies with a strong female lead' on netflix was so low, but they knew that's what I liked, they started recommending to me rom-coms such as 'Bridget Jones Diary'
  • by evolutionary ( 933064 ) on Friday March 17, 2017 @06:23PM (#54062417)
    So basically, they figured that 3/5 starts is a "thumbs up" and there will be more positive ratings (and making their offerings look better) while abstracting (hiding) the "meh" factor. Gotta love marketing.. I remember an old phrase from my math tutors: Statistics can lie. :D
    • by ChromeAeonium ( 1026952 ) on Friday March 17, 2017 @07:39PM (#54062841)

      Could be. I'm playing a game I got on Steam right now. It is thoroughly mediocre. I want to rate it, but Steam has only a positive/negative system as well, and this game deserves neither. So which do I give it? If I round up, it makes the game look better than it is. If I round down, I am being giving an inaccurate portray of how I really feel. I either am inaccurate, or I make tht game look better than it actually is in the rating aggregate, which will increase the likelihood of Steam making a sale.

      In Netflix's case though, seeing as how you've already bought the subscription, I don't see how this will help them, but maybe that is their angle. Either way, as someone who gives far more 2-4 stars for things than 1 and 5 stars (because most things tend to fll somewhere between total crap and absolutely amazing), I do not welcome this change.

      • If I round up, it makes the game look better than it is. If I round down, I am being giving an inaccurate portray of how I really feel.

        You are but one drop of rain in a monsoon. Ratings *should* appropriately dither over the aggregate, so the few who are in the middle will likely half vote up, while half vote down.

        While up/down may not be entirely fair, there's really more options for manipulation in a star rating system. By removing zero-star ratings as an option, they can artificially inflate scores.

        • You've marked me as a foe, but I definitely agree with you on this.

          Unless you account for the total number of ratings, a title with three 5-star ratings (5.0 average) will rank higher than a title with ninetyfive 5-star ratings and five 3-star ratings (4.9 average), and that's obviously an issue.

          I think moving to a thumbs up/down system will be good. Personally, I don't really care about rating stuff in detail, I just want to mark whether I liked it or disliked it.

      • I plan to simply give anything I would have rated 1-3 stars a thumbs down, and only give something I rated 4 or 5 stars a thumbs up. If they would to adjust the narrative, so will I.
      • You basically just have to decide for yourself, whether you liked it or disliked it overall?

    • Re: (Score:2, Insightful)

      by Anonymous Coward

      If they're going to convert my 2000+ ratings to thumbs up / thumbs down, then they better choose 4 as the cutoff for thumbs up.

      5 = great ("top 100 of all time; would recommend to everyone")
      4 = very good ("thumbs up / would recommend to other people, but only if I know their taste in movies")
      3 = okay ("I've seen it / it's not good enough to recommend, but not bad enough to badmouth")
      2 = bad ("thumbs down / would badmouth to anyone that asks")
      1 = awful ("one of the worst things I've ever watched / never watch

    • by antdude ( 79039 )

      I'd prefer up, down, and medium. Just three choices.

    • I see something that looks slightly interesting, maybe I'll waste a couple hours on it. Then I see the 1/2 star rating and I stop. Maybe it really is bad. Or maybe people are just stupid and rating it without seeing it, or rating it to get it off their queues, or... Popularity isn't a good indicator of quality or enjoyment. Sometimes a really bad movie is just the thing for saturday morning. I need to train myself not to look at the ratings.

      • I don't think the Netflix ratings you enter actually count unless you've watched some significant part of the title in question.

  • by uCallHimDrJ0NES ( 2546640 ) on Friday March 17, 2017 @06:24PM (#54062427)

    I'm sure there are many exceptions amongst the Slashdot user base, but hardcore critical thinkers aside, people don't use anything but "5 Stars" and "1 Star" anyway. They love it or they hate it. This change will make the experience more we KNOW it's just a bunch of crap. I'll be much less tempted to believe the recommendations now. That said...the recommendations Netflix used to give me, using the algorithm they used back in 2005...those were uncannily accurate.

    • by PRMan ( 959735 )
      Now Netflix just gives me a bunch of 1- and 2- star rated shows that they think I will... I don't know... hate?
    • I disagree here. Netflix's system is a recommendation engine and not a rating engine. Yes, you are correct that a lot of morons misuse the 5 star system, only voting 1 or 5. However, a lot of people also use stuff n the middle. I think for recommendations you could get away with three levels.

      0-2 can all be summarized as bad/didn't like. For recommendations, I don't see a lot of value in differentiating between complete garbage and bad stuff I don't want to watch. That could all be boiled down into thum

  • A web-site changes its interface... Stuff that matters? Seriously?

    • Are you new here? []

      Fortunately there are scripts on github to download the data. A sad day for the non-Bayesians.

      • by mi ( 197448 )
        Well, that [] was a mathematically-interesting story. And today the individual subscriber match part of TFA is interesting too, even for those of us, who do not use Netflix:

        Netflix is also introducing a new percent-match feature that shows how good of a match any given show or movie is for an individual subscriber

        But the change from stars to thumbs?..

  • Netflix becomes PC (Score:5, Insightful)

    by Silvergoat ( 75465 ) on Friday March 17, 2017 @06:33PM (#54062485)

    Great. Now the PC world has invaded online movie reviews. Sorry, haven't seen Lena Dunham or Amy Shumer, mostly because I dislike their material. Maybe it's a generational thing. Netflix is now invalidating their recommendations because some whiny Hollywood types got their feelings hurt. Has anyone ever seen Heaven's Gate? Seriously, if Netflix can't stand the heat of their ratings system, why even have a ratings system? Is Reed Hastings (Co-founder and CEO) so needy for West Coast adulation that he's gutting their system? Sad.

    • by hawk ( 1151 )

      >Lena Dunham or Amy Shumer, mostly because I dislike their material.

      wait a minute, "their"?

      Are you saying that she is, er they are, not the same person?


    • What the fuck does this even have to do with Lena Dunham or Amy Schumer? They're not mentioned anywhere in the summary, nor in the article.

      So why the rant? What's the point?

  • The current rating system works for me. I generally only consider shows with 4 stars or above. It seems to correlate with my expectations. I have tried to watch stuff with 3 stars only and found it lacking. If the new system blurs 3 stars versus 4 then I will end up watching more crud that I don't want to. However, I guess we will have to see how it plays out.
  • They gave thumbs up/down ratings for movies, instead of ratings out of 4 or 5 stars (or A-F) like everyone else

    One of the reasons why I think thumbs up/down works better than 5 stars is that everyone has a different idea of what 2, 3, and 4 stars should mean, and tend to skew towards 4 or even 4.5 stars as a midpoint ("average" product) instead of 3 as you'd expect from the scale's range. e.g. Amazon's 5 star system sounds like a really effective tool at first glance, until you learn that the average ra []
    • Siskel and Ebert each used a thumb, don't know what they did with the other two, but it wasn't a binary verdict. If ratings skew high this change means more of Netflix's content will have the highest rating, which of course is in their interest, though this seems really about eliminating public ratings, in which case 5 or 10 star private rating scales would have provided more precision at least for some users, but apparently the data shows that their customers find it too mentally challenging or that the ca
  • by Anonymous Coward

    In physics, many a times S/N ratio is extremely poor for an individual bit of data. In such cases, it is better to 1 bit quantization rather than multi-bit for optimal performance. In star rating, you are giving more weight to people rating it at extremes. With thumbs up and thumbs down, the weight is same for everyone.

    Theoretically, the optimum performance is to use "e" levels of quantization which I am not sure how to achieve. For integers, the 3 levels (like, dislike, no-opinion) is the most optimum when

  • Unqualified ratings systems are painfully limited whether they are stars or thumbs. Context is needed to know *why* people liked or disliked something. I can read reviews to get a small sample of this (assuming people have taken the time to write good reviews), but letting users tag content allows us find stuff we like so much more effectively. Tags for sub-genre, themes, memes, good acting, bad SFX or anything else people might be looking for. I wish Amazon would do the same for books too.
  • I think it was 2 years ago, it's so long I don't remember it anymore, but I really hated when that happened.

    I liked to see what stuff had one star rating, because it usually told me it wasn't worth watching, now I have a HUGE list of "continue watching..." that won't go away, simply because there's so much trash that Netflix "guesses" that I want to watch.

    Bet the thumb system was introduced to stop people from complaining about too few movies and series, and watch "whatever" is available instead of being selective about what we watch.

  • by Anonymous Coward

    Yes, I know there is probably a team responsible for the interface.
    The problem is that when you have a well thought out interface and that team still has a job, they have do work this means they have to make it worse (Hi Windows!). "We have created the perfect interface, now the team has some proposed changes.."
    So they will go about endlessly fixing what ain't broke and then not going back to what was working because that would admit fault.
    So now on PS3 the movies begin playing before you can read all of wh

  • This is obviously what Reed Hastings meant as innovation that traditional theaters couldn't deliver.
  • Use IMDB (Score:4, Informative)

    by hcs_$reboot ( 1536101 ) on Friday March 17, 2017 @09:33PM (#54063287)
    Use an extension like this one [] that gives IMDb ratings on the Netflix page.
  • by UnknownSoldier ( 67820 ) on Friday March 17, 2017 @09:36PM (#54063299)

    What's next -- removing the thumbs down???

    What the fuck is the point of having ratings if you are just going to make them homogeneous???

    Just because _you,_ Netflix, don't let me rate a movie 0/5 doesn't mean it deserves a 1/5. IF I hate a movie it should get 0/5.

    The WHOLE point of a 5 star rating is to provide fine-grained-ratings not some bullshit dual artificial rating. There is a HUGE difference between me liking a movie 50% and 100%. Some movies are 3/5 (60%), 4/5 (80%), and very few are 5/5 (100%). Lumping them these ALL together is fucking retarded. HOW is that helping the system find stuff I _really_ like vs stuff I _kind of_ like???

    • by dohzer ( 867770 )

      Isn't the next step having a thumbs down button that does nothing? Like YouTube comments. Yay!
      Even FB doesn't have one. What a waste.

  • Right, because we only ever "like" or "dislike" things. As humans, we never have any ambiguity in how we feel about things.

    This is stupid, and a ham-handed attempt to "dumb it down" in order to boost the perceived rating. Ugh.

  • And we continue the dumbing down of america.

"For a male and female to live continuously together is... biologically speaking, an extremely unnatural condition." -- Robert Briffault