Is Rotten Tomatoes Still Reliable? A Statistical Analysis (statsignificant.com) 50
An analysis of Rotten Tomatoes data reveals average Tomatometer scores have climbed steadily since Fandango's 2016 acquisition of the review aggregation platform. The average number of reviewers per mainstream film release increased by 40 to 70 critics following the purchase. New additions to the critic pool include smaller outlets such as Denerstein Unleashed and KKFI-FM Kansas City. Prior to 2016, critic and audience scores demonstrated stable correlation year-over-year. Post-acquisition data shows the two metrics diverged sharply as Tomatometer ratings rose.
Fandango, America's largest movie-ticketing platform, is partially owned by NBCUniversal and Warner Bros. Discovery. In 2023 Vulture reported PR firms court reviewers from smaller outlets to secure higher Tomatometer scores before film releases.
Fandango, America's largest movie-ticketing platform, is partially owned by NBCUniversal and Warner Bros. Discovery. In 2023 Vulture reported PR firms court reviewers from smaller outlets to secure higher Tomatometer scores before film releases.
It never was reliable (Score:5, Informative)
It still baffles me that RottenTomatoes ever became regarded with any degree of respectability. Their whole system is a statistical nightmare. I remember shortly after the site came out, reading reviews that included both positive and negative features, rendered into binary classification as either "fresh" or "rotten" despite what the reviewer said. Even overall positive reviews being rated as "rotten", for example, or vice versa. It's a shit show. (Presumably the site rose to prominence simply because it somehow outcompeted other review aggregators)
Samples (Score:2)
IMHO Equally as bad is the sample set. Either you shape your sample set to include, maybe, a few dozen very reliable sources, or you hoover up hundreds of sources, and the wheat and chaff will be separated out statistically.
There seems to be a mishmash of both approaches. You get reviews from Variety, The New York Times and The Miami Herald mixed in with reviews from BuzzFeed NYC Movie Guru.
Re:Samples (Score:4, Insightful)
I'd rather get reviews from "regular people" and not "professional reviewers" with a silver spoon up their nose. I can't recall ever agreeing with any movie review from a place like the NYT.
Re:Samples (Score:5, Insightful)
I can't recall ever agreeing with any movie review from a place like the NYT.
Your taste in cinema must be pretty out there then because reviewers like NYT give widely popular movies good reviews all the time.
My big problem with fan reviews are two things. The first is the modern political stuff, I don't care what busy bodies think of a movie on either side of the political spectrum. Review bombing is a big problem with fan reviews.
The other problem I have with fan reviews is for more niche productions they can't be trusted because niche productions tend to overwhelmingly attract enthusiasts of said niche as opposed to a proper sampling of the general public. This makes the fans reviews largely useless for those with more mainstream tastes as the really big enthusiasts in a genre or other categorization of movies will give almost anything a good score within the context they're excited about
For that last one what I think of the most is anime reviews. Most fan reviews of anime are done by hardcore anime fans (outside of the rare, really big production). I am not one of those though so their reviews are useless to me as in my ample experience hardcore anime fans don't give great advice on what anime to watch to those of us who only enjoy a very small percent of this genre.
Re: (Score:3)
I can't recall ever agreeing with any movie review from a place like the NYT.
Your taste in cinema must be pretty out there then because reviewers like NYT give widely popular movies good reviews all the time.
My big problem with fan reviews are two things. The first is the modern political stuff, I don't care what busy bodies think of a movie on either side of the political spectrum. Review bombing is a big problem with fan reviews.
The other problem I have with fan reviews is for more niche productions they can't be trusted because niche productions tend to overwhelmingly attract enthusiasts of said niche as opposed to a proper sampling of the general public. This makes the fans reviews largely useless for those with more mainstream tastes as the really big enthusiasts in a genre or other categorization of movies will give almost anything a good score within the context they're excited about
For that last one what I think of the most is anime reviews. Most fan reviews of anime are done by hardcore anime fans (outside of the rare, really big production). I am not one of those though so their reviews are useless to me as in my ample experience hardcore anime fans don't give great advice on what anime to watch to those of us who only enjoy a very small percent of this genre.
I get that; however, why would I care about a reviewer of a movie that *isn't* a fan? For instance, I am a fan of science fiction and superhero movies. I don't much care if the general public would like it or not; I want to know what the fans think. If, for instance, a new Star Trek movie is getting poor reviews from critics, yet good reviews from most who watched it? I can safely say that I will at least be entertained.
I remember back in the day, when a critic for TV Guide wrote an article reviewing Star
Re: (Score:2)
I get that; however, why would I care about a reviewer of a movie that *isn't* a fan? For instance, I am a fan of science fiction and superhero movies. I don't much care if the general public would like it or not; I want to know what the fans think. If, for instance, a new Star Trek movie is getting poor reviews from critics, yet good reviews from most who watched it? I can safely say that I will at least be entertained.
In that case, sure, I agree. Why would you even look at any reviews (fan or professional) if you've already decided to watch the movie though? To have parts of the movie spoiled for you before you watch it?
What if you were only a casual fan of super hero movies like most people are though? Why would you want the opinion of an avid fan who likes everything (or almost everything) they watch in the genre when their tastes clearly don't match yours. In that case I would rather take the word of the more generali
Re: (Score:2)
Tuvok is a Vulcan. Being wooden and emotionless is kind of his thing.
Re: (Score:2)
Hahaha, I didnt remember the character's name as I havent seen the show since the 90's but that's hilarious now that you tell me that. It's not like Vulcans arent easy to spot, even if they'd never seen a Star Trek show or movie before (which would be pretty bad for a professional reviewer) the ears should give away that something is going on there that makes that character different.
This is why checking multiple reviews is helpful, no one is going to get it right all the time, tastes vary, and some people
Re: (Score:2)
I can't recall ever agreeing with any movie review from a place like the NYT.
The other problem I have with fan reviews is for more niche productions they can't be trusted because niche productions tend to overwhelmingly attract enthusiasts of said niche as opposed to a proper sampling of the general public. This makes the fans reviews largely useless for those with more mainstream tastes as the really big enthusiasts in a genre or other categorization of movies will give almost anything a good score within the context they're excited about
It would be nice to have Netflix-like personalized reviews or ratings. So, the rating is not so much what the general population likes but what people "similar" to you like. Rotten Tomatoes doesn't have the same amount of viewing history that Netflix has, but maybe people would be willing to give detailed ratings if that led to personalized ratings/recommendations.
Re: (Score:3)
Because what I want to see is Billy Bob telling me how "woke" (whatever that means) a movie is because the female lead had short hair and didn't cook in the kitchen.
Has Rotten Tomatoes– (Score:2)
Ever been a reliable source of statistical analysis?
Re:Has Rotten Tomatoes– (Score:4, Insightful)
This is one of those recurring things in America where even things that already suck ass are proving themselves perfectly able to get worse to sell out what little value they have for pennies of extra profit.
Re: (Score:2)
This is one of those recurring things in America where even things that already suck ass are proving themselves perfectly able to get worse to sell out what little value they have for pennies of extra profit.
This should be +5 insightful.
Still? (Score:2)
I'm still trying to figure out if a lot of rotten tomatoes is good or bad.
"Goodhart's Law" in action (Score:5, Insightful)
Goodhart's law states that "when a measure becomes a target, it ceases to be a good measure." This leads us to the exact problem described above: A Rotten Tomatoes score can be useful for audiences determining whether to see a movie. Studios know this, and thus will juke the stats by whatever means necessary to increase that score, which means that the RT score is no longer useful.
Re: (Score:2)
If say the main thing that changed is people started review bombing RT and IMDB.
Re: "Goodhart's Law" in action (Score:2)
Thanks. I've never heard this law name before, even though it was unsubstantiated, instinctive somewhere in my mind all the time.
Reviewers are not reliable, period (Score:1)
Look at how bad the "reviews" are for War of the Worlds. A movie of that quality doesn't deserve such hate
Re: (Score:2)
I don't really understand the point of movie reviews. If i see a trailer and think "hmm i wonder if thats good?" ill watch it and figure out if i like it
if i see a trailer and think "i think i might like that" ill watch it and figure out if i like it
what other scenarios are there?
You see numerous negative reviews for a movie you thought looked interesting and realize that you were just buying into a marketing push and skip the movie that had initially looked good.
Most folks don't like having their time wasted by a crap movie, reviews are a helpful (although not infallible) aid in avoiding that.
Re: (Score:2)
but you werent the one that formed your own opinion. why are you taking other peoples opinion for things that you should decide for yourself?
You were just talking about having your mind made up in regards to watching a movie by watching marketing materials (trailers). How is listening to the informed opinions of several people worse then blindly following marketing? I've seen plenty of movies where all the best parts were in the trailer thus the whole movie was ruined for me as I had already seen the good parts. Other times what I think I'm shown in trailers isnt what I end up getting and sometimes I don't like what I get. Even after that, somet
Re: (Score:3)
Roger Ebert was always a good indicator for movies. He was honest and never held back. Read his legendary review of the steaming turd known as Milk Money https://www.rogerebert.com/rev... [rogerebert.com]
Re: (Score:2)
Must be nice (Score:2)
Must be nice to have the time and money to see every movie with an interesting trailer. For the rest of us, reviews (either official or word-of-mouth from someone we know) is how to avoid wasting an evening and $50 on a stinker.
Re: (Score:2)
You really think you can measure the quality of a movie from a trailer? Like, you really believe that? Or do you just not care if you watch horrible movies?
Corporate meddling for years (Score:2)
A while back, RT tried different techniques to appease their corporate overlords.
The most egregious I recall was when they tried to introduce manufactured, positive reviews well ahead of the general release window. They did this to trick early bird types that would be looking for an RT score and maybe not come back to check for updates. Bad movies would have an 80%+ score until a few days before when the real reviews would blast the movie and drop it into rotten territory. That was kind of obvious to regula
Imdb feels like it's been going this way (Score:1)
for a while.
I don't think online reviews should be trusted anymore. Advertisers have found all the tricks to make ratings look one way or another.
It really sucks because I just want good legitimate information, but everywhere I find we're just bombarded with sponsored content. This trend of 'influencers' is a perfect example.
Re: (Score:2)
This trend of 'influencers' is a perfect example.
It's another triumph of marketing that "influencers" are even called this. How the hell did people swallow such a disgusting re-brand of the far more accurate term "paid shill"?
This is a cultural problem, not just a RT one (Score:5, Informative)
In the last fifteen years, critics have leaned more and more heavily into telling people what they ought to like, as opposed to how likely they are to like something.
Things like the Mario movie, that are enjoyable and escapist tend to get panned by the critics. Conversely, a movie like The Last Jedi that turned a formerly enjoyable, escapist series on its head (the 8th part of a 9 part series isn't the time to do that) has a critic rating of 95% versus an audience rating in the 40s, because the critics straight up don't care about the movie actually making sense in the context of the previous 7 entries (8 counting Rogue One).
Critics were, as far as I can remember, generally against the release of the Snyder cut of Suicide Squad, and also against the rework of Sonic's appearance in the Sonic movies (now widely considered to be a very wise decision by the studio).
It used to be that critics would catch the occasional good, intelligent movie that parts of the audience didn't really get, but recently what they like doesn't seem to have any correlation with what general audiences will like *or* whether a movie is intelligent (The Last Jedi was quite stupid but critics glazed it anyway).
So yeah, it's not so much that Rotten Tomatoes has brought in too many random critics, it's that today's literary "elite" enjoy fart-sniffing more than they enjoy actual entertainment.
Re: (Score:2)
In the last fifteen years, critics have leaned more and more heavily into telling people what they ought to like, as opposed to how likely they are to like something.
This has always been true, not just the last 15 years.
Re: (Score:2)
There have always been some critics like that, yes, but it's a lot more universal now. It wasn't nearly this bad in the 90s and 00s.
No, it's not. (Score:2)
Fandango is owned by the studios. Rotten Tomatoes is owned by Fandango. Do you really expect them to say "yeah, that last film we pushed out, wow, that one really sucked"?
"Professional" critics are kept in line by threatening to limit their access, without which they can't do their job. Huge negative differentials between critic ratings and audience ratings are passed off as review bombing.
Not to say this always happens. But if there's a half billion dollars at stake, you can bet the studios will call Fa
All internet review sites (Score:2)
All internet review sites have been enshittified.
I can't think of one - movies, restaurants, amazon, hotels, whatever - that has reliable reviews.
Usually, it's the site that is to blame, but often it's the vendor or the entitled reviewers.
And now: featuring AI-generated reviews!
No. (Score:2)
From the moment they changed their 'rating' requirements beyond 'i have seen the film' its been highly rated garbage.
Two other points: (Score:1)
I thought this was a statistically sound blog but there are two points worth considering (and not mentioned):
1) RT is an average of an average. That is, it's a binary variable, good or bad, of a review and this is then averaged for all reviewers. It's worth considering if Fandango changed either or both of these measurements. In some cases, for instance, it's not clear if a review is positive or negative. It could have been, if an analysis found that there was an equal amount of positive and negative sentim
Better Alternatives? (Score:2)
Burning question (Score:2)
I use it as pass/fail. (Score:2)
If I'm considering watching a movie, and I have no other context, I'll look it up on RT as the final gate. Under 50%, I'll pass. 50% or better, I'll give it a go. But that's as far as I'll go with it. I'm not considering whether a movie that would have historically got a 70 now gets an 85.
I don't trust the opinions of people in groups at all. Why would these groups be different?
I miss Siskel and Ebert (Score:2)
I was a kid when I watched them. I always felt I learned something about the film in particular but also films in general.
I like Rotten Tomatoes (Score:2)
I never read the critics' reviews.
I read the audience reviews.
I try to write informative reviews about why I liked/disliked a movie/TV show.
I do not patronize movie theaters. I watch everything at home. We did not spend $8K during Covid-time to not make use of a very nice setup.
Re: (Score:2)
I read both. If it has poor critic reviews but a good audience score, it's probably trash. If both are good, that's a good sign.
Critics don't know what you like (Score:2)
If critics knew anything about making good movies, they would have been hired by a studio to do exactly that.
No sommelier can tell you if you will enjoy a wine, just as no critic can can tell you if you will enjoy a given movie.
Having said that, I wish there was a site to compare trailers with the actual movie. Especially for so-called "comedy" movies where the only funny moments are in the trailer and the other 100 minutes of the movie are just filler.
Made a check.... (Score:2)
Can confirm, tomatoes still reliably rotten.
Gaming it (Score:2)
Considering that they actively and openly manipulate the public score when they don't like the result (Ms Marvel, She-Hulk, Cinderella, etc) I don't think anyone's considered it worthwhile for ages.
Rule of thumb for current times. (Score:2)
Assume a movie is bad unless you heard otherwise.