Samsung Caught Cheating in TV Benchmarks (flatpanelshd.com) 124
Samsung has been caught cheating by designing its TVs to recognize and react to test patterns used by reviewers. The company promises to provide software updates to address the situation. From a report: Reviewers, calibrators and certification bodies typically use a 10% window for HDR testing, which simply means that it takes up 10% of the screen. In this window multiple steps from black to white as well as a set of colors are measured. Samsung has designed its TVs to recognize this and other commonly used window sizes, after which the TV adjusts its picture output to make measurements appear more accurate than the picture really is. When using a non-standard window such as 9% (everything else equal), the cheating algorithm can be bypassed so the TV reveals its true colors. This is deliberate cheating, an orchestrated effort to mislead reviewers. Vincent Teoh of HDTVTest first identified and documented the issue on Samsung's S95B QD-OLED TV. FlatpanelsHD has since identified and documented the issue on Samsung's QN95B 'Neo QLED' LCD TV where it gets even worse. QN95B not only changes its color and luminance tracking during measurements to appear very accurate, it also boosts peak brightness momentarily by up to 80%, from approx. 1300 nits to 2300 nits. This is possible because the power supply can send short bursts into the miniLED backlight -- these cannot be sustained without damaging the panel. In our QN95B review we found no evidence of the TV surpassing 1300 nits with real content.
Time and again, some idiot does this (Score:5, Insightful)
And short-term it often works. But in a networked global world, somebody smart soner or later looks at it and finds out and publishes. One of the instances where the Internet works and actually provides positive value.
Re:Time and again, some idiot does this (Score:5, Insightful)
It's 2022 and Samsung currently have the single largest market share of any vendor in phones. Why is this relevant? Well they've been caught cheating on nearly every benchmark published for well over a decade. Even now the Galaxy S22: https://finance.yahoo.com/news... [yahoo.com] but we even covered this back in 2013 where they (and others) were caught cheating on the Galaxy S4. https://mobile.slashdot.org/st... [slashdot.org]
I think the question is, does it make a difference... The typical buyer these days goes into a white goods store and stares at the pretty screens. They don't know what a nit is or what it means to have twice as high of a number. They oooh and aaah at the pretty colours and tell the service guy: I'll have that one.
Re: (Score:3)
Well, yes and no. The typical buyer that (mistakenly) thinks they have a clue buys the one with the higher numbers and psyches themselves up to perceive it looks better. Hence this is anti-competitive behavior and should come with a steep penalty.
Re:Time and again, some idiot does this (Score:5, Interesting)
Well, yes and no. The typical buyer that (mistakenly) thinks they have a clue buys the one with the higher numbers and psyches themselves up to perceive it looks better. Hence this is anti-competitive behavior and should come with a steep penalty.
Stop calling everything "Anticompetitive Behavior". It isn't "Anticompetitive"; it is Fraudulent Behavior.
Right sentiment; wrong crime.
Re: (Score:2)
Actually, right sentiment. "Anticompetitive" is a form of fraud.
Anticompetitive behavior isn't always illegal (e.g. predatory pricing is legal unless you're a monopoly or nearly so). Fraud is.
And lying about your specs isn't inherently anticompetitive, because it doesn't tend to reduce competition in the marketplace.
So I agree with the GP.
Re: (Score:2)
You think there is a competition as to who lies best?
Re: (Score:2)
Re: (Score:2)
Actually, right sentiment. "Anticompetitive" is a form of fraud.
Actually, no it isn't. Anticompetitive is not always a crime. Fraud is always a crime.
A quote for you: "When an honest man discovers he is mistaken, he will either cease being mistaken, or he will cease being honest. "
Re: (Score:2)
Actually, right sentiment. "Anticompetitive" is a form of fraud.
No, it isn't.
Re: (Score:2)
You're defrauding your customers by misrepresenting your product's capabilities; this unfairly reduces your rivals' sales, so it's also anti-competitive.
Anti-competitive and fraud are not the same, you can have either without the other. But sometimes, one act can be both.
Like a dog with a bone. . .
Re: (Score:2)
Thing is Samsung QD-OLED TVs already have the best numbers without cheating. They have better colour and better brightness compared to normal OLED screens. The difference is big enough to be immediately obvious in a side-by-side comparison.
Nobody else uses that panel yet, although Sony has a model due this year.
Re: (Score:2)
Well, that makes this even more stupid. Lying to the customer when there is no reason to makes me suspect it is a reflex.
Re: Time and again, some idiot does this (Score:2)
Perhaps they have the best numbers because they cheat and don't really have the best numbers.
Re: (Score:2)
Some reviewers have professional colour grading monitors that are the same reference displays that the film makers use. They compare them to the TVs they review and without doing any measurements the Samsung panels are noticeably closer to the reference panels.
Re: Time and again, some idiot does this (Score:2)
That doesn't mean it will hit the right numbers during testing. You're presupposing the idea that the test results are a very good representation of actual performance. If they're not, then they could be delivering a superior product that the tests can't capture.
Re: (Score:2)
Perhaps they have the best numbers because they cheat and don't really have the best numbers.
Perhaps you didn't read the parent's comment... no need to read it all, just the first 11 words would have sufficed.
Re: (Score:2)
Re: (Score:3)
Except even though they cheat on benchmarks they consistently get the highest marks from third party reviewers. Apparently it is their corporate culture because they have no reason to cheat in the TV market but they do repeatedly.
From third-party reviewers doing actual objective measurements, or from third-party reviewers merely using their eyes? Because sunk cost effects and confirmation bias are real.
Re: Time and again, some idiot does this (Score:2)
Neither of those fallacies operate when you're shopping. Only after your purchase.
Re: (Score:2)
Look at Lance Armstrong doping in cycling. It's not like the guy was un-talented or lazy - just the opposite. On top of intense conditioning and nutrition, they had the most professional doping program.
Re: (Score:2)
Re: (Score:2)
Yes that's my point. No one is going to care that Samsung was cheating on test based reviews because the overwhelming majority of customers don't read them.
That said a doubling in nits would definitely be detectable, and in some houses quite a welcome benefit. But what is a nit? Look at your living room, could you tell me how many nits you need to get a TV bright enough to overcome the glare of the light coming through the window? It's an important number that informed shoppers have no idea what to do with.
Re: (Score:2)
or not. A nit is important fir display brightness because a normal display is around 400=500 nits, maybe higher and it's "painfully bright".
But a technolog
Re: (Score:2)
or not. A nit is important fir display brightness because a normal display is around 400=500 nits, maybe higher and it's "painfully bright".
Or yes. Painfully bright varies with how your viewing condition. If you only watch TV at night you are correct a normal display is more than fine. On the other hand my TV is next to a 4m x 2.6m window which gets full sun half the day. At max brightness the display isn't even properly visible without closing the curtains.
The thing about your technically correct post is that you assume TVs follow standards. They may have the capability to do so but back in reality the first thing people do is bust out the rem
No Nit Wit eh?? (Score:2)
Re: (Score:2)
Samsung has been consistently cheating for a very long time, and that is well known.
Yes, they do make nice shiny devices, but no, you should never trust them.
Also, iPhones use Samsung screens (Score:2)
Re: (Score:2, Funny)
Re: (Score:2)
The thing is, we only hear about the ones that get caught. I'd be willing to bet every single manufacturer is doing something like this.
In the software world, have you ever faced a compliance or security audit? Standard procedure is to show the auditor the parts of the company's process that DOES comply with the regulations or best practices, and simply don't mention the "legacy" apps and processes that violate it. Those "legacy" apps are where all the risk is, but because it's too expensive to replace them
Re: (Score:2)
In the software world, have you ever faced a compliance or security audit? Standard procedure is to show the auditor the parts of the company's process that DOES comply with the regulations or best practices, and simply don't mention the "legacy" apps and processes that violate it. Those "legacy" apps are where all the risk is, but because it's too expensive to replace them all, businesses keep them around way, way too long. But they can claim to have "passed" the audit in order to look good and retain customers that require certification.
I never faced one of these, but I have done several of them by now. All in a regulated environment and with me being internal audit. As in lying to me comes with a very real probability if getting fired or worse. Don't know whether it is me or the situation, but I am pretty sure I got told about all the legacy crap. At least I cannot believe things could be even worse than some of the stuff I have been told and I have been active in the IT security space for more than 30 years by now. I think I have seen h
Re: (Score:2)
It's usually not so cut-and-dried as lying vs. telling the truth. It's all in the interpretation of the regulations and implementation details of the audit responses.
Auditor question: Do you perform regular penetration testing?
Answer: Yes.
But how thorough? How effective? Penetration tests come in many flavors.
Auditor question: Do you require multifactor authentication?
Answer: Yes, our Windows logins and internal apps all require multifactor authentication.
But...what about external third-party apps like CRM
Any other cheating (Score:5, Interesting)
Re: (Score:3)
Yes. Latest phones: https://finance.yahoo.com/news... [yahoo.com] But the reality is that this is rife in the industry and has been for a long time. Slashdot even covered a story about it in 2013: https://mobile.slashdot.org/st... [slashdot.org] spoiler: everyone cheats.
Re:Any other cheating (Score:4, Insightful)
Pretty mich "everyone cheats", yes. And it is bad for the market because the one with the better show makes the sale instead of the one with the better product. On the political side, nobody cares as they are all bought or hope to get bought.
Re:Any other cheating (Score:4, Insightful)
Yes. Latest phones: https://finance.yahoo.com/news... [yahoo.com] But the reality is that this is rife in the industry and has been for a long time. Slashdot even covered a story about it in 2013: https://mobile.slashdot.org/st... [slashdot.org] spoiler: everyone cheats.
Liar.
The article clearly states that Google doesn't cheat with the Nexus; and the Comments confirm that Apple doesn't cheat, either.
So, it is actually "Every Android phone manufacturer except Google cheats at benchmarks."
Re: (Score:2)
LIAR LIAR PANTS ON FIRE. If you're going to call me out on the details you better actually be right. Motorola doesn't cheat either.
You are a hilarious idiot.
Re: (Score:2)
LIAR LIAR PANTS ON FIRE. If you're going to call me out on the details you better actually be right. Motorola doesn't cheat either.
You are a hilarious idiot.
"Motorola" hasn't existed as other than a nostalgic brand name for nearly 20 years now.
That's why I didn't include them.
Re: (Score:2)
Re: (Score:2)
Motorola doesn't have to cheat. The only phones they make these days are devices made for police and fire departments that double as high-powered mobile and handheld radios and FirstNet devices. The screens need to be daylight readible, and the devices have to withstand either incoming fire, or fire fire at least as well as their operator. Nobody else competes for this market, so Motorola can just phone it in, after a fashion.
Well, they basically invented that market many moons ago. In a way, I am glad to see that the name
Motorola still means something in the First Responders' world!
Thanks for the correction. . .
Re: (Score:2)
So then you're calling me a liar while at the same time noting the fact that we're talking about a decade old article, but you're selectively applying whatever facts you want from it? You are a hilarious idiot.
Re: (Score:2)
So then you're calling me a liar while at the same time noting the fact that we're talking about a decade old article, but you're selectively applying whatever facts you want from it? You are a hilarious idiot.
I agree it is an old article, now that you mention it. But I didn't originally post the link to the old article.
Ok, we're done here. I agree that arguing over essentially ancient data is silly.
Re: (Score:2)
Re: (Score:2)
Wait, Apple doesn't cheat on benchmarks. *cough* "faster than a 3090" *cough*. Everyone cheats
Wrong, that was Specsmanship.
Apple created a chart that wasn't what you thought it was. You are just butthurt that Apple is smarter than you. That's not Lying; that's Mathematics.
Pay close attention to the X and Y Scales of the M1 Ultra vs. the RTX 3090. They are about Relative "Performance per Watt"; not "Absolute TFLOPS" or anything like that.
I agree it was a little too clever by half; but it wasn't outright lying. Just a reading comprehension test.
Guess what? You failed!
So that means iPhones are impacted (Score:2)
Re:Any other cheating (Score:5, Informative)
Given that Samsung has had multiple heads of the company serve prison terms over corruption issues in the last few decades [1][2] and has been implicated in cheating on everything from display panel metrics (see above) to phone benchmarks [3] to prison sentences [4] to bribing heads of state [5] (seriously, a former President of South Korea is currently serving a double-digit prison term over a conviction that, in part, is due to them being a beneficiary of a widely reported slush fund that Samsung’s leaders were using to bribe political figures, including using the slush fund to bribe said President to pardon their convicted leader so they could get out of jail early after being sent to jail for using said slush fund), it’s probably faster to list the areas where Samsung isn’t cheating than to list the areas where they’ve been caught cheating.
I remember even reading some reporting a few years back that tied the conglomerate to the sex trade, which I’d normally dismiss as spurious, but Samsung and its leadership is so corrupt from the core that the more you read about what’s already been confirmed and what they’ve already been convicted of in court, the more you’re willing to just accept that anything ludicrous you might read about them is likely to be true.
[1] https://en.wikipedia.org/wiki/... [wikipedia.org]
[2] https://en.wikipedia.org/wiki/... [wikipedia.org]
[3] https://finance.yahoo.com/news... [yahoo.com]
[4] https://en.wikipedia.org/wiki/... [wikipedia.org]
[5] https://en.wikipedia.org/wiki/... [wikipedia.org]
Re: (Score:2)
Does Samsung cheat anywhere else? They're a huge panel manufacturer.
Let's just be happy they don't make diesel emission systems [wikipedia.org].
Re: (Score:3)
Samsung has grown a reputation for extremely buggy launches of their high-end gaming monitors. It's pretty obvious they don't give their firmware dev & QA team enough time to iterate.
Re: (Score:2)
The UI on these TVs is total crap, and also quite buggy too.
It's sad really, the hardware is top notch and the video processing is competitive, if not top tier. Right now Samsung has the best TV display on the market thanks to that QD-OLED panel, with LG giving OLED at least another year before releasing a similar product.
If they just stopped cheating and switched their OS to Android TV they would have some really good products.
Re: (Score:2)
If they just stopped cheating and switched their OS to Android TV they would have some really good products.
They would just fuck up the UI like they do with phones.
Re: (Score:2)
I'm told that the latest Samsung phones are actually quite good, with minimal changes to stock Android. You still get the Samsung Store but almost all the Samsung apps are optional installs from it, not baked into the OS.
Re: (Score:2)
I love the Samsung hardware but I hate their take on Android. That's why I was glad that the Pixel 6 Pro was close enough to the same. If their apps are optional, they're definitely default. They have their own launcher, dialer, contacts, keyboard, smart assistant, messages and email apps and they're all installed and set as default and they're worse than the stock Android software. It would take quite a while to go through and rip out and replace those with their stock Android equivalents and set them
Re: Any other cheating (Score:2)
How about a TV without all the software? Please.
Re: (Score:2)
As much as I'd like that it wouldn't be cheap. Those features subsidise the cost of the TV.
You can usually disable it all by simply not agreeing to the EULA. If you decline it none of the smart functions can operate and it acts as just a basic display with tuner.
Re: (Score:2)
I guess I'll find this out at some point my Samsung TV from 2009 bites the bullet - I guess I just don't care enough for newer features :). Maybe by then the BT.2020 wide colour gamut support will be more complete and they'll finally be compliant with the HDR specs (the DCI-P3 colour gamut they're currently at is a bit smaller).
Yes, they sneaked in advertisements. (Score:5, Interesting)
Even on high end televisions of thousands of euros/dollars/dinglehoppers.
It was delayed so as not to appear in reviews, but soon enough to make extra money and to force those in the know to use pi-holes with their TVs.
Re: (Score:2)
Add it to the list (Score:3)
Re: (Score:2)
Add Samsung to the long list of has-been dishonorable companies.
Wait, there are honorable companies?
Re:Add it to the list (Score:5, Insightful)
Add Samsung to the long list of has-been dishonorable companies.
Wait, there are honorable companies?
Some small(er) family owned companies actually have some honor. The rest does not. And because the market rewards dishonesty, anybody that is honorable is on borrowed time.
Re: (Score:2)
Yeah you build up honour so that someone can buy you out and you can live your life rolling in metaphorical bloodmoney.
There's no such thing as a smaller family owned company in the TV panel game.
Re: (Score:2)
There's no such thing as a smaller family owned company in the TV panel game.
True. But the question was more general. Also, quite a few smaller family owned companies refuse to sell, no matter what. That is the primary reason they still exist.
Re: (Score:2)
... the market rewards dishonesty, anybody that is honorable is on borrowed time.
George, a business acquaintance of mine, made a great deal of money out of being honest. He sourced electronic and electrical components in the far east, to package and sell to the servicing industry, in the days when that existed. In those days, a lot of people were ripped of by fly-by-night suppliers in the far east. George somehow sorted out who was trustworthy, and also made sure that they trusted him. It was maybe not mega-bucks, but George retired to a nice villa on a hill in Cyprus, with the most ex
Wanted: Unethical SW Engineer. Must know Yocto... (Score:5, Funny)
To be fair (Score:2)
Its been shown the VW engineers just did what they were told by management. Perhaps they should have said "hold on a minute, is this legal?" but when you have bills to pay and perhaps a family to support sometimes people just turn a blind eye.
Re: (Score:3)
VW engineers just did what they were told by management
Yeah, corporate culture there indeed is to "just follow orders".
And before anybody quotes Godwyn to me, just look up the early history of Volkswagen :-)
Re: (Score:2)
True, but there's a world of difference between nazism and cheating at a few tests to get better reviews.
Re: (Score:3)
True, but there's a world of difference between nazism and cheating at a few tests to get better reviews.
In both cases, they were gassing people.
(Although, the nazis did'nt actually use Volkswagens for that purpose, they preferred other makes: Gas Vans [deathcamps.org])
Re: To be fair (Score:2)
You think deliberately gassing people in a basement is the same as a bit of extra pollution from a car? Get a reality check you nauseating moron.
Re: (Score:2)
This isn't about Godwyn, all of Germany has a corporate culture rife with "just follow orders". The problem is half the time those orders are given by an old school class of German, the kind who justifies everything by "it's always worked in the past".
Of all of our plants around the world the Germans are the hardest to deal with, and we have nothing to do with Volkswagen.
Re: To be fair (Score:2)
And if not following orders means you and your family are shot? It's very easy to be judgemental when you're sitting in a nice cosy house with nothing to worry about.
Typical (Score:2)
And they apologize ... for getting caught.
Re: (Score:2)
I suggest we raise the fine by 1% international gross revenue each time they get caught. Their stockholders will start to pay attention.
Okay but (Score:2)
Why can they cheat to have color accuracy, and then color accuracy goes out the window later?
Re: (Score:2)
I don't get it either. It's not like a VW car where respecting the pollution regulations have a real impact on engine performance and fuel consumption. Here it's just a digital device doing a digital thing. Why can't they just show accurate colors all the time?
Re: (Score:3)
They boost brightness in the benchmark to a level that damages the display if done longer. Apparently that gives them better color accuracy for the benchmark only?
Re: (Score:2)
You'd think the testers might notice that though. Unless its all done by machine vision.
Re: (Score:2)
The referenced article has some details. Apparently, the standard test done only used 10% of the screen and specific patterns.
Re: (Score:2)
Yeah, I read that. But you'd expect a human to notice the brightness suddenly going up to absurd screen damaging levels.
Re: (Score:2)
Reviewer sloppiness seems to be a factor, yes.
Re: (Score:3)
QD-OLED panels have only blue OLED pixels, and then a Quantum Dot (QD) layer in front. The quantum dots change the wavelength of the blue light to red or green, with very little loss.
Conventional OLED TVs use white OLED pixels and colour filters over them. The problem with colour filters is that they block some of the light, i.e. it's wasted. Many have a white pixel as well as RGB, with the white being used to provide extra brightness at the expense of colour accuracy. The losses due to the filters also mea
Re: (Score:3)
Because they push components past their limits to do it. In regular service they'd burn out in days but they hold together long enough to complete the tests.
Re: (Score:2)
Sounds like lazy benchmarking. Like they only tested one thing in a tiny, consistent area of the panel.
It's the age-old story (Score:2)
"We'll fix it in post".
Just as in the emissions scandal... (Score:3)
... this simply demonstrates poorly designed tests that can be cheated easily. There should be far more variability and this would be far easier to achieve with display testing than with car emissions. The testers only have themselves to blame.
Re: (Score:2)
Re: (Score:2)
People only watch 10% of their screen at a time, so clearly you should only test 10% of the screen at once. And they only watch white rectangles on that 10%, so that's the best thing to display in your test.
It's only fair.
THX-1138 would make for a good test (Score:2)
Part of the film takes place in an all white environment.
THX 1138 (8/10) Movie CLIP - Doorway to Chaos (1971) HD
https://www.youtube.com/watch?... [youtube.com]
Re: (Score:2)
Not the first time (Score:5, Interesting)
Re: (Score:2)
To be fair they aren't the only ones that cheat on TV efficiency tests.
A few years back I bought an LG LCD model. Out of the box the picture was very dim and flat. You have to turn off eco mode to make it look decent, and it warns you that it won't live up to its claimed efficiency rating if you do. That one got returned two years later when half the screen glitched out.
I've seen it with cable boxes too. They go into low power mode by default, but that takes literally ten minutes to wake up again. No exagge
Re: (Score:2)
LG and Samsung are both complete shit from stem to stern. They only make flaky shit. Well, that's not true, Samsung makes flammable flaky shit.
Literally every LG or Samsung product I have ever owned has been a massive disappointment in some way or another, usually multiple ways.
I frankly do not understand why people think these companies' products are worth one tenth of one shit.
Re: (Score:2)
The TV was okay for the low price I paid, except that it broke. I have a couple of LG 4k monitors and they have been decent. Picture is good, menu system is about as crap as the average monitor. There are a couple of issues, firstly sometimes when it turns on there is no picture. It goes into "slow mode" where it takes seconds to respond to button presses. A power cycle fixes it. Secondly it doesn't seem to like some 1080p modes as it cuts the bottom of the image off. It's usually an issue when booting Linu
Re: (Score:2)
I have a couple of LG 4k monitors and they have been decent. Picture is good, menu system is about as crap as the average monitor. There are a couple of issues, firstly sometimes when it turns on there is no picture. It goes into "slow mode" where it takes seconds to respond to button presses. A power cycle fixes it. Secondly it doesn't seem to like some 1080p modes as it cuts the bottom of the image off. It's usually an issue when booting Linux.
So it's decent, except it doesn't display an image properly, and sometimes it doesn't respond to button presses? It sounds like a complete pile of shit to me.
Re: (Score:2)
Admittedly a Dell would probably be better in that respect, but would also cost twice as much. I can live with the issues given the price I paid.
If I'd paid 1500 Euro for a high end OLED TV though I'd be sending it back.
Re: (Score:2)
Re: (Score:3)
As long as they disclose it, that isn't cheating.
It is willfully deceptive to advertise a bunch of specifications together that you can't achieve at the same time. They should either have to advertise the highest power consumption seen during benchmarking, or advertise the stats seen while achieving the power consumption advertised in the big text, not the small print. Anything else is based on fraudulent intent.
Software update already available (Score:2)
Only top leadership can prevent it (Score:2)
This kind of cheating is endemic to any large organization in a competitive environment.
*Unless* top management makes it crystal-clear that chating is not acceptable. I'm happy to see both Google and Apple appear to have done this.
It's just human nature - we all want to win, and many of us are rewarded career-wise for winning.
So some people cheat if they think they can get away with it. Others with better morals will object internally and the conflict will bubble up thru the organization all the way to the
rtings.com indicates that this is irrelevant (Score:2)
Assuming it's true. I'm not connected to Samsung nor do I get or have gotten freebies, but I guess I'm a fan as I have two of their high end tablets and a high end phone. Though I just bought a LG TV. :)
Right, disclaimer aside, here's the link. https://www.rtings.com/tv/revi... [rtings.com]
"There's some concern that some Samsung TVs detect when they're being calibrated or tested for a review, and adjust the output to be more accurate. We measured the PQ EOTF with different window sizes and found that there's very little
need better testing (Score:2)
I evaluate TVs myself with my eyes. No manufacturer has tricked my testing yet.