Nvidia accused of fudging tests

[Tweaker]

First off, it's a pretty well-known fact among any non-"casual" gamers that drivers are often "optimized" to look good on benchmarks. Sometimes, it's subtle enough to go unnoticed, and in other cases it's so blatant that it's impossible not to see it. For example, do a google search on "ati quack" to find info on the incident where ATI was caught adding checks for quake3 and, upon detecting, cranking down the mipmap and detail levels for a performance boost.

http://firingsquad.gamers.com/hardwa...uack/page3.asp has all the smoking guns on that particular case. Futuremark has its own issues. Any nvidia card owner can recall the insane bias futuremark showed in favor of ATI card because of some programming issues. Yet another case again the benchmark credibility could be seen in one of the early previews of GFFX. Maximum PC got a hold of one and tested it against ATI's 9700pro using 3dmark 2001:se and q3 plus ut2003. During the game tests, GFFX had a 40% and 20% FPS leads (although at those framerates, it hardly makes any visual difference) but it was beaten by 10% in the game4 3dmark test. read more at
http://www.maximumpc.com/features/fe...003-01-03.html

So folks, take those benchmark results with a grain of salt, since both nvidia and ATI fudge on them if they think they can get away with it.

Source: C/NET News

[MishY]

Interesting read thankyou Tweaker!

[yuppicide]

I don't believe that they fudge their tests. The thing I am kinda worried about is that EA and Nvidia teamed up and future EA games will have "nividia only content" -- fine with me because I have Nvidia, but my friend has ATI now so he's worried.

Actually, though here's some more info on the accusations:

More NVIDIA vs. Futuremark [May 23, 2003, 5:05 PM EDT] - 50 Comments
Nvidia accused of fudging tests (thanks Mike Martinez) is the report of the latest salvos fired in what's become a bit of a feud between the graphics chipset creators and the benchmark designers (story). In related news, the Futuremark Website (thanks Tony!!!) has a 3DMark03 patch for download with the following note: "We have now established that NVIDIA’s Detonator FX drivers contain certain detection mechanisms that cause an artificially high score when using 3DMark®03. We have just published a patch 330 for 3DMark03 that defeats the detection mechanisms in the drivers and provides correct results." Here's a bit of point/counter-point from the CNET article:
"Recently, there have been questions and some confusion regarding 3DMark 03 results obtained with certain Nvidia" products, Futuremark said in the statement. "We have now established that Nvidia's Detonator FX drivers contain certain detection mechanisms that cause an artificially high score when using 3DMark 03."

A representative at Nvidia questioned the validity of Futuremark's conclusions. "Since Nvidia is not part of the Futuremark beta program (a program which costs of hundreds of thousands of dollars to participate in), we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer," the representative said. "We don't know what they did, but it looks like they have intentionally tried to create a scenario that makes our products look bad."