Punch For nVidia.....

[Embattle]

....smack for ATI.

Futuremark today released a patch for 3DMark 03 and have thus readjusted the base on nVidia cards using the most recent FX driver.

3DMark®03 incorporates many ground breaking new features and technologies and sets the standard for easily and objectively measuring and comparing modern PCs’ 3D graphics performance. Recently, there have been questions and some confusion regarding 3DMark®03 results obtained with certain NVIDIA drivers.

We have now established that NVIDIA’s Detonator FX drivers contain certain detection mechanisms that cause an artificially high score when using 3DMark®03. We have just published a patch 330 for 3DMark03 that defeats the detection mechanisms in the drivers and provides correct results.

Genuine or the continuation of a bitch fight?

 

[smurkin]

Naughty NVidia. I know this is different....but have been plenty allegations in the past that they have been streamlining their drivers to work well with benchies. I guess the bottom line is...if a card has 5% faster performance in a benchie....then yoou'll buy it...its common sense.

But NVidia are definitely full of it....another good example was recent benchmarking of the Nv35 on D3...NVidia turned up with a demo they had prepared...but Id insisted on running their own benchie for fairness (the Nv35 still out performed the best ati tho). There was also that issue with NVidia pooh-poohing MadOnion...saying its 3dmark03 sucked (well, urmm everyone said it sucked) ... so they then sneak in and tweak their drivers for the benchmark !doh!

I dont trust ATI or Nvidia....card prices seems way too high for me...I'm not sure if the more sophistcated technology costs more or they are fixing the market...I mean some of the best cards/bundles are weighing in at £400...ffs..these are GAMING cards....they wont be l33t in six months time ! [/rant off]

*edit....are there any reviews of ati/nv cards with the patch yet ??*

 

[Testin da Cable]

like....wooo, who whould have thought?

 

[Jonty]

Good evening

For once I don't really support nVidia in their battle to save face, but, for what it's worth, here's part their response:

nVidia
A representative at Nvidia questioned the validity of Futuremark's conclusions. "Since Nvidia is not part of the Futuremark beta program (a program which costs of hundreds of thousands of dollars to participate in), we do not get a chance to work with Futuremark on writing the shaders like we would with a real applications developer," the representative said. "We don't know what they did, but it looks like they have intentionally tried to create a scenario that makes our products look bad."

Be back soon.

Good luck and God's speed

 

[Deadmanwalking]

We really shouldnt be putting so much space to a benchmark. So ill stop

 

[smurkin]

hmmm...I was wondering how mad onion made money

 

[Embattle]

They generate money through the sale of certain Beanchmarks (Bapco), Beta programs, Web analyser and even through the selling of information regarding hardware distribution.

 

[Xavier]

Originally posted by Embattle
They generate money through the sale of certain Beanchmarks (Bapco), Beta programs, Web analyser and even through the selling of information regarding hardware distribution.

well, their main product is now the performance knowledgebase they've accumulated thanks to everyone submitting their benchmark scores... that data is like gold to many system builders...

of course the hundreds of thousands of dollars they demand for membership to the likes of NVIDIA and ATI probably makes them a pretty penny too

BAPCO are a separate company who make their money simply by selling their applications and membership to various companies - and have no products in the consumer space.

 

[smurkin]

!DoH! ATI have been at it too !

 

[Jonty]

Oh dear I wonder where, if at all, one can draw the line between 'optimisation' and 'cheating' I'm really not surprised people are getting disenchanted with benchmarks. To be honest, things are no taken so out of context it's laughable. The other week, for example, I was reading about a 'massive descrease' in performance following the nForce drivers. This massive decrease, according to The Inquirer, occured in Quake III Arena and say the frame rate drop from around ~315 to ~298 fps, or there abouts. Now yes, that is an unacceptable drop, but I doubt it's humanly possibly to tell, without benchmarking, that you've dropped like that. Sure, if it had gone from 45 to 25 fps, may be, but taken out of context like that, it just made me chuckle

Kind Regards

Jonty

P.S. I was reading the other day that, prior to all this debacle, it was remains an nVidia card powering the highest ever 3DMark score system in both 2001 and 2003 tests

 

[bodhi]

Originally posted by smurkin
!DoH! ATI have been at it too !

ATi's is an optimisation which will have an effect in proper games, but have said that they will remove it anyway. nVIDIA's method was a flat out cheat which brings absolutely no benefit whatsoever to any other game, yet don't have the common decency to admit it. Quite sad really.

 

[Jonty]

Actually, just so we're clear, these optimisations were aimed just at 3DMark and not at games in general.

The Register
ATI has admitted that it tweaked its Catalyst drivers to generate better 3DMark 03 scores

ATi's response said that these kind of optimisations are often used in games, but it didn't say that the code in question was designed to improve anything but 3DMark 2003. Hats off to ATi for removing the code, but if we're being cynical (and I'm sure we don't want to be) you could say ATi are only removing the code because they have been found out Now if nVidia would do the same that would be the end of all this

Kind Regards

 

[Embattle]

Sadly no one escapes this fiasco that well with nVidia coming out of it worse than ATi but by far the biggest loser will be Futuremark.

Many doubts had already been cast about 3DMark 2003 as a useful benchmarking tool, partly by a number of hardware reviewers and then by nVidia...although some would say this was down to the less than stunning performance of the FX 5800 in 3DM2003. It would seem that Futuremark then did a classic of saying that nVidia had cheated there way to a better result for FX 5900, ATI then tells the truth that it tweaked its drivers to produce better scores. This all leaves Futuremark with a problem, while it was not directly to blame, the validity of their most popular consumer product is now languishing in the toilet....unsurprisingly it joins CS.

 

[Xavier]

I have read some threads in our forum and here too, where people speculate that we are out to do some damage to Nvidia. Let me ensure you that is not the case. If it was up to me, I would only patch 3DMark03 when some critical bug is found, or the hardware detection needs to be updated. Otherwise I would like to concentrate on making the next 3DMark, once again a better benchmark than ever before. I'm doing my best to keep the next 3DMark on track despite this PR mess, but it's not easy. We are a small company (26 employees tot.) in Finland and a 3 person sales office in the US. We have no desire to confront a huge >1500 employee market leader of high end PC graphics. Some of our documents may be seen as harmful to Nvidia, and I find that regrettable. We only want to defend the validity of our benchmark products, since our business depends on it.

I wish we could somehow leave behind us our differences in opinions, again work together on a new 3DMark version, and again do our best to make it reflect future 3D game performance. Even though Nvidia left our beta program, we will make sure the next 3DMark is an impartial benchmark that genuinely measures 3D hardware performance. We will test it also on any Nvidia hardware available, but if Nvidia are out of the beta program, they will not get any pre-releases. You should therefore not judge them even if the next 3DMark doesn't run that well on their hw right after we launch. Give them some time to get possible driver issues fixed, and start comparing scores only when the rendering is correct.
_________________
Patric Ojala - 3DMark Producer

Taken from http://www.beyond3d.com/forum/viewtopic.php?t=6082

now, I don't know about you, but that says to me that FutureMark themselves now omit that their current DX9 'gamers benchmark' may carry some bias.

 

[Jonty]

What a mess, eh? hehe. Here are some interesting words from John Carmack on the whole issue of ATi/nVidia performance, this time in Doom (this links up with what you were saying the other day Xavier):

John Carmack
The significant issue that clouds current ATI / Nvidia comparisons is fragment shader precision. Nvidia can work at 12 bit integer, 16 bit float, and 32 bit float. ATI works only at 24 bit float. There isn't actually a mode where they can be exactly compared. DX9 and ARB_fragment_program assume 32 bit float operation, and ATI just converts everything to 24 bit. For just about any given set of operations, the Nvidia card operating at 16 bit float will be faster than the ATI, while the Nvidia operating at 32 bit float will be slower. When DOOM runs the NV30 specific fragment shader, it is faster than the ATI, while if they both run the ARB2 shader, the ATI is faster.

Kind Regards

 

[Embattle]

A rescue too late?

http://hardocp.com/article.html?art=NDgx

 

[Jonty]

Futuremark
Futuremark now has a deeper understanding of the situation and NVIDIA's optimization strategy. In the light of this, Futuremark now states that NVIDIA's driver design is an application specific optimization and not a cheat.

Well, quite how it came about, I'm not sure, but I'm happy nonetheless Let us all hope this doesn't happen again

Futuremark and nVidia
Both NVIDIA and Futuremark want to define clear rules with the industry about how benchmarks should be developed and how they should be used. We believe that common rules will prevent these types of unfortunate situations moving forward.

Kind Regards

 

[Embattle]

I reckon its just a cover up after some sort of deal

 

[Jonty]

Originally posted by Embattle
I reckon its just a cover up after some sort of deal

hee hee, I'm inclined to agree, although I doubt it's so sinister Futuremark know that the credibility of their benchmarking suite, despite the number of downloads, has been badly damaged by nVidia, and that they need them 'onboard' if matters are to be rectified. nVidia know that, like it or not, 3DMark is popular and that their actions over this affair has won them no fans. But whatever the case, all's well that ends well

Kind Regards

 

[john_kyle]

Originally posted by Jonty
Well, quite how it came about, I'm not sure, but I'm happy nonetheless Let us all hope this doesn't happen again
Kind Regards

Threats of legal action from big fish tend to make the little fish change their minds.

 

[Jonty]

Originally posted by john_kyle
Threats of legal action from big fish tend to make the little fish change their minds.

Quite possibly Then again, neither nVidia nor Futuremark could really be said to be 'little', more 'big' and 'bigger' Anyway, it's over with, let's just be thankful for that.

Kind Regards