[fjf]

I though you all would enjoy this one. Click on Blind Tests: Matrix-Hifi

 

[nick_charles]

Quote:

Originally Posted by fjf /img/forum/go_quote.gif I though you all would enjoy this one. Click on Blind Tests: Matrix-Hifi

These chaps have done a whole load of ABX tests between all sorts of different components many which end up with

... NO SE ENCONTRARON DIFERENCIAS.

sadly most are in Spanish so I cannot (My O level Spanish was a long time ago) work out the details too well.

The one that is the most interesting is the test between the 12000 Euro Oracle CD player and the 200 Euro Pioneer DVD player where

NO SE ENCONTRARON DIFERENCIAS.

However flawed a test may be if item A that costs 60x item B cannot be told apart then........

 

[fjf]

It is in english. Try this one: Matrix-Hifi: Blind Tests. Open it in a new window. Weird html!.

Or go here: http://www.matrixhifi.com/ENG_index.htm choose the red pill and go to blind tests.

 

[nick_charles]

Quote:

Originally Posted by fjf /img/forum/go_quote.gif It is in english. Try this one: Matrix-Hifi: Blind Tests. Open it in a new window. Weird html!. Or go here: Matrix-HiFi choose the red pill and go to blind tests.

Sorry, I did not explain clearly. I have sen the one you point to.

If you go to Matrix-HiFi (Spanish) click on the red pill then select Pruebas Ciegas you get to a whole raft of ABX tests.

The English version has only one blind test in it - the Spanish version has several including CD players before and after clock modding - laptop/M-Audio vs expensive CD player and others.

 

[wavoman]

This is a perfectly done test. They did EXACTLY what I have been yelling about in all my posts -- they ditched the lame A/B/X question of:

"which is X more like, A or B",

and the asked the correct question:

"which do you like better, A or B, and it is OK to say 'no difference' ".

There was no X sample. Bravo!! They used "X" to mean "no preference" ... and that's perfect. Again so there is no misunderstanding -- although they wrote about ABX they did not do classic A/B/X testing (good for them!), they did the correct (IMO) test: do you prefer A to B, and no forced choice.

I have read deeply in the sensory discrimination literature, and have credentials in statsitics. Trust me, these guys did it right! The question they used is the best question to eliminate repsonse bias -- better than "do you hear any difference", better than "do these sound the same", and better than "do you prefer A or B, and you must pick one", and way better than A/B/X and a number of similar tests.

Again, Bravo! They did everything else right too -- no peer pressure, no rush, etc. Perfect.

I accept their conclusion 100%. Let's review it.

Most of us here don't put a lot of stock in high end power cords, high end RCA analog interconnects, or high-end anti-vibration platforms. This test shows that in an ordinary, even a really good ordinary system, such things do not make an audible difference for many users, even ones with good ears.

They also showed that Sony makes damn good CD players, which we already knew. And that for bookshelf speakers on stands, a good power amp is as good as a great one, to coin a phrase!

Of course 10 out of 38 people did pick the better system, so we can't say "the systems sound identical". We can say "the great majority of the people shouldn't spend the money on the high end system". I think we already knew that too. Sales of iPods prove it.

I am sure that for some people, and some CDs, and some systems, the Wadia will sound better than the Sony -- heck it did for 10 people. The key would now be to continue the test using "play the winner". Make the 10 people come back, and see if they can do it again -- pick the high end system. In my design, some of them would be fooled (both choices would actually be the low end system).

Of course one bloke might get it right over and over by dumb luck, and we want to be careful and not imbue him with special powers -- that's called the selection fallacy --but with good experimental design this can be prevented.

We need to do more studies just like this one, with more play-the-winner, etc. By March 2009 I expect to have a set-up where I can do this for redbook vs 24/96 WAV files, for high-end vs low-end DACS, and some other comparisons. I'll be inviting guys from the NJ Meet over to try.

Read the tests carefully -- it's all about "Do you prefer A or B, and OK to say 'no preference' ". That's the way to do listener blind testing!

My hat is off to these guys.

 

[fjf]

They have many others, although they are written in Spanish. Specially funny is the ABX between the Benchmark DAC1 (well known and with a very good reputation here) and the Behringer DAC2496 DACs. From 14 people (audiophiles), 10 heard no differences. The other 4 tried to differentiate them blindly but they did not get any better results than you could do by chance.

This is not to say there are no differences. Just that if they exist, most of the time they are not audible.

 

[fjf]

More on this: Are You On The Road To... Audio Hell? by Leonard Norwitz and Peter Qvortrup

More ABX data: ABX Double Blind Comparator Data

Quote:

Originally Posted by wavoman /img/forum/go_quote.gif This is a perfectly done test. They did EXACTLY what I have been yelling about in all my posts -- they ditched the lame A/B/X question of: "which is X more like, A or B", and the asked the correct question: "which do you like better, A or B, and it is OK to say 'no difference' ". There was no X sample. Bravo!! They used "X" to mean "no preference" ... and that's perfect. Again so there is no misunderstanding -- although they wrote about ABX they did not do classic A/B/X testing (good for them!), they did the correct (IMO) test: do you prefer A to B, and no forced choice. I have read deeply in the sensory discrimination literature, and have credentials in statsitics. Trust me, these guys did it right! The question they used is the best question to eliminate repsonse bias -- better than "do you hear any difference", better than "do these sound the same", and better than "do you prefer A or B, and you must pick one", and way better than A/B/X and a number of similar tests. Again, Bravo! They did everything else right too -- no peer pressure, no rush, etc. Perfect. I accept their conclusion 100%. Let's review it. Most of us here don't put a lot of stock in high end power cords, high end RCA analog interconnects, or high-end anti-vibration platforms. This test shows that in an ordinary, even a really good ordinary system, such things do not make an audible difference for many users, even ones with good ears. They also showed that Sony makes damn good CD players, which we already knew. And that for bookshelf speakers on stands, a good power amp is as good as a great one, to coin a phrase! Of course 10 out of 38 people did pick the better system, so we can't say "the systems sound identical". We can say "the great majority of the people shouldn't spend the money on the high end system". I think we already knew that too. Sales of iPods prove it. I am sure that for some people, and some CDs, and some systems, the Wadia will sound better than the Sony -- heck it did for 10 people. The key would now be to continue the test using "play the winner". Make the 10 people come back, and see if they can do it again -- pick the high end system. In my design, some of them would be fooled (both choices would actually be the low end system). Of course one bloke might get it right over and over by dumb luck, and we want to be careful and not imbue him with special powers -- that's called the selection fallacy --but with good experimental design this can be prevented. We need to do more studies just like this one, with more play-the-winner, etc. By March 2009 I expect to have a set-up where I can do this for redbook vs 24/96 WAV files, for high-end vs low-end DACS, and some other comparisons. I'll be inviting guys from the NJ Meet over to try. Read the tests carefully -- it's all about "Do you prefer A or B, and OK to say 'no preference' ". That's the way to do listener blind testing! My hat is off to these guys.

 

[JadeEast]

Any idea of the number of trials that were done?

 

[b0dhi]

This test is pretty good, but still flawed IMO. The reason for this is: -

Quote:

b.2) The sound samples will be of minimum 10 sec. duration

Although there has been an effort by the experimenters to reduce the effects of aural memory (by using LANCIS), I don't think it goes far enough. In my experience, being able to switch instantly from one component to another is not always useful if the audio being compared last occured on the other component at least 10 seconds ago.

A better method would be to have a preliminary trial which isn't scored, whereby the listener identifies a short (1 to 2 second) part (or parts) they believe best exposes differences in sound between the two components. Then this short part is looped in sync and compared, while being able to switch instantly between the two. This is both practical and would expose differences in sound much more effectively, IMO.

 

[XXII]

Quote:

Originally Posted by wavoman /img/forum/go_quote.gif Read the tests carefully -- it's all about "Do you prefer A or B, and OK to say 'no preference' ". That's the way to do listener blind testing!

Maybe I'm missing something...If they say "no preference" to one trial, what is the point of doing another trial? They took as much time as they want to come to the decision of "no preference" in the first trial; how is it possible that just by relabelling A and B they can come to a different decision?..and if they do, doesn't it mean that their decision making is flawed??

It doesn't make sense to me. Can you explain?

 

[nick_charles]

Quote:

Originally Posted by XXII /img/forum/go_quote.gif Maybe I'm missing something...If they say "no preference" to one trial, what is the point of doing another trial? They took as much time as they want to come to the decision of "no preference" in the first trial; how is it possible that just by relabelling A and B they can come to a different decision?..and if they do, doesn't it mean that their decision making is flawed?? It doesn't make sense to me. Can you explain?

As I read it they changed the identity of A and B for each test session but each person in a given test session would have the same A and B consistently. Once a listener stated a preference the test was over for them.

However if you told subjects to come back for say 9 more test sessions but let them believe that they were listening to wholly different A and B systems for each session ( a lie) and randomised the identity of A and B then you want to see if the preference was consistent.

Preference may not be consistent, I might prefer my Senns for some things and my ATs for others, this may change on a day by day or hour by hour basis or on my mood and so on.

 

[b0dhi]

whoops

 

[upstateguy]

Quote:

Originally Posted by wavoman /img/forum/go_quote.gif ...Of course 10 out of 38 people did pick the better system... ...Make the 10 people come back, and see if they can do it again --...

I'd like to see that too......

If 14 people picked the better system out of 2 choices, would it not still be random?

USG

 

[wavoman]

usg wrote:

if 14 people picked the better system out of 2 choices, it would it not still be random?

Yah, maybe. Repeat with those 14.

 

[Oublie]

I could tell the difference but then again i've had my ear wax cryo treated