Home » Audio » General » Blind testing and what I would like to see done...
Blind testing and what I would like to see done... [message #2124] Mon, 22 August 2005 09:04 Go to next message
Mr Vinyl is currently offline  Mr Vinyl
Messages: 407
Registered: May 2009
Illuminati (1st Degree)
Hi,

First a little background on this subject. I am not an expert on blind testing so take everything below for what it's worth. But as I see it, people who believe in the benefits of blind testing audio equipment basically say that if you can't hear the difference in a double blind test then you're only fooling yourself. There is no difference. The people against using blind testing have a host of reasons why they aren't accurate. Such as length of time, unfamiliar equipment and rooms etc. etc. They believe there are audible differences in say cables etc that are masked by the blind test procedure.

Thinking on this subject I realized that I have never read about a blind test procedure where proven differences were heard with audio components. In other words, let's take a component in the audio chain where almost everyone will agree there are sonic differences such as speakers or cartridges. I don't know of anyone who would argue that there are no audible differences between different brands of speakers. So lets use them for starters. One would assume that just about anyone could tell the difference very close to 100% of the time, in a blind test, between a full range floor standing speaker and a mini monitor. But before we assume, I would like to see it done. If that assumption turns out to be correct then let's try two different speakers with similar bass responses. I think this would be considerably more difficult. But again you would "assume" that most people would be able to tell the difference almost 100% of the time between these speakers as well. Then drop down to testing preamps, amplifiers etc.

My point is this, Having the bench marks created using the tests above, we could then move on to more difficult to hear components such as cables etc. In other words if people had trouble telling apart even speakers, cartridges and preamps using blind testing, this would help prove the point of view that the tests are not an accurate way of showing audible differences. If the above tests show that people can consistently hear differences between speakers, cartridges and even amps and preamp, then this would be a mark for the double blind testers. But instead of assuming people can hear gross differences from components such as speakers I would like to see it done first.

Maybe I'm wrong. Can someone show me a double blind test using audio components of any kind that shows a consistent verifiable difference was heard by a number of people.

Just food for thought.

Re: Blind testing and what I would like to see done... [message #2125 is a reply to message #2124] Mon, 22 August 2005 13:24 Go to previous messageGo to next message
Wayne Parham is currently online  Wayne Parham
Messages: 18671
Registered: January 2001
Illuminati (33rd Degree)

I think blind testing makes a lot of sense. The thing it (obviously) doesn't consider is aesthetics. It doesn't take into consideration status symbols and things like that either, but I guess that's the point. For sound only, and people's perception of sound, I think it makes a lot of sense. But if something sounds good and looks horrible, maybe it shouldn't be surprising that some people don't like it.

I think the usual findings are kind of no-brainers. When blind tests don't find differences in cables, that should be no surprise. Take a 50 foot run of 36 guage wire and use it for speaker connections, that's something you can probably blind test a positive result compared with say a 12 guage wire. But comparing two brands of same size copper wire is going to give a null result. The same could be said about other components that have similar performance levels. Some things are just more noticeable than others. That doesn't mean they sound or measure the same in all characteristics, but there comes a point where most people can't tell, just like a size of print they can't read.

I think speakers are the easiest components to identify in a blind test. There is such a wide range of performance levels and optimizations. Speakers are electro-mechanico-acoustic devices, so even though they have relatively simple construction, they have an impossible job. That's why each designer chooses trade-offs he optimizes for, and it's those trade-offs that expose the speaker and make it possible to identify. Mini-monitors have less bass, so you can tell them right away when compared with a speaker with a larger woofer. Horns are more directional, so they have audible cues of their own, lack of reflections and what not. As long as the differences are pretty large, you can tell them right off. But once the differences become subtle, then obviously a blind test will yield a null result.


Re: Blind testing and what I would like to see done... [message #2126 is a reply to message #2124] Tue, 23 August 2005 23:30 Go to previous messageGo to next message
Mike.e is currently offline  Mike.e
Messages: 471
Registered: May 2009
Illuminati (1st Degree)
Of particular interest to me is the time domain rather than the frequency domain. It also seems people have sensitivities in one or the other.

Other interesting things such as Low Q resonances being more audible that high Q ones.

One thing is for sure - The room and loudspeakers contribute the most distortion. As for the rest well,you can do your own blind testing but certain smart alecs may dismiss the results. Does it matter if they do?

Just like politics and religion,such is audio _ it gets people passionate.


Re: Blind testing and what I would like to see done... [message #2127 is a reply to message #2126] Wed, 24 August 2005 00:47 Go to previous messageGo to next message
Wayne Parham is currently online  Wayne Parham
Messages: 18671
Registered: January 2001
Illuminati (33rd Degree)

One of these times, build yourself a sawtooth wave generator capable of making a wave with the peak on the left or the right. The difference between the two is reverse sawtooth waves have even harmonics phase shifted. Listen to the two and see if you can hear a difference.

This is a good test because it uses a pair of signals with exactly the same frequency harmonic content but having different phase. It prevents the listener from obtaining frequency cues that expose a difference in phase. Cancellations of certain frequencies, beat frequencies and things that make response aberrations are pretty easy to detect, but absolute phase isn't. So you're going to have a very hard time finding people that can detect a sawtooth from a reverse sawtooth in a blind test. Try it and see.


Re: Blind testing and what I would like to see done... [message #2128 is a reply to message #2124] Wed, 24 August 2005 11:19 Go to previous messageGo to next message
Manualblock is currently offline  Manualblock
Messages: 4973
Registered: May 2009
Illuminati (13th Degree)
Mr. V. I hope some take this particular aspect of our hobby seriously because it is the only important test there is. It doesn't matter how a unit is measured or how well it eventually can be made to measure if no one wants to listen to it.
Since assigning a value to a subjective perception is a science that is still very much in it's infancy then all we have to go by is personal preference.
Therefore it is my contention that blind testing is useless.
There are so many variables intrinsic to individual perception alone; forget about those variables we can control, that to create some sort of clearly controlled set of variables is impossible at this time in our history.
Which brings me back to a simple observation. If you gather a group of people and assign them a task of numerically rating the quality of sound exposed to the group; how do you seperate out the nine thousand issues of hearing loss/mood/the simple emotional effect the chosen piece of music has on the individual,,on and on.
Not to beat a very dead horse but this I know to be true. What sounds good is recognised by everyone. If you take a De'Angelis Guitar played by Julian Bream and sit people in front of it. Even if they don't like classical music, hate guitar, hate Julian Bream... I don't think you will get anyone in the group to say the sound itself is bad.
Thats a reference standard and we all can recognise one when we hear it. Anyone ever say that Pavorrotti can't sing?
My point? What would be the criteria that indicates a unit is better sounding than another? It sounds more musical? Even if all agreed one unit sounded better than another in a blind test; what happens when you switch that unit into another system and situation? Where maybe it doesn't synergise?
So I think what sounds good is recognised by all; what sounds best is a meaningless approbation. Comparing five amps is beyond pointless it seems to me.
Just my opinion after doing this many times and hearing many points of view,( some that caused my head to spin.)

You have it backwards [message #2129 is a reply to message #2124] Wed, 24 August 2005 17:25 Go to previous messageGo to next message
Dean Kukral is currently offline  Dean Kukral
Messages: 177
Registered: May 2009
Master
You suggest that if someone cannot tell the difference in a DBT between copper cables and $100/foot silver cables, then DBT is useless.

I say that if someone cannot tell the difference in a DBT between copper cables and $100/foot silver cables, then who in the world could possibly be stupid enough to pay $100/foot for silver cables, when there is no audible difference??


You might want to re-read my post.... [message #2130 is a reply to message #2129] Thu, 25 August 2005 07:25 Go to previous messageGo to next message
Mr Vinyl is currently offline  Mr Vinyl
Messages: 407
Registered: May 2009
Illuminati (1st Degree)
I didn't suggest any such thing. I am saying that if people can't hear the difference between speakers or cartridges in DBT's then maybe there is something else going on. I didn't say whether I was for or against DBT's. I only suggested that we try double blind testing with components that everyone would agree has sonic differences as a reference point.

Re: Blind testing and what I would like to see done... [message #2131 is a reply to message #2128] Thu, 25 August 2005 07:33 Go to previous messageGo to next message
Mr Vinyl is currently offline  Mr Vinyl
Messages: 407
Registered: May 2009
Illuminati (1st Degree)
Hi Manualblock,

As I said earlier I am no expert in DBT's. However I don't think these tests are trying to show which component sounds better but instead if a difference can be heard at all. In other words, listen to Amp "A" then switch to Amp "B" did you hear a difference, yes or no. First you would have to determine if a difference between say amps or cables can be heard at all, and then you could go from there.

Re: Blind testing and what I would like to see done... [message #2132 is a reply to message #2131] Thu, 25 August 2005 08:36 Go to previous messageGo to next message
Manualblock is currently offline  Manualblock
Messages: 4973
Registered: May 2009
Illuminati (13th Degree)
I know Mr. V. because I am no expert either. I think what I was trying to say was these DBT have been examined to death in many venues starting back in the 60's with Floyd Toole at JBL.
I have experienced this. When sitting in a room with some friends; relaxed and enjoying the company and the surroundings there are very easily heard perceptual differences agreed to by all between different components. Soon as you line them up in a straight row with organised criteria and defined critical aspects to be expressed the whole thing goes south. You get results that make your hair stand on end. Solid State amps suddenly sound best; CD's beat vinyl etc.
Now the reductionists will state that that proves their point; that in a familiar group, group dynamics superceded the facts. All kinds of emotional stuff interferes with good results in a DBT. Well I argue that the opposite is true and that is my position.
So; Then you go back home and sit down and voila' it's all back where it should be soundwise.
My point? The tests are useless.

You don't understand why DBT's [message #2133 is a reply to message #2132] Thu, 25 August 2005 09:25 Go to previous messageGo to previous message
Bob Brines is currently offline  Bob Brines
Messages: 186
Registered: May 2009
Location: Hot Springs Village, AR
Master
Beautiful! JFB!!

If there ever was a post that demonstrated why I will never allow my equipment get involved in a shoot-out, this is it.

The basic premise of the post is that since DBT's don't produce the expected results, the the TESTs are in error. Everyone knows that tubes sound better than SS, vinyl better than CD's, silver wire better than copper, etc.

DBT's are designed to identify differences between systems, not to assign rankings. If the tests you were involved in demonstrated "A" was better than "B", then the tests were flawed. Rankings are subjective and cannot be tested. Sure, some equipment will be universally agreed to be poor, or good, whatever, but this is NOT the result of a DBT test.

You like tubes and vinyl. Maybe I don't. Therefore, either my ears are flawed, I am a fool or both. Or perhaps my program material is different from yours and might need different reproduction equipment? Maybe I just don't like the fat sound of small SET's?

It has been suggested that at the next GPAF we arrange a shoot-out between Wayne's speakers and mine and whoever else wants to participate. Not going to happen. Do our speakers sound different? Sure. Do we need A-B testing to prove so. I don't think so. Our design philosophies are very different and our target audience is very different. If you were at the last GPAF, you would have seen that some folks gravitated to my suite and some gravitated to Wayne's suite. Different folks. Different ears. And this is the way it will always be with audio equipment.

Bob


Previous Topic: Tascam 133
Next Topic: Hey Charlie, man are you out there?
Goto Forum:
  


Current Time: Fri Mar 29 08:29:29 CDT 2024

Sponsoring Organizations

DIY Audio Projects
DIY Audio Projects
OddWatt Audio
OddWatt Audio
Pi Speakers
Pi Speakers
Prosound Shootout
Prosound Shootout
Smith & Larson Audio
Smith & Larson Audio
Tubes For Amps
TubesForAmps.com

Lone Star Audiofest