Home » Audio » General » Blind testing and what I would like to see done...
Here it is - YOU wrote it! [message #2134 is a reply to message #2130] Thu, 25 August 2005 11:04 Go to previous messageGo to next message
Dean Kukral is currently offline  Dean Kukral
Messages: 177
Registered: May 2009
Master
"In other words if people had trouble telling apart even speakers, cartridges and preamps using blind testing, this would help prove the point of view that the tests are not an accurate way of showing audible differences."

It seems to me if you were to define "audible difference," it would be what could be heard in a well-run DBT in an acoustically dead room. Of course, this oversimplifies the problem of directionality of speakers. You might have to take sampling points in different parts of the room.

I think that "if people had trouble telling apart even speakers, cartridges and preamps using blind testing, this would help prove the point of view that" there were no audible differences between the things being measured.

audible differences or not? [message #2135 is a reply to message #2128] Thu, 25 August 2005 11:19 Go to previous messageGo to next message
Dean Kukral is currently offline  Dean Kukral
Messages: 177
Registered: May 2009
Master
Mr. V is talking about audible differences,not quality, as he ably points out below.

Let us start with cables. Put on Romex, 16ga speaker wire, and $100/foot silver cable.

First, do you hear an audible difference??? In not, then the issue of quality is irrelevant!! If you DO hear a difference, then the rest is subjective as you point out.

But, if you do NOT hear a difference, you can save a lot of money!

(Of course, older people like myself may not hear a difference, while younger people may. This kind of factor along with statisitcal variation and room environment have to be taken into account in a well-run DBT.)

To say that DBT's are "useless," is, wellll, is just silly.

Re: You don't understand why DBT's [message #2136 is a reply to message #2133] Thu, 25 August 2005 14:09 Go to previous messageGo to next message
Manualblock is currently offline  Manualblock
Messages: 4973
Registered: May 2009
Illuminati (13th Degree)
Mr. Brines allow me to state catagorically that in as far as audio goes you do not appear in any way to be a "fool". I ask your indulgence because it is apparent that a conversation like this will never happen outside of ART and as such is a priviledge for me at least to participate.
This cannot be an argument because I personally have no standing upon which to argue. I hold no advanced degree in Psycho-Acoustics or Physics and therefor rely solely on personal observation and extensive listening experience and exposure. My only reason for being hear is a love of music and home reproduction.
If I read you correctly we seem to be at agreement at times and disagreement at other times in the discussion.
It would seem to me that to assign differences in sound implies ranking's; in order to have a difference you must have a quality descision that implicates the difference.
If I gave the impresssion that I personally required expected results I have mis-lead readers of the post. As per standard test procedure I would not know if a tube or SS device was used. Since there would have to be a certified difference between components to make the test of any use and since obviously one can hear the difference between tube and SS gear that alone seems to me to negate the purpose of the test; hence it follows my statement that the test is useless for any meaningfull conclusions.
Program material; here we agree. I posted a while back concerning a session I was at where the SD speaker at test reproduced some material consisting of modern pop jazz/vocals and light chamber music to a startlingly musical level. But had a hard time with very good and well recognised recordings of older jazz and more complex classical stuff.
That is what piqued my interest. The vast gap between ability regarding musical choice.
My likes; I listen to tubes and vinyl but have SS and CD on hand. I listen to tubes and vinyl but that doesn't mean I like it; they are a PITA and expensive. I would much rather spend money on something else.
Shootouts; well I can see some value because after all; there you are truth be told, I mean at some point you have to race the horse.
So thanks first of all for taking this little opinion of mine seriously because I assure you I do yours.
Then in simple language I don't believe the DBT tell's you anything at all; I think was what I was trying to say originally and it almost sounds like we agree on that point.
I have referenced my personal beliefe in a reference standard for all things of quality. I hold that beliefe sincerely and without question. I think everyone knows what sounds good intrinsically, that while you may prefer one musical sounding unit to another the same as you would enjoy piano sonata's more so than violin concerto's you will always recognise a good unit from a weak one. I think that is a function of human nature and it is why we have it all over measurement systems and blind testing when it comes to deciding issues of sound/art/architecture etc. to be able to make those distinctions that unite opinions into critical determinants of universal value.

Re: audible differences or not? [message #2137 is a reply to message #2135] Thu, 25 August 2005 14:27 Go to previous messageGo to next message
Manualblock is currently offline  Manualblock
Messages: 4973
Registered: May 2009
Illuminati (13th Degree)
That next to last statement you make; where you state what variations both statistically and environmentally must be taken into account in all DBT's is the difficult part for me. The possible variants multiplied by infinite sets of conditions revolving around perceptuall and cognitive modulations by the subjects; wheew, it's exhausting just to think of it. So how do they accomplish this Sisyphean task?
Allow me not to appear patronising. There are so many possible variables just in the existing neural conscripts of each individual unique at the time of the test it just seems impossible. And I believe you create a musical artifact internally whenever you listen. That takes in the surroundings and mood et all. Who hasn't heard a song that seemed blase' initially only to become a favorite over time. That is the best and simplest analogy I can come up with.
But like I say; nobody implies Caruso can't sing; or Renoir can't paint. So we know instinctively what is good. That I think is why you build your system over time and experience. Those requisites must be accepted and that waters down any effect of DBT's.

obviously a reading problem on your part... [message #2138 is a reply to message #2134] Thu, 25 August 2005 20:17 Go to previous messageGo to next message
Mr Vinyl is currently offline  Mr Vinyl
Messages: 407
Registered: May 2009
Illuminati (1st Degree)
Show me where I said "You suggest that if someone cannot tell the difference in a DBT between copper cables and $100/foot silver cables, then DBT is useless."

I said...

"If the above tests show that people can consistently hear differences between speakers, cartridges and even amps and preamp, then this would be a mark for the double blind testers."

I am only suggesting that if people couldn't hear a difference between speakers in a double blind test then maybe the tests are not accurate for this kind of thing. That's all I said. I have no idea how a DBT would come out using speakers. Don't be so combative and try understanding the post before you jump to conclusions.

You say...

"I think that "if people had trouble telling apart even speakers, cartridges and preamps using blind testing, this would help prove the point of view that" there were no audible differences between the things being measured."

So the test must be accurate no matter what the circumstances? So if in a double blind test people could not tell the difference between water and wine then there must be no difference?

Makes sense to me.

Re: Blind testing and what I would like to see done... [message #2139 is a reply to message #2132] Thu, 25 August 2005 20:28 Go to previous messageGo to next message
Mr Vinyl is currently offline  Mr Vinyl
Messages: 407
Registered: May 2009
Illuminati (1st Degree)
We are in agreement I think. Useless? I don't know about that. But I know I hear what I consider a significant difference between cables and amps. However I doubt I could consistently and correctly pick out these differences in a DBT. Does that mean I am just fooling myself. Maybe. Does that mean the tests are flawed? Maybe. Frankly I don't care if I am fooling myself or there is a real difference. If it sounds good to me then I'm happy. And it's money well spent.

Re: You don't understand why DBT's [message #2140 is a reply to message #2136] Thu, 25 August 2005 20:58 Go to previous messageGo to next message
Bob Brines is currently offline  Bob Brines
Messages: 186
Registered: May 2009
Location: Hot Springs Village, AR
Master
"It would seem to me that to assign differences in sound implies ranking's; in order to have a difference you must have a quality descision that implicates the difference."

No. Absolutely not.This is a binary choice. 1=There is a difference, 0=There is not a difference. The choices are not A=It sounds good, B=It sounds bad, C=There is a difference but I like both systems, D=..... Quality is not an issue with DBT, only a detectable difference.

"If I gave the impresssion that I personally required expected results I have mis-lead readers of the post."

Well, you said: "You get results that make your hair stand on end. Solid State amps suddenly sound best; CD's beat vinyl etc." Perhaps I misread this, but it sure sounds like you expect tubes to sound better than SS and vinyl better than CD's. This was the waving red flag that caused my perhaps impolite response. What sounds best is a highly individualistic, very subjective concept and each and every serious audiophile will have a different answer.

"Since there would have to be a certified difference between components to make the test of any use...."

Again, no. If the difference was so large that there is general agreement upon casual listening, there is no need for a test. You are again making the mistake that a DBT is to determine which is best. It is not. The value of a DBT is to determine if a difference exists at all.

"Shootouts; well I can see some value because after all; there you are truth be told, I mean at some point you have to race the horse."

This is pure opinion and my opinion differs. My speakers are designed for a very specific niche. When I demo them, I choose conditions that present the speakers in their best light. I choose amps, program material, SPL levels, etc. In any valid shootout, the conditions for all speakers have to be the same. This gives the real potential for all the speakers to sound crappy. No, it's best to listen to good audio equipment it a proper environment, then move on to the next room with a properly optimized environment.

Bob

Definition of DBT [message #2141 is a reply to message #2140] Fri, 26 August 2005 07:46 Go to previous messageGo to next message
colinhester is currently offline  colinhester
Messages: 1349
Registered: May 2009
Location: NE Arkansas
Illuminati (3rd Degree)
From everything I remember about DBT, there is nothing that says the response must be pure binary (1/0 response.) Why are not answers A-D (reference your prior post) just as valid? Am I missing something?......Colin



Re: Definition of DBT [message #2142 is a reply to message #2141] Fri, 26 August 2005 09:34 Go to previous messageGo to next message
Manualblock is currently offline  Manualblock
Messages: 4973
Registered: May 2009
Illuminati (13th Degree)
Thanks for bringing that up. The other concern I have and I don't want it to appear confrontational so I will just state simply and honestly this; if you make a distinction between two units, that distinction requires you have a basis of comparison. If that shows a difference then by definition a quality determination must be made. You may say you like both and they are just different; but with a difference by definition there must be good aspects and bad aspects assigned. Otherwise there would be no discernable difference to speak about.
It's a judgement call indicating one is something that the other is not.
Thats a quality descision.


Re: obviously a reading problem on your part... [message #2143 is a reply to message #2138] Fri, 26 August 2005 11:32 Go to previous messageGo to previous message
tomservo is currently offline  tomservo
Messages: 7
Registered: May 2009
Esquire
Hi

First, blind testing is useful because it can eliminate the inputs from the senses not in question.

Your example of not being able to tell any difference between water and wine is a good one. In this case one is talking about what you can “taste” as opposed to “hear”.

So lets look at that, lets say by blind testing, you gradually diluted real red wine with tap water and you do this progressively until the people say there is no difference between it and water half of the time.
Now, at this dilution, the question is since there is no statistical correlation one way or the other, does this mean it is the same as tap water?
Also, what if your one of the people who can still tell there is wine present even though most can’t?

Statistically speaking, in the first question, it means that in that group, there was no actual correlation and so for that group (not individuals), there is statistically no “taste” difference.

Allowing the subjects to “see” the shade of the liquid would certainly produce a vastly higher positive because they could also “see” that the water is slightly “pink”, giving them another mental data point for the subjective judgement.
Going further, if they watched the dispensing process they would “know” even more about what they were tasting and only those daydreaming would get it wrong.

This part is what’s wrong about “not” blind testing, you include many other inputs other than just “what you hear” and one cannot remove that subconscious bias, one can only test “blind” where you cut off from those other inputs.

So what if your one of the people in that group that could reliably tell which was wine or water?
You have (statistically) a greater sensitivity than the group had, but if one continued to dilute it further, you would reach a dilution where you could not tell which was which.
At that point, even for you, there would then be no taste difference even though there were still X molecules of wine in the water.
On the other hand, if you were allowed to see the water’s color or you watched it being diluted and distributed, you could still “taste” which was wine.

Also, in medicine were big money is at stake, it is well known that “doing anything” can produce a placebo reaction. Taking a “sugar pill” that is thought to be helpful medicine causes a positive outcome between ¼ and 1/3 of the time. It is the act of “doing something” thought to have a specific outcome, that causes a false positive.
In the case of drug testing, progress was VERY slow until blind testing was used to compare the result of the test drug against the placebo while no one knew which was which.

In audio, big money or life and death is not the issue and in some ways getting to the bottom of things would not improve sales figures, so it is popular to cite the approach as flawed. I suppose it really is flawed too if sales is the measure of success and you make costly hifi trinkets

I would put it more like this.
IF you can’t hear any difference between X and Y with your ears alone, when not knowing which was which ahead of time (blind), what are you really “hearing” when the differences are then audible “with” prior knowledge (not blind)?

Your looking at the wine and watching it being dispensed that’s all..

Tom Danley



Previous Topic: Tascam 133
Next Topic: Hey Charlie, man are you out there?
Goto Forum:
  


Current Time: Fri Nov 29 16:16:35 CST 2024

Sponsoring Organizations

DIY Audio Projects
DIY Audio Projects
OddWatt Audio
OddWatt Audio
Pi Speakers
Pi Speakers
Prosound Shootout
Prosound Shootout
Miller Audio
Miller Audio
Tubes For Amps
TubesForAmps.com

Lone Star Audiofest