Home » Audio » Measurement » To measure or not to measure (and what good is it anyway?)
To measure or not to measure (and what good is it anyway?) [message #25183] Fri, 03 December 2004 00:30 Go to next message
Wayne Parham is currently offline  Wayne Parham
Messages: 18792
Registered: January 2001
Illuminati (33rd Degree)
Any good engineer or scientist will tell you that the proof of any concept requires measurements. Measurements provide verification. Without validation of a concept, it is just a hypothesis.

But where does that leave the DIY builder? If building loudspeakers, can good results be obtained without the ability to perform measurements? Are their properties so complex that one is completely blind without measurements? If on a budget, which ones measured data is most important? What kinds of tests can be done reliably?

These are some of the things I've pondered over the years. When I first built loudspeakers, I had an oscilloscope, meters, good microphones, an accurate signal generator and an SPL meter. Having good microphones and an oscilloscope and signal generator put me ahead of the game, with more visibility than 99% of the hobbyists that might undertake building their own loudspeakers at that time. But I still wouldn't have wanted to publish response graphs made by plotting individual data points. It is too coarse, and doesn't give an accurate picture.

Now days, a guy can use the same PC he uses to play video games and check E-mails and have a pretty good measurement system. That wasn't so just a few years ago, and it was difficult to obtain measurements that were worth doing. But several measurement system programs have been written that work pretty well. Using a PC and its built-in sound card, you can connect a $10.00 microphone and actually perform some pretty good measurements.

But how good are they? The answer, in my opinion, is that they are very good for helping a hobbyist find response between 500Hz and 5kHz, where crossover points are likely to be. That in itself is worth the admission price, because it makes good design work much easier. But I would still be leery of using them for exchanging with others to make overall performance comparisons. There are too many places for indefinites and ambiguity to creep in.

Hobbyist measurements performed on homebrew test equipment are useful, but probably should be limited to fine-tuning and not for critical comparisons and performace evaluations, in my opinion. At least one should be careful to realize what they are looking at when interpreting data like this. Understand that what is measured by one person with his test setup may be wholly different than what another person finds, even if they are testing the exact same loudspeakers on the very same sound system. So comparisons should only be made if the conditions of the test can be controlled, or measurement data should be taken with a grain of salt.

I've made comments like this before, and some have characterized me as a person that doesn't like measurements. Nothing could be further from the truth. The problem is that I don't like ambiguous measurements.

Set the clock back to a time before PC's. This is when the only measurement systems were pretty expensive. A DIY audio hobbyist couldn't make measurements reliably, so he had to depend on shops that could. But he could pretty well trust reactive circuit formulas, so he could expect to understand the crossover. He could use Thiele/Small data and model a sealed or ported box with reasonable certainty. Electro-mechanical measurements aren't nearly so hard to obtain on a budget. So these are things that were reasonable to do. Using the data that was available, one could get an accurate picture of some features, and use mathematical models to determine the rest.

Using limited measurement equipment, one probably couldn't see the fine-grain features of response that would illustrate anomalies due to crossover interaction. Maybe a lucky hit on a certain frequency might show a null, but there was much more a chance they would be missed. Even if you've done the math and know what to look for, the tolerances involved would make it difficult to hit the frequencies of interest within less than about 10% accuracy. The tolerance of the parts and the system involved prevents it. So the best thing, in my opinion, was to use mathematical models to determine crossover and physical placements that would work best.

Now fast-forward to today, when measurement software is available to everyone. A motivated person can setup a measurement system and generate useful data with practically no investment. It is still difficult to measure some things, but it's worlds easier than it was before. An example of something that I don't expect to measure is the response of a basshorn in eighth-space, decoupled from the room its placed in. The two conditions are tied together. So this is an example of something I still find some merit in comparing models as opposed to measurements, to prevent two people from comparing their rooms more than comparing their speakers. But as for checking response through the crossover region, it's much easier to measure than to model these days. You can do the math to know what to expect, and depend on measurements to confirm or disprove your models, making changes if necessary.

There are still problems to avoid though. This is probably what was most important for me to say, and why I took the time to write this here.

Testing is a science all by itself. Engineers certainly need to perform tests to validate and confirm predictions, and scientists perform tests to see if they're on the right track. But it is important to realize that testing is a pretty sophisticated thing all by itself. Doing a test right makes all the difference.

As an engineer, I think test results are extremely helpful. An example is putting a part that will be subjected to physical stress in a hydraulic press to see how much it deflects. Another example would be to check airflow through an orifice or current flow through an electronic component at specific voltages and frequencies. These kinds of tests are just part and parcel of the design process.

Where complications arise is when a system is to be tested for overall performance evaluation. Here again, there are some places where performance testing is appropriate and unambiguous. I might time an automobile through the traps or check its gas mileage. Or I might test the speed of a computer doing a particular set of instructions. That can give me a good feel for the design, and can assure me that the design goals have been met. But after the design is completed and it comes to the point where a battery of "signoff" tests be done, this is where I think the engineer or scientist must hand off to an unbiased test group. The engineer is too close to his work and a true evaluation really cannot be performed by the designer or design team. It should be done by an impartial and qualified testing person or organization.

If an impartial test can't be done, as is often the case with small shops and individuals, then other measures might be considered. It might be cost prohibitive to send out products to an independent testing facility, for example. But the problem still remains: How reliable is a comparative performance evaluation when the test is performed by someone who is affected by the test outcome? Should it be done by an engineer or design group that wants their project to be successful? Should it be done by a competitor? Even if everyone is objective and ethical, you can see how there is a problem here, or at least the potential for one.

Hobbyists are sometimes just as emotionally tied to their favorite equipment as designers are, sometimes even more so. They are certainly not immune to bias. And another complication presents itself, which is their varied abilities. Some hobbyists are technically inclined, but others are not.

Even though loudspeakers are very simple, acoustics testing requires many variables be considered. It's a little like the weather, in that it's a simple subject but it isn't easy to nail down. Engineers and scientists have a hard time dealing with some of the issues because they can be difficult to solve. Things like boundary reinforcement and reflections can make certain kinds of tests impossible or at least somewhat ambiguous. And emotional attachment can blind even the most objective people and tempt them to see what they want to see instead of what is.

Now add to this the fact that some hobbyists obtain tools to perform acoustic tests with their PCs, but may be entirely unqualified to do them. This adds a whole new layer of ambiguity to the picture. In one sense, it is very good that these affordable measurement systems are available to hobbyists, but on the other hand, it makes it possible to put a credible face on an entirely bogus dataset.

I've seen more than one occasion where wholly false data was presented in a pretty format and was more believable to laymen than better, more accurate data presented in raw form, which is less impressive looking. It isn't always a case where data is falsified on purpose, although that is sometimes the case. Sometimes it is just wishful thinking, throwing out good datasets in favor of less accurate data that looks better. The opposite case can be made too, if someone has a bone to pick. Sometimes erroneous charts are just a result of improper setup or calibration. But whatever the case, the point is that even a good measurement system can provide ambiguous data if used in the wrong way.

Just like a mechanic's tool can be misused and broken, so too can a tool like an audio measurement system. A system like this in the hands of an amateur will make professional looking charts that lend credibility to the dataset. But the data is no good whatsoever if the system isn't setup right. The environment may be unsuitable, the hardware may not be adequate or the system may just be misconfigured.

Sophisticated test equipment in the hands of hobbyists is a two-edged sword. Not everyone can use the equipment properly and yet practically anyone can make a professional looking chart. It's easy to think that test data is accurate when it is presented to you in a professional format, but if you don't know the conditions of the test, it really should be considered with skepticism.

So that's my problem with trusting measurements. I guess what I'm saying is that if testing isn't done by someone I know to be qualified, reliable and unbiased, I am skeptical. And even if I know and trust the person doing it, if they are too close to the subject emotionally, it may be difficult for them to be objective. Test results reflect conditions under test as well as devices under test, so a person wanting to find a particular outcome may throw out important test results that contradict the expected outcomes. But those may be the most reliable datasets. And when you put this in front of amateurs, the problem becomes even more acute.

If you get a group that is already bent on finding superiority in their particular pet project, it makes it pretty difficult to overcome this prejudice. As an example, it's almost a foregone conclusion that Chevy guys will like their Chevy better than a Ford, even if the Ford is a better product. Probably best to let an independent and unbiased testing group do the tests than a Chevy club or an engineer for Chevrolet. Otherwise, even if the charts and graphs look very professional and the people involved have unimpeachable credentials, it's easy to see how they might find the Chevy as the better car and produce data to back it up.

Take it or leave it. Certainly there are plenty of people that can perform good tests, and that are objective enough to come up with something useful. But do be careful and dilligent because there are plenty of ways to screw it up. And there are also lots of people with emotional attachments that make it very difficult to really get to the bottom of things.

Re: To measure or not to measure (and what good is it anyway?) [message #25184 is a reply to message #25183] Sat, 04 December 2004 13:18 Go to previous messageGo to next message
harmony is currently offline  harmony
Messages: 1
Registered: May 2009
Esquire
Are you saying that the average sound man or weekend warrior shouldn't bother with measurements? Do you think there is no point for audiophiles to measure equipment in home? Should they disregard their RT analyzers?

Re: To measure or not to measure (and what good is it anyway?) [message #25185 is a reply to message #25184] Sat, 04 December 2004 13:33 Go to previous messageGo to next message
Wayne Parham is currently offline  Wayne Parham
Messages: 18792
Registered: January 2001
Illuminati (33rd Degree)
I'm suggesting that a person be realistic. Be dilligent, know your limitations and watch out for these things.

Re: To measure or not to measure (and what good is it anyway?) [message #25281 is a reply to message #25185] Tue, 03 February 2009 22:17 Go to previous messageGo to next message
Psychoacoustic is currently offline  Psychoacoustic
Messages: 75
Registered: May 2009
Viscount
Can absolute newcomers to DIY, with no experience in electronics, uneducated in acoustics- either formally nor self-taught, get too far ahead of themselves by considering testing their DIY speakers (bearing in mind the limitations presented in the article) or is it the next logical step?
At this crossroads right now. Mr Parham's article assists thinking on the subject by returning to the basics of the scientific method; experimentor bias, cross-test reliablity, does the test test what it intends to test, replicability, etc.
Does measuring the room fall under similar auspices? Can a 'layman' use (e.g.) a RS SPL meter, external soundcard and PC to gain accurate enough information to be practically useable?
Opinions seem to differ here.

Re: To measure or not to measure (and what good is it anyway?) [message #25282 is a reply to message #25281] Wed, 04 February 2009 22:28 Go to previous messageGo to next message
Wayne Parham is currently offline  Wayne Parham
Messages: 18792
Registered: January 2001
Illuminati (33rd Degree)

Excellent questions.

I think good measurements are extremely helpful, truly mandatory for proper optimizations. Without measurements, you're really just guessing. One can go to great lengths to model the system mathematically, and that's a good way to estimate or approximate things. You can do a lot with a model. But nothing beats accurate measurements. Whether you're trying to optimize a loudspeaker design or an in-room installation, good measurements are important.

The good news for the serious DIY'er is good measurement equipment is relatively cheap these days. A $1000 investment buys measurement equipment that's better than anything available in the pioneering days of audio, even up to the 1970's.

That said, I've seen a lot of people over the years trying to quantify things with tools that weren't up to the job. An improperly taken measurement, or a measurement with inaccurate or uncalibrated equipment can give false readings. The novice is then armed with a false sense of security, thinking he knows things he really doesn't. So in some cases, a model is better than a measurement.

One must know what they're looking for too. That sort of goes hand in hand with learning how to measure - knowing what to measure. It's tempting for the novice to take a new piece of test gear and measure everything in sight, posting charts that really say nothing. That's probably one of the most common things I see these days. An indoor measurement done without proper gating, for example.

Generally, if you're trying to measure a loudspeaker, you want to do it anechoically or pseudo-anechoically. You want to know what the speaker is doing, not the room. So either measure it outdoors or indoors with gating to ignore reflections. This requires some knowledge of boundary conditions and reflections, of free-space, half-space (ground plane) and how to calculate time to first reflection.

If you're trying to measure a room, you want an averaged measurement. An RTA may suffice for that, but I'd prefer having something a little more sophisticated with gating capability. That way, you can decide to gate or not, to average or not. Some might measure only at the listening position but I prefer to measure at several points, to know what is happening throughout the room, or at least at several places in the listeneing area. You can then develop a database of information that shows energy distribution at all frequencies, at all positions in the room. This can be seen in high resolution, or it can be averaged to find overall trends.


Re: To measure or not to measure (and what good is it anyway?) [message #64221 is a reply to message #25282] Fri, 08 October 2010 05:38 Go to previous messageGo to next message
jazzlover is currently offline  jazzlover
Messages: 41
Registered: September 2010
Baron
HI Wayne! The essay is long and full of passion about the subject. That made me realize that as an enthusiast, the way I (and my friends) have been measuring sound is by sensing it, by closing our eyes and letting the sound fill us. That kind of measurement would be the way Master Yoda would have it. But it falls short of quantifying sound.

Now, would a $1,000 investment be worth it? I'm still pondering about it Smile
Re: To measure or not to measure (and what good is it anyway?) [message #64222 is a reply to message #64221] Fri, 08 October 2010 10:54 Go to previous messageGo to next message
Wayne Parham is currently offline  Wayne Parham
Messages: 18792
Registered: January 2001
Illuminati (33rd Degree)

If you plan on designing speakers and want them to be really good, a measurement system is very important. You can do a very good with computer models, but that will only give you ~90% as good as it gets. if you want to rise to a 99% solution, measurements are in order.

Re: To measure or not to measure (and what good is it anyway?) [message #65025 is a reply to message #64222] Tue, 30 November 2010 15:49 Go to previous messageGo to next message
Adveser is currently offline  Adveser
Messages: 434
Registered: July 2009
Location: USA
Illuminati (1st Degree)
Wayne Parham wrote on Fri, 08 October 2010 10:54

If you plan on designing speakers and want them to be really good, a measurement system is very important. You can do a very good with computer models, but that will only give you ~90% as good as it gets. if you want to rise to a 99% solution, measurements are in order.




Yeah.

I think too that once someone knows what good sound is like and are very familiar with it, you can rely on your ears a lot more.


Re: To measure or not to measure (and what good is it anyway?) [message #65225 is a reply to message #65025] Fri, 10 December 2010 02:41 Go to previous messageGo to next message
woodfree is currently offline  woodfree
Messages: 30
Registered: December 2010
Baron
I believe that in any activity that you do, you got to know what you're doing! Accuracy is very important, and in order to be accurate, you have to research and find truthful knowledge so you can make informed decisions.

But I also believe the definition of what passes as "good sound" still depends on the individual. I appreciate good sound quality from my audio equipment, but then again, I don't want to be constantly obsessing about what's better out there, when the music I'm hearing from my current equipment is enough to get me through the day!
Re: To measure or not to measure (and what good is it anyway?) [message #68137 is a reply to message #25183] Sat, 11 June 2011 14:01 Go to previous messageGo to previous message
DJ Dave is currently offline  DJ Dave
Messages: 111
Registered: October 2010
Viscount
If you want something that you are custom-making for your own enjoyment - you can do whatever you want with it. But still, there are certain measurement guidelines you will want to stick with, to make sure it all works in the end.
Previous Topic: Good Speaker Bad Speaker
Next Topic: The Acoustic Center: How it applies to Loudspeaker Measurements
Goto Forum:
  


Current Time: Sun Dec 08 12:11:41 CST 2024

Sponsoring Organizations

DIY Audio Projects
DIY Audio Projects
OddWatt Audio
OddWatt Audio
Pi Speakers
Pi Speakers
Prosound Shootout
Prosound Shootout
Miller Audio
Miller Audio
Tubes For Amps
TubesForAmps.com

Lone Star Audiofest