Entering the comment area of audio forums or audio websites, you may not have seen such a sentence: "Some products with good test data sound bad, and some products with bad test data sound good." This is at best. If you don't understand the relevant knowledge, the worst case is a lie. The specific type depends on who said it. This is a lie and sometimes makes audio equipment bought by audiophiles perform poorly.
The first half of this sentence is incorrect, which can be shown in examples in scientific papers and informal literature. The lower part may be true, if it is for the individual. I always think this is a common saying in the audio world, and in this case, it is wrong. This cliché reflects not wisdom, but people’s rejection of science. As far as I know, these people are too lazy to study science and have no measurement experience.
One of the most obvious examples of this sentiment appeared this month, in the July 2020 issue of Stereophile magazine, Herb Reichert's evaluation of Tianlang Revolution XT 6 speakers. The first sentence of this review read: "I have been wrestling with my predecessors on new methods of measuring loudspeakers, and persuading people to find ways to integrate more directly with the audience’s experience." In this article, the same author stated his point more directly: “As an evaluation tool, or as a predictor of user satisfaction, today’s measurement procedures are almost useless.” As we will see, this article The overview clearly shows why measurement is so important when evaluating audio products.
Both of the author's statements reflect ignorance of this issue. Taking speakers as an example, the measurement method has been proven to predict 86% correlation of user satisfaction, which was established more than 30 years ago. This is mainly based on the extensive research conducted by Dr. Floyd Toole at the National Research Council of Canada and continued at Harman International. Now, countless speaker companies use these methods as design guidelines. This is because they know that speakers measured according to these standards are good for most listeners.
Some people will point out that this model will have a 14% failure rate. However, 14% speakers with good sound quality but not universally loved by the audience are unlikely to sound "bad" unless they have higher distortion. Some measurement methods can be easily detected. In any case, it is absurd to claim that an 86% success rate is "almost useless".
Recently, scientific research has produced measurement results of earphones and earphones, which can accurately predict user satisfaction. For example, in the AES (International Audio Engineering Association) paper 9878, "Statistical Models for Predicting the Rating of Listeners' Preference for In-Ear Headphones: Part 2-Model Development and Validation", by Dr. Sean Olive and Harman International Research The team’s Todd Welti, Omid Khonsaripour reported that in the evaluation of 30 earphones using 71 listeners, the correlation between test data and listener preferences was 91%.
I agree that test data does not predict which power amplifiers, DACs and other electronic products people will like. But this is not because of flaws in the measurement, but because listeners rarely agree with which audio electronics they like. For example, blind listening tests rarely show obvious differences or preferences between certain models, brands, or types of amplifiers. The evaluation of these products does not show the preference trend of the reviewers; they tend to praise some power amplifiers and DACs. If in the controlled hearing test, a considerable number of participants did not show their liking for certain audio electronic devices and their dislike for other audio electronic devices, then the test data or subjective evaluation cannot predict the listener's preference.
This is actually a very euphemism.
What about the idea of "some products with bad test data sound good"? Stereophile technical editor (former editor) John Atkinson (John Atkinson) put forward a strong argument against this statement, he said in the abstract of the 1997 AES speech, "Once the response flatness deviates from a certain level (for example, The frequency-weighted standard deviation between 170Hz and 17kHz is about 3.5dB), the speaker is unlikely to sound very good and impossible to be recommended." He is talking about the speaker recommended by Stereophile writer. Studies have shown that a group of multiple listeners may be less tolerant of speakers with poor test data in blind listening tests.
Of course, even an apparently defective audio product may sound good to some people. To find such an example, just look at the above comment on Tianlang speakers. Atkinson’s measurement results show that, as he said, “The level of the tweeter seems to be balanced between 3dB and 5dB higher than other frequency bands.” This will cause excess energy in these frequency bands. When I measure, I can hear MLSSA pseudo-random noise signal. "
To get a rough idea of what the sound is, just turn up the treble knob of the audio system by 4dB. This sound is not soft at all, nor is it pleasant. I have to consider that the factory used the wrong treble impedance to view the measurement results. In a blind listening test with multiple listeners, such as the evaluation conducted by the National Research Council of Canada or Harman International, such speakers will almost certainly not get good scores.
However, I found that this flaw was not mentioned in the subjective evaluation. In fact, the reviewer described the sound of the speakers as "slightly soft" and ended with "strongly recommended". At least, according to this review, if a measurement method can be found that can reliably predict which speaker the reviewer likes, most listeners may not like those same speakers.
Fortunately, those who read the measurement data got the true story. Those who ignore the measurement results because they are told they are "almost useless" may end up buying a loudspeaker with a significantly wrong three-band equalization.
Don't get me wrong-I don't mind someone touting an audio product with huge flaws, just like I like to listen to Kiss's Alive albums occasionally, I hope no one will mind! I have read many such comments, and few people have the inspiration to comment on them. But denying the decades of work of some of the most talented audio scientists in the world just because it doesn’t fit your personal feelings is as frivolous as claiming that Gene Simmons is the greatest bassist of all time.
I hope the audio reviewers are curious about their hobbies and want to know everything about it as much as possible, but a large number of them have shut themselves out of any new information that might shake their beliefs . In their rejection of science, they let readers and their industry fall into nonsense--in many cases, they trap readers in buying poorly performing products, and then sell these products to buy other defective products. Products, rather than simply understanding the key facts about audio so that they can buy good equipment the first time.