- Μηνύματα
- 2.331
- Reaction score
- 3.635
Sound reproduction – art and science/opinions and facts
Most visitors online was 2716 , on 5 Dec 2023
-
Καλώς ήρθατε στο AVsite! Ελάτε στην παρέα μας κάνοντας εγγραφή!
Floyd Toole lecture
- Thread starter m1ts0s
- Ημερομηνία δημιουργίας
- Μηνύματα
- 2.331
- Reaction score
- 3.635
'The overall thrust of the lecture is, I would say, “objectivity works”. We can make measurements of the hardware, judge them against some fixed criteria, and then demonstrate that these correlate with real, human preferences in blind tests. Hurrah! The age-old debate is over, and we can improve our new speaker designs by building them to maximise their objective scores in the full knowledge that this would correlate with human preferences.
However, I am not convinced by the criteria that were specified in the lecture; it seemed to me that there were holes in the argument and that the case was rather circular. As I wrote before, simply through the design of the experiments and what they are testing for, biases may be baked into such results just as badly as the sighted listening tests Toole dismisses.
What he showed:
I have no doubt that flat frequency response and smooth off-axis response are essential, as he says, but might there be more to it than just that? Any unexplained deviations between the listening tests and the measurements (it isn’t a perfect correlation) could be explained by the speaker’s transient response which, after all, is a straightforward difference between what was recorded and what the speaker emits – it is just that someone around 1936 declared that ‘phase doesn’t matter’. Until recently it has not been possible to verify this ‘fact’. Comparing different speakers all of which have phase/time distortion and other problems, and finding that listeners cannot tell them apart (in mono using someone’s idea of ‘typical’ music), does not tell us that a speaker without those distortions would not sound better.
Correlation is not causation, but people are talking about the Harman method as if it is. So, if I were a speaker designer doing things by the Toole book, I would always use bass reflex without thinking, as this would have no effect on the Spin-o-rama score but would result in a smaller box. And I would be supremely relaxed about crossover design, ensuring only that it matched the phases of drivers through the crossover. Phase correction and sealed enclosures wouldn’t get a look-in because they offer nothing extra in terms of the Spin-o-rama score but cost more to manufacture.
My opinion is of no consequence, of course, but there are some serious people who do suggest that transient response matters*, and it would have been nice if the guru of gurus could have mentioned it, if only to dismiss it with reasons.
* Just writing those words seems absurd. How can transient response not be important, and yet in a forum debate on the topic of the Toole lecture this is what Amir Majidimehr (onetime corporate vice president of Microsoft’s Consumer Media Technology group, and now running companies involved in audio and video) has just said:
"As to effect of phase, and time domain correction, there is little objective data to back any of those techniques. Indeed, listening tests shows that we are exceptionally insensitive in this domain. In sharp contrast, we hear frequency response variations readily. It is for that reason that I so emphasize frequency response measurements and results. Time domain does not enter my vocabulary…"
You will be aware of my opinion of the listening tests to which he refers. Clearly the industry could do nothing to correct phase in the past, and in a perfect circularity, using (in my inconsequential opinion) questionable listening tests with uncorrectable speakers, ‘proved’ that we are insensitive to phase, therefore it doesn’t matter. This is now baked into speaker design forever!'
That Floyd Toole lecture
However, I am not convinced by the criteria that were specified in the lecture; it seemed to me that there were holes in the argument and that the case was rather circular. As I wrote before, simply through the design of the experiments and what they are testing for, biases may be baked into such results just as badly as the sighted listening tests Toole dismisses.
What he showed:
- Sighted listening tests can be flawed (implied: all sighted listening tests are unreliable).
- Some measurements can be carried out on speakers in an anechoic chamber (dubbed ‘Spin-o-rama’) and munged together to create a performance index related to flatness of frequency response and smoothness of off-axis response. Transient response is not a factor. At all.
- In listening tests, speakers with the ‘best’ Spin-o-rama score are usually preferred by listeners over the opposite (implied: all else being equal).
- Mono allows maximum discernment of difference, and does not contradict stereo listening test results in the above tests (implied: therefore mono should be used for all listening tests)
- Trained professional listeners give the ‘statistically healthiest’ range of scores, and do not contradict ordinary listeners in the above tests (implied: therefore trained professionals should be used for all listening tests)
- That it is valid to use the Spin-o-rama score in reverse i.e. as a tool for designing a speaker. He implies it is, but does not prove it. In other words, can it be guaranteed that a truly terrible speaker could not be designed that, nevertheless, achieved an exemplary Spin-o-rama score?
- That transient response doesn’t matter – it is simply ignored. The speakers tested may have had good transient responses, or not, but as most of them were of conventional design they may all have been much of a muchness.
- That various speaker technologies are inherently better or worse than others i.e. no view on whether sealed cabinets are better than bass reflex, or active crossovers better than passive – and his performance index is indifferent to this, assuming that flat steady-state frequency response is all that matters.
- That mono speakers and trained professionals are the best choice for all listening tests.
I have no doubt that flat frequency response and smooth off-axis response are essential, as he says, but might there be more to it than just that? Any unexplained deviations between the listening tests and the measurements (it isn’t a perfect correlation) could be explained by the speaker’s transient response which, after all, is a straightforward difference between what was recorded and what the speaker emits – it is just that someone around 1936 declared that ‘phase doesn’t matter’. Until recently it has not been possible to verify this ‘fact’. Comparing different speakers all of which have phase/time distortion and other problems, and finding that listeners cannot tell them apart (in mono using someone’s idea of ‘typical’ music), does not tell us that a speaker without those distortions would not sound better.
Correlation is not causation, but people are talking about the Harman method as if it is. So, if I were a speaker designer doing things by the Toole book, I would always use bass reflex without thinking, as this would have no effect on the Spin-o-rama score but would result in a smaller box. And I would be supremely relaxed about crossover design, ensuring only that it matched the phases of drivers through the crossover. Phase correction and sealed enclosures wouldn’t get a look-in because they offer nothing extra in terms of the Spin-o-rama score but cost more to manufacture.
My opinion is of no consequence, of course, but there are some serious people who do suggest that transient response matters*, and it would have been nice if the guru of gurus could have mentioned it, if only to dismiss it with reasons.
* Just writing those words seems absurd. How can transient response not be important, and yet in a forum debate on the topic of the Toole lecture this is what Amir Majidimehr (onetime corporate vice president of Microsoft’s Consumer Media Technology group, and now running companies involved in audio and video) has just said:
"As to effect of phase, and time domain correction, there is little objective data to back any of those techniques. Indeed, listening tests shows that we are exceptionally insensitive in this domain. In sharp contrast, we hear frequency response variations readily. It is for that reason that I so emphasize frequency response measurements and results. Time domain does not enter my vocabulary…"
You will be aware of my opinion of the listening tests to which he refers. Clearly the industry could do nothing to correct phase in the past, and in a perfect circularity, using (in my inconsequential opinion) questionable listening tests with uncorrectable speakers, ‘proved’ that we are insensitive to phase, therefore it doesn’t matter. This is now baked into speaker design forever!'
That Floyd Toole lecture
- Μηνύματα
- 2.331
- Reaction score
- 3.635
- Μηνύματα
- 2.331
- Reaction score
- 3.635
The Curse of Dimensionality
What has been called The curse of dimensionality is a phenomenon that has some relevance to Floyd Toole’s work and to many of the ideas that flourish in those audiophile forums that think of themselves as being at the scientific end of the spectrum. In a nutshell:
when the dimensionality increases, the size of the space increases so fast that the available data become sparse. This sparsity is problematic for any method that requires statistical significance.
Having followed a few online discussions in recent days, it seems clear to me that many people think that Floyd Toole’s experiments were/are controlling only a single variable: the speakers’ ‘directionality’. This is far from the truth*. And even if the tests had been genuinely controlling a single variable, the fixing of the other variables would have restricted the experiments to such a tiny subset of the overall problem space as to make them potentially meaningless**. In reality, the experiments created a few sparse pieces of data within a small fraction of the overall problem space.
In this particular case the research was attempting to confirm the answers to questions which, without doing any experiments at all, most people would have given the ‘correct’ answers to anyway:
“Which do you think is better? Flat on-axis frequency response or not? Flat off-axis response or not? Smooth off-axis response or not?”
Which is not to say that identifying the questions in the first place was not a significant achievement. It is just the pseudo-science of the experiments, the implication that the methodology can be extended to ‘solve’ all audio problems, and the conclusions that people draw from the results that I object to.
* Directionality was not a direct variable. Instead, a selection of existing speakers with varying characteristics were listened to, and their directionality measured. A ‘directionality index’ was created ‘heuristically’. Many other incidental variables related to the speakers were not controlled in the experiments.
** For example, if we restrict all the speakers to the same position in the room regardless of whether they work best there or not, restrict the choice of music to a few pieces of audiophile music, restrict the listening to mono, restrict SPL to the capabilities of the smallest speaker in the selection etc. etc. To test the entire ‘problem space’ meaningfully would require thousands of experiments – the curse of dimensionality.
What has been called The curse of dimensionality is a phenomenon that has some relevance to Floyd Toole’s work and to many of the ideas that flourish in those audiophile forums that think of themselves as being at the scientific end of the spectrum. In a nutshell:
when the dimensionality increases, the size of the space increases so fast that the available data become sparse. This sparsity is problematic for any method that requires statistical significance.
Having followed a few online discussions in recent days, it seems clear to me that many people think that Floyd Toole’s experiments were/are controlling only a single variable: the speakers’ ‘directionality’. This is far from the truth*. And even if the tests had been genuinely controlling a single variable, the fixing of the other variables would have restricted the experiments to such a tiny subset of the overall problem space as to make them potentially meaningless**. In reality, the experiments created a few sparse pieces of data within a small fraction of the overall problem space.
In this particular case the research was attempting to confirm the answers to questions which, without doing any experiments at all, most people would have given the ‘correct’ answers to anyway:
“Which do you think is better? Flat on-axis frequency response or not? Flat off-axis response or not? Smooth off-axis response or not?”
Which is not to say that identifying the questions in the first place was not a significant achievement. It is just the pseudo-science of the experiments, the implication that the methodology can be extended to ‘solve’ all audio problems, and the conclusions that people draw from the results that I object to.
* Directionality was not a direct variable. Instead, a selection of existing speakers with varying characteristics were listened to, and their directionality measured. A ‘directionality index’ was created ‘heuristically’. Many other incidental variables related to the speakers were not controlled in the experiments.
** For example, if we restrict all the speakers to the same position in the room regardless of whether they work best there or not, restrict the choice of music to a few pieces of audiophile music, restrict the listening to mono, restrict SPL to the capabilities of the smallest speaker in the selection etc. etc. To test the entire ‘problem space’ meaningfully would require thousands of experiments – the curse of dimensionality.
- Μηνύματα
- 2.331
- Reaction score
- 3.635
Frequency domain dogma
"Reading around various technical-ish audio forums, I think it is clear that there is close to zero awareness of the over-simplification inherent in thinking about audio only in terms of the frequency domain. While it is true that the frequency and time domains are equivalent – with a reversible mathematical transform between the two – the majority of people discard the phase components when they convert to (and think in) the frequency domain. They are absolutely and wholly convinced that everything can be measured and divined from a phase-free steady-state sinusoidal view of the world. This gives all speaker manufacturers carte blanche to use phase-shifting crossover filters just as their distant ancestors did.
Some audiophiles like to dabble in ‘room correction’ these days, and in a further over-simplification deriving from the frequency domain-centric view, they are unaware of the difference between physical delays and phase shifts. If we have a steady sinusoidal signal then, indeed, a physical delay is indistinguishable from a phase shift. Mix a delayed sine wave with a direct sine wave and we get a partial cancellation or reinforcement. Measure a speaker’s output in a room at the listener’s position using a sinusoidal sweep (or mathematical equivalent) and we get a very wobbly frequency response curve. For continuous waveforms, this can be inverted at the speaker using a bank of tone controls. Hey presto!, a flat response at the listener’s position. Job done. But this does not mean it is correct for real music. Real music does not comprise continuous waveforms, but rather dynamically-changing signals (‘transients’). The effect of delays on transients is not the same as tone control filters. The result at the listener’s position sounds more coloured as a result (the error is less obvious at lower frequencies, however). In fact, the human ability to discern delayed reflections from the direct sound means that the only natural-sounding result is to use no EQ at all! – except for ensuring that the speaker itself is ‘flat’ in terms of amplitude and, in the 21st century, phase.
How has this happened? Surely the time domain view is how you would expect most people to think intuitively.
I think there are several reasons:
What is happening is that their audio signals are falling apart internally. The signals measure perfectly in the world of the magnitude-only, steady-state, frequency domain, but on real music what comes out of the speaker bears no resemblance to what the microphone picked up. Mention this to them, and the response is positively hostile: humans are not sensitive to phase! Understandably, they cite the killer fact: the most expensive and highly regarded speakers in the world distort the phase and ‘enhance’ the bass with acoustic resonators just like any other conventional speaker. Quite so. But they also, probably, have other positive characteristics that partially offset the phase issue. It is also undeniable that the people who own these wonder-speakers seem just as keen to ‘churn’ their equipment as everyone else…"
"Reading around various technical-ish audio forums, I think it is clear that there is close to zero awareness of the over-simplification inherent in thinking about audio only in terms of the frequency domain. While it is true that the frequency and time domains are equivalent – with a reversible mathematical transform between the two – the majority of people discard the phase components when they convert to (and think in) the frequency domain. They are absolutely and wholly convinced that everything can be measured and divined from a phase-free steady-state sinusoidal view of the world. This gives all speaker manufacturers carte blanche to use phase-shifting crossover filters just as their distant ancestors did.
Some audiophiles like to dabble in ‘room correction’ these days, and in a further over-simplification deriving from the frequency domain-centric view, they are unaware of the difference between physical delays and phase shifts. If we have a steady sinusoidal signal then, indeed, a physical delay is indistinguishable from a phase shift. Mix a delayed sine wave with a direct sine wave and we get a partial cancellation or reinforcement. Measure a speaker’s output in a room at the listener’s position using a sinusoidal sweep (or mathematical equivalent) and we get a very wobbly frequency response curve. For continuous waveforms, this can be inverted at the speaker using a bank of tone controls. Hey presto!, a flat response at the listener’s position. Job done. But this does not mean it is correct for real music. Real music does not comprise continuous waveforms, but rather dynamically-changing signals (‘transients’). The effect of delays on transients is not the same as tone control filters. The result at the listener’s position sounds more coloured as a result (the error is less obvious at lower frequencies, however). In fact, the human ability to discern delayed reflections from the direct sound means that the only natural-sounding result is to use no EQ at all! – except for ensuring that the speaker itself is ‘flat’ in terms of amplitude and, in the 21st century, phase.
How has this happened? Surely the time domain view is how you would expect most people to think intuitively.
I think there are several reasons:
- Frequency domain techniques are tremendously powerful, and allow us to ‘X-Ray’ a continuous signal, showing us minuscule harmonic distortion components and frequency response errors. (This is great for testing the basic functionality of amps and DACs, for example). But with such a powerful tool under our belts, it is too tempting to try to mould the world the to the tool rather than the other way round.
- Audio systems of the past could not actually preserve phase information without arbitrary errors, so there seemed to be little point in measuring it. In reverse, this has meant that (almost) no one has tried to make a system that preserves the phase information, to this day.
- There is a belief that the human hearing system is “insensitive” to phase. Quite possibly in relative terms, but it is also “insensitive” to considerable levels of distortion, noise and all the other minutiae that Golden-Eared people tell us we should worry about. Based on experimental evidence, humans are completely “insensitive” to the errors introduced by 16 bit 44.1 kHz audio, but ‘real’ audiophiles don’t believe that. Why so blasé about phase?
What is happening is that their audio signals are falling apart internally. The signals measure perfectly in the world of the magnitude-only, steady-state, frequency domain, but on real music what comes out of the speaker bears no resemblance to what the microphone picked up. Mention this to them, and the response is positively hostile: humans are not sensitive to phase! Understandably, they cite the killer fact: the most expensive and highly regarded speakers in the world distort the phase and ‘enhance’ the bass with acoustic resonators just like any other conventional speaker. Quite so. But they also, probably, have other positive characteristics that partially offset the phase issue. It is also undeniable that the people who own these wonder-speakers seem just as keen to ‘churn’ their equipment as everyone else…"