Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add precomputations to HAAQI #363

Open
groadabike opened this issue Feb 8, 2024 · 4 comments · May be fixed by #364
Open

Add precomputations to HAAQI #363

groadabike opened this issue Feb 8, 2024 · 4 comments · May be fixed by #364
Assignees
Labels
Cadenza enhancement New feature or request refactor Refactor of existing code

Comments

@groadabike
Copy link
Contributor

groadabike commented Feb 8, 2024

There are two banches working with HAAQI to improve the metric. However, after several analysis, I realised that many of the slowest parts can be improved by precomputing values. These values correspond computations that are not related to the signals. Additionally, as HAAQI works at 24000 Hz, many of the computations that depend on the sample rate can also be precomputed:

  1. Filter coefficients in eb.env_compress_basilar_membrane:
    Depends on a constant (flp = 800) and sample rate

  2. Filter coefficients in eb.middle_ear
    Depends on sample rate

  3. Group delay vector in eb.group_delay_compensate
    These are constant

  4. Sin and Cos in eb.gammatone_bandwidth_demodulation
    This is a recursive function (every 300 samples) that depends on the length of the audio signal (i.e., 24000 samples per second of audio) and the sample rate. However, considering values to the 10th decimal place, the function is periodic.

image

image

@groadabike groadabike added the enhancement New feature or request label Feb 8, 2024
@groadabike groadabike self-assigned this Feb 8, 2024
@jonbarker68
Copy link
Contributor

Yes. This is all true. The solution is probably to make an OOP version of these metrics. If you look at the MSBG code you will see that there is an Ear class which has a Cochlea object. A set_audiogram method will build a new Cochlea for the Ear. This allows all the precomputation to be done in the object constructors so that the Ear's process() method doesn't repeat the parts that are signal independent. It would be great if the Kates' models were all rewritten in this style.

Note, also that the MSBG code has its own implementation of a gammatone filterbank. See gammatome_filterbank in cochlea.py. This looks like the cascaded filter implementation. Here it is just a loop of lfilter commands, so no use of any njit code. This would probably be much faster if we could show that it gave similar results. But I haven't benchmarked it.

@groadabike
Copy link
Contributor Author

I checked the periodicity and found that some are not "perfect" I mean, you can find the period (no more than 300 samples) However, the values for some bands are the same in each period with variation in the 15th decimal place but, for other bands, the variations are in the 3rd or 4th decimal place. repeating the first period resulted in a reduction of 0.02 HAAQI in the example I was running from 0.12 to 0.10.

Regarding refactoring HAAQI, there are many computations that can be done in the constructor and others that could be saved:

  1. There are many constants and 3 or 4 filters' numerator and denominator that depends only on sample rate that can be precomputed and saved as constant in the class.

  2. The sin and cos for the gammatone depends only on the length of the signal, so they could be computed only the first time the metric is called. Pre-saving this computation for a specific max length can be another option but, may increase the clarity package size. We could add this as part of the constructor and compute only if precomputations are not given.

  3. Profiling the original HAAQI, one third of the computation are the number of correlations.

@groadabike
Copy link
Contributor Author

groadabike commented Feb 14, 2024

HAAQI's Ear model is hard to separate as the MSGB does.
The main issue is an alignment step that run for all the bands (32 times)

# Correct for the delay between the reference and output
processed_cochlear_compression = envelope_align(
reference_cochlear_compression, processed_cochlear_compression
) # Align processed envelope to reference
processed_b[n] = envelope_align(
reference_b[n], processed_b[n]
) # Align processed BM motion to reference

In this part of the Ear model, the processed signal is aligned with the reference signal. This makes this step the reference signal's ear model computation a requirement of the processed signal's ear model computation.

As we can't really make the ear model independent of the signals we could have the following structure.
Note, from now on, I will called the processed signal as enhanced signal to avoid confusions

EarModel Class:

  • __init__(): In the constructor we can compute all the parameters that depends on frequency sample to avoid repetitions.
  • set_audiogram(): Here we can compute parameters that depend on audiogram that are common to reference and enhanced
  • process_reference(): Here we compute the reference signal ear model.
  • process_enhanced(): Here we compute the enhanced signal ear model. We can condition this method to the execution of process_reference()

To me, it is not an ideal structure but I can't think in another right now. @jonbarker68, any suggestions?

@groadabike
Copy link
Contributor Author

groadabike commented Feb 14, 2024

Analysing the ear model, it has 2 functions using almost all the execution time: envelope_align and gammatone_basilar_membrane.

The next figure show the execution time for 15-second signals. The total execution takes 8.3 seconds.
image

  • The envelope_align takes 2.6 seconds. However, it may not be necessary to use the whole signal for the alignment, maybe we can select one second to find the delay, which it is not expected to be more then 200 samples
  • The gammatone_basilar_membrane is the green box next to envelope_align and it takes 1.7 seconds to execute.

This affects all three metrics, HAAQI, HASQI and HASPI

@groadabike groadabike linked a pull request Feb 20, 2024 that will close this issue
2 tasks
@groadabike groadabike added refactor Refactor of existing code Cadenza labels Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Cadenza enhancement New feature or request refactor Refactor of existing code
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants