Running outlier_detection with in_memory=False reads all models into memory

[stscijgbot-jp]

Issue JP-3619 was created on JIRA by Brett Graham:

Running outlier detection with the recent main commit: 13e0927

With an association containing 33 members (with no custom grouping or tweakreg_catalog entries for any member).

And with "in_memory=False" results in the following memory usage:

The attached graph shows that even with "in_memory=False" all models are read into memory (the largest plateau in the graph). The main cause is that the implementation of ModelContainer.models_grouped (which is used in resample) opens all models and keeps them in memory.

[jdavies-st]

I will add here that when in_memory=True for outlier_detection, it still writes out the blot files and then reads them back in.

2024-05-15 15:48:21,169 - stpipe.Image3Pipeline.outlier_detection - INFO - Blotting (2048, 2048) <-- (6321, 9320)
2024-05-15 15:48:21,455 - stpipe.Image3Pipeline.outlier_detection - INFO - Saved model in jw02321001009_02101_00006_nrcblong_c1001_blot.fits

So it is writing to disk when it should be doing everything in memory. This is with release 1.14.0.