Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

numpy.memmap not returned when mmap_mode passed to memory.cache decorator #1522

Open
tinducvo opened this issue Nov 17, 2023 · 0 comments
Open

Comments

@tinducvo
Copy link

Evening!

I found:

  • numpy.memmap IS returned when mmap_mode passed to joblib.Memory constructor
  • numpy.memmap IS NOT returned when mmap_mode passed to memory.cache decorator
import numpy
import joblib
from tempfile import TemporaryDirectory


def numpy_function(length: int) -> numpy.ndarray:
    return numpy.zeros((length, length))


def test_memory_mmap_mode():
    with TemporaryDirectory() as directory:
        mmap_memory = joblib.Memory(directory, mmap_mode="r", verbose=0)
        cached_numpy_function = mmap_memory.cache(numpy_function)
        assert type(cached_numpy_function(1)) == numpy.memmap


def test_cache_mmap_mode():
    with TemporaryDirectory() as directory:
        mmap_memory = joblib.Memory(directory, verbose=0)
        cached_numpy_function = mmap_memory.cache(numpy_function, mmap_mode="r")  # type: ignore
        assert type(cached_numpy_function(1)) == numpy.memmap

I expected for both cases to return a numpy.memmap object. Is this expected behaviour?
If so, perhaps MemorizedFunc needs a modified location argument?

    def __init__(self, location=None, backend='local',
                 mmap_mode=None, compress=False, verbose=1, bytes_limit=None,
                 backend_options=None):
        ...
        self.store_backend = _store_backend_factory(
            backend, location, verbose=self._verbose,
            backend_options=dict(compress=compress, mmap_mode=mmap_mode,
                                 **backend_options))
    def cache(self, func=None, ignore=None, verbose=None, mmap_mode=False,
              cache_validation_callback=None):
        ...
        return MemorizedFunc(
            func, location=self.store_backend, backend=self.backend,
            ignore=ignore, mmap_mode=mmap_mode, compress=self.compress,
            verbose=verbose, timestamp=self.timestamp,
            cache_validation_callback=cache_validation_callback
        )
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant