disabling memory mapping as default in asdf 4.0.0

[braingram]

Memory mapping (controlled by the copy_arrays argument to asdf.open) can be useful but increases the risk of segfaults and other unrecoverable errors.

We are considering deprecating copy_arrays (possibly in favor of a memory_map argument) and making the new default that a file will not use memory mapping. For a user who wants to use memory mapping they will have to override this default (perhaps by calling asdf.open(fn, memory_map=True) or something similar).

[CagtayFabry]

Would this mean memory_map=False - the new default - would load all data into memory - as per the current copy_arrays=True?
Or is there some other mechanism that will load arrays only on request?

The new memory_map argument name is certainly more descriptive compared to copy_arrays 👍

[braingram]

Sort of :)

When a file is first opened (as long as lazy_load is kept as the default True) all arrays would not be read into memory (until they are accessed). With memory_map=False accessing an array would load that array into memory. The in-memory array would be re-used for subsequent accesses to the same array. However, the first and last blocks are often read into memory when a file is opened (to partially verify the block index).

[CagtayFabry]

Ah yes, somehow I forgot about the lazy_load option during all this, thanks!