data problem after upgrading

[t-reents]

Hi,

this is somehow a follow-up of #6017 but I also post this in a separate discussion as I am not sure whether #6017 is the reason for the following problem.
With respect to #6017, I was able to rerun the verdi storage maintain --full command and it seems like it worked after deleting the temporary pack -1. All loose files were packed, the repacking was applied and afterwards the loose files were deleted.

I upgraded AiiDA to version 2.3 (I was still running 2.0.4) and ran the storage migrate command. Afterwards, I checked the availability of my data and in case of the PDOS, I encountered the following error.

I tried to load the separate pdos_arrays using projections_up.get_array("pdos_array_i") and this works for the majority of indices i (in this example, only 2 out of 64 arrays failed). However, for the different nodes I find (different) indices where the aforementioned error occurs.

There were just two things that I noticed afterwards and they might be related:

1. Due to a RabbitMQ version conflict, the daemon is not running at the moment but I think this should not affect the interaction with the data
2. After upgrading AiiDA and before running the storage migrate command, I ran verdi profile show to check my profile name again. I did not notice that this already migrates the configuration file. I compared the back-up configuration file with the one that was present before running storage migrate and they show no difference. Therefore, I would expect that this should not cause the problems but maybe something else happens in the background of which I am not aware.

Based on the observation that the error occurs only for some arrays and only for the pdos arrays, I would think that there is some problem with the corresponding files. I think it is a little bit confusing that this happens for different nodes (with different creation time) and just for specific files. If this would be related to the points that I mentioned above, I would assume that this should happen to all arrays of a given node.

Maybe someone has some thoughts on this as I am trying to avoid a reinstallation.

[mbercx]

@t-reents did you manage to fix the issue? I think I once ran into something, and it was related to my numpy version. If the problem is solved, could you tell us how you solved it? Thanks!

EDIT: Ah, only now saw your note about closing this dicussion in https://github.com/orgs/aiidateam/discussions/6017 😅

[t-reents]

@mbercx sorry! I should have mentioned it here, too. I briefly tried a different numpy version and it didn't work. As I mentioned in #6017, one pack file is actually missing (the largest index is 43 but 25 is not present), so I think that the error is raised due to the missing file. However, I would expect a different error when the whole pack file is missing.