I have a list of
42000 numpy arrays (each array is
240x240) that I want to save to a file for use in another python script.
I've tried using
numpy.savez_compressed and I run into Memory Errors (I have 16gb DDR3). I read that
hdf5 which is commonly used for deep learning stuff cannot save lists so I'm kind of stuck.
Does anyone have any idea how I can save my data?
EDIT: I previously saved this data into a numpy array onto disk using
np.save and it was around 2.3GB but my computer couldn't always handle it so it would sometimes crash if I tried to process it. I read lists might be better so I have moved to using lists of numpy arrays