FileNotFoundError

[pdebuyl]

Describe the bug

Satpy errors with a FileNotFoundError when using the xrit decompression feature for the reader seviri_l1b_hrit.

To Reproduce

import satpy
import argparse

SEVIRI_CHANNELS = ['VIS006', 'VIS008', 'IR_016', 'IR_039', 'WV_062', 'WV_073', 'IR_087',
                   'IR_097', 'IR_108', 'IR_120', 'IR_134']

parser = argparse.ArgumentParser()
parser.add_argument('input', nargs='+')
args = parser.parse_args()
composite = 'natural_color'

sev_scene = satpy.Scene(reader='seviri_l1b_hrit', filenames=args.input)
sev_scene.load(SEVIRI_CHANNELS)
sev_scene.load([composite])

sev_scene.show(composite)

Calling the code above with a list of HRIT files as argument produces sometimes the traceback below. Sometimes (for the same input) the code runs fine.

Expected behavior

satpy should place the decompressed segments in the proper location.

Actual results

Traceback (most recent call last):
  File "/tmp/bug_for_satpy/seviri_show.py", line 18, in <module>
    sev_scene.show(composite)
  File "/sdata/anaconda/envs/py311/lib/python3.11/site-packages/satpy/scene.py", line 1012, in show
    img.show()
  File "/sdata/anaconda/envs/py311/lib/python3.11/site-packages/trollimage/xrimage.py", line 1464, in show
    self.pil_image().show()
    ^^^^^^^^^^^^^^^^
  File "/sdata/anaconda/envs/py311/lib/python3.11/site-packages/trollimage/xrimage.py", line 923, in pil_image
    img = img.compute()
          ^^^^^^^^^^^^^
  File "/sdata/anaconda/envs/py311/lib/python3.11/site-packages/dask/base.py", line 342, in compute
    (result,) = compute(self, traverse=False, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sdata/anaconda/envs/py311/lib/python3.11/site-packages/dask/base.py", line 628, in compute
    results = schedule(dsk, keys, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sdata/anaconda/envs/py311/lib/python3.11/site-packages/satpy/readers/hrit_base.py", line 342, in _read_data
    return HRITSegment(filename, mda).read_data()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sdata/anaconda/envs/py311/lib/python3.11/site-packages/satpy/readers/hrit_base.py", line 373, in read_data
    data = self._read_data_from_file()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sdata/anaconda/envs/py311/lib/python3.11/site-packages/satpy/readers/hrit_base.py", line 382, in _read_data_from_file
    return self._read_data_from_disk()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sdata/anaconda/envs/py311/lib/python3.11/site-packages/satpy/readers/hrit_base.py", line 392, in _read_data_from_disk
    return np.fromfile(filename,
           ^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/H-000-MSG3__-MSG3________-VIS006___-000005___-202401261100-__'

Environment Info:

• OS: Linux
• Satpy Version: 0.44.0
• PyResample Version: 1.27.1
• Readers and writers dependencies (when relevant): seviri_l1b_hrit: ok (from a run of from satpy.utils import check_satpy; check_satpy()).

Additional context

When the error occurs, the missing segment is found in the current working directory (and is not deleted) instead of going to /tmp and being deleted.

[sfinkens]

Does it also happen when you explicitly set the temporary directory?

satpy.config.set(tmp_dir="my_temp_dir")

[pdebuyl]

It seems to help for this specific problem. However, I realize that in some case there is a broader issue with os.chdir. I attempt to create an output file (i.e. out/name_of_file.h5) and have an error. The code is actually trying to write to /tmp/out/name_of_file.h5 and that does not work if /tmp/out does not exit.

In the case of the XRIT decompressor, it indeed uses as os.chdir as the program xRITDecompress output only in the current directory so you need it. Maybe passing cwd to subprocess.Popen would fix this in the specific case of XRIT.