Import with wildcard not working on Windows

[dagewa]

This works:

dials.import C:\Users\fcx32934\.cache\dials_data\thaumatin_i04\th_8_2_0001.cbf

but this does not:

dials.import C:\Users\fcx32934\.cache\dials_data\thaumatin_i04\*.cbf
DIALS (2018) Acta Cryst. D74, 85-97. https://doi.org/10.1107/S2059798317017235
DIALS 3.dev.366-gf8623246a
Traceback (most recent call last):
  File "C:\Users\fcx32934\sw\cctbx\build\..\modules\dials\command_line\dials_import.py", line 954, in <module>
    run()
  File "C:\Users\fcx32934\sw\cctbx\conda_base\lib\contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "C:\Users\fcx32934\sw\cctbx\build\..\modules\dials\command_line\dials_import.py", line 950, in run
    script.run(args)
  File "C:\Users\fcx32934\sw\cctbx\build\..\modules\dials\command_line\dials_import.py", line 781, in run
    params, options = self.parser.parse_args(args=args, show_diff_phil=False)
  File "C:\Users\fcx32934\sw\cctbx\modules\dials\util\options.py", line 842, in parse_args
    params, args = self._phil_parser.parse_args(
  File "C:\Users\fcx32934\sw\cctbx\modules\dials\util\options.py", line 547, in parse_args
    importer = Importer(
  File "C:\Users\fcx32934\sw\cctbx\modules\dials\util\options.py", line 206, in __init__
    self.unhandled = self.try_read_experiments_from_images(
  File "C:\Users\fcx32934\sw\cctbx\modules\dials\util\options.py", line 277, in try_read_experiments_from_images
    experiments = ExperimentListFactory.from_filenames(
  File "C:\Users\fcx32934\sw\cctbx\modules\dxtbx\model\experiment_list.py", line 525, in from_filenames
    for db in DataBlockFactory.from_filenames(
  File "C:\Users\fcx32934\sw\cctbx\modules\dxtbx\datablock.py", line 1483, in from_filenames
    importer = DataBlockFilenameImporter(
  File "C:\Users\fcx32934\sw\cctbx\modules\dxtbx\datablock.py", line 915, in __init__
    format_class = find_format.find_format(filename)
  File "C:\Users\fcx32934\sw\cctbx\modules\dxtbx\datablock.py", line 326, in find_format
    self._format_class = get_format_class_for_file(filename)
  File "C:\Users\fcx32934\sw\cctbx\modules\dxtbx\format\Registry.py", line 124, in get_format_class_for_file
    if scheme in format_class.schemes and format_class.understand(image_file):
  File "C:\Users\fcx32934\sw\cctbx\modules\dxtbx\format\FormatCBF.py", line 26, in understand
    with FormatCBF.open_file(image_file, "rb") as fh:
  File "C:\Users\fcx32934\sw\cctbx\modules\dxtbx\format\Format.py", line 559, in open_file
    return cls.get_cache_controller().check(filename, fh_func)
  File "C:\Users\fcx32934\sw\cctbx\modules\dxtbx\filecache_controller.py", line 69, in check
    self._cache = dxtbx.filecache.lazy_file_cache(open_method())
OSError: [Errno 22] Invalid argument: 'C:\\Users\\fcx32934\\.cache\\dials_data\\thaumatin_i04\\*.cbf'

[dagewa]

Windows shell does not expand wildcards 🤔

[dagewa]

Can we glob if a * makes it into the OptionParser? (Rapidly realising this should have been a dxtbx issue...)

[graeme-winter]

Up to the point where some brightspark calls their images haha*_1_master.h5 this will work fine 🙂

Your proposal would be something like:

if "*" in arg:
  glob()

?

[graeme-winter]

Also OptionParser is a dials thing not dxtbx -> this is the right home

[dagewa]

if "*" in arg:
 glob()

I don't know if this is a good idea, if it should be under an if os.name == "nt" block or what. I do feel it would be nice to allow the wildcard form of import on Windows, so that processing instructions and tutorials work similarly to POSIX-ey systems

[elena-pascal]

It is expanding on getting experimental data.

[dagewa]

Ah nice. I wonder why it is broken then 🤔

[elena-pascal]

next function is not expanding though

[rjgildea]

Ah nice. I wonder why it is broken then 🤔

Because:

>>> urlparse(r"C:\Users\fcx32934\.cache\dials_data\thaumatin_i04\*.cbf").scheme
'c'

I.e. it's the same issue as resolved here: cctbx/dxtbx@219e0c7

[dagewa]

Easily fixed in that case, thanks.

[dagewa]

There's 8 instances of urlparse(something).scheme across dxtbx and DIALS. Is there any reason not to replace these all with a function that includes the if "://" in something else "" treatment?