Retrieving data with intake esgf fails

[schlunma]

Describe the bug

While testing #2946 with

esmvaltool run --max_parallel_tasks=1 ~/ESMValTool/esmvaltool/recipes/examples/recipe_preprocessor_derive_test.yml

(minus some commented-out datasets where the data could not be found)

I ran into

File "/home/manuel/ESMValCore/esmvalcore/_main.py", line 156, in process_recipe
    recipe.run()
    ~~~~~~~~~~^^
  File "/home/manuel/ESMValCore/esmvalcore/_recipe/recipe.py", line 1329, in run
    file.prepare()
    ~~~~~~~~~~~~^^
  File "/home/manuel/ESMValCore/esmvalcore/io/intake_esgf.py", line 125, in prepare
    self.catalog.to_path_dict(minimal_keys=False)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/manuel/ESMValCore/esmvalcore/io/intake_esgf.py", line 96, in to_path_dict
    self._result[key] = super().to_path_dict(**kwargs)
                        ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/home/manuel/micromamba/envs/esm/lib/python3.13/site-packages/intake_esgf/catalog.py", line 655, in to_path_dict
    raw_infos = self._get_file_info(separator=separator, quiet=quiet)
  File "/home/manuel/micromamba/envs/esm/lib/python3.13/site-packages/intake_esgf/catalog.py", line 563, in _get_file_info
    index_infos = list(
        tqdm(
    ...<8 lines>...
        )
    )
  File "/home/manuel/micromamba/envs/esm/lib/python3.13/site-packages/tqdm/std.py", line 1181, in __iter__
    for obj in iterable:
               ^^^^^^^^
  File "/home/manuel/micromamba/envs/esm/lib/python3.13/multiprocessing/pool.py", line 873, in next
    raise value
  File "/home/manuel/micromamba/envs/esm/lib/python3.13/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ~~~~^^^^^^^^^^^^^^^
  File "/home/manuel/micromamba/envs/esm/lib/python3.13/site-packages/intake_esgf/catalog.py", line 512, in _get_file_info
    info = index.get_file_info(list(dataset_ids.keys()), **search_facets)
  File "/home/manuel/micromamba/envs/esm/lib/python3.13/site-packages/intake_esgf/core/globus.py", line 182, in get_file_info
    info["path"] = base.get_content_path(content)
                   ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/home/manuel/micromamba/envs/esm/lib/python3.13/site-packages/intake_esgf/base.py", line 614, in get_content_path
    path = match.group(1).replace(project.lower(), project)
           ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'replace'

when trying to use intake_esgf.

Please attach

• Recipe: recipe_preprocessor_derive_test.yml
• Log: main_log_debug.txt

[bouweandela]

I suspect this with would be fixed by esgf2-us/intake-esgf#160, but please open an issue with intake-esgf.

[schlunma]

I don't feel familiar enough with intake-esgf to open a descriptive issue yet. If you're 100% certain that it's an issue on their side, we can ignore this issue for now and I can move on with the testing of #2946 with local data.

[schlunma]

Moving this to v2.15.0.

[valeriupredoi]

Manu, Nathan at intake-esgf is a great bloke - you can always open an issue and expect good treatment from him, don't be afraid. As long as you give him a decent MRE 🍺