Skip to content

Double asterisks in path used by PMP was not supported by xarray #601

@minxu74

Description

@minxu74

PMP used double asterisks in its file paths to expand directories and files, for example, the infile: /global/cfs/projectdirs/m4931/gsharing/css03_data/CMIP6/CMIP/INM/INM-CM5-0/amip/r1i1p1f1/**/ta_*12.nc was used, then xr.open_mfdataset was used to open it. It seemed that xarray does not support it. The glob expansion is needed before the file was opened by xarray.

start and end are  1981-01   2005-12                                                                                      │
│ variable list:  ['ta']                                                                                                    │
│ ver: v20260319                                                                                                            │
│ var: ta                                                                                                                   │
│ infile: /global/cfs/projectdirs/m4931/gsharing/css03_data/CMIP6/CMIP/INM/INM-CM5-0/amip/r1i1p1f1/**/ta_*12.nc             │
│ outfile:                                                                                                                  │
│ /pscratch/sd/m/minxu/mytest_ref_2026-03-14/config/scratch/pmp/annual-cycle/09e02b3b305852e52faac1746181d4d39d223e8c/ta_IN │
│ M-CM5-0_amip_r1i1p1f1_clims.nc                                                                                            │
│ outfilename: None                                                                                                         │
│ outpath: None                                                                                                             │
│ ver: v20260319                                                                                                            │
│ infilename: ta_*12.nc                                                                                                     │
│ Traceback (most recent call last):                                                                                        │
│   File                                                                                                                    │
│ "/pscratch/sd/m/minxu/mytest_ref_2026-03-14/config/software/conda/pmp-a0cd98ec96f7c858f788439a4f0b4951fe2d7a55/bin/pcmdi_ │
│ compute_climatologies.py", line 91, in <module>                                                                           │
│     calculate_climatology(                                                                                                │
│   File                                                                                                                    │
│ "/pscratch/sd/m/minxu/mytest_ref_2026-03-14/config/software/conda/pmp-a0cd98ec96f7c858f788439a4f0b4951fe2d7a55/lib/python │
│ 3.10/site-packages/pcmdi_metrics/mean_climate/lib/calculate_climatology.py", line 33, in calculate_climatology            │
│     d = xcdat_open(infile, data_var=var)                                                                                  │
│   File                                                                                                                    │
│ "/pscratch/sd/m/minxu/mytest_ref_2026-03-14/config/software/conda/pmp-a0cd98ec96f7c858f788439a4f0b4951fe2d7a55/lib/python │
│ 3.10/site-packages/pcmdi_metrics/io/xcdat_openxml.py", line 56, in xcdat_open                                             │
│     ds = xc.open_mfdataset(                                                                                               │
│   File                                                                                                                    │
│ "/pscratch/sd/m/minxu/mytest_ref_2026-03-14/config/software/conda/pmp-a0cd98ec96f7c858f788439a4f0b4951fe2d7a55/lib/python │
│ 3.10/site-packages/xcdat/dataset.py", line 234, in open_mfdataset                                                         │
│     ds = xr.open_mfdataset(                                                                                               │
│   File                                                                                                                    │
│ "/pscratch/sd/m/minxu/mytest_ref_2026-03-14/config/software/conda/pmp-a0cd98ec96f7c858f788439a4f0b4951fe2d7a55/lib/python │
│ 3.10/site-packages/xarray/backends/api.py", line 1597, in open_mfdataset                                                  │
│     raise OSError("no files to open")                                                                                     │
│ OSError: no files to open                                                                                                 │
│                                                                                                                           │
│ stderr: None    

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions