|
| 1 | +[PnetCDF](https://parallel-netcdf.github.io) Version 1.15.0 Release Notes (July 1, 2026) |
| 2 | +------------------------------------- |
| 3 | + |
| 4 | +* New feature |
| 5 | + + Intra-node aggregation for read requests is added. This is the complement |
| 6 | + to the write counterpart first added to version 1.14.0. Now the intra-node |
| 7 | + aggregation feature supports both write and read operations. This feature |
| 8 | + can be enabled by setting hint `nc_num_aggrs_per_node` to the desired |
| 9 | + number of aggregators per compute node. |
| 10 | + |
| 11 | +* New optimization |
| 12 | + + When creating a new file on the Lustre file system, PnetCDF will try to set |
| 13 | + the Lustre file striping count to the number of compute nodes (NUMA nodes) |
| 14 | + allocated to the MPI communicator passed to the call of `ncmpi_create()`. |
| 15 | + See the section of "New PnetCDF Hints" below for more information. |
| 16 | + |
| 17 | +* New I/O driver |
| 18 | + + A submodule named "GIO" has been added as the PnetCDF's default I/O driver. |
| 19 | + PnetCDF has been using MPI-IO as its only I/O driver since its first |
| 20 | + release. With the GIO driver, PnetCDF users now have an additional option |
| 21 | + to tune the performance. GIO is a parallel I/O library, sharing the design |
| 22 | + concept of the MPI-IO. GIO implements several performance improvement, such |
| 23 | + as: |
| 24 | + * automatically sets `cb_nodes` based on the number of compute nodes and |
| 25 | + number of MPI processes running per node. |
| 26 | + * supports hint `cb_buffer_size` to be a multiple of file striping unit |
| 27 | + size for Lustre. |
| 28 | + * supports Lustre overstriping through hint `overstriping_ratio`. See the |
| 29 | + "New PnetCDF hints" section below. |
| 30 | + |
| 31 | +* New configure options |
| 32 | + + `--disable-gio` disables the GIO driver and uses MPI-IO. The default is to |
| 33 | + enable using the GIO driver. |
| 34 | + |
| 35 | +* APIs deprecated |
| 36 | + + The "vard" family APIs introduced in version 1.6.0 are now deprecated. |
| 37 | + These are the APIs that take an argument of MPI derived data type |
| 38 | + describing the file access layout, which is used to set the fileview |
| 39 | + through a call to `MPI_File_set_view()`. The deprecation is because direct |
| 40 | + file access can be a security risk and error prone. |
| 41 | + * ncmpi_get_vard() |
| 42 | + * ncmpi_get_vard_all() |
| 43 | + * ncmpi_put_vard() |
| 44 | + * ncmpi_put_vard_all() |
| 45 | + |
| 46 | +* API semantics updates |
| 47 | + + API `ncmpi_inq_header_size()` now can be called in the define mode. This |
| 48 | + API returns the file header size with metadata defined by the time of the |
| 49 | + call. The inquired file header size can be helpful to pick proper values |
| 50 | + for arguments `h_minfree`, `v_align`, `v_minfree`, `r_align` when calling |
| 51 | + API `ncmpi__enddef()` which can allow to preserve a sufficiently large free |
| 52 | + space for file header extent and variable data sections to grow later |
| 53 | + without moving data already stored in the file, i.e. when adding new |
| 54 | + variables, dimensions, or attributes. |
| 55 | + See [PR #201](https://github.com/Parallel-NetCDF/PnetCDF/pull/201). |
| 56 | + |
| 57 | +* New error code |
| 58 | + + `NC_EDRIVER` indicates an invalid PnetCDF I/O driver is set in I/O hint, |
| 59 | + `nc_driver`. The current valid drivers are "gio" and "mpiio". |
| 60 | + + `NC_EFSTYPE` indicates a PnetCDF internal error when an invalid file system |
| 61 | + type is detected. |
| 62 | + + `NC_EFILEVIEW` indicates a PnetCDF internal error when creating an MPI |
| 63 | + fileview whose offsets violate the MPI standard requirement of being in a |
| 64 | + monotonically non-decreasing order. |
| 65 | + |
| 66 | +* New PnetCDF hints |
| 67 | + + `nc_data_move_chunk_size` -- When adding new data objects into an existing |
| 68 | + file, the data sections may need to be moved to a higher file offset. The |
| 69 | + movement is performed in chunks. This hint allows users to customized the |
| 70 | + chunk size. The default is 1048576 bytes, i.e. 1 MiB. |
| 71 | + See [PR #203](https://github.com/Parallel-NetCDF/PnetCDF/pull/203). |
| 72 | + + `file_striping` -- When creating a new file on the Lustre file system, this |
| 73 | + hint advises PnetCDF to set the new file's striping configuration. The hint |
| 74 | + value is either "auto" or "inherit". The default is "auto". The former sets |
| 75 | + the new file's striping unit size to 1 MiB and striping count to the number |
| 76 | + of compute nodes found in the MPI communicator passed to the call of |
| 77 | + `ncmpi_create()`. The latter sets the striping to inherit the new file's |
| 78 | + parent folder's settings, if the folder's striping has been set. Note that |
| 79 | + if users also set the MPI-IO hint `striping_factor` or `striping_unit`, |
| 80 | + then these MPI-IO hints will take a higher precedence. |
| 81 | + See [PR #222](https://github.com/Parallel-NetCDF/PnetCDF/pull/222). |
| 82 | + + `nc_driver` -- To select an internal I/O driver. The value is either string |
| 83 | + "gio" or "mpiio". The default is "gio", which advises PnetCDF to use the |
| 84 | + GIO driver, an external library built as a sub-module of PnetCDF. The other |
| 85 | + is "mpiio", advising PnetCDF to use the MPI-IO driver. |
| 86 | + + `overstriping_ratio` -- This hint is passed to the GIO driver and instructs |
| 87 | + it to make use of Lustre's overstriping feature which allows to create a |
| 88 | + new file to have more than one stripe per OST. For instance, when hints |
| 89 | + `striping_factor` and `overstriping_ratio` are set to 16 and 2, |
| 90 | + respectively, the newly created file will be striped across 16/2=8 OSTs. |
| 91 | + The advantage of Lustre overstriping is to allow a file's striping count to |
| 92 | + be larger than the total number of OTS available on the system. |
| 93 | + |
| 94 | +* New run-time environment variables |
| 95 | + + `GIO_ONLY` - when testing PnetCDF by running command `make check` and `make |
| 96 | + ptest`, setting this environment variable will run all the tests using the |
| 97 | + default GIO driver and skip MPI-IO driver. Without setting this environment |
| 98 | + variable, the default is to test both I/O drivers. Note this environment |
| 99 | + variable only affect when testing PnetCDF and takes no effect on PnetCDF |
| 100 | + library itself. |
| 101 | + |
| 102 | +* Build recipes |
| 103 | + + `doc/NERSC.md` - note for building PnetCDF on machines at NERSC |
| 104 | + + `doc/ALCF.md` - note for building PnetCDF on machines at ALCF |
| 105 | + |
| 106 | +* Updated utility programs |
| 107 | + + `pnetcdf_version` - add information about MPI compiler vendors, supported |
| 108 | + MPI standard version, names of base compilers, and LMOD PrgEnv module |
| 109 | + loaded if available. |
| 110 | + + `ncmpidiff` now will error out when the two input file names are identical. |
| 111 | + |
| 112 | +* Other updates: |
| 113 | + + Running `make check` and `make ptest` takes much longer time to complete. |
| 114 | + This is because the testing mechanism has been revised to become more |
| 115 | + thorough than before. |
| 116 | + |
| 117 | +* Bug fixes |
| 118 | + + Fix problem of building PnetCDF shared library when using NVIDIA HPC |
| 119 | + compilers, nvc, nvc++, and nvfortran. |
| 120 | + + Fix data movement when new record variables are added to an existing file |
| 121 | + that does not change the starting offset of record variable section. |
| 122 | + See [PR #199](https://github.com/Parallel-NetCDF/PnetCDF/pull/199). |
| 123 | + |
| 124 | +* New I/O benchmarks |
| 125 | + + benchmarks/C/indep_data_obj_create.c - It evaluates the performance of |
| 126 | + creating a large number of data objects (dimensions and variables) in an |
| 127 | + output file. This benchmark is used in a paper published in the PDSW |
| 128 | + Workshop of SC 2025. DOI: 10.1145/3731599.3767512 |
| 129 | + |
| 130 | +* New test programs |
| 131 | + + test/testcases/tst_grow_data.c -- It tests the case when adding new |
| 132 | + variables by re-entering the define mode multiple time without causing the |
| 133 | + file header extent to grow. It also tests a case when adding a new record |
| 134 | + variable that does not change the starting offset of the record variable |
| 135 | + section in the file. |
| 136 | + |
0 commit comments