Handling timezone information absence

[lnett99]

What happened?

Hello,

I am currently trying to read an NWB file created with MATLAB in Python. While trying to process the file, I've encountered a problem related to the lack of timezone information in the NWB file.
Is there a way to read a file without this information?

Any help you could provide on this matter would be greatly appreciated. Thanks in advance for your time and help.

Steps to Reproduce

filepath = "C:/Users/Lenovo/Documents/test.nwb"
io = NWBHDF5IO(filepath, mode="r", load_namespaces=True)
nwbfile = io.read()

Traceback

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File ~\anaconda3\Lib\site-packages\hdmf\build\objectmapper.py:1258, in ObjectMapper.construct(self, **kwargs)
   1257 try:
-> 1258     obj = self.__new_container__(cls, builder.source, parent, builder.attributes.get(self.__spec.id_key()),
   1259                                  **kwargs)
   1260 except Exception as ex:

File ~\anaconda3\Lib\site-packages\hdmf\build\objectmapper.py:1271, in ObjectMapper.__new_container__(self, cls, container_source, parent, object_id, **kwargs)
   1269 # obj has been created and is in construction mode, indicating that the object is being constructed by
   1270 # the automatic construct process during read, rather than by the user
-> 1271 obj.__init__(**kwargs)
   1272 obj._in_construct_mode = False  # reset to False to indicate that the construction of the object is complete

File ~\anaconda3\Lib\site-packages\hdmf\utils.py:664, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
    663 pargs = _check_args(args, kwargs)
--> 664 return func(args[0], **pargs)

File ~\anaconda3\Lib\site-packages\pynwb\file.py:479, in NWBFile.__init__(self, **kwargs)
    478 elif timestamps_reference_time.tzinfo is None:
--> 479     raise ValueError("'timestamps_reference_time' must be a timezone-aware datetime object.")
    481 # convert file_create_date to list and add timezone if missing

ValueError: 'timestamps_reference_time' must be a timezone-aware datetime object.

The above exception was the direct cause of the following exception:

ConstructError                            Traceback (most recent call last)
Cell In[10], line 4
      2 # Open the file in read mode "r",
      3 io = NWBHDF5IO(filepath, mode="r", load_namespaces=True)
----> 4 nwbfile = io.read()

File ~\anaconda3\Lib\site-packages\hdmf\utils.py:664, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
    662 def func_call(*args, **kwargs):
    663     pargs = _check_args(args, kwargs)
--> 664     return func(args[0], **pargs)

File ~\anaconda3\Lib\site-packages\pynwb\__init__.py:304, in NWBHDF5IO.read(self, **kwargs)
    301         raise TypeError("NWB version %s not supported. PyNWB supports NWB files version 2 and above." %
    302                         str(file_version_str))
    303 # read the file
--> 304 file = super().read(**kwargs)
    305 return file

File ~\anaconda3\Lib\site-packages\hdmf\backends\hdf5\h5tools.py:481, in HDF5IO.read(self, **kwargs)
    478     raise UnsupportedOperation("Cannot read from file %s in mode '%s'. Please use mode 'r', 'r+', or 'a'."
    479                                % (self.source, self.__mode))
    480 try:
--> 481     return super().read(**kwargs)
    482 except UnsupportedOperation as e:
    483     if str(e) == 'Cannot build data. There are no values.':  # pragma: no cover

File ~\anaconda3\Lib\site-packages\hdmf\utils.py:664, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
    662 def func_call(*args, **kwargs):
    663     pargs = _check_args(args, kwargs)
--> 664     return func(args[0], **pargs)

File ~\anaconda3\Lib\site-packages\hdmf\backends\io.py:60, in HDMFIO.read(self, **kwargs)
     57 if all(len(v) == 0 for v in f_builder.values()):
     58     # TODO also check that the keys are appropriate. print a better error message
     59     raise UnsupportedOperation('Cannot build data. There are no values.')
---> 60 container = self.__manager.construct(f_builder)
     61 container.read_io = self
     62 if self.herd_path is not None:

File ~\anaconda3\Lib\site-packages\hdmf\utils.py:664, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
    662 def func_call(*args, **kwargs):
    663     pargs = _check_args(args, kwargs)
--> 664     return func(args[0], **pargs)

File ~\anaconda3\Lib\site-packages\hdmf\build\manager.py:284, in BuildManager.construct(self, **kwargs)
    280     result = self.__type_map.construct(builder, self, parent)
    281 else:
    282     # we are at the top of the hierarchy,
    283     # so it must be time to resolve parents
--> 284     result = self.__type_map.construct(builder, self, None)
    285     self.__resolve_parents(result)
    286 self.prebuilt(result, builder)

File ~\anaconda3\Lib\site-packages\hdmf\utils.py:664, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
    662 def func_call(*args, **kwargs):
    663     pargs = _check_args(args, kwargs)
--> 664     return func(args[0], **pargs)

File ~\anaconda3\Lib\site-packages\hdmf\build\manager.py:795, in TypeMap.construct(self, **kwargs)
    793     raise ValueError('No ObjectMapper found for builder of type %s' % dt)
    794 else:
--> 795     return obj_mapper.construct(builder, build_manager, parent)

File ~\anaconda3\Lib\site-packages\hdmf\utils.py:664, in docval.<locals>.dec.<locals>.func_call(*args, **kwargs)
    662 def func_call(*args, **kwargs):
    663     pargs = _check_args(args, kwargs)
--> 664     return func(args[0], **pargs)

File ~\anaconda3\Lib\site-packages\hdmf\build\objectmapper.py:1262, in ObjectMapper.construct(self, **kwargs)
   1260 except Exception as ex:
   1261     msg = 'Could not construct %s object due to: %s' % (cls.__name__, ex)
-> 1262     raise ConstructError(builder, msg) from ex
   1263 return obj

ConstructError: (root GroupBuilder {'attributes': {'namespace': 'core', 'neurodata_type': 'NWBFile', 'nwb_version': '2.6.0', 'object_id': '0ef5b482-2997-4584-a162-cc0f384239e0'}, 'groups': {'acquisition': root/acquisition GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}, 'analysis': root/analysis GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}, 'general': root/general GroupBuilder {'attributes': {}, 'groups': {'devices': root/general/devices GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}, 'extracellular_ephys': root/general/extracellular_ephys GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}, 'intracellular_ephys': root/general/intracellular_ephys GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}, 'optogenetics': root/general/optogenetics GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}, 'optophysiology': root/general/optophysiology GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}}, 'datasets': {'experiment_description': root/general/experiment_description DatasetBuilder {'attributes': {}, 'data': 'Electrophysiological recordings and behavioral data from the Concept Museum task performed in patients with drug-resistant epilepsy implanted with Behnke-Fried hybrid depth electrodes in the human medial temporal lobe'}, 'institution': root/general/institution DatasetBuilder {'attributes': {}, 'data': 'Department of Epileptology, University Hospital Bonn'}, 'keywords': root/general/keywords DatasetBuilder {'attributes': {}, 'data': <StrDataset for HDF5 dataset "keywords": shape (1,), type "|O">}, 'notes': root/general/notes DatasetBuilder {'attributes': {}, 'data': 'Session start time has been set to January 1st 1900 to avoid disclosure of patient information'}}, 'links': {}}, 'intervals': root/intervals GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}, 'processing': root/processing GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}, 'scratch': root/scratch GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}, 'stimulus': root/stimulus GroupBuilder {'attributes': {}, 'groups': {'presentation': root/stimulus/presentation GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}, 'templates': root/stimulus/templates GroupBuilder {'attributes': {}, 'groups': {}, 'datasets': {}, 'links': {}}}, 'datasets': {}, 'links': {}}}, 'datasets': {'file_create_date': root/file_create_date DatasetBuilder {'attributes': {}, 'data': <StrDataset for HDF5 dataset "file_create_date": shape (1,), type "|O">}, 'identifier': root/identifier DatasetBuilder {'attributes': {}, 'data': 'Subject_001_Session_001'}, 'session_description': root/session_description DatasetBuilder {'attributes': {}, 'data': 'Concept Museum - Subject 1 on Session 1'}, 'session_start_time': root/session_start_time DatasetBuilder {'attributes': {}, 'data': '1900-01-01'}, 'timestamps_reference_time': root/timestamps_reference_time DatasetBuilder {'attributes': {}, 'data': '1900-01-01'}}, 'links': {}}, "Could not construct NWBFile object due to: 'timestamps_reference_time' must be a timezone-aware datetime object.")

Operating System

Windows

Python Executable

Python

Python Version

3.11

Package Versions

No response

Code of Conduct

• I agree to follow this project's Code of Conduct
• Have you checked the Contributing document?
• Have you ensured this bug was not already reported?

[CodyCBakerPhD]

Also found in catalystneuro/tye-lab-to-nwb#40

@rly @oruebel @mavaylon1 Any ideas?

[rly]

@lnett99 Sorry for our delay in response. We'll working on a fix for this.

As an aside, I'm surprised this has not come up before when reading NWB files generated by matnwb in pynwb or nwbinspector - the code to raise an error on the missing timezone is 2 years old: https://github.com/NeurodataWithoutBorders/pynwb/blame/dev/src/pynwb/file.py#L480

[rly]

Ah, some MatNWB tutorials create an NwbFile like so:

nwb = NwbFile( ...
    'session_start_time', datetime(2018, 4, 25, 2, 30, 3, 'TimeZone', 'local'), ...
    'timestamps_reference_time', datetime(2018, 4, 25, 3, 0, 45, 'TimeZone', 'local'), ...

Other tutorials create the datetime object without the time zone.

We are working to remove the time zone requirement from datetime objects in NWB Schema and PyNWB. NeurodataWithoutBorders/nwb-schema#542

[CodyCBakerPhD]

As an aside, I'm surprised this has not come up before when reading NWB files generated by matnwb in pynwb or nwbinspector - the code to raise an error on the missing timezone is 2 years old:

Our case came up via someone using NeuroConv (Python) so not just MatNWB - not sure why CI hasn't caught it otherwise