Release 1.4.5

Release 1.4.5 is a bugfix release, that includes the following fixes:

• Fix the TextRebuilder.deduplicateAndSort() method that was unsorting the table prior to deduplicating samples based on the datetime column.

Release 1.4.4

Release 1.4.4 is a bug fix release. The following problems were fixed:

• The FileArchiverSink was not archiving files on a UTC day. Fixed.
• The TextSinkApp was setting a timer at the wrong hour (not UTC 00:00) for daily and weekly exports. Fixed.
• The TextRebuilder was not deduplicating samples with the exact timestamp instant. Fixed.

Release 1.4.3

Release 1.4.3 is a bug fix release. The following problems were fixed:

• The TextSinkApp was not archiving processed files correctly. Fixed.
• The TextRebuildApp was not archiving hour-23 data (11:00 pm to 11:59 pm). Fixed
• Improved documentation

Release 1.4.2

Release 1.4.2 is a bug fix release. The following problems were fixed:

• Rebuilding instrument archives misses the last hours of the last day in the archive. Fixed.
• Configuration files had <dataPattern> fields with loose regular expressions. Fixed.

Release 1.4.1

Release 1.4.1 is a bug fix release. The following problems were fixed:

• The manage-instruments script had a regression where the -a flag or multiple -i flags failed to start or stop drivers or archivers.
• The manage-instruments -c reloader command failed due to formatting issues of datetimes in Java.

Release 1.4.0

The 1.4.0 release improves support for managing the data archive and instrument drivers. Features and bug fixes include:

• Refactors the TextRebuildApp to create a TextRebuilder class used in both the TextRebuildApp and the new TextReloaderApp
• Adds a new TextLoaderApp that takes an instrument ID and raw data directory and uploads the latest year of data to the Data Turbine
• Adds a new manage-instruments command called reloader that calls the TextLoaderApp
• Adds a short name field to the XML configuration file used in the manage-instruments -l command (e.g. shows NS01, NS02, etc.)
• Removes redundant files found in /data/processed and fixes the code issue creating extra files

See the new manage-instruments usage help:

$ manage-instruments -h

Usage:
manage-instruments -l
manage-instruments -c driver|archiver -o start|stop [-a] [-h] [-i instr1] [-i instr2] [-i instrN]
manage-instruments -c rebuilder -i instr -d directory
manage-instruments -c reloader -i instr -d directory

List the status of all instrument drivers and archivers, or
start or stop one or more instrument source drivers or archivers by optionally providing the instrument id.

Alternatively, rebuild an instrument archive by providing the instrument id, and an existing raw archive directory path.

Lastly, reload an instrument's data into the Data Turbine by providing an instrument id and raw data directory.

OPTIONS:

    -a  Start, stop or show status for all configured instruments
    -c  The command to run, 'driver', 'archiver', 'rebuilder', 'reloader', to stream, archive, rebuild or reload data.
    -h  Show this message.
    -i  The instrument id. Use -i multiple times for multiple instruments. Use once for a rebuild or reload.
    -l  List the status of all drivers and archivers.
    -o  Indicate which operation to perform, start, stop, or status (requires 'driver' or 'archiver' commands).
    -d  The existing raw instrument archive to rebuild (requires 'rebuilder' or 'reloader' command).
    -V  Show the version (1.4.0)

Release 1.3.0

The 1.3.0 release improves support for managing the data archive and instrument drivers. Features and bug fixes include:

• Better handling of the pacioos-2020-format for nearshore instrument data.
• The manage-instruments includes a new status option (-l) to list the status of each instrument
• The SimpleTextSource class includes better handling of streaming data when the instruments are sporadically online.
• Other minor bug fixes

See the new manage-instruments usage help:

$ manage-instruments -h

Usage:
manage-instruments -l
manage-instruments -c driver|archiver -o start|stop [-a] [-h] [-i instr1] [-i instr2] [-i instrN]
manage-instruments -c rebuilder -i instr [-d directory]

List the status of all instrument drivers and archivers, or
start or stop one or more instrument source drivers or archivers by optionally providing the instrument id.

Alternatively, rebuild an instrument archive by providing the instrument id, and either a file
with samples to add to the archive, or an existing raw archive directory path.

OPTIONS:

    -a  Start, stop or show status for all configured instruments
    -c  The command to run, 'driver', 'archiver', 'rebuilder', to stream, archive, or rebuild data.
    -h  Show this message.
    -i  The instrument id. Use -i multiple times for multiple instruments. Use once for a rebuild.
    -l  List the status of all drivers and archivers.
    -o  Indicate which operation to perform, start, stop, or status (requires 'driver' or 'archiver' commands).
    -d  The existing instrument archive to rebuild (requires 'rebuilder' command).
    -V  Show the version (1.3.0)

Release 1.2.0

The 1.2.0 release includes improvements in archive management for the PacIOOS data.

Features include:

• A new TextRebuilderApp that allows an entire raw data archive to be reprocessed, sorted, deduplicated, and re-written with correct UTC file names/timestamps.
• The TextRebuilderApp also converts legacy data to the pacioos-2020-format, a CSV format common to all nearshore instrument to help ease the production of upstream products like NetCDF data files.
• An improved manage-instruments script that incorporates rebuilding of archives:

$ manage-instruments -h

Usage:
manage-instruments -c driver|archiver -o start|stop [-a] [-h] [-i instr1] [-i instr2] [-i instrN]
manage-instruments -c rebuilder -i instr [-d directory]

Start or stop one or more instrument source drivers or archivers by optionally providing the instrument id.

Alternatively, rebuild an instrument archive by providing the instrument id, and either a file
with samples to add to the archive, or an existing raw archive directory path.

OPTIONS:

    -a  Start or stop all configured instruments
    -c  The command to run, 'driver', 'archiver', 'rebuilder', to stream, archive, or rebuild data.
    -h  Show this message.
    -i  The instrument id. Use -i multiple times for multiple instruments. Use once for a rebuild.
    -o  Indicate which operation to perform, start or stop (requires 'driver' or 'archiver' commands).
    -d  The existing instrument archive to rebuild (requires 'rebuilder' command).
    -V  Show the version (1.2.0)

Release 1.1.0

The 1.1.0 release includes improvements in instrument driver and archiver management for the PacIOOS text-based drivers.

Features include:

• An improved manage-instruments scripts used to start and stop drivers and archivers
• Support for archiving data to a common CSV format with all UTC date timestamps
• Handling of all data as UTC timestamps in the Data Turbine, archive file names, and sample entries
• A Converter interface that allows multiple converters to be developed and deployed
• A RawToPacIOOS2020Converter that standardizes CSV output files into an agreed upon format
• A Configuration class with convenience methods that hide the details of accessing the XML configuration file settings

Release 1.0.0

A snapshot of the realtime-data codebase as it was used prior to migrating to git, and was known as the BBL project.