All notable changes to this project will be documented in this file.
- Support for MWA Average Embedded Element beams in UVBeam.
- Support for python 3.14.
- Added improved handling of a few warning messages from h5py and numpy.
- A bug with reading ms caltables with multiple closely-spaced times.
- Significant reduction in memory when reading UVFITS files, particularly when reading the entire file (no selection on read).
- Added a
UVData.flip_conjugationmethod and more detail to the uvw check warning message to inform users when a conjugation flip might be needed. - When reading MWAX correlator files ,
remove_coarse_bandnow defaults toFalseif DERIPPLE is on. Otherwise it defaults toTrue. - If the user has set
remove_coarse_band=Truewhen DERIPPLE is on, a warning is raised, and no pfb shape is removed.
- Support for wide-band
UVCalgain objects withuvcalibrate. - A colormap option to the UVBeam and AnalyticBeam
plotmethods, along with better default colormaps for phase. - A NotImplementedError to
mwa_corr_fits.pythat is thrown when trying to read fringe-stopped data.
- A backwards compatibiilty issue with feed arrays in old beamfits files which could specify feeds as "e" or "n".
- Compatiblity with numpy>=2.3
- Added a
plotmethod to UVBeam and to AnalyticBeam to make visualizing beams easier.
- Moved test data to a new repo (https://github.com/RadioAstronomySoftwareGroup/rasg-datasets/), use pooch to download and cache the test data in testing and in tutorials.
- Added pooch as a testing dependency, added a new optional dependency option
called
tutorialwhich currently adds pooch and matplotlib.
- A bug that caused an error when setting the pixel_coordinate_system in
UVBeam.newandAnalyticBeam.to_uvbeam.
- Support for reading FEKO ffe files into UVBeam.
- Analytic ShortDipoleBeam objects can now accept arbitrary wraps of 2pi in their feed angles. The feed angles that are stored on the object have their feed angles normalized to be between 0 and pi/2.
- A bug in the UVBeam yaml constructor that caused the
freq_rangeoption to be ignored. - A bug in
utils.io.ms.read_ms_historywhere reading the table caused an IndexError due to the "APP_PARAMS" and "CLI_COMMAND" columns being populated in a non-standard fashion. - A bug in UVBeam's CST reader that caused the bandpass array to be incorrectly set to 0 except for the first frequency.
- A bug in
UVParameter.get_from_formandUVParameter.set_from_formthat caused errors when using astropy SkyCoord, Quantity or Time objects in UVParameters, arose when usingUVBase._select_along_param_axis.
- numba added as a dependency for pyuvdata.
- Refactored
utils.bls, replacing cython-based extensions with numba-based routines. - Updated minimum dependency versions: h5py>=3.7, python>=3.11, scipy>=1.9
- Updated minimum optional dependency versions: hdf5plugin>=3.3.1, pytest-cases>=3.9.1
- A bug where the
Telescope.get_x_orientationwould not return"east"or"north"if the feed angles if outside of the range of 0 to 90 degrees (e.g. 180 and 270 degrees for x- and y-polarization should return "north".). - A bug in reading in uvfits files with baseline coordinates that have the '--' suffix, which is allowed in uvfits files.
- A bug in reading in uvfits files where the antenna frame is given by an arbitrary number of repeated '?', which is allowed in uvfits.
- A new page to the docs labeled "Conventions" has been added.
- The methods
Telescope.__add__andTelescope.__iadd__have been added to allowTelescopeobjects with different antennas or feeds to be combined. - The methods
Telescope.reorder_antennasandTelescope.reorder_feedshave been added to allow reordering of antenna metadata. - The methods
UVParameter.get_from_formandUVParameter.set_from_form, which allows one to perform selection/setting operations based on theUVParameter.formattribute. - Handling for MeasurementSet calibration subtypes "T Jones" (non-pol-specific antenna
gains) and "D Jones" (polarization leakages) has been added to
MSCal. - The
UVBeam.x_orientationparameter has been deprecated, superseded by two new required parameters:UVBeam.feed_arrayandUVBeam.feed_angle, which describe the polarization and orientation of the beam (the former of which was already present but only required for efield beams). - The
Telescope.x_orientationparameter has been deprecated, superseded by two new parameters:Telescope.feed_arrayandTelescope.feed_angle, which describe the polarization and orientation of the detectors for a given antenna. - The
Telescope.mount_typeandUVBeam.mount_typeparameters have been added, which describe the mount and optics of a given antenna (e.g., HERA is "fixed", MWA is "phased",VLA/ALMA are "alt-az"). These new parameters are required wheneverfeed_arrayandfeed_angleare set on these objects (always forUVBeam). - The
Telescope.get_x_orientation_from_feedsmethod has been added, which returns a string (either "east" or "north") based on values present inTelescope.feed_arrayandTelescope.feed_angle(mimicking the behavior of getting the now-defunctTelescope.x_orientationparameter). A method of the same name and functionality has also been added toUVBeam. - The
Telescope.set_feeds_from_x_orientationmethod has been added, which sets values inTelescope.feed_arrayandTelescope.feed_anglebased on a string describing the x-orientation (mimicking the behavior of setting the now-defunctTelescope.x_orientationparameter). A method of the same name and functionality has also been added toUVBeam. - New
near_fieldoption added to thephasemethod'scat_typekeyword attribute. Passingnear_fieldfirst performs sidereal far-field corrections using fixed RA/Dec, then applies geometric near-field corrections using thedistancekeyword, which can either be specified as a floating-point number (which, combined with thenear_fieldtag, will be interpreted in meters) or as anastropy.units.Quantityobject. - A new utility function
utils.pol.convert_feeds_to_polsto get a polarization array given a feed array (previously existed as a private function in uvbeam.py). - New
strictkeyword added toUVData.select,UVBeam.select,UVFlag.select, andUVFlag.select, which allows the user to specify whether to warn or error when supplied criteria only partially match (default being to warn). - New
invertkeyword added toUVData.select,UVBeam.select,UVFlag.select, andUVFlag.select, which allows the user to specify data to deselect rather than select. - New method
UVBase._select_along_param_axis, which allows for more uniform selection behavior across parameters withinUVData,UVCal,UVBeam,UVFlag, andTelescopeclasses. - Several new selection helper and check functions have been added to
utils. - New
warn_spacingkeyword added toselect,__add__,fast_concatmethods ofUVData,UVCal,UVBeam, andUVFlag, which allows the user to specify whether or not to warn based on spacing errors that would prevent writing out to e.g., FITS-based file formats. Default is typicallyFalse(with the exception ofUVBeam, where the default isTrue), such that most warnings about frequency/polarization/time spacing will not normally be raised.
- Default redundancy finding algorithm in
UVData.compress_by_redundancyto match the default inUVData.get_redundanciesand utils redundancy functions. - Started the process of changing the default for the
include_conjugatesinUVData.get_redundanciesfrom False to True (with a deprecation process) to match the behavior ofUVData.compress_by_redundancy. - Warnings related to setting of parameters from known telescopes when instantiating
UVData,UVCal, andUVFlagobjects are now turned off by default. feed_array(and by association,feed_angleandNfeeds) is now a required parameter forUVBeam.- Default for
UVParameter.tolswhen the expected type is int, bool, or str is now set to(0, 0), was originally(1e-5, 1e-8)(relative and absolute tolerance, respectively). - Flex-jones
UVCalobjects now recorded to MeasurementSet format as "T Jones" subtype. - Only import lunarsky if needed.
UVData.select,UVBeam.select,UVCal.select,UVFlag.selecthave been significantly refactored and made to behave more uniformly.- Allowing
UVParameter.__eq__to useUVParameter.compare_valueif the item being compared andUVParameter.valueshare the same class. - Warnings about
extra_keywordshave been removed fromUVData.check,UVCal.check, andUVBeam.check.
- Bug in
UVData.write_mswhere datasets with a single spectral window were being written with the wrong conjugation when settingflip_conj=True. - Bug in
UVData.write_mswhere the baseline conjugation scheme did not conform to what CASA nominally expects. - Bug in
utils.tools.slicifyandutils.tools._convert_to_sliceswhere reverse-ordered slices (i.e., where the step was negative) were not correctly handled. - Bug in MWA beams that caused beams pointed away from zenith to be wrong because the delays were not assigned to the right dipoles.
- Bug in
UVData.sum_viswhere it errored if there were different filenames on the input objects. Now the filename lists are combined on the output object. - Bug in
UVBeam.selectwherepolarization_arraycould be incorrectly ordered after selection (if input topolarizationskeyword was unordered).
- The
x_orientationattribute onTelescopeandUVBeam. - The options
"e"and"n"for elements offeed_arrayinTelescopeandUVBeam.
- The
future_array_shapesattribute onUVBaseobjects. - The
use_future_array_shapeskeyword in various class methods. - The
use_future_array_shapesmethod onUVBaseobjects. - Support for accessing the telescope-related metadata through their old attribute
names on
UVData,UVCalandUVFlagrather than via their attributes on the attachedTelescopeobject (e.g.UVData.telescope_name->UVData.telescope.nameandUVData.antenna_positions->UVData.telescope.antenna_positions). - Support for passing telescope-related metadata as separate parameters to
UVData.newandUVCal.newrather thanTelescopeobjects. - The
UVData.get_ENU_antposmethod in favor ofUVData.telescope.get_enu_antpos. - The
Telescope.telescope_locationandTelescope.telescope_nameattributes in favor ofTelescope.locationandTelescope.name. - The
get_telescopefunction in favor of theknown_telescope_locationfunction and theTelescope.from_known_telescopesclassmethod. - The KNOWN_TELESCOPE dict in favor of the
known_telescope_locationfunction and theTelescope.from_known_telescopesclassmethod.
- Option to write the uvw_array as double precision in UVFITS files even when the data array are single precision. Default is set to write them as doubles.
- ATA has been added to the list of known telescopes.
- Bug in
calc_frame_pa_anglewhich gave rise to an indexing error at positions close to the Southern celestial pole. Additionally fixed a bug where these values were incorrectly calculated. - A sign flip of the MWA beam response to azimuthally aligned polarization. This was caused by the difference in coordinates used in UVBeam and the mwa_pb repo, the coordinates themselves were properly accounted for but the flip in orientation of the basis vectors was not properly handled.
- Bug in the
uvh5._addmethod that expected keys which could be optional for some phase-center catalog-entry types. This was noticed as an issue when adding two uvh5 objects that were of type "sidereal" which did not have "info_source" entries in their phase-center catalogs. - Bug in the
look_in_catalogutility function that expected keys which could be optional for some phase-center catalog-entry types. This was noticed as an issue when adding two uvh5 objects that were of type "sidereal" which did not have "cat_times" entries in their phase-center catalogs. - Bug in selecting baselines on a UVData object using
blskeyword with 3-tuples and more than one polarization (introduced in 3.1.2).
- New keyword
check_kwtonew_uvdatathat lets the user specify options to the checking of the created object. - New keyword
all_times_uniquetocalc_app_coordsthat short-cuts the calculation when it is known that all input times are unique. - New convenience methods on
BeamInterfaceto simplify the handling of analytic vs UVBeam objects. - Added support for partial read for MWA correlator FITS files.
- Added
antenna_names,time_range,lstsandlst_rangeparameters toUVFlag.selectto match UVData and UVCal select methods.
- Made it possible to not return the
interp_basis_vectorarray from beam interpolators.
- Initialization of analytic beams with single feed.
- New test failures related to incorrect tolerances for floating point comparisons in tests. Floating point comparisons have been updated throughout the code base.
- New analytic beam classes: AiryBeam, GaussianBeam, ShortDipoleBeam, UniformBeam that support evaluating either the efield or power beam in any direction and frequency.
- A new BeamInterface class that provides a unified interface to UVBeam and analytic beam objects to get beam responses in any direction and frequency (via interpolation or evaluation as appropriate).
- New optional spatial interpolation method,
interpolation_function="az_za_map_coordinates"that improves the linear interpolation speed for data inaz_zacoordinates. - New UVParameter
pol_conventiononUVDataandUVCal. This specifies the convention assumed for converting linear to stokes polarizations -- either "sum" or "avg". Also added touvcalibrateto apply from theUVCalto theUVData. - New
ignore_telescope_param_update_warnings_forfunction that globally ignores warnings for specific telescopes. - New
.telescopecached property on theFastUVH5Metaobject that auto-creates a telescope object from the metadata. UVData.get_enu_data_antsmethod to get east, north, up positions only for antennas with data.
- Updated minimum optional dependency versions: lunarsky>=0.2.5, pytest>=8.2.0
- A bug in the MWA beam reader that resulted in the wrong polarization response (the azimuthal-aligned response was swapped with the zenith angle-aligned response).
- A bug in reading UVH5 files with antenna names saved as variable length strings that was introduced in v3.0.0.
- Several new methods on
Telescopeobjects, including the classmethodsfrom_known_telescopesandnewto instantiate new objects and theget_enu_antposmethod to get antenna positions in East, North, Up coordinates. - Support for writing "MODEL_DATA" and "CORRECTED_DATA" columns has been added to the
UVData.write_msmethod. - Support for "flexible-Jones"
UVCalobjects -- where different spectral windows can contain different Jones vectors/polarization information -- has been added, along with methods for convertingUVCalobjects to and from flex-Jones format (remove_flex_jones,_make_flex_jones, andconvert_to_flex_jones). Support for flex-Jones has been added toMSCalandCalH5subclasses. - A new class called
MSCalhas been added as a subclass ofUVCal, which adds support for reading and writing of CASA Measurement Set gains, bandpass, and delays tables. To support this, several new optional parameters have been added to theUVCalclass, includingphase_center_catalog,phase_center_id_array,antenna_diameters,scan_id_array, andref_antenna_array(which affords the ability to record a different reference antenna per time-axis entry). - A new HDF5 file format for calibration solutions called
calh5, which supports writing out all types of UVCal objects and supports partial reads (select on read). - A
UVCal.get_lst_arraymethod similar to theget_time_arraymethod that either returns the LST of the mean of the start and stop time for each time range or the lst_array (if there's a lst_array and no lst_range). - A
UVCal.get_time_arraymethod that either returns the mean of the start and stop time for each time range or the time_array (if there's a time_array and no time_range). - Added new keyword handling for v.6 of the MIR data format within
MirParser.
- Updated minimum dependency versions: scipy>=1.8, setuptools_scm>=8.1
- Updated minimum optional dependency versions: lunarsky>=0.2.4
- Restructured
utils.pyinto a new submoduleutilswith functions split among several submodules. Utility functions used widely are still available from pyuvdata.utils, although this access pattern is deprecated for some of them. - Modified
UVBeam.interpto speed up processing whencheck_azza_domain=True. - Updated minimum dependencies: setuptools>=64, setuptools_scm>=8.0
- Restructured to a
srclayout. This should not affect most users, but thecheck_warningsfunction has moved frompyuvdata.teststopyuvdata.testing. - Most packaging information has now moved to the
pyproject.toml. - Future array shapes are now the only supported standard for
UVData,UVCal,UVFlag,UVBeamclasses. UVCalobjects where the cal type is delay must now havewide_band=True.- The
flex_spw_id_arrayis now required forUVCalobjects werewide_band=False. UVCalobjects withwide_band=Truemay only have thefreq_rangeparameter set and thefreq_arrayandchannel_widthparameters. Ifwide_band=False, then thefreq_arrayandchannel_widthparameters must be set, while thefreq_rangeparameter must be left unset.- For
UVCalobjects, only one of the time-based parameters (time_arrayandtime_range) may be set on a given object. - Telescope-related metadata (including antenna metadata) on
UVData,UVCalandUVFlaghave been refactored into aTelescopeobject (attached to these objects as thetelescopeattribute) and most of the code related to these attributes from the three objects has been consolidated on theTelescopeobject. These attributes are still accessible via their old names on the objects, although accessing them that way is now deprecated. Note that the telescope locations are now stored under the hood as astropy EarthLocation objects (e.g.UVData.telescope.location) (or as MoonLocation objects for simulated arrays on the moon iflunarskyis installed). - The
UVData.newandUVCal.newmethods now accept a telescope object for setting the telescope-related metadata (the older parameters are still accepted but deprecated.) - When FHD calibration solutions are read in, the
time_rangeis now set to be one quarter of an integration time before and after the earliest and latest times respectively. This is a change from extending it by half an integration time to prevent apparent overlaps in neighboring calibration solutions due to jd precision loss. - Improved performance for
antpair2indand_key2indsby using theblts_are_rectangularparameter, and also by caching the results. To improve performance of the cache, the resulting indices are returned as slices whenever possible. - Changed the way files are passed for reading FHD files. Rather than one list including
all the types of files, each file type (e.g. params, obs, layout) must be passed to
their own parameter. This makes the API for reading FHD data files more similar to the
API for reading FHD cal files. Also removed the
use_modelparameter because it is no longer needed (only data or model files should be passed as visibility files). - Require keyword arguments rather than allowing for passing arguments by position for functions and methods with many parameters.
- Only one of
time_arrayandtime_range(and similarlylst_arrayandlst_range) can be set on a UVCal object. - If
time_rangeis set it must be 2D with a shape of (Ntimes, 2) where the first axis gives the number of different time solutions and the second axis gives the start/stop times for those solutions.
- The
future_array_shapesattribute onUVBaseobjects has been deprecated, as pyuvdata now exclusively uses future array shapes. - The
use_future_array_shapeskeyword in several different class methods, as well as theuse_future_array_shapesmethod onUVBaseobjects. - Accessing the telescope-related metadata through their old attribute names on
UVData,UVCalandUVFlagrather than via their attributes on the attachedTelescopeobject (e.g.UVData.telescope_name->UVData.telescope.nameandUVData.antenna_positions->UVData.telescope.antenna_positions). - Passing telescope-related metadata as separate parameters to
UVData.newandUVCal.newrather thanTelescopeobjects. - The
UVData.get_ENU_antposmethod in favor ofUVData.telescope.get_enu_antpos. - The
Telescope.telescope_locationandTelescope.telescope_nameattributes in favor ofTelescope.locationandTelescope.name. - The
get_telescopefunction in favor of theknown_telescope_locationfunction and theTelescope.from_known_telescopesclassmethod. - The KNOWN_TELESCOPE dict in favor of the
known_telescope_locationfunction and theTelescope.from_known_telescopesclassmethod.
- Fixed a minor bug in
utils._check_freq_spacingthat raised issues which raised an error when evaluating a "flex-spw" dataset where Nspws was 1. - Fixed a bug in
UVBasewhereallowed_failureswas being ignored if a parameter hadrequired=Trueset. - Fixed a bug where selection and addition/concat methods did not operate correctly on
flex-spw
UVCalobjects whereNspws>1.
- Support for current array shapes has been removed on
UVData,UVCal,UVFlag, andUVBeamobjects, as well as theuse_current_array_shapesmethod for these classes. - The
flex_spwattribute has been removed onUVDataandUVCalobjects. - Support for using the old phasing attributes (
phase_center_ra,phase_center_dec,phase_center_frame,phase_center_epoch,phase_type, andobject_name) inUVData. - Support for handling of the
input_flag_arrayparameter forUVCalobjects. - Support for providing capitalized variants of
UVBeam.feed_array. - Support for 'unknown' cal type in
UVCalobjects.
- numpy c-api for 2.0 compatibility
- numpy.string_ calls to numpy.bytes_, np.in1d to np.isin and a few other changes for numpy 2.0 compatibility.
- Fixed #1445 in which in some corner cases the
antpairsarray was not correctly interpreted as int.
- updated minimum cython dependency to 3.0.
- Updated minimum optional dependency versions: astropy-healpix>=1.0.2
- Fixed a bug where UVFITS header parameters RDATE and DATE-OBS were being improperly set.
- Fixed a bug with calculating Miriad antenna and baseline numbers, which properly accounts for MIRIAD antennas being 1-indexed not 0-indexed.
- Fixed a bug writing UVData objects to UVFITS formats with extra phase center catalog entries resulted in an error on read.
- Fixed a bug where calling
UVData._set_app_coords_helperwith extra phase center catalog entries resulted in an error. - Bug in which
correct_cable_lendefaulted toNoneinstead ofTrueforread_mwa_corr_fits.
- Support for new lunar ellipsoids in
lunarskyversion 0.2.2. - Added new keyword handling for v.6 of the MIR data format within
MirParser.
- Updated minimum dependency versions: astropy>=6.0, h5py>=3.4, numpy>=1.23, pyerfa>=2.0.1.1, scipy>=1.7.3, python>=3.10
- Updated minimum optional dependency versions: hdf5plugin>=3.2.0, pytest>=6.2.5, python-casacore>=3.5.2, pyyaml>=5.4.1, lunarsky>=0.2.2
- Speed up for reading MWA correlator FITS files into
UVData. - Now compatible with pytest>=8.0 but require pytest-cases>=3.8.3.
- Fixed a bug where SMA-based coordinates were recorded as FK5 for sidereal sources (should be ICRS).
- Bug in which passing 'ee' or 'nn' pols to UVData.new() would error, even if
x_orientationwas passed.
- Added a new redundancy finding algorithm based on the HERA gridding algorithm. This new algorithm is used by default but the older clustering algorithm is still available.
- Added support for initializing
MirParserandMirMetaDataobjects fromPathobjects. - Added a brief tutorial on how to convert SMA MIR data into MS format (with some notes on SMA specific keywords than can be used).
- Added the
Mir.generate_sma_antpos_dictmethod for reading in SMA-formatted antenna position files into a dict (which can be passed toUVData.update_antenna_positions). - Added handling for older formatted (so called "ASIC-era") MIR files from SMA, as well as more recent file formats that preceded pyuvdata support (circa 2020, v1/v2 of the MIR file format).
- Added the ability to save/load selection masks in
MirParserobjects. - Fixed a bug when trying to write a MS with a legacy array shape (but note legacy shape will be deprecated in upcoming releases).
- Improved support for MIRIAD UV files, and added
use_miriad_conventionflag to the UVFITS writer code, so files with the MIRIADBASELINEconvention can be created. - A new
freq_interp_kindparameter toUVBeam.interp,UVBeam._interp_az_za_rect_splineandUVBeam._interp_healpix_bilinearto allow the frequency interpolation specification to be passed into the methods. Note this defaults to "cubic" rather than "linear" (the old default for the attribute of the same name on UVBeam objects) because several groups have found that a linear interpolation leads to nasty artifacts in visibility simulations for EoR applications. - A new
UVBeam.new()method (based on new functionnew_uvbeam) that creates a new, self-consistentUVBeamobject from scratch from a set of flexible input parameters. - Added a the
UVData.update_antenna_positionsmethod to enable making antenna position updates with corresponding updates the uvw-coordinates and visibility phases. - Added a switch to
UVData.write_mscalledflip_conj, which allows a user to write out data with the baseline conjugation scheme flipped from the standardUVDataconvention. - Added the
utils.determine_pol_ordermethod for determining polarization order based on a specified scheme ("AIPS" or "CASA"). - Added a
check_surface_based_positionspositions method for verifying antenna positions are near surface of whatever celestial body their positions are referenced to (either the Earth or Moon, currently).
- Made the determination of whether or not to create a flex-pol dataset when reading in a MIR more robust (particularly with pre-V3 data formats).
- Reading in of MIR data sets into
UVDataobjects will now use pyuvdata-calculated values forlst_arrayvia (UVData.set_lsts_from_time_array) instead of those read in from the file due to known precision issues in the later (~25 ms), so long as the two agree within this known precision limit. UVFlag.to_baselineandUVFlag.to_antennaare now more robust to differences in antenna metadata sorting.- Made
MirParsermore robust against metadata indexing errors. - Made
MirParserhandling of autocorrelation data more robust. - Improved hooks for handling and loading of COMPASS solutions into
MirParserobjects. - Fixed handling of weights to be consistent with tsys scaling method in
MirParser. - Changed
MWACorrFits.corrcorrect_simpsmethod to use thescipy.integrate.simpsonmethod rather than thescipy.integrate.simpsmethod to fix deprecation warnings. - added support for python 3.12, dropped support for python 3.8.
- Updated minimum dependency versions: pyyaml>=5.3
- Changed
UVData.write_msto sort polarizations based on CASA-preferred ordering. - Added some functionality to the
utils._convert_to_slicesmethod to enable quick assessment of whether an indexing array can be replaced by a single slice. - Increased the tolerance to 75 mas (equivalent to 5 ms time error) for a warning about
values in
lst_arraynot conforming to expectations forUVData,UVCal, andUVFlag(was 1 mas) inside ofcheck. Additionally, added a keyword tocheckenable the tolerance value to be user-specified. - Changed the behavior of checking of telescope location to look at the combination of
antenna_positionsandtelescope_locationtogether forUVData,UVCal, andUVFlag. Additionally, failing this check results in a warning (was an error).
- The
freq_interp_kindattribute on UVBeams. - The
spw_arrayandNspwsattributes on UVBeam objects. Also theunset_spw_paramsandset_spw_paramsparameters to theuse_future_array_shapesanduse_current_array_shapesmethods on UVBeam objects. - Upper case feed names (e.g. "N" or "E") in UVBeam.feed_array. This was never fully tested and didn't work properly.
- Having
freq_rangedefined on non-wide-band gain style UVCal objects. - Having
freq_arrayandchannel_widthdefined on wide-band UVCal objects.
- A bug where
time_arraywas not being correctly calculated for older (pre-V3) MIR data formats. - A small bug in UVFlag that could occur when reading in an older UVFlag HDF5 file with missing antenna metadata.
- A small bug (mostly affecting continuous integration) that threw an error when the
IERS service was down and when tests were using
test.check_warningsto look for no warnings thrown (i.e., wherematch=None). - A couple of small bugs related to handling of the
freq_rangeparameter in thereorder_freqsand__add__methods onUVCal.
- Added a
uvw_track_generatormethod withinutilsfor calculating the expected uvws (and a few other values) without needing to instantiate a wholeUVDataobject. - Added a convenience function called
compare_valueinUVParameterthat enables one to do value checking with tolerances accounted for. - New
mwa_metafits_fileandtelescope_nameoptional parameters toUVFlag.readandUVFlag.__init__to help with setting telescope metadata for old UVFlag files that are missing it. - MWA antenna metadata to our known telescopes to allow them to be filled in for old
UVFlag files. This is a stopgap solution, a better approach is to pass an MWA metafits
file to the new
mwa_metafits_fileparameter. - Support for recarrays in
UVParameterobjects and inUVBase, needed for pyradiosky. - Support for setting the astrometry library for various object methods including
set_lsts_from_time_array, file read methods and others. - Properly round-trip the telescope frame through UVH5, UVFITS and MS files.
- A bug in apparent coordinate calculation that resulted in small errors/loss of
precision due to the way times were passed when using
erfaastrometry library. - A small correction due to polar drift for LST calculation when using the
erfaastrometry library. - Fixed a bug in
utils.calc_app_coordsthat occurred when supplying an astropyTimeobject for thetime_arrayargument. - Fixed a bug in
UVData.write_msthat caused the writer to crash if prior measurement set history was not formatted in the currently expected fashion.
- Support for multiple spectral windows in
UVData.frequency_average, including a new parameterrespect_spwsfor controlling whether averaging crosses spectral window boundaries. - Better handling in
UVData.frequency_averagewhen averaging by a number of channels that does not divide evenly into the number of channels in each spectral window. - Compatibility with Python 3.11
- The following
UVFlagparameters are now required:Nspws,channel_width,spw_array,telescope_name,telescope_location,antenna_names,antenna_numbers, andantenna_positions. - The
quality_arrayon UVCal objects is no longer required.
- The
sky_fieldattribute onUVCal.
- A bug in LST calculation that led to small discontinuities in LSTs calculated using
the
erfaornovasastrometry libraries. - Error when setting
UVBeam.freq_interp_kindto an integer. - Error when reading
mwa_corr_fitsfiles from the new MWAX correlator
- The
phase_uvwandunphase_uvwutility methods associated with old style phasing. - The
with_conjugatesoption to theget_baseline_redundanciesutility method in favor of theinclude_conjugatesoption. - Support for the
interpolation_functionattribute on UVBeams. - Support for the "unphased" catalog type in UVData.phase_center_catalog in favor of the "unprojected" catalog type.
- The
UVData.unphase_to_driftmethod in favor ofUVData.unproject_phasemethod. - Support for using old style phasing on UVData objects (the
UVData.fix_phasemethod to fix datasets that were phased with the old style of phasing remains as well as thefix_old_projoption on read for uvfits, miriad and uvh5 file types). - Phasing related keywords in the following
UVDatamethods:__add__,__iadd__,fast_concat,readandfrom_file. Multiple phase centers are now fully supported so datasets can always be combined without phasing. If desired, the full object can be rephased to a single phase center afterwards. - The
use_cotter_flagsandflag_small_sig_antsoptions to theUVData.readandUVData.from_filemethods for MWA correlator fits files in favor of theuse_aoflagger_flagsandflag_small_auto_antsoptions respectively. - The
spoof_nonessentialoption toUVData.write_uvfitsmethod as the previously spoofed parameters are now properly calculated.
- New dependency
docstring-parserto enable dynamic docstring generation for methods that mostly wrap other methods. - A new
UVCal.new()method (based on new functionnew_uvcal) that creates a new, self-consistentUVCalobject from scratch from a set of flexible input parameters. - A new
UVData.new()method (based on new functionnew_uvdata) that creates a new, self-consistentUVDataobject from scratch from a set of flexible input parameters. - A new
fast_concatmethod onUVCal. - A new generic
readmethod onUVCalthat, like thereadmethods on our other objects, supports all file types and a newfrom_fileclass method to allow one-line reading of UVCal files.
- The
input_flag_arrayattribute onUVCal. - Support for the 'unknown' cal_type in UVCal.
- Reading in multiple files to
UVCalusing file-type specific read methods (e.g.read_calfits) in favor of the genericreadmethod.
- Removed error when
time_axis_faster_than_bls=TrueandNtimes=1. In the case ofNtimes=1, it is inconsequential whether time-axis moves first or not, so it should not be an error.
- The
catalog_namekeyword has been added to theUVData.readandUVData.selectmethods to allow users to select on source name.
- The keywords for
UVData.readhave been changed for MIR-specific values to more generally match those founds with other data types. - The
UVData.read_mirmethod has been overhauled to mak reading in of MIR data more efficiency in memory/processor usage.
- Frequency frame information for MS datasets is now correctly recorded as "TOPO".
- A new
FastUVH5Metaobject that enables quick partial reading of uvh5 file metadata. This is used by default in theUVData.read_uvh5method to initialize aUVDataobject, without any apparent change for the user. To get the benefits of the new class, use it directly to interface with the metadata portion of a.uvh5file.
- Updated numpy requirements to
>=1.20. - Updated scipy requirements to
>=1.5.
- Handling of antenna_names and antenna_numbers in
read_fhdandread_fhd_cal.
- A new
interpolation_functionparameter toUVBeam.interpandUVBeam.to_healpixto allow the function name to be passed into the methods, with sensible defaulting. - Support for selecting on phase center IDs, including on read.
- Several new attributes to UVFlag:
telescope_name,telescope_location,antenna_names,antenna_numbers,antenna_positions,channel_width,spw_array,flex_spw_id_array, enabling support for flexible spectral windows. All of these exceptflex_spw_id_arraywill eventually be required for all UVFlag object types, as willNspws(which used to only be required for baseline and antenna types). These parameters all already exist on UVData and UVCal objects and UVFlag objects initialized from them will inherit them. UVFlag objects will also inherit these parameters when they are converted between types using UVData and UVCalobjects. - The
UVFlag.set_telescope_paramsmethod, similar to the ones on UVData and UVCal, to set several of these new parameters.
- Increases the h5py minimum version to 3.1.
- Increase the minimum compatible version for lunarsky to 0.2.1 to fix astropy deprecation warnings.
- Fixed some bugs related to handling of the
UVCal.time_rangeparameter inUVCal.__add__and other methods. Ensured that it is a list whenUVCal.future_array_shapesis False. - Fixed a bug in
UVCal.__add__when data are sorted differently along any axis or interleaved along an axis. - Fixed a bug in reading MS files with non-UTC time scales.
- Fixed some bugs in UVData, UVFlag and UVCal
__add__andselectmethods for objects with only one spectral window andflex_spw_id_arraydefined. Also fixed some handling in those methods on UVFlag for objects with multiple spectral windows. - Fix a bug when reading FHD files into UVData objects when the layout file was not passed but the antenna_positions are in the known telescopes.
- Fixed a bug where the beamfits partial read parameters were not available in the
UVBeam.from_fileclass method and were not passed properly when reading multiple files. - Fixed a bug where objects created from scratch with the old phase attributes weren't
properly getting converted to the new
phase_center_catalogrepresentation. That conversion now happens in the check so will happen before anywrite. There could still be errors if some methods are called before a check is done (or if the check is turned off). - Fix a bug in how
frame_pawas calculated inphaseand_set_app_coords_helperfor multi_phase_center objects (it was using the oldphase_center_frameattribute). - Fix a bug where trying to select lsts or lst_ranges on read didn't work for some file types.
- Severe performance hit when calling
polnum2strand its variants for many baselines.
- Reading files into objects without setting
use_future_array_shapesnow results in deprecation warnings. - The utility functions
phase_uvwandunphase_uvwassociated with the deprecated old phasing method. - The
flex_spw_id_arraywill be required on all UVData and UVFlag and all non-wide-band UVCal objects in version 3.0. - Deprecated the
interpolation_functionattribute on UVBeams. - Deprecated the older phase attributes (
phase_type,phase_center_ra,phase_center_dec,phase_center_frame,phase_center_epoch,object_name) in favor of thephase_center_catalog. The older phase attributes will be removed in version 3.0. - The
lst_from_uvfunction in the uvflag module.
- Removed deprecated handling for UVCal objects without
telescope_location,antenna_positionsandlst_arrayparameters defined.
- Updated how wheels for PyPI are built.
- A frame attribute to the telescope_location parameters on UVData, UVCal and UVFlag objects to support observatories on the moon (or anywhere not on earth).
- A bug in
UVData.read_uvh5wheremulti_phase_centerwas being used to imply the existence of thephase_center_catalogheader item, rather than checking for the presence of that item. This conflicted with the uvh5 memo.
- A new subclass of UVParameter for SkyCoord objects, needed in pyradiosky.
- Added
UVData.convert_to_flex_polmethod to enable writing uvh5 files with the polarization axis changing slowest as desired by HERA. Also updatedUVData.remove_flex_polto properly undo theUVData.convert_to_flex_poloperation. - Added an option to
UVData.readandUVData.from_filemethods when reading uvh5 type files to callremove_flex_polas part of the read. This is defaulted toTrueso that when HERA data is written with flex_pol the change will not disrupt users, but other users of uvh5 may want to set it toFalse. - Added support to make bitshuffle compression easier when writing UVData objects to uvh5 files
- Support for partial reads on beamfits files. This can be done on frequency, azimuth or zenith angle (if beam is in az/za coordinates).
- Logging statements (INFO level) in
UVData.check.
- Significantly increased uvh5 reading and writing speed.
- Added
use_future_array_shapesmethod to UVBeam to allow users to convert to using the future array shapes now, with support throughout UVBeam methods and related utility functions. Also addeduse_current_array_shapesto revert to the standard shapes. - Major performance improvement in the
UVBeam.to_healpixmethod. - Performance improvement when doing a
UVData.selectusing theblsparameter (~70% improvement in the_select_preprocessfunction)
- A bug in
UVData.checkwhencheck_autosis True and the object is a flex pol object.
- Added
use_future_array_shapesmethod to UVFlag to allow users to convert to using the future array shapes now, with support throughout UVFlag methods and related utility functions. Also added ause_future_array_shapesparameter to UVFlag's__init__,read,from_uvdataandfrom_uvcalmethods to allow the conversion to be done at the time the object is made. Addeduse_current_array_shapesto revert to the standard shapes. - Baseline number calculations for antenna numbers > 2047.
- New uvh5 version 1.1 definition in memo and code to use the future phasing info (i.e. the phase_center_catalog) rather than the older parameters.
- Added the
normalize_by_autosmethod toUVData, which allows one to normalize cross-correlations (i.e., covert values from arbitrary scale to correlation coefficients) if auto-correlations are present in the data. - Added the
writemethod toMirParser, which allows forMirParserto write out Mir-formatted data to disk. - Added the
selectmethod toMirParser, which provides a much simpler interface for selecting subsets of data to load. - Added the
rechunkmethod toMirParser, which allows data to be spectrally averaged (either after already loading or to be averaged "on-the-fly" when data are read from disk). - Added the
redoppler_datamethod toMirParser, which enables frequency shifting of the data. - Added new
__add__and__iadd__methods toMirParserfor combining data sets. - Added new functionality to
UVBeam.checkto verify that power beams for the auto polarizations (and pstokes) are real-only, along with an option to force them to be real-only if non-zero imaginary components are detected. - Coarse band correction for mwax to
mwa_corr_fits - Reordering methods for UVCal:
reorder_antennas,reorder_freqs,reorder_timesandreorder_jones.
- Always write out uvh5 version 1.0 or greater files (which use the future array shapes) with
UVData.write_uvh5. - Use "unprojected" rather than "unphased" for the cat_type in the phase_center_catalog to describe zenith driftscan data that have no w-projections applied.
- The
MirParserclass has been significantly overhauled, which significantly reduces memory usage and improves processing speed. - Updated minimum dependency versions: numpy>=1.19, scipy>=1.3, optional dependencies: python-casacore>=3.3, pyyaml>=5.1, astropy-healpix>=0.6
UVBaseobject now require that individual attribute names match that given inUVParameter._name.UVData.fix_phasenow raises a warning when called.- Changed
UVData.write_uvfitsto allow for one to write out datasets in UVFITS format without "spoofing" (via settingspoof_nonessential=True) UVFITS-specific values. - Methods that ensure that attributes are set up properly on
UVBeamare now called in theUVBeam.checkmethod, making it easier to fill in UVBeam objects from scratch. - Changed uvfits reading and writing to no longer subtract/add 1 to the antenna numbers. The subtraction/addition of 1 was initially done because FITS uses one-based indexing but antenna numbers are physical numbers not indices, so the numbers should not be adjusted.
- Changed the way baseline numbers are calculated from antenna numbers to conform to the standard in uvfits and miriad. The inconsistency with those files was noticed when we corrected the handling of antenna numbers in uvfits.
- The
spw_orderparameter for theUVData.reorder_freqsmethod now accepts an index array rather than an array of spw numbers, making it match the other reorder methods. - Updated the astropy requirement to >= 5.0.4
- Dropped support for python 3.7
- Store the phase_center_catalog as a set of nested datasets in uvh5 files rather than as a json blob.
- A bug in
UVData.__add__where flexible spectral window datasets were not combined correctly if they contained overlapping frequencies in different spectral windows. - A bug in
UVData.print_phase_center_infothat occasionally resulted in incorrect values being reported for RA/Az/Longitudinal coordinates. - A bug in
UVData.selectthat could causeUVData.checkto fail ifUVData._scan_number_arraywas set. - A bug in
UVBeam.selectwhere after selecting down to only auto polarization power beams theUVBeam.data_arrayremained complex instead of real. - A bug in
UVBeam.__add__where adding an object with cross pol power beams to an object with only auto pol power beams could result in loss of the imaginary part of the cross pol power beams. - Testing of piping keywords through
readbytest_mwa_corr_fits. - Incorrect piping of
flag_dc_offsetkeyword. mwa_corr_fitshanding of theBSCALEkeyword in gpubox files.
- The "unphased" cat_type in phase_center_catalog.
- Parameter
check_azza_domaininUVBeam.interp(), which can turn off the (relatively expensive) check on input az/za values.
- The
UVCal.initialize_from_uvdatamethod to initialize a UVCal object using metadata from a UVData object. - An option to
UVData.reorder_bltsto sort the autos before the crosses. - Added new functionality to
UVData.checkto verify that auto-correlations are real-only, along with an option to force them to be real-only if non-zero imaginary components are detected.
UVFlag.to_baseline()uses internal time tolerances (both rtol and atol) stored in the UVParameter_time_array.tolsto check whether times in the UVFlag object and the input UVFlag/UVData object match.
- The
with_conjugateskeyword in favor of theinclude_conjugateskeyword in theutils.get_baseline_redundanciesfunction.
- A new attribute to
UVDatacalledflex_spw_polarization_array, of type=int and shape=(Nspws,), which allows for individual spectral windows to carry different polarization data. - Added the
_make_flex_polandremove_flex_polmethods toUVData, which allows for one to convert a standardUVDataobject to one with "flexible-polarization". - Added a new method to
MirParsercalled_apply_tsys, which will convert MIR visibility data from correlation coefficients to pseudo-Jy. - Added a method to
Mircalled_init_from_mir_parser, which allows for one to pass aMirParserobject to be converted into a UVData object (rather than reading from a file on disk). - Added the
flex_spwattribute to theUVCalclass, which can be set to True by using the new_set_flex_spwmethod. - Added the optional
flex_spw_id_arrayattribute to UVCal class, of type=int and shape=(Nfreqs,), which indexes individual channels along the frequency axis tospw_array. - Added
use_future_array_shapesmethod to UVCal to allow users to convert to using the future array shapes now, with support throughout UVCal methods and related utility functions. Also addeduse_current_array_shapesto revert to the standard shapes. - Added support for wide-band gain calibrations via the
wide_bandattribute on theUVCalclass, which can be set using the new_set_wide_bandmethod. - Added
time_range,lsts, andlst_rangekwargs from UVH5.write_uvh5_part() to UVData.write_uvh5_part(). - Added
time_range,lsts, andlst_rangekwargs from UVH5.write_uvh5_part() to UVData.write_uvh5_part().
- General performance improvements in the
read_mirmethod. - Enabled
read_mirto read in dual- and full-polarization data.
- Fixed a bug where
UVData.compress_by_redundancysometimes produced incorrectly conjugated visibilities. - Fixed a bug where
lstsandlst_rangegot ignored when doing partial i/o with multiple files.
- Support for reading MWAX correlator fits files.
- Added a warning when using
UVData.write_uvfitswhen thevis_unitsattribute is anything other thanJy. - Added a warning when using
UVData.write_uvfitsif a UVData object has > 256 antennas, recommending use ofUVData.write_msif intending to import the data into CASA. - Reading writing of scan numbers for MS files as
UVData.scan_number_array. - Grouping of contiguous integrations for a phase center into "scan numbers" in
UVData._set_scan_numbers. This grouping definesUVData.scan_number_arraywhen not originally present in the data (e.g. reading in a non-MS file) and is used when writing to an MS file. - Added a
filenameattribute to UVCal, UVBeam and UVFlag objects. - Support for reading MWAX/birli uvfits files.
- Flexible spectral windows to
read_mwa_corr_fits.
- Changed the
UVData.write_msto reduce processing time and memory footprint. - Changed the order in which data are written in Measurement Set format via the method
UVData.write_ms, where data are now first grouped together byscan_number_array. - Changed the defaults by which the
UVData.antenna_namesattribute is set when reading measurement sets (used to default to station names, now uses antenna names if available and the MS file was not written using the CASAimportuvfitsroutine). - Fixed a bug where
UVData.x_orientationandUVData.vis_unitswere not being written to / read from measurement sets. - Updated the astroquery requirement to >= 0.4.4, due to changes in the API handling for calls to JPL Horizons.
- Assumes uvfits files are in ITRF frame unless explicitly stated otherwise. Consistent with AIPS 117.
- Improved readability, functionality, and memory usage in
read_mwa_corr_fits.
- A bug in writing calfits files when the optional
time_rangeparameter is not set. - A bug where the
TIME_CENTROIDfield was not being filled in data sets written byUVData.write_ms, which caused odd behavior in some CASA routines (e.g.,gaincal). - A bug in reading in uvfits files with baseline coordinates that have suffixes of '---SIN' or '---NCP' which are allowed in uvfits files.
- A bug that could cause some routines in CASA to fail when using data sets written by
UVData.write_ms. - A bug that could have resulted in
UVData.__add__combining objects together incorrectly when containing overlapping time-baselines/polarization/frequency channels together that were ordered differently for the twoUVDataobjects. - A bug that prevented
extra_keywordskeys with a value ofNonefrom being saved to UVH5 files. - A bug that resulted in the wrong expected shapes for data-like arrays when metadata only UVData objects were set to use future array shapes.
- A bug that caused an error when writing non-double antenna diameters to measurement set files
- Added a warning in
utils.uvcalibratewhen uvdata x_orientation is not set. - Fixed a bug in
UVBeam.efield_to_powerwhen there is only one feed.
- Fixed a bug in converting non-crosspol power beams to real in
UVBeam.efield_to_power.
- A bug that resulted in coordinate queries via for solar-system objects
utils.lookup_jplhorizonsto throw an error. The workaround temporarily requires the precision for ephemerides to be limited to 0.05 arcseconds. - A bug that resulted in a potential error when attempting to write out multi-phase-center data sets into the UVH5 file format.
- Added the
UVData.write_msmethod, a UVData measurement set writer. - Added an option to the
UVData.checkmethod to flip the uvws and conjugate the data_array to align with theUVDataconvention on baseline orientation. - Added the
flip_gain_conjoption toutils.uvcalibrateto use the opposite gain conjugation convention in calibration. - Support for selecting on polarization strings on
UVData,UVCal,UVBeamandUVFlag. from_fileclass function on UVData and UVBeam. Allows users to instantiate an object from a file directly from each Class.- Generic 'read' function for UVBeam objects.
- UVData methods
set_data,set_flags, andset_nsampleswhich allow the user to update values in the corresponding arrays. - pyERFA was added as an explicit dependency (previously it was implicit because it is an astropy dependency).
- A bug in the way times were written to uvfits files that could cause a loss of precision.
- A bug that allowed the check to pass on
UVData,UVCalandUVFlagobjects if theflag_arraycontained integers instead of booleans. - A bug in equality testing for
UVParameterobjects with strict types when the parameters contain arrays. - Provide a more useful error message if
UVCal.read_fhd_calis called withread_data=Falseand a settings file is not provided. - Fixed a bug in
UVCal.read_fhd_calwhere the reader crashed on a metadata only read if the settings file had an empty field for the diffuse model. - Fixed a bug in
UVBeam.efield_to_powerwhere a thedata_arrayremained complex rather than real because the tolerance was too low in the numpyreal_if_closecall.
- The
__eq__method on UVBase objects (and subclasses) now supports anallowed_failureskeyword for parameters which are allowed to be unequal without failing the check. This includes thefilenameparameter ofUVDataobjects by default.
- Added
filenameattribute for UVData objects. - Added support for multiple sources/phase centers in a single UVData object, which can
be enabled using the
_set_multi_phase_centermethod. Using this method will set the new attributemulti_phase_centertoTrue(otherwise set toFalse.). - Added the optional attribute
Nphase, of typeintand required ifmulti_phase_center=True, which records the number of different phase centers recorded in the UVData object. - Added the optional attribute
phase_center_catalog, a dictionary of dictionaries that is required ifmulti_phase_center=True, which records various meta-information about the different phase centers used within the UVData object. The keys of individual entries of phase_center_catalog are strings representing the names of the individual phase centers. These keys are matched to individual dictionaries which store various information about the phase centers, including coordinate information (e.g.,cat_lat,cat_lon,cat_frame,cat_epoch) and a unique identification number (cat_id, typeint). Catalog information can be printed to the terminal using the methodprint_phase_center_info. - Added the optional attribute
phase_center_id_array, of typeintand shape(Nblts,)required ifmulti_phase_center=True, which records which phase center the individual baseline-time record is phased to. The individual values of are matched tocat_idfrom the individual dictionary entries ofphase_center_catalog. - Added the attributes
phase_center_app_raandphase_center_app_dec, which records he topocentric apparent positions of the phase center as seen from the positiontelescope_location. Both are of typefloat, shape(Nblts,). - Added the attribute
phase_center_frame_pa, of typefloat, shape(Nblts,), which encodes the position angle between the topocentric apparent frame and the celestial frame of phase center coordinates, as recorded inphase_center_frame(e.g., FK5, ICRS). UV-coordinates are rotated byphase_center_frame_pa, such that - Added a switch called
ignore_nameto__add__which allows one to ignore whether theobject_nameattribute matches when attempting to combine objects (default isignore_name=False, which will cause an error to be thrown ifobject_namedoes not match). This parameter is also available with thereadmethod, when reading in multiple files. - Added a switch called
make_multi_phaseto the__add__method, which will convert the data to a multi-phase-center-enabled dataset, allowing for UVData objects with different phase centers to be combined. This parameter is also available with thereadmethod, when reading in multiple files. - Added the
fix_phasemethod, which converts UVData objects from the "old" phasing framework to the "new' phasing framework. - Added support for looking up ephemeris information of solar system objects from
JPL-Horizons via the optionally required
astroquerypackaged. - Added option to select based on LST or an LST range in UVData objects.
- Added
get_lstsmethod on UVData objects for retrieving LST corresponding to data.
utils.uvcalibratewill error rather than warn if some of the newly added checks do not pass, including if antenna names do not match between the objects.- Phasing methods (e.g.,
UVData.phase,UVData.unphase_to_drift,UVData.set_uvws_from_antenna_positions) have undergone a significant revision, which change the way in UVW coordinates are calculated. Specifically, these methods will now use the topocentric apparent positions (stored in the attributesphase_center_app_ra,phase_center_app_dec). As a result, phasing of data and calculating of uvw-coordinates are significantly faster. phasemethod now supports phase centers of different types, specified by thecat_typeparameter. Supported types include "sidereal" (default; fixed position in RA/Dec), "ephem" (position in RA/Dec which moves with time), and "driftscan" (fixed position in Az/El, NOT the same asphase_type=“drift”). The "ephem" and "driftscan" types can only be used whenmulti_phase_center=True.phasenow uses antenna positions for deriving uvw-coordinates by default.- Decreased the standard tolerances for angle-based parameters to 1 mas (was 10 mas).
- Updated the astropy requirement to >= 4.2.1
- Unknown phasing types (i.e.,
phase_type="unknown") are no longer supported,read(and the associated file-reading methods) will defaultphase_typeto "drift" when unable to definitively ascertain the phasing type.
- A bug in
aipy_extracts.pipethat caused an error when usingraw=True. - A bug in
aipy_extracts.writewhich caused an error when using certain masked arrays. - Phasing accuracy/UVW coordinate calculation no longer limited to 2 cm on a 3 km baseline (i.e.,~1 part in 1e5; new accuracy is better than 1 part in 1e8).
- A bug in reading DSB data from Mir filetypes, that resulted in visibilities being matched to the wrong baselines.
- A bug in
UVData.phase_to_timewhich prevented users from rephasing phased objects. - A bug in
UVBeam.read_cst_beamwhen specifying beams with a single file using a yaml.
- The UVBeam methods
set_cs_params,set_efield,set_power,set_simple, andset_phased_arrayhave been removed. - The UVCal methods
set_gain,set_delay,set_unknown_cal_type,set_sky, andset_redundanthave been removed. - The UVData methods
set_phasedandset_drifthave been removed. - The functions
utils._str_to_bytesandutils._bytes_to_strhave been removed. - The
flag_missingkeyword of theutils.uvcalibratefunction has been removed.
- Added
use_future_array_shapesmethod to allow users to convert to using the future array shapes now, with support throughout UVData methods and related utility functions. Also addeduse_current_array_shapesto revert to the standard shapes. - Added versioning to UVH5 files. Files with future array shapes are version 1.0, files with the current shapes are are version 0.1.
- Added
UVData.reorder_freqsmethod to reorder the data along the frequency axis. - Better re-initialization of UVParameters on UVBase objects when using pickle. Improves compatibility of UVBase objects with MPI.
- Added option to apply a Van Vleck correction to mwa_corr_fits files.
- A bug in mwa_corr_fits that resulted in an error when using
memap=truewhile reading files containing the 'BSCALE' keyword. - A bug in mwa_corr_fits that didn't properly test against 2015 data, and so didn't
catch the error when using
memap=truewhile reading files containing the 'BSCALE' keyword. - A bug in mwa_corr_fits that didn't round start_flag to the nearest multiple of integration time when using goodtime, resulting in an error when reading data taken at 2 seconds.
- A bug in reading in the MWA beam that caused the beam to be rotated by 90 degrees.
- Added
propagate_coarse_flagsoption toread_mwa_corr_fits, as well as new flagging defaults. - Added support for
extra_keywordsin UVFlag construction from UVData objects (and paths)
- Improved memory usage in reading MWA correlator fits files.
- Speed improvement in redundant group finder.
- Fixed a bug in
compress_by_redundancywithmethod='average'that resulted in more unique times on the compressed object than on the original object. - Fixed bug causing MemoryError when finding redundant baselines for over 100 antennas.
- Added support for
telescope_location,antenna_positionsandlst_arrayin UVCal objects and file types. - Added support for a
metadata_onlymode in UVCal, including options to only read the metadata when reading in calibration files. - Added a
copymethod for UVCal objects. - Added
antenna_positions,antenna_namesandantenna_numbersas optional metadata to our known telescopes list and added HERA antenna position information. - Added handling for
extra_keywordsin UVFlag objects, including read/write to/from files. - Added handling for
object_nameandextra_keywordstosum_visanddiff_vismethods and added theoverride_paramsoption to override other parameters.
- Changed to use Astropy sites for telescope locations when avaliable. This results in a small change for our known position for the MWA.
- Modified
UVData.readto do faster concatenation of files, changed the interface toUVData.fast_concatto allow lists ofUVDataobjects to be passed in.
- Fixed a bug where telescope positions from MWA uvfits files created by Cotter were not identified as being in the ITRF frame because of a missing FITS keyword.
- Fixed a bug where
antenna_positionsfrom FHD files were interpreted as being in the relative ECEF frame rather than the rotated ECEF frame - Fixed a bug where the
lst_arraywas not updated incompress_by_redundancywhen usingmethod='average'. - Fixed an undefined parameter bug when interpolating beams that do not cover the sky.
- Fixed an indexing error bug in
compress_by_redundancywhen usingmethod='average'.
- UVCal objects now require
telescope_location,antenna_positionsandlst_arrayparameters. Not setting them will cause an error starting in pyuvdata 2.3.
- Added support for multiple spectral windows for UVH5, UVFITS, MIR, and MIRIAD files.
- Added the
flex_spwattribute to theUVDataclass, which can be set to True by using the new_set_flex_spwmethod. - Added the optional
flex_spw_id_arrayattribute, of type=int and shape=(Nfreqs,), which indexes individual channels along the frequency axis tospw_array. - Adjustment to digital gain removal from mwa_corr_fits files to account for a division by 64 due to a bit selection shift.
- Options to remove coarse band shape and digital gains from mwa_corr_fits files.
- Support for cotter flags in mwa_corr_fits files.
- When
flex_spw=True, thechannel_widthattribute of UVData is expected to be an array of type=float and shape=(Nfreqs,). Individual channels are allowed to have different channel widths. - Changed
freq_arrayto be of shape (1,Nfreqs) (formerley (Nspws,Nfreqs)). - Changed
data_array,flag_array,nsample_arrayto be of shape (Nblts, 1,Nfreqs,Npols) (formerly (Nblts,Nspws,Nfreqs,Npols)). - Changed UVH5 metadata byte conversion method from tobytes() to bytes()
- Added read-only support for MIR files, adding the
Mirclass toUVData. - Added the
MirParserclass, which allows for python access to MIR files. - A new
check_warningsmethod in our test module that behaves likepytest.warnsbut adds the ability to check for multiple warnings. - The 'ant_str' option when making selects on UVFlag objects
- A check that the uvws match the antenna positions, as part of the acceptability checking.
- Updated the numpy requirement to >= 1.18
- Moved
parse_antstoutils.pyto allow any UVBased object the potential to use it.
- The
checkWarningsfunction in our test module, use the newcheck_warningsfunction instead.
UVFlagcan now takePosixPathobjects as input.- An option to average rather than select a single baseline in
UVData.compress_by_redundancy, controlled by themethodkeyword. nsample_array_dtypekeyword forUVData.read, which sets the datatype for the nsample_array. Currently only used for mwa_corr_fits files.utilscython module to speed up some common utility functions.- Added support for installing the package on Windows.
- 'background_lsts' keyword in
uvdata.readto calculate lsts in the background while reading. Defaults to True. backgroundkeyword inuvdata.set_lsts_from_time_arrayto allow lst calculation in a background thread.read_datakeyword to read_mwa_corr_fits, allows from metadata only reads of MWA correlator fits files.propagate_flagskeyword forUVData.frequency_averagewhich flags averaged samples if any contributing samples were flagged
- Nants data calculation changed to use numpy functions for small speed up.
- Changed
inputvariable toindatain UVFlag to avoid shadowing python builtin. selectnow also accepts a list of baseline indices for theblsparameter.data_array_dtypekeyword forUVData.readis now also respected by mwa_corr_fits files. Previously it was only used by uvh5 files.- The
time_rangeparameter on UVCal is no longer required and it is suggested to only set it if the Ntimes axis is length one. - FHD now supports metadata only reads.
- Updated formatting and added more explicit typing in corr_fits.pyx
- Updated Nants calculations for speed up.
- updated HERA telescope location to match the HERA defined center of array.
utils.uvcalibratenow incorporates many more consistency checks between the uvcal and uvdata object. The new keywordstime_checkandant_checkwere added to control some of these checks.
- The UVBeam methods
set_cs_params,set_efield,set_power,set_simple, andset_phased_arrayhave been made private. The public methods will be removed in version 2.2. - The UVCal methods
set_gain,set_delay,set_unknown_cal_type,set_sky, andset_redundanthave been made private. The public methods will be removed in version 2.2. utils.uvcalibratewill error rather than warn if some of the newly added checks do not pass, including if antenna names do not match between the objects, starting in version 2.2. In addition, theflag_missingkeyword is deprecated and will be removed in version 2.2.
- UVFlag.add now properly concatenates all existing data-like parameters of the object, including optional ones.
- A bug in
UVData.downsample_in_timewhere the data were not being properly weighted by the nsample array and the nsample_array was not being properly weighted by the integration times. - A bug in
UVData.downsample_in_timethat lead to duplicated data on the final object if a baseline had varying integration times and some integration times were greater than or equal to the requested minimum integration time.
- New
_corr_fitsC extension for performing the index and conjugation mapping calculation for read_mwa_corr_fits copymethod to UVCal and UVBeam objectschunkstoUVH5.write_uvh5andUVData.write_uvh5for HDF5 dataset chunking.multidim_indextoUVH5.read_uvh5andUVData.readfor multidimensional slicing into HDF5 datasets
- Various serialized calculations in uvdata.py, uvbeam.py, and utils.py to be vectorized with numpy
- Miriad interface was re-written from a hand written CPython interface to a Cython interface which dynamically creates the CPython during package setup/build.
- uvfits, calfits and beamfits files should now be read faster due to a simplified handling of fits header objects, especially when the history is very long.
- The UVData methods
set_drift,set_phased, andset_unknown_phase_typehave been made private. The public methods will be removed in version 2.2.
- UVData allows conjugation of metadata only objects.
- Handling of strings in UVFlag files has been made more widely compatible.
- Option to provide parameters for RectBivariatespline through interp
weights_square_array(optional) parameter on UVFlag - stores sum of squares of weights when converting to waterfallfrequency_averagemethod on UVData to average data along the frequency axis.
- A bug in
selectwhere baseline-polarization tuples would not be conjugated correctly inUVDataandUVFlag, and select on read for Miriad files. metafits_ppds.fitsfiles can now be passed tomwa_corr_fits.readwithout throwing an error.- UVParameters that are array_like and have NaNs are now properly identified as equal if the NaNs are in the same locations and all non-NaN entries are equal.
- A bug in
mwa_corr_fits.readin filling flag and nsample arrays. - A bug in
UVData.downsample_in_timein calculating the number of new blts.
- All references to Python 2 removed from codebase.
- Routine flagging of MWA coarse band edges and center channels, as well as beginning and end integrations. Only done during
read_mwa_corr_fits. - Added a return_weights_square option to
utils.collapsefunction, which returns the sum of the squares of the weights.
- Previously deprecated code marked for removal in version > 1.5:
- reading multiple files with file format specific read functions (e.g. read_mirad). Multi-file reads can only be performed with
uvdata.read read_metadatakeyword in various specific read functions (e.g.read_miriad)metadata_onlykeyword inselectandget_redundanciesuvdata.miriad.read_miriad_metadataphase_centerkeyword inuvdata.read_mwa_corr_fitsphase_datakeyword inuvdata.read_mwa_corr_fitsuvdata.get_antenna_redundanciesuvdata.get_baseline_redundanciesuvdata.uvfits.read_uvfits_metadatauvdata.uvfits.read_uvfits_data
- Support for reading the MWA full embedded element beam into a UVBeam object.
- New
time_rangekeyword toselectso exact times don't need to be specified (also added toreadmethods for select on read). - Support for rephasing phased data including on
read,__add__andfast_concatso that files with different phasing can be read in together. sum_visanddiff_visfor summing or differencing visibilities in the data_array.read_mwa_corr_fitsfor reading in MWA correlator gpubox files and applying cable corrections.eq_coeffsfor storing equalization coefficients andremove_eq_coeffsfor removing them.utils.apply_uvflagfor applying UVFlag objects to UVData objects
- Fixed
utils.uvcalibrateto handlex_orientationattribute - Arrays of file names can now be passed to
read(not just lists). - run_check is no longer turned off in
read_uvfitsandread_uvh5whenread_datais False. - Redundancy finder will now error if any baselines appear in multiple groups.
- A bug in
UVCalobjects that prevented them from properly getting data withee/nn-style polarizations. - a bug in
UVFlagwherex_orientationwas not set during initialization. - A bug in
UVCal.read_fhd_calthat caused calibration solutions to be approximately doubled. - A bug in UVFlag where polarization array states were not updated when using
force_polkeyword into_antennaandto_baseline - A bug in UVFlag.to_baseline() where force_pol kwarg did not work for UVData Npols > 1
UVData.read_uvfitsno longer breaks if there are non-ascii bytes in antenna names (which CASA sometimes writes).
- Reading in multiple files (or file sets) using file-type specific read methods (e.g.
read_uvfits) in favor of the genericreadmethod. - The
phase_centerandphase_datakeywords toread_mwa_corr_fitsin favor ofphase_to_pointing_centerand thephase_center_radeckeyword in the genericreadmethod. - Support for reading only the header (not all the metadata) of uvfits files.
- The
read_uvfits_metadataandread_uvfits_datamethods on the UVFITS object. - The
read_miriad_metadatamethod on the Mirad object.
copymethod forUVData, which can optionally make a copy of just metadataupsample_in_time,downsample_in_time, andresample_in_timemethods onUVDataobjectsutils.uvcalibrate(.., undo=True)kwarg for undo-ing a calibration.utils.uvcalibrateupdatesUVData.vis_unitsifUVCal.gain_scaleis set.UVCal.gain_scalenon-required attribute- UVData.get_redundancies method to replace old get_baseline_redundancies and get_antenna_redundancies.
- option for
UVBeam.interpto return a new beam object. UVFlaginformation on Read The Docs
UVData.phase_to_timenow accepts a float as an input. Assumes float represents a JD.- Added optional acceptability check for
utils.LatLonAlt_from_XYZ - Use
astropy_healpixrather thanhealpyfor HEALPix functionality becausehealpyhas a GPL license which is incompatible with ours. h5pyis now a required package instead of an optional one.- Phasing now supports metadata only
UVDataobjects - utils.get_baseline_redundancies uses scipy pdist functions instead of for loops (faster)
- UVData.get_antenna_redundancies will no longer automatically conjugate baselines.
- UVData.get_baseline_redundancies and UVData.get_antenna_redundancies have been combined.
UVFlaginherits fromUVBaseobject.UVFlagobjects can now convert from antenna type to baseline typeUVFlagobjects can now be initialized without inputs
- A bug in UVBeam._interp_freq where kind parameter was not passed for real-only beams
- A bug in get_antenna_redundancies for nearly N-S baselines.
- A bug where
conj_polcould not handle cardinal direction polarizations. - A bug that gave the wrong error message when calling
UVData.phase_to_timewithout an Astropy Time object.
- UVData.get_baseline_redundancies and UVData.get_antenna_redundancies will be deprecated in version 1.6.1
metadata_onlyproperty onUVDatato automatically detect if data-like arrays are present- support for combining metadata only objects and reading in multiple files as metadata only
utils.uvcalibrateflag propagation bug fixUVCal.ant2indindexing bug fixUVCal.get_*methods for accessing data arrays with antenna-polarization keysutils.uvcalibratefor automated calibration ofUVDatabyUVCal
- Fixed a bug in select that caused bls and antenna_names/numbers to be or'ed rather than and'ed together.
- Fixed a bug where
baseline_to_antnumscould accept a numpy array as input but not other array_like objects.
- removed
new_objectkeyword fromUVBeam._interp_freqin favor of new functionality inUVBeam.interp
- Option in UVBase.check() to ignore whether required parameters are set.
- made an option to not save the
lst_arrayto uvfits files. conjugate_blsoption toUVData.get_antenna_redundanciesUVData.conjugate_blsmethod to conjugate baselines to get the desired baseline directions.UVData.reorder_bltsmethod to reorder the data along the blt axis (and optionally also conjugate baselines), and a newblt_orderoptional parameter on UVData objects to track the ordering (including through read/writes).lst_arrayis now saved to UVFITS files (even though it's not a standard parameter) so that it doesn't have to be recalculated
- Fixed init logic in UVFlag.
- Fixed a bug in how FHD uvw vectors were oriented (and visibilities were conjugated)
- Fixed a bug in
UVData.inflate_by_redundancywhen Nblts is not equal to Nbls * Ntimes. - Fixed a bug in UVData when reading in an FHD file with a single time integration.
- Fixed a bug in how the longitudinal branch cut was handled in beam interpolation
- Changed the way interpolation splines are saved in UVBeam to fix errors related to polarization selections.
- Python 3:
np.string_call now useskeepdims=Trueto guard againstantenna_namesbeing cast as an array.
- Testing framework changed from
nosetopytest. uvdata.set_lsts_from_time_arrayonly calculates lsts for unique elements intime_array.
- Optional
x_orientationparameter to utils functions polstr2num, polnum2str, jstr2num and jnum2str to allow for E/N based polarization strings (rather than just x/y based ones) - New optional
x_orientationparameter on UVBeam (paralleling UVData and UVCal), with read/write support in beamfits - Added
x_orientationas an optional parameter in read_cst_beam and in cst settings yaml files. - All str2num or num2str calls on UVData and UVBeam pass the object's x_orientation
- New
UVData.fast_concatmethod to allow fast concatenation of UVData objects (or files) along a particular axis. - Added preliminary
UVFlagmodule from hera_qm to pyuvdata. Will eventually promote toUVBaseobject, but for now this is undocumented functionality.
- Defined 'east' and 'north' as the allowed 'x_orientation' values in UVData and UVCal, Backwards compatiblity support exists for 'E' and 'N' values
UVData.order_polsmethod in favor ofUVData.reorder_pols.
- Building pyuvdata on macOS now targets minimum macOS 10.9 if run on macOS 10.9 or above
- Possible bug where
check_variablesdictionary can change size duringread_miriadcall
- Added
add_to_historykwarg to UVH5.write_uvh5_part _healpix_interp_bilinearas a new interpolation method inUVBeamfreq_interpolation_kindadded as an attribute toUVBeamtoladded as keyword argument toUVBeam._interp_freqwhich allows for a fast return ofdata_arrayslice if nearest-neighbor frequencies are all within the distance tolerance.polarizationsadded as keyword argument toUVBeaminterpolation methods.- Support for a yaml settings file to collect and propagate metadata for CST beam files.
UVBeam._interp_freqreturns bothinterp_dataandinterp_bandpass, instead of just the former.
- Combining overlapping data along multiple axes (most common when reading in multiple files) no longer errors.
keep_all_metadatakeyword for optionally discarding unused metadata when performing a select operation.
- Extends
run_acceptability_checkfor UVH5 metadata incheck_headerfunction.
- Antenna numbering bug in redundancy methods. It wasn't using the correct antenna numbers to make baseline indices.
- Redundancy code returns one group if all baselines are redundant. Previously returned each baseline as a separate group
- Redundancy code finds unique baselines along baseline_array without assuming Nblts = Nbls * Ntimes. Previously assumed Nblts = Nbls * Ntimes and attempted to slice array.
- "inflate_by_redundancy" method errored when phase_type == phased, due to _set_u_positive using phased uvw coordinates. It now uses ENU frame uvw coordinates.
- Methods on UVData objects to compress/inflate data by redundant baselines.
- Convenience functions on UVData for finding redundant baselines (calling the corresponding utils functions)
- memo describing the UVH5 format
- read/write support for uvh5 files with integer datatypes for visibilities
- Option to only do the select on the metadata. This is useful for partially defined objects as in pyuvsim setup or after reading only the metadata from a file.
- support for python3.7
- UVdata.get_ENU_antpos() now defaults to using the telescope_location as the center rather than the median antenna position.
- UVBeam.efield_to_pstokes() no longer restricted to healpix coordinates
- latitude and longitude in uvh5 files are written in degrees instead of radians.
- Fixes a bug in redundancy methods for when there are no redundant baselines.
_key2indsnow properly reorders polarization axis for conjugated visibilities. This also effects theget_datafunction.- long strings are saved correctly in miriad files from python3
- option to save splines for reuse in UVBeam.interp function
- improve string handling for uvh5 files
- changed top-level import structure to exclude file-specific class (e.g.
UVFITS,CALFITS) and base classes (UVBase,UVParameter) and to not import utility functions into the top-level namespace
- Support for UVData objects without antenna_positions. Antenna positions will be required in a future version.
- Utility functions to identify redundant baselines (either by baseline or antenna pair)
- Reading FHD layout files to get antenna positions
- Polarization dict constants and parsing functions in utils.py for mapping between polarization strings and numbers
- LST array is now optional in uvh5 files
- polarization string capitalization was unified throughout: use lower case except for Stokes parameters
- integration_time is now a vector of length NBlts instead of a scalar
- Support for FHD inputs without layout files (containing antenna positions).
- flags are always returned as a boolean array from
UVData.get_flags - integration_time, uvw_array and lst_array are now always checked for consistency when adding UVData objects
- consistency checks on baseline lengths now uses the uvw_array tolerances
- compatibility with python 3 (retaining python 2 compatibility)
- generic
UVData.readmethod for all input file types, including select on read options - partial write support for uvh5 file format
- partial read support for uvfits, miriad and uvh5 file formats (including only reading metadata and select on read)
- new uvh5 file format: an HDF5 file standard matched to UVData objects
- new method to calculate uvws from antenna positions
UVBeam.get_beam_areaandUVBeam.get_beam_sq_areafunctions to calculate beam integrals, including for pseudo-Stokes beams- beam interpolation methods, to any set of points and to healpix pixel centers
- a script to renumber antennas for CASA compatiblity if there are fewer than 256 antennas but numbers higher than that
- memo describing the beam fits file format
- method to peak normalize UVBeam objects
- support for reading FHD calibrations into UVCal objects
- support for sky-based calibration metadata in UVCal and the calfits file format
- method to convert E-field beams to power beams
UVData.get_ENU_antposmethod to get ENU coordinates from antenna positions- support for
extra_keywordson UVCal objects
- Major overhaul of phasing code, see the phasing memo in docs/references for more details
- Make all earth location coordinate conversions use same axes order (n_points, 3)
- replaced
ant_pair_numskeyword inUVData.selectand partial read methods withblswhich supports lists like [(0,1,'xx'), (2,3,'yy')] - extracted miriad wrappers from aipy, removing aipy as a dependency
- Fixed error when combining auto-correlation only and cross-correlation only UVData objects
- Fixed error with the
UVData.selectfunction using a single antenna name - Fixed bugs with
UVData.get_data,UVData.get_flags, andUVData.get_nsamplesfor conjugated baseline polarizations - Fixed a bug that caused a memory error in
UVData.write_uvfits - Fixed bugs in interpreting the uvw direction convention for uvfits
- Fixed a bug in reading azimuth locations from CST beam files
- Fixed a bug reading in single frequency uvfits files
- Fixed a bug in reading MWA Cotter measurement sets
- Fixed units errors in cal fits files
- Fixed a serious bug where data was overwritten in the add functions if the axes were out of canonical order
- Fixed scrambled data ordering in add function
- Fixed a bug in parsing frequencies from CST file names
- support doctest in the tutorial
- utility functions for converting between Jones numbers and polarization strings
- support for
antdiamvariable in miriad files - module for reading a CST file into a UVBeam object
- utility functions for converting to rotated ECEF coordinates from ECEF coordinates
- support reading in a list of calfits files
- include
extra_keywordsto miriad files __add__method for UVCal objectsantenna_diametersvalue to UVData objectsparse_antsmethod on UVData objectsx_orientationvalue to UVData objects- "smart slicing" functionality to UVData objects
- convenience methods on UVData objects for easily getting data and metadata
- UVBeam object
- in-place selection for UVData objects
total_quality_arrayvalue on UVCal objects__add__method for UVData objects- utility functions for converting to local ENU coordinates from ECEF coordinates
convert_to_gainmethod on UVCal delay-type objects- read-only support for CASA measurement sets into UVData objects
selectmethod on UVCal objects
- antenna names in miriad saved as strings instead of arrays of ASCII hex values
- baseline-time axis mis-ordering in add function
- handling of antenna positions in miriad and uvfits files
- selecting autocorrelation data from UVData objects
- indexing of spectral windows in calfits files
- handling of
total_quality_arrayin UVCal objects when selecting a subset of data
(historical information needs to be filled in)