diff --git a/Prompt/100_How_to_use_RSP_tools/101_Alert_archive/101_1_Alert_retrieval_service_Herald.ipynb b/Prompt/100_How_to_use_RSP_tools/101_Alert_archive/101_1_Alert_retrieval_service_Herald.ipynb index 00f6144..03daa09 100644 --- a/Prompt/100_How_to_use_RSP_tools/101_Alert_archive/101_1_Alert_retrieval_service_Herald.ipynb +++ b/Prompt/100_How_to_use_RSP_tools/101_Alert_archive/101_1_Alert_retrieval_service_Herald.ipynb @@ -22,7 +22,7 @@ "Data Release: [Prompt Products](https://prompt-products.lsst.io/)\\\n", "Container Size: Large\\\n", "LSST Science Pipelines version: r29.2.0\\\n", - "Last verified to run: 2026-04-24\\\n", + "Last verified to run: 2026-05-04\\\n", "Repository: [github.com/lsst/tutorial-notebooks](https://github.com/lsst/tutorial-notebooks)\\\n", "DOI: [10.11578/rubin/dc.20250909.20](https://doi.org/10.11578/rubin/dc.20250909.20)" ] @@ -59,10 +59,10 @@ "**Alert packets** are files of measurements for sources detected in difference images that are streamed to brokers within a few minutes of image acquisition.\n", "Alerts are a Prompt Product, and are the result of Rubin's Difference Image Analysis (DIA) and Alert Production (AP) pipelines.\n", "\n", - "**Brokers** are the recommended way to do time-domain science with alerts because their functionality includes, e.g., filtering, cross-match, classification, and users can explore and retrieve scientifically-relevant subsets via their web-based user interfaces or via API clients.\n", + "**Brokers** are the recommended way to do time-domain science with alerts because their functionality includes, e.g., filtering, cross-match, and classification, and users can explore and retrieve scientifically-relevant subsets via their web-based user interfaces or API clients.\n", "\n", "**Why use the RSP's alert retrieval service, Herald?** \\\n", - "When only the contents of a single alert packet are desired, and the alert ID is known.\n", + "Herald is useful when only the contents of a single alert packet are desired, and the alert ID is known.\n", "Otherwise, brokers are the best way to do real-time science with the alerts, and scientific analyses on longer timescales (days to weeks) can use the other Prompt Products (queryable databases, images).\n", "\n", "**Additional resources:**\n", @@ -302,7 +302,7 @@ "source": [ "#### 2.1.2. Lightcurve\n", "\n", - "Extract the band, MJD, and forced PSF difference-image flux from the alert record as `numpy` arrays, and plot the lightcurve." + "Extract the band, MJD, and forced PSF difference-image fluxes from the alert record as `numpy` arrays, and plot the lightcurve." ] }, { @@ -456,7 +456,7 @@ } }, "source": [ - "Avro schemas are written in JSON, so convert in order to explore." + "Avro schemas are written in JSON, so use the `response.json()` method to convert to a Python dict in order to explore." ] }, { @@ -474,7 +474,7 @@ "id": "633b43f1-db19-444b-95b4-19e14b84379a", "metadata": {}, "source": [ - "This converts to JSON but use native or something else in the Avro section " + "Print some fields from the schema." ] }, { @@ -597,7 +597,7 @@ "id": "5da256b2-fe69-4dae-b3e7-ba1d5472c4fa", "metadata": {}, "source": [ - "> **Figure 3:** The science, template, and difference-image stamps." + "> **Figure 3:** The difference-image, science, and template stamps." ] }, { @@ -625,7 +625,7 @@ "source": [ "## 3. JSON format\n", "\n", - "The JSON response is the full deserialized alert record." + "The JSON response is the full deserialized alert record. Request the JSON formatted response by using the `RESPONSEFORMAT` keyword." ] }, { @@ -719,7 +719,7 @@ "id": "13083cd9-5ee4-40f8-bca1-ef31c1762f72", "metadata": {}, "source": [ - "Option to display the stamps. The code is very similar to that used for the Avro packets, except in the JSON formatted packet the image stamps are base64-encoded bytes. They need to be first decoded with `base64.b64decode` then read as bytes with `io.BytesIO` by `fits.open`." + "Option to display the stamps. The code is very similar to that used for the Avro packets, except in the JSON formatted packet the image stamps are base64-encoded bytes. They need to be first decoded with `base64.b64decode` then read as bytes with `io.BytesIO` by `fits.open`. Note also that the stamps are in a different order than in the Avro packets." ] }, { @@ -732,7 +732,7 @@ "# stamps = [base64.b64decode(record.get(\"cutoutDifference\")),\n", "# base64.b64decode(record.get(\"cutoutScience\")),\n", "# base64.b64decode(record.get(\"cutoutTemplate\"))]\n", - "# stamp_names = ['Science', 'Template', 'Difference']\n", + "# stamp_names = ['Difference', 'Science', 'Template', ]\n", "\n", "# fig, ax = plt.subplots(1, 3, figsize=(9, 3))\n", "# for s, (stamp, name) in enumerate(zip(stamps, stamp_names)):\n", @@ -844,7 +844,24 @@ "outputs": [], "source": [ "hdu = hdul['PRIMARY']\n", - "hdu.header\n", + "hdu.header" + ] + }, + { + "cell_type": "markdown", + "id": "5ba9785f-a260-4bca-ba22-e762cee48762", + "metadata": {}, + "source": [ + "Clean up." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8985dd43-b8c5-4a85-8f99-8c8444f5ff19", + "metadata": {}, + "outputs": [], + "source": [ "del hdu" ] }, @@ -932,7 +949,7 @@ "# mew=1, alpha=0.7, mec=filter_colors[filt], color='None', label=filt)\n", "# del fx\n", "# plt.xlabel('MJD')\n", - "# plt.ylabel('forced difference-image flux [nJy]')\n", + "# plt.ylabel('difference-image PSF flux [nJy]')\n", "# plt.legend(bbox_to_anchor=(1.05, 1), handletextpad=0, loc='upper left')\n", "# plt.tight_layout()\n", "# plt.show()\n", @@ -945,12 +962,80 @@ "id": "88fe362b-74f3-47e2-9e12-e90329c4cfce", "metadata": {}, "source": [ - "**FORCEDPHOT** \\\n", - "Contains the forced photometry history.\n", + "**FORCEDPHOT**\n", + "\n", + "The `FORCEDPHOT` extension contains forced measurement data on all previous visits at the position of the `diaSource` (the lightcurve). This includes measurements on the difference images and the science images.\n", + "\n", + "Option to extract the `diaSource` data and plot the forced photometry lightcurve." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f14e6bb2-6daf-425f-acfb-08d23044fedd", + "metadata": {}, + "outputs": [], + "source": [ + "# hdu = hdul['FORCEDPHOT']\n", + "# data = hdu.data\n", + "# fig = plt.figure(figsize=(6, 3))\n", + "# for f, filt in enumerate(filter_names):\n", + "# fx = np.where(data['band'] == filt)[0]\n", + "# if len(fx) > 0:\n", + "# plt.plot(data['midpointMjdTai'][fx], data['psfFlux'][fx], filter_symbols[filt], ms=5,\n", + "# mew=1, alpha=0.7, mec=filter_colors[filt], color='None', label=filt)\n", + "# del fx\n", + "# plt.xlabel('MJD')\n", + "# plt.ylabel('forced difference-image flux [nJy]')\n", + "# plt.legend(bbox_to_anchor=(1.05, 1), handletextpad=0, loc='upper left')\n", + "# plt.tight_layout()\n", + "# plt.show()\n", + "# del data\n", + "# del hdu" + ] + }, + { + "cell_type": "markdown", + "id": "d8f5c41d-3ade-48ca-acec-f50c6f6ea080", + "metadata": {}, + "source": [ + "**SSSOURCE**\n", "\n", - "**SSSOURCE** \\\n", - "For alerts associated with moving objects.\n", - "Contains the LSST-computed per-source instantaneous quantities at the time of observation (e.g., heliocentric and topocentric positions and velocities)." + "The `SSSOURCE` extension contains the LSST-computed per-source instantaneous quantities at the time of observation (e.g., heliocentric and topocentric positions and velocities) for alerts associated with moving objects (i.e., Solar System objects).\n", + "\n", + "Using an ID of a known Solar System object (SSO), retrieve the `SSSOURCE` information." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5521e26b-372f-4bb3-b7b4-0b8dff89b19c", + "metadata": {}, + "outputs": [], + "source": [ + "sso_alert_id = \"170059294376985743\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b2591fd5-d81f-4b86-a63d-bc038e9fedc9", + "metadata": {}, + "outputs": [], + "source": [ + "sso_response = await client.get(url, params={\"ID\": sso_alert_id, \"RESPONSEFORMAT\": \"fits\"})\n", + "sso_response.raise_for_status()\n", + "sso_hdul = fits.open(io.BytesIO(sso_response.content))\n", + "for hdu in sso_hdul:\n", + " print(hdu.name)" + ] + }, + { + "cell_type": "markdown", + "id": "08ccc9f5-c81c-4a30-9370-c31875505774", + "metadata": {}, + "source": [ + "Note that the header cards are the same for all alerts. However, this alert will have data in the `SSSOURCE` extension. Extract that extension, and examine the `ssObjectId` of the alert." ] }, { @@ -959,6 +1044,37 @@ "id": "71595613-25b8-45c3-a730-b24e215a5284", "metadata": {}, "outputs": [], + "source": [ + "hdu = sso_hdul['SSSOURCE']\n", + "sso_data = hdu.data\n", + "print(sso_data['ssObjectId'])" + ] + }, + { + "cell_type": "markdown", + "id": "409d9c8b-b503-4b6c-8691-a57a40d0e915", + "metadata": {}, + "source": [ + "Option to list the names of all the columns in `SSSOURCE`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "804f1a16-59bc-444c-9760-d01ea196d349", + "metadata": {}, + "outputs": [], + "source": [ + "# for col in sso_hdul['SSSOURCE'].columns:\n", + "# print(col)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cfda84a2-1323-4625-9abd-de3ddf9ae8d9", + "metadata": {}, + "outputs": [], "source": [] } ], diff --git a/Prompt/200_Data_products/201_Alerts/201_1_Alert_packets.ipynb b/Prompt/200_Data_products/201_Alerts/201_1_Alert_packets.ipynb index 33224bb..02cf136 100644 --- a/Prompt/200_Data_products/201_Alerts/201_1_Alert_packets.ipynb +++ b/Prompt/200_Data_products/201_Alerts/201_1_Alert_packets.ipynb @@ -31,7 +31,7 @@ "Data Release: [Prompt Products](https://prompt-products.lsst.io/)\\\n", "Container Size: Large\\\n", "LSST Science Pipelines version: r29.2.0\\\n", - "Last verified to run: 2026-04-24\\\n", + "Last verified to run: 2026-05-05\\\n", "Repository: [github.com/lsst/tutorial-notebooks](https://github.com/lsst/tutorial-notebooks)\\\n", "DOI: [10.11578/rubin/dc.20250909.20](https://doi.org/10.11578/rubin/dc.20250909.20)" ] @@ -80,8 +80,8 @@ "4. Image cutouts (stamps) from the science, template, and difference images, for the triggering `diaSource`.\n", "\n", "**How do alerts relate to Rubin's other Prompt Products?** \\\n", - "The data in the alert packets will also stored in a queryable database, the Prompt Products Database (PPDB), which will update on a ~24-hour timescale and will available via the Rubin Science Platform (RSP).\n", - "The contents of the PPDB are public but RSP access is not (unlike broker access).\n", + "The data in the alert packets will also be stored in a queryable database, the Prompt Products Database (PPDB), which will update on a ~24-hour timescale and will be available via the Rubin Science Platform (RSP).\n", + "The *contents* of the PPDB are public but RSP access is not (unlike broker access).\n", "The processed visit images and difference images become available via the RSP after an 80-hour embargo, and are subject to the two-year proprietary period. (See the [Rubin Data Policy](https://rubinobservatory.org/for-scientists/data-products/data-policy).)\n", "\n", "**Additional resources:**\n", @@ -234,8 +234,8 @@ "Other formats (JSON, FITS) are possible, as demonstrated in the 100-level tutorial on the alert retrieval service.\n", "\n", "Pass the `url` to the `client` along with the `alert_id`.\n", - "Use the `await` command to pause execute of the cell's code until a response is received, then show the `response`.\n", - "Retreive the alert for the `snia_alert_id`." + "Use the `await` command to pause execution of the cell's code until a response is received, then show the `response`.\n", + "Retrieve the alert for the `snia_alert_id`." ] }, { @@ -332,7 +332,7 @@ "source": [ "### 2.2. Packet schema\n", "\n", - "The `fastavro.reader` used above reads the embedded schema automatically\n", + "The `fastavro.reader` used above reads the embedded schema automatically.\n", "\n", "Extract the schema, and show that the keys and fields are the same for each (but for triggering `diaSources` that are associated with a `diaObject` instead of a `ssObject`, the fields `ssSource` and `mpc_orbits` won't be populated)." ] @@ -493,7 +493,7 @@ "id": "aae8cf9b-c0ab-40d3-95f4-dacdc5a9ebe7", "metadata": {}, "source": [ - "Option to print the keys of `diaSource`, which are the same all objects." + "Option to print the keys of `diaSource`, which are the same for all objects." ] }, { @@ -516,7 +516,7 @@ "* `band`: LSST filter of the observation, one of: $ugrizy$.\n", "* `midpointMjdTai`: The MJD at the midpoint of the exposure.\n", "* `psfFlux`, `psfFluxErr`: The PSF flux and its error, measured on the difference image, in nJy.\n", - "* `scienceFlux`, `scienceFluxErr`: This PSF flux and its error, measured on the science image, in nJy.\n", + "* `scienceFlux`, `scienceFluxErr`: The PSF flux and its error, measured on the science image, in nJy.\n", "\n", "> **Warning:** while the science flux is useful for variable stars, for supernovae it is typically contaminated by host galaxy light and is inappropriate for use in a lightcurve.\n", "\n", @@ -767,11 +767,11 @@ "\n", "* `ssObjectid`: Unique ID in the `ssObject` table.\n", "* `phaseAngle`: Phase angle between the Sun, object, and observer, in degrees.\n", - "* `eclBeta`, `eclLambda`: The ecliptic latitude and logintude, in degrees.\n", + "* `eclBeta`, `eclLambda`: The ecliptic latitude and longitude, in degrees.\n", "* `elongation`: Solar elongation, in degrees.\n", "* `ephVmag`: Predicted magnitude in V band.\n", "\n", - "The heliocentric and topocentric positions and velocities (`helio_` and `topo_` `x`, `y`, `z`, `vx`, `vy`, `vz`) and also included, among other parameters.\n", + "The heliocentric and topocentric positions and velocities (`helio_` and `topo_` `x`, `y`, `z`, `vx`, `vy`, `vz`) are also included, among other parameters.\n", "\n", "Print a few of the key column values." ] @@ -929,7 +929,7 @@ "id": "ea1a9163-5706-4a55-ae75-da94df78f3a1", "metadata": {}, "source": [ - "Plot separately the lightcurves using the detections (plus the triggering detection), and using the force photometry.\n", + "Plot separately the lightcurves using the detections (plus the triggering detection), and using the forced photometry.\n", "\n", "The fluxes in the g, r, and i bands overlap, so impose a flux offset by filter when plotting." ] @@ -1114,7 +1114,7 @@ "id": "aa4d7992-b4f8-4725-bd8a-e01a90b2badb", "metadata": {}, "source": [ - "> **Figure 2:** Image stamps from the SNIa alert packet. The literature coordinats of the SNIa are marked with a yellow circle, and the north direction indicated with a cyan line." + "> **Figure 2:** Image stamps from the SNIa alert packet. The literature coordinates of the SNIa are marked with a yellow circle, and the north direction indicated with a cyan line." ] }, {