Skip to content

U/jcarlin/patch alert n bs #147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jeffcarlin wants to merge 2 commits into
base: tickets/SP-3129
Choose a base branch
from
Open

U/jcarlin/patch alert n bs #147

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
89252d8
Add full version of 101_1 alerts NB
jeffcarlin May 5, 2026
4e0f66a
Add full version of 201_1 alerts NB
jeffcarlin May 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 133 additions & 17 deletions Prompt/100_How_to_use_RSP_tools/101_Alert_archive/101_1_Alert_retrieval_service_Herald.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
"Data Release: [Prompt Products](https://prompt-products.lsst.io/)\\\n",
"Container Size: Large\\\n",
"LSST Science Pipelines version: r29.2.0\\\n",
"Last verified to run: 2026-04-24\\\n",
"Last verified to run: 2026-05-04\\\n",
"Repository: [github.com/lsst/tutorial-notebooks](https://github.com/lsst/tutorial-notebooks)\\\n",
"DOI: [10.11578/rubin/dc.20250909.20](https://doi.org/10.11578/rubin/dc.20250909.20)"
]
Expand Down Expand Up @@ -59,10 +59,10 @@
"**Alert packets** are files of measurements for sources detected in difference images that are streamed to brokers within a few minutes of image acquisition.\n",
"Alerts are a Prompt Product, and are the result of Rubin's Difference Image Analysis (DIA) and Alert Production (AP) pipelines.\n",
"\n",
"**Brokers** are the recommended way to do time-domain science with alerts because their functionality includes, e.g., filtering, cross-match, classification, and users can explore and retrieve scientifically-relevant subsets via their web-based user interfaces or via API clients.\n",
"**Brokers** are the recommended way to do time-domain science with alerts because their functionality includes, e.g., filtering, cross-match, and classification, and users can explore and retrieve scientifically-relevant subsets via their web-based user interfaces or API clients.\n",
"\n",
"**Why use the RSP's alert retrieval service, Herald?** \\\n",
"When only the contents of a single alert packet are desired, and the alert ID is known.\n",
"Herald is useful when only the contents of a single alert packet are desired, and the alert ID is known.\n",
"Otherwise, brokers are the best way to do real-time science with the alerts, and scientific analyses on longer timescales (days to weeks) can use the other Prompt Products (queryable databases, images).\n",
"\n",
"**Additional resources:**\n",
Expand Down Expand Up @@ -302,7 +302,7 @@
"source": [
"#### 2.1.2. Lightcurve\n",
"\n",
"Extract the band, MJD, and forced PSF difference-image flux from the alert record as `numpy` arrays, and plot the lightcurve."
"Extract the band, MJD, and forced PSF difference-image fluxes from the alert record as `numpy` arrays, and plot the lightcurve."
]
},
{
Expand Down Expand Up @@ -456,7 +456,7 @@
}
},
"source": [
"Avro schemas are written in JSON, so convert in order to explore."
"Avro schemas are written in JSON, so use the `response.json()` method to convert to a Python dict in order to explore."
]
},
{
Expand All @@ -474,7 +474,7 @@
"id": "633b43f1-db19-444b-95b4-19e14b84379a",
"metadata": {},
"source": [
"This converts to JSON but use native or something else in the Avro section "
"Print some fields from the schema."
]
},
{
Expand Down Expand Up @@ -597,7 +597,7 @@
"id": "5da256b2-fe69-4dae-b3e7-ba1d5472c4fa",
"metadata": {},
"source": [
"> **Figure 3:** The science, template, and difference-image stamps."
"> **Figure 3:** The difference-image, science, and template stamps."
]
},
{
Expand Down Expand Up @@ -625,7 +625,7 @@
"source": [
"## 3. JSON format\n",
"\n",
"The JSON response is the full deserialized alert record."
"The JSON response is the full deserialized alert record. Request the JSON formatted response by using the `RESPONSEFORMAT` keyword."
]
},
{
Expand Down Expand Up @@ -719,7 +719,7 @@
"id": "13083cd9-5ee4-40f8-bca1-ef31c1762f72",
"metadata": {},
"source": [
"Option to display the stamps. The code is very similar to that used for the Avro packets, except in the JSON formatted packet the image stamps are base64-encoded bytes. They need to be first decoded with `base64.b64decode` then read as bytes with `io.BytesIO` by `fits.open`."
"Option to display the stamps. The code is very similar to that used for the Avro packets, except in the JSON formatted packet the image stamps are base64-encoded bytes. They need to be first decoded with `base64.b64decode` then read as bytes with `io.BytesIO` by `fits.open`. Note also that the stamps are in a different order than in the Avro packets."
]
},
{
Expand All @@ -732,7 +732,7 @@
"# stamps = [base64.b64decode(record.get(\"cutoutDifference\")),\n",
"# base64.b64decode(record.get(\"cutoutScience\")),\n",
"# base64.b64decode(record.get(\"cutoutTemplate\"))]\n",
"# stamp_names = ['Science', 'Template', 'Difference']\n",
"# stamp_names = ['Difference', 'Science', 'Template', ]\n",
"\n",
"# fig, ax = plt.subplots(1, 3, figsize=(9, 3))\n",
"# for s, (stamp, name) in enumerate(zip(stamps, stamp_names)):\n",
Expand Down Expand Up @@ -844,7 +844,24 @@
"outputs": [],
"source": [
"hdu = hdul['PRIMARY']\n",
"hdu.header\n",
"hdu.header"
]
},
{
"cell_type": "markdown",
"id": "5ba9785f-a260-4bca-ba22-e762cee48762",
"metadata": {},
"source": [
"Clean up."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8985dd43-b8c5-4a85-8f99-8c8444f5ff19",
"metadata": {},
"outputs": [],
"source": [
"del hdu"
]
},
Expand Down Expand Up @@ -932,7 +949,7 @@
"# mew=1, alpha=0.7, mec=filter_colors[filt], color='None', label=filt)\n",
"# del fx\n",
"# plt.xlabel('MJD')\n",
"# plt.ylabel('forced difference-image flux [nJy]')\n",
"# plt.ylabel('difference-image PSF flux [nJy]')\n",
"# plt.legend(bbox_to_anchor=(1.05, 1), handletextpad=0, loc='upper left')\n",
"# plt.tight_layout()\n",
"# plt.show()\n",
Expand All @@ -945,12 +962,80 @@
"id": "88fe362b-74f3-47e2-9e12-e90329c4cfce",
"metadata": {},
"source": [
"**FORCEDPHOT** \\\n",
"Contains the forced photometry history.\n",
"**FORCEDPHOT**\n",
"\n",
"The `FORCEDPHOT` extension contains forced measurement data on all previous visits at the position of the `diaSource` (the lightcurve). This includes measurements on the difference images and the science images.\n",
"\n",
"Option to extract the `diaSource` data and plot the forced photometry lightcurve."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f14e6bb2-6daf-425f-acfb-08d23044fedd",
"metadata": {},
"outputs": [],
"source": [
"# hdu = hdul['FORCEDPHOT']\n",
"# data = hdu.data\n",
"# fig = plt.figure(figsize=(6, 3))\n",
"# for f, filt in enumerate(filter_names):\n",
"# fx = np.where(data['band'] == filt)[0]\n",
"# if len(fx) > 0:\n",
"# plt.plot(data['midpointMjdTai'][fx], data['psfFlux'][fx], filter_symbols[filt], ms=5,\n",
"# mew=1, alpha=0.7, mec=filter_colors[filt], color='None', label=filt)\n",
"# del fx\n",
"# plt.xlabel('MJD')\n",
"# plt.ylabel('forced difference-image flux [nJy]')\n",
"# plt.legend(bbox_to_anchor=(1.05, 1), handletextpad=0, loc='upper left')\n",
"# plt.tight_layout()\n",
"# plt.show()\n",
"# del data\n",
"# del hdu"
]
},
{
"cell_type": "markdown",
"id": "d8f5c41d-3ade-48ca-acec-f50c6f6ea080",
"metadata": {},
"source": [
"**SSSOURCE**\n",
"\n",
"**SSSOURCE** \\\n",
"For alerts associated with moving objects.\n",
"Contains the LSST-computed per-source instantaneous quantities at the time of observation (e.g., heliocentric and topocentric positions and velocities)."
"The `SSSOURCE` extension contains the LSST-computed per-source instantaneous quantities at the time of observation (e.g., heliocentric and topocentric positions and velocities) for alerts associated with moving objects (i.e., Solar System objects).\n",
"\n",
"Using an ID of a known Solar System object (SSO), retrieve the `SSSOURCE` information."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5521e26b-372f-4bb3-b7b4-0b8dff89b19c",
"metadata": {},
"outputs": [],
"source": [
"sso_alert_id = \"170059294376985743\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b2591fd5-d81f-4b86-a63d-bc038e9fedc9",
"metadata": {},
"outputs": [],
"source": [
"sso_response = await client.get(url, params={\"ID\": sso_alert_id, \"RESPONSEFORMAT\": \"fits\"})\n",
"sso_response.raise_for_status()\n",
"sso_hdul = fits.open(io.BytesIO(sso_response.content))\n",
"for hdu in sso_hdul:\n",
" print(hdu.name)"
]
},
{
"cell_type": "markdown",
"id": "08ccc9f5-c81c-4a30-9370-c31875505774",
"metadata": {},
"source": [
"Note that the header cards are the same for all alerts. However, this alert will have data in the `SSSOURCE` extension. Extract that extension, and examine the `ssObjectId` of the alert."
]
},
{
Expand All @@ -959,6 +1044,37 @@
"id": "71595613-25b8-45c3-a730-b24e215a5284",
"metadata": {},
"outputs": [],
"source": [
"hdu = sso_hdul['SSSOURCE']\n",
"sso_data = hdu.data\n",
"print(sso_data['ssObjectId'])"
]
},
{
"cell_type": "markdown",
"id": "409d9c8b-b503-4b6c-8691-a57a40d0e915",
"metadata": {},
"source": [
"Option to list the names of all the columns in `SSSOURCE`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "804f1a16-59bc-444c-9760-d01ea196d349",
"metadata": {},
"outputs": [],
"source": [
"# for col in sso_hdul['SSSOURCE'].columns:\n",
"# print(col)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cfda84a2-1323-4625-9abd-de3ddf9ae8d9",
"metadata": {},
"outputs": [],
"source": []
}
],
Expand Down
24 changes: 12 additions & 12 deletions Prompt/200_Data_products/201_Alerts/201_1_Alert_packets.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
"Data Release: [Prompt Products](https://prompt-products.lsst.io/)\\\n",
"Container Size: Large\\\n",
"LSST Science Pipelines version: r29.2.0\\\n",
"Last verified to run: 2026-04-24\\\n",
"Last verified to run: 2026-05-05\\\n",
"Repository: [github.com/lsst/tutorial-notebooks](https://github.com/lsst/tutorial-notebooks)\\\n",
"DOI: [10.11578/rubin/dc.20250909.20](https://doi.org/10.11578/rubin/dc.20250909.20)"
]
Expand Down Expand Up @@ -80,8 +80,8 @@
"4. Image cutouts (stamps) from the science, template, and difference images, for the triggering `diaSource`.\n",
"\n",
"**How do alerts relate to Rubin's other Prompt Products?** \\\n",
"The data in the alert packets will also stored in a queryable database, the Prompt Products Database (PPDB), which will update on a ~24-hour timescale and will available via the Rubin Science Platform (RSP).\n",
"The contents of the PPDB are public but RSP access is not (unlike broker access).\n",
"The data in the alert packets will also be stored in a queryable database, the Prompt Products Database (PPDB), which will update on a ~24-hour timescale and will be available via the Rubin Science Platform (RSP).\n",
"The *contents* of the PPDB are public but RSP access is not (unlike broker access).\n",
"The processed visit images and difference images become available via the RSP after an 80-hour embargo, and are subject to the two-year proprietary period. (See the [Rubin Data Policy](https://rubinobservatory.org/for-scientists/data-products/data-policy).)\n",
"\n",
"**Additional resources:**\n",
Expand Down Expand Up @@ -234,8 +234,8 @@
"Other formats (JSON, FITS) are possible, as demonstrated in the 100-level tutorial on the alert retrieval service.\n",
"\n",
"Pass the `url` to the `client` along with the `alert_id`.\n",
"Use the `await` command to pause execute of the cell's code until a response is received, then show the `response`.\n",
"Retreive the alert for the `snia_alert_id`."
"Use the `await` command to pause execution of the cell's code until a response is received, then show the `response`.\n",
"Retrieve the alert for the `snia_alert_id`."
]
},
{
Expand Down Expand Up @@ -332,7 +332,7 @@
"source": [
"### 2.2. Packet schema\n",
"\n",
"The `fastavro.reader` used above reads the embedded schema automatically\n",
"The `fastavro.reader` used above reads the embedded schema automatically.\n",
"\n",
"Extract the schema, and show that the keys and fields are the same for each (but for triggering `diaSources` that are associated with a `diaObject` instead of a `ssObject`, the fields `ssSource` and `mpc_orbits` won't be populated)."
]
Expand Down Expand Up @@ -493,7 +493,7 @@
"id": "aae8cf9b-c0ab-40d3-95f4-dacdc5a9ebe7",
"metadata": {},
"source": [
"Option to print the keys of `diaSource`, which are the same all objects."
"Option to print the keys of `diaSource`, which are the same for all objects."
]
},
{
Expand All @@ -516,7 +516,7 @@
"* `band`: LSST filter of the observation, one of: $ugrizy$.\n",
"* `midpointMjdTai`: The MJD at the midpoint of the exposure.\n",
"* `psfFlux`, `psfFluxErr`: The PSF flux and its error, measured on the difference image, in nJy.\n",
"* `scienceFlux`, `scienceFluxErr`: This PSF flux and its error, measured on the science image, in nJy.\n",
"* `scienceFlux`, `scienceFluxErr`: The PSF flux and its error, measured on the science image, in nJy.\n",
"\n",
"> **Warning:** while the science flux is useful for variable stars, for supernovae it is typically contaminated by host galaxy light and is inappropriate for use in a lightcurve.\n",
"\n",
Expand Down Expand Up @@ -767,11 +767,11 @@
"\n",
"* `ssObjectid`: Unique ID in the `ssObject` table.\n",
"* `phaseAngle`: Phase angle between the Sun, object, and observer, in degrees.\n",
"* `eclBeta`, `eclLambda`: The ecliptic latitude and logintude, in degrees.\n",
"* `eclBeta`, `eclLambda`: The ecliptic latitude and longitude, in degrees.\n",
"* `elongation`: Solar elongation, in degrees.\n",
"* `ephVmag`: Predicted magnitude in V band.\n",
"\n",
"The heliocentric and topocentric positions and velocities (`helio_` and `topo_` `x`, `y`, `z`, `vx`, `vy`, `vz`) and also included, among other parameters.\n",
"The heliocentric and topocentric positions and velocities (`helio_` and `topo_` `x`, `y`, `z`, `vx`, `vy`, `vz`) are also included, among other parameters.\n",
"\n",
"Print a few of the key column values."
]
Expand Down Expand Up @@ -929,7 +929,7 @@
"id": "ea1a9163-5706-4a55-ae75-da94df78f3a1",
"metadata": {},
"source": [
"Plot separately the lightcurves using the detections (plus the triggering detection), and using the force photometry.\n",
"Plot separately the lightcurves using the detections (plus the triggering detection), and using the forced photometry.\n",
"\n",
"The fluxes in the g, r, and i bands overlap, so impose a flux offset by filter when plotting."
]
Expand Down Expand Up @@ -1114,7 +1114,7 @@
"id": "aa4d7992-b4f8-4725-bd8a-e01a90b2badb",
"metadata": {},
"source": [
"> **Figure 2:** Image stamps from the SNIa alert packet. The literature coordinats of the SNIa are marked with a yellow circle, and the north direction indicated with a cyan line."
"> **Figure 2:** Image stamps from the SNIa alert packet. The literature coordinates of the SNIa are marked with a yellow circle, and the north direction indicated with a cyan line."
]
},
{
Expand Down
Loading