From 5265567650a5499efb9d76c5bc3541faac781267 Mon Sep 17 00:00:00 2001 From: MelissaGraham Date: Mon, 10 Aug 2026 18:09:23 +0000 Subject: [PATCH] feedback from Slack --- .../301_4_Scheduler_information.ipynb | 61 ++++++++++++++++--- 1 file changed, 51 insertions(+), 10 deletions(-) diff --git a/DP2/300_Science_demos/301_DP2_observations/301_4_Scheduler_information.ipynb b/DP2/300_Science_demos/301_DP2_observations/301_4_Scheduler_information.ipynb index 69238c7a..9c35ffdc 100644 --- a/DP2/300_Science_demos/301_DP2_observations/301_4_Scheduler_information.ipynb +++ b/DP2/300_Science_demos/301_DP2_observations/301_4_Scheduler_information.ipynb @@ -22,7 +22,7 @@ "Data Release: [Data Preview 2](https://dp2.lsst.io/)\\\n", "Container Size: Large\\\n", "LSST Science Pipelines version: r30.0.10\\\n", - "Last verified to run: 2026-08-07\\\n", + "Last verified to run: 2026-08-10\\\n", "Repository: [github.com/lsst/tutorial-notebooks](https://github.com/lsst/tutorial-notebooks)\\\n", "DOI: [10.11578/rubin/dc.20250909.20](https://doi.org/10.11578/rubin/dc.20250909.20)" ] @@ -57,7 +57,7 @@ "In the Early Data Preview 2 (EDP2) Visit table ([schema](https://sdm-schemas.lsst.io/dp2.html#Visit)) does not include scheduler information such as the target name, observation reason, or science program.\n", "These values were omitted partially because they were not standardized during commissioning, not used consistently throughout the commissioning period, and are thus of limited use for EDP2.\n", "\n", - "**Warning: for EDP2, when \"all the visits of a particular field\" are desired, it is recommended to follow the guidance in tutorial \"301.1. DP2 overview\"** and search for visits using the list of field center coordinates and boresight radii provided there in Section 1.\n", + "**Warning: for EDP2, when \"all the visits of a particular commissioning field\" are desired, it is recommended to follow the guidance in tutorial \"301.1. DP2 overview\" and search for visits in a given region by coordinates and offsets.** Note also that these \"commissioning fields\" were defined only for the commissioning (pre-LSST) era; with the exception of the deep drilling fields, the LSST does not have defined fields.\n", "\n", "However, because some _limited_ and _non-standardized_ scheduler information does exist and is accessible via the Butler dimension records, in order to assuage curiosity and provide guidance, this tutorial demonstrates how to retrieve and interpret the scheduler-related information.\n", "\n", @@ -160,6 +160,8 @@ "source": [ "## 2. Retrieve visit information\n", "\n", + "### 2.1. Visit table (TAP)\n", + "\n", "Using the TAP service, retrieve the five columns for all rows of the `Visit` table: the visit identifier, the coordinates RA and Dec in degrees, and the modified julian date at the midpoint of the exposure.\n", "Store the results in `tap_results_df` as a pandas dataframe." ] @@ -186,7 +188,40 @@ "id": "b8463e55-6045-4e6f-b61f-38219d3e63a7", "metadata": {}, "source": [ - "Retrieve the visit dimension records for all visits from the Butler." + "### 2.2. Visit dimension records\n", + "\n", + "Visit dimension records can be retrieved from the Butler by individual visit identifier number.\n", + "Retrieve the records for three randomly selected visits by their visit identifier." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cf14c84b-d9e0-4c1b-9d8d-5efc2b81160f", + "metadata": {}, + "outputs": [], + "source": [ + "visits_list = [2025042800283, 2025042800284, 2025042800285]\n", + "visits_string = \"(\" + \", \".join(str(value) for value in visits_list) + \")\"\n", + "query = \"visit.id IN \" + visits_string\n", + "visit_dimrecs = butler.query_dimension_records(\"visit\", limit=None, where=query)\n", + "\n", + "print(query)\n", + "print('Number of visit dimension records: ', len(visit_dimrecs))\n", + "print(' ')\n", + "print('Notice the order can change between query and results:')\n", + "for r, rec in enumerate(visit_dimrecs):\n", + " print(r, rec.id)\n", + "\n", + "del visits_list, visits_string, query, visit_dimrecs" + ] + }, + { + "cell_type": "markdown", + "id": "049ae9dc-41d9-4bd6-891a-1c4efbe15225", + "metadata": {}, + "source": [ + "Retrieve the visit dimension records for all visits from the Butler, for use in the rest of this tutorial." ] }, { @@ -275,6 +310,8 @@ "id": "32967464-5085-4a2c-8a13-3021f3dc79c2", "metadata": {}, "source": [ + "### 2.3. Join Visit table and dimension records dataframes\n", + "\n", "Do an inner join on the two dataframes, which means keeping only the rows of the `temp_df` that are matched to rows of the `tap_results_df`.\n", "Store the result in `df`." ] @@ -331,7 +368,7 @@ "id": "16d36732-dc7d-4eb0-8ce6-0c1a84046330", "metadata": {}, "source": [ - "### 2.1. Fix non-standard DDF names\n", + "### 2.4. Fix non-standard DDF names\n", "\n", "As mentioned in Section 1, the scheduler metadata was not standardized during commissioning and multiple formats exist for the deep drilling field (DDF) names.\n", "Convert the target names for the DDFs to be all lowercase and use underscores instead of spaces." @@ -364,13 +401,13 @@ "\n", "**Caveat: A single visit can be associated with multiple target names.**\n", "\n", - "**Caveat: These target names are not necessarily: (1) standard names; (2) used consistently; (3) to be used in the future.**\n", + "**Caveat: These target names are not necessarily: (1) standard names; (2) used consistently; (3) to be used in the future.** New values for target names might be introduced as the LSST strategy evolves.\n", "\n", "The [wide-fast-deep regions](https://dp2.lsst.io/overview/observations.html#wide-fast-deep-region) covered during commissioning are:\n", "\n", "* `lowdust` - low dust sky region\n", "* `dusty_plane` - dusty regions of the Galactic plane\n", - "* `bulgy` - Galactic bulge region\n", + "* `bulgy` - Galactic bulge and plane regions\n", "* `nes` - north ecliptic spur\n", "* `LMC_SMC` - Large and Small Magellanic Clouds\n", "* `scp` - south celestial pole\n", @@ -489,7 +526,7 @@ "id": "0e8207c6-0412-43ee-9cde-1a3819761ed5", "metadata": {}, "source": [ - "> **Figure 1:** The density of the number of visits, in any filter, with target name of `dusty_plane` (purples), `bulgy` (oranges), or `lowdust` (blues). Notice that there are a small number of visits labeled as being Galactic bulge visits that are not near the Galactic center (yellow star)." + "> **Figure 1:** The density of the number of visits, in any filter, with target name of `dusty_plane` (purples), `bulgy` (oranges), or `lowdust` (blues). Notice that `bulgy` does not strictly mean `Galactic bulge` but extends into regions of the Galactic plane; the Galactic center is marked with a yellow star for reference." ] }, { @@ -501,7 +538,7 @@ "\n", "The `observation_reason` column records the source of the visit in the Feature Based Scheduler (FBS).\n", "\n", - "**Caveat: These observation reasons are not necessarily: (1) standardized in name or meaning; (2) used consistently over commissioning; (3) to be used in the future.**\n", + "**Caveat: These observation reasons are not necessarily: (1) standardized in name or meaning; (2) used consistently over commissioning; (3) to be used in the future.** New values of the observation reason might be introduced as the LSST strategy evolves.\n", "\n", "The feature-based scheduler (FBS) and Rubin operations were in the early stages when these visits were obtained, and the values of the `observation_reason` column exhibit diversity; explanations for every single value are not provided, but in general \n", "\n", @@ -555,9 +592,13 @@ "source": [ "### 3.3. Science program\n", "\n", - "For commissioning, the science program is just an internal designation with no science application.\n", + "For commissioning, the science program is a designation with no science application.\n", "It is accessible as column `science_program`, but it is just the word \"BLOCK\" and then a number.\n", "The observing block was simply the bundle of science-motivated commissioning observations used during a given era of commissioning.\n", + "For example, the \"BLOCK-365\" was used initially for commissioning observations.\n", + "Then the \"BLOCK-400\" programs were defined for the pre-LSST observations, and the different blocks were just configurations for different conditions (e.g., good and bad seeing).\n", + "**Caveat:** This will not be how the feature-based scheduler works for the LSST, and the **science program should not be used to constrain data for analyses**.\n", + "Use, e.g., the measured image quality parameters in the `VisitDetector` table instead.\n", "\n", "Option to print the unique values of the `science_program` column." ] @@ -612,7 +653,7 @@ "id": "dcf22eaf-f8f7-4fc8-8e13-3606cb3b35eb", "metadata": {}, "source": [ - "> **Figure 2:** Distribution of visit dates by \"science_program\", illustrating how different \"program\" block names were used during the commissioning period to indicate \"science observations\" in general. The point being made with this plot is that \"science_program\" is not a scientifically useful piece of scheduler info for EDP2." + "> **Figure 2:** Distribution of visit dates by \"science_program\", illustrating how different \"program\" block names were used during the commissioning period. The point being made with this plot is that \"science_program\" is not a scientifically useful piece of scheduler info for EDP2." ] }, {