Skip to content

XRT DEM Iterative Solver Module#367

Open
joyvelasquez wants to merge 215 commits into
HinodeXRT:mainfrom
joyvelasquez:xray-plasma-decoder-dem-solver
Open

XRT DEM Iterative Solver Module#367
joyvelasquez wants to merge 215 commits into
HinodeXRT:mainfrom
joyvelasquez:xray-plasma-decoder-dem-solver

Conversation

@joyvelasquez

Copy link
Copy Markdown
Contributor

Creating a tool to estimate differential emission measures (DEMs) from Hinode/XRT observations using temperature response functions and spline-based fitting techniques. It is a Python implementation inspired by the IDL routine xrt_dem_iterative2.pro

@joyvelasquez joyvelasquez self-assigned this Jul 16, 2025
@joyvelasquez joyvelasquez added this to the 0.6.0 milestone Jul 16, 2025
@joyvelasquez

Copy link
Copy Markdown
Contributor Author

Ensure the function checks for minimum required inputs (filters, observed intensities, etc.) and validates that all observation values are non-zero before proceeding.

@codecov

codecov Bot commented Mar 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 2106 lines in your changes missing coverage. Please review.
✅ Project coverage is 40.48%. Comparing base (1ac0dc1) to head (5fad287).

Files with missing lines Patch % Lines
xrtpy/xrt_dem_iterative/test/test_dem_solver.py 0.00% 503 Missing ⚠️
xrtpy/xrt_dem_iterative/dem_solver.py 0.00% 386 Missing ⚠️
...rative/test/older_testing_methods/plots_results.py 0.00% 294 Missing ⚠️
...ive/test/older_testing_methods/idl_vs_xrtpy_dem.py 0.00% 265 Missing ⚠️
...py/xrt_dem_iterative/test/plot_idl_vs_xrtpy_dem.py 0.00% 153 Missing ⚠️
...rative/test/older_testing_methods/utils_case_io.py 0.00% 117 Missing ⚠️
.../xrt_dem_iterative/test/test_dem_idl_comparison.py 0.00% 114 Missing ⚠️
xrtpy/xrt_dem_iterative/dem_plotting.py 0.00% 99 Missing ⚠️
xrtpy/xrt_dem_iterative/test/utils_sav_io.py 0.00% 84 Missing ⚠️
xrtpy/xrt_dem_iterative/utils_sav_io.py 0.00% 84 Missing ⚠️
... and 2 more
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #367       +/-   ##
===========================================
- Coverage   87.35%   40.48%   -46.88%     
===========================================
  Files          26       38       +12     
  Lines        1819     3925     +2106     
  Branches      106      277      +171     
===========================================
  Hits         1589     1589               
- Misses        184     2290     +2106     
  Partials       46       46               

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread docs/gallery/data_processing/deconvolving.py
Comment thread docs/gallery/data_processing/remove_lightleak.py
Comment thread docs/gallery/data_processing/temperature_from_filter_ratios.py
Comment thread docs/gallery/instrument_response/effective_area.py
Comment thread docs/about_xrt.rst
@joyvelasquez joyvelasquez force-pushed the xray-plasma-decoder-dem-solver branch from bd4687b to 582c8f7 Compare July 2, 2026 16:49
pre-commit-ci Bot and others added 4 commits July 2, 2026 16:54
* Enable parallel test execution in noxfile

* Ignore sphinx-gallery scripts in pytest

---------

Co-authored-by: joyvelasquez <joy.velasquez@cfa.harvard>
Comment thread docs/conf.py
Comment on lines -73 to +77
ogp_custom_meta_tags = ('<meta property="og:ignore_canonical" content="true" />',)
# ogp_custom_meta_tags = ('<meta property="og:ignore_canonical" content="true" />',)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why this line was commented out. 🤔

Comment thread docs/dem_overview.rst
The core solver is implemented in :class:`xrtpy.xrt_dem_iterative.XRTDEMIterative`.

Conceptually, the solver:
1. Builds a regular grid in log10(T) between user-specified bounds.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Builds a regular grid in log10(T) between user-specified bounds.
1. Builds a regular grid in :math:`\log_{10}(T)` between user-specified bounds.

I'm not sure if the \log will work. 🤔

Comment thread docs/dem_overview.rst
Conceptually, the solver:
1. Builds a regular grid in log10(T) between user-specified bounds.
2. Interpolates the filter temperature responses onto that grid.
3. Represents log10(DEM) as a spline in log10(T).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
3. Represents log10(DEM) as a spline in log10(T).
3. Represents :math:`\log_{10}(DEM)` as a spline in :math:`\log_{10}(T)`.

Comment thread docs/dem_overview.rst
The DEM class requires temperature response functions for each filter, which
describe the instrument sensitivity as a function of temperature. These
responses can be generated for a chosen observation date using utilities
provided in ``xrtpy.response.tools``.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
provided in ``xrtpy.response.tools``.
provided in `xrtpy.response.tools`.

This would make it into a link.

Comment thread docs/dem_overview.rst

dem_solver = XRTDEMIterative(
observed_channel=filters,
observed_intensities=intensities,

@namurphy namurphy Jul 2, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we provide some sample intensities for a toy problem, maybe within something like xrtpy.data or xrtpy.examples? 🤔

There might be parallels in SunPy.

Comment thread docs/dem_overview.rst
Comment on lines +175 to +177
The arrays ``dem_solver.mc_dem``, ``dem_solver.mc_chisq``,
``dem_solver.mc_base_obs``, and ``dem_solver.mc_mod_obs`` are then available
for custom analysis.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might be able to put links here like `~xrtpy.dem.dem_solver.mc_dem` (though my example here probably has the wrong namespace).

Comment thread docs/dem_overview.rst

Small numerical differences between the Python and IDL implementations can arise due to:

* Different interpolation choices (for example, cubic splines from SciPy).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"cubic splines from SciPy" compared to what in IDL? 🤔

Comment thread docs/dem_overview.rst
Small numerical differences between the Python and IDL implementations can arise due to:

* Different interpolation choices (for example, cubic splines from SciPy).
* Differences in optimization libraries (lmfit versus IDL MPFIT).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a link to MPFIT?

Comment thread docs/dem_overview.rst
XRTpy DEM solver. It is intended for orientation rather than as a full
mathematical derivation.

The DEM inversion problem is mathematically ill posed, meaning that multiple

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The DEM inversion problem is mathematically ill posed, meaning that multiple
The DEM inversion problem is underdetermined, meaning that multiple

maybe? up to you.

Comment thread docs/dem_overview.rst
curves provides an estimate of the uncertainty in :math:`DEM(T)`.


Extended example with options

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if this would be best in an example notebook since that way the code will be executed to make sure it works. 🤔

Comment thread docs/dem_overview.rst
Comment on lines +277 to +290
minimum_bound_temperature=5.5, # Minimum log T (default: 5.5)
maximum_bound_temperature=8.0, # Maximum log T (default: 8.0)
logarithmic_temperature_step_size=0.1, # Bin width in log T (default: 0.1)
monte_carlo_runs=100, # Number of Monte Carlo runs (default: 0 -(disabled))
max_iterations=2000, # Solver max iterations (default: 2000)
normalization_factor=1e21, # Normalization scaling factor (default: 1e21)
)

dem_solver.solve()
dem_solver.plot_dem_mc()

.. note::
The values shown above correspond to the solver defaults and are written
out here to illustrate which parameters can be tuned. You can adjust these

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would help to put a comment near the class definition (especially __init__) that if the default values are changed, then the arguments in this example should be updated.

Comment thread docs/dem_overview.rst
Comment on lines +293 to +301

.. Acknowledgement
.. ---------------
.. *Development of the DEM solver in XRTpy has been supported in part by
.. a NASA Heliophysics Tools and Methods (HTM) program grant (ROSES-2025,
.. element B.20). This effort reflects the ongoing transition of DEM
.. capabilities from legacy IDL routines into modern, open-source Python
.. tools for the solar physics community.*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would likely go best onto a separate acknowledgements page accessible from the landing page.

Comment thread docs/glossary.rst

Monte Carlo DEM
A set of DEM solutions computed by adding random noise (based on intensity errors) to the observed intensities and re-solving the DEM multiple times.
The spread of these Monte Carlo solutions provides an estimate ofuncertainty in the DEM at each temperature.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The spread of these Monte Carlo solutions provides an estimate ofuncertainty in the DEM at each temperature.
The spread of these Monte Carlo solutions provides an estimate of uncertainty in the DEM at each temperature.

Comment thread docs/glossary.rst
Point Spread Function — describes the response of the telescope to a point source of light. In XRTpy, it is used in deconvolution routines to sharpen images.

Response Matrix
A two-dimensional array containing the temperature response of each XRT filter interpolated onto the solver's regular log10 temperature grid.This matrix connects

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A two-dimensional array containing the temperature response of each XRT filter interpolated onto the solver's regular log10 temperature grid.This matrix connects
A two-dimensional array containing the temperature response of each XRT filter interpolated onto the solver's regular log10 temperature grid. This matrix connects

"""
Test suite for xrtpy.xrt_dem_iterative and related DEM tools.

This file marks the 'tests' directory as a Python package and allows

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This file marks the 'tests' directory as a Python package and allows
This file marks the `tests/` directory as a Python package and allows

Mostly a personal stylistic preference to have the / and the end of a directory name to communicate that it's a directory. 🎨

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants