Skip to content

BUG: Fix DLE.compute_sequence under NumPy >= 2.4 (#839)#842

Open
mmcky wants to merge 3 commits into
mainfrom
fix-839-dle-numpy24-scalar
Open

BUG: Fix DLE.compute_sequence under NumPy >= 2.4 (#839)#842
mmcky wants to merge 3 commits into
mainfrom
fix-839-dle-numpy24-scalar

Conversation

@mmcky

@mmcky mmcky commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #839. DLE.compute_sequence raises under NumPy 2.4 when it computes the asset-price terms R1_Price, R2_Price, R5_Price. The same code runs under NumPy 2.3.x, so this is a 2.3 → 2.4 regression.

Root cause

Each price expression evaluates to a size-1 array, not a Python scalar: self.beta is a (1, 1) array and e1 is a row vector, so the matmul chain reduces to shape (1,) or (1, 1). Assigning such an array into a scalar slot like self.R1_Price[i, 0] used to work because NumPy silently converted a size-1 ndim > 0 array to a scalar. That conversion was deprecated in NumPy 1.25 and became a hard error in NumPy 2.4 (listed under Expired deprecations in the 2.4.0 release notes), so the assignment now raises ValueError: setting an array element with a sequence (underlying TypeError: only 0-dimensional arrays can be converted to Python scalars).

Fix

Coerce each price expression to a scalar with .item() before assignment. This works regardless of the intermediate shape and reproduces the pre-2.4 values exactly.

Note: redefining e1 as 1-D (suggested in the issue discussion) is not sufficient on its own — because self.beta is itself a (1, 1) array, self.beta * e1 @ … re-introduces a 2-D shape and the result stays ndim > 0. I verified this empirically: under both the current 2-D e1 and the proposed 1-D e1, R1 is shape (1,) and R2/R5 are shape (1, 1). .item() addresses the conversion at the point of assignment and is robust to all of these shapes.

The same size-1 assignment pattern also exists in the optional Pay branch (Pay_Price / Pay_Gross), which is only reached when a Pay array is passed. It is fixed here too, so compute_sequence is robust regardless of arguments.

Verification

The validated MWE from the issue (Hall's permanent-income economy) now runs, and the .item() path reproduces the pre-regression values exactly:

numpy before this PR after this PR
2.3.5 ✅ runs — R1_Price[:3] = [0.95238095, …] ✅ runs — identical values
2.4.6 ❌ raises ✅ runs — identical values

Tests

compute_sequence previously had no test coverage. This PR adds two regression tests to TestDLE:

  • test_compute_sequence — exercises the default (risk-free price) path, checks shapes and finiteness, and anchors R1/R2/R5_Price[0, 0] to the closed form β**J at t = 0.
  • test_compute_sequence_with_pay — exercises the Pay branch (Pay_Price / Pay_Gross).

All 10 tests in quantecon/tests/test_dle.py pass locally (NumPy 2.3.5, Python 3.13).

Downstream impact

This unblocks 5 lectures in lecture-python-advanced.myst under anaconda=2026.06 (numpy 2.4.6): cattle_cycles, growth_in_dles, irfs_in_hall_model, lucas_asset_pricing_dles, permanent_income_dles.

🤖 Generated with Claude Code

The asset-price terms in `DLE.compute_sequence` (`R1_Price`, `R2_Price`,
`R5_Price`, and the optional `Pay_Price` / `Pay_Gross`) each evaluate to a
size-1 array rather than a Python scalar: `self.beta` is a (1, 1) array and
`e1` is a row vector, so the matmul chain reduces to shape (1,) or (1, 1).

Assigning such an array into a scalar slot used to work because NumPy
silently converted a size-1 `ndim > 0` array to a scalar. That conversion
was deprecated in NumPy 1.25 and became a hard error in NumPy 2.4, so the
assignments now raise `ValueError: setting an array element with a sequence`
(underlying `TypeError: only 0-dimensional arrays can be converted to Python
scalars`). NumPy 2.3.x still works, which is why this is a 2.3 -> 2.4
regression.

Coerce each price expression to a scalar with `.item()` before assignment.
This works regardless of the intermediate shape (unlike redefining `e1` as
1-D, which is insufficient because `self.beta` is itself a (1, 1) array) and
reproduces the pre-2.4 values exactly. Also covers the `Pay` branch, which
has the same size-1 assignment pattern.

Adds regression tests for both the default and `Pay` paths of
compute_sequence, which previously had no coverage.

Fixes #839

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings June 27, 2026 06:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a NumPy 2.4 regression in quantecon.DLE.compute_sequence where size-1 arrays (from matmul chains involving (1, 1) beta and a 2-D selector vector) can no longer be implicitly assigned into scalar array slots, causing ValueError: setting an array element with a sequence.

Changes:

  • Coerce computed bond-price and Pay-asset expressions to Python scalars via .item() before assignment (robust across NumPy versions).
  • Add regression tests covering both the default risk-free price path and the optional Pay branch.
  • Anchor risk-free prices at t=0 to the closed-form β**J values and assert finiteness/shape invariants.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
quantecon/_dle.py Extracts scalars with .item() for bond-price and Pay-asset assignments to avoid NumPy 2.4 size-1-array assignment errors.
quantecon/tests/test_dle.py Adds regression tests for compute_sequence (default and Pay branch) validating shapes, finiteness, and closed-form pricing at t=0.

Comment thread quantecon/_dle.py Outdated
@coveralls

coveralls commented Jun 27, 2026

Copy link
Copy Markdown

Coverage Status

coverage: 93.517% (+0.5%) from 93.008% — fix-839-dle-numpy24-scalar into main

Per Copilot review: precompute `e1 @ self.Mc` and the matrix powers of
`self.A0` once (they do not depend on `i`), and compute the denominator
`e1 @ self.Mc @ xp[:, i]` once per iteration instead of three times. The
results are bit-identical to the previous code (verified on a fixed
trajectory); the change avoids redundant work and clarifies intent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@mmcky

mmcky commented Jun 27, 2026

Copy link
Copy Markdown
Contributor Author

A note on commit 4737381 (the loop-invariant hoist from Copilot's suggestion): the main driver here is readability, not performance. Pulling e1 @ self.Mc and the matrix_power(self.A0, k) terms out of the loop, and computing the denominator once per i, makes it explicit that those quantities don't depend on the iteration index. The actual speedup is negligible in practice — the matrices are tiny and the loop is short, and this loop is a small fraction of compute_sequence's total cost (dominated by the LQ solve, the simulation, and the Lyapunov solve). The change is bit-identical to the previous code (verified on a fixed trajectory), so it carries no numerical risk; the #839 fix itself lives in 68ba04f.

@mmcky mmcky requested a review from oyamad June 27, 2026 06:39

@oyamad oyamad left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • My opinion is that conceptually scalar objects (such as beta) should be implemented as scalars; conceptually 1-d vectors should be implemented as 1-d arrays. The whole code in this file is very "matrix-oriented", which is very different from the other part of this library, and to me is very confusing. But this would require a global full re-write of the whole code.
  • So I understand the preference for keeping this PR local, only applying a minimal "workaround" that removes the error.

Comment thread quantecon/_dle.py Outdated
Move ``.item()`` outside the division in the R1/R2/R5 price terms so the
division is performed by NumPy rather than on extracted Python floats.
Dividing two Python floats raises ``ZeroDivisionError`` when the
denominator (consumption, ``e1 @ Mc @ x``) is zero, whereas the pre-2.4
code produced ``inf``/``nan`` with a ``RuntimeWarning``. Keeping the
division inside NumPy restores that behaviour and makes these lines
consistent with the Pay branch.

Addresses review feedback from @oyamad on #842.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@oyamad oyamad left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mmcky Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DLE.compute_sequence fails under NumPy 2.4 (setting an array element with a sequence)

4 participants