Skip to content

Bernoulli Expansion based Unitary Coupled-Cluster - #581

Draft
ajay-mk wants to merge 39 commits into
ajay/feat/cc-rdmfrom
ajay/feat/bernoulli-v2
Draft

Bernoulli Expansion based Unitary Coupled-Cluster#581
ajay-mk wants to merge 39 commits into
ajay/feat/cc-rdmfrom
ajay/feat/bernoulli-v2

Conversation

@ajay-mk

@ajay-mk ajay-mk commented Jul 27, 2026

Copy link
Copy Markdown
Member

Adds the Bernoulli expansion of the UCC similarity-transformed Hamiltonian $\bar{H} = e^{-\sigma} H e^{\sigma}$, $\sigma = T - T^\dagger$, as an alternative to the BCH expansion in mbpt::CC. Ranks $\bar H^0$$\bar H^4$, Eqs. (45)–(50) of Liu, Asthana, Cheng & Mukherjee, JCP 148, 244110 (2018)

Companion PR: MPQC4 #792.

Changes

  • Adds the Bernoulli-number expansion of the UCC similarity-transformed Hamiltonian $\bar H = e^{-\sigma}He^{\sigma}$ as an alternative to BCH in mbpt::CC, plus per-block $\bar H$ truncation for block-truncated EOM (qUCCSD and its IP/EA analogues).
  • CC::Options::hbar_expansion: Selects BCH or Bernoulli; the former is the default. Bernoulli requires a
    unitary ansatz and hbar_comm_rank.
  • New Bernoulli expansion related logic:
    • bernoulli::hbar(N, rank, skip1) assembles $\bar H^0$$\bar H^4$ (Eqs. (45)–(50) of Ref. 1) rank by rank. $H$ splits as $F+V$, and every operator splits into its pure excitation/de-excitation part ($O_N$) and the rest ($O_R$).
    • detail::wick_reduce — Wick-reduces while keeping partial contractions
      (needed so nesting produces operators, not scalars). Must run with use_topology(false): its weight bookkeeping is only valid on the fully-contracted path, and it defaults to on.
    • detail::N_part / detail::R_part — the $O_N$ / $O_R$ split. Only $N$ is block-resolved; $R$ stays in compact general-index form, which keeps the nested commutators that consume it cheap.
    • bernoulli::hbar() returns a tensor-level expression (BCH's hbar() returns an operator-level one). Callers have to take care of this.
  • CC::eom_r: per-block H̄ truncation (block_ranks):
    • New optional argument: a row-major $K\times K$ matrix, one commutator truncation rank per block of the EOM secular matrix instead of one uniform $\bar H$.
    • Manifolds are ordered by ascending rank, so the same matrix serves EE, IP and EA. qUCCSD is {2,1,1,0}. See Liu & Cheng, JCP 155, 174102 (2021), Sec. II C; Zhang & Liu, JCTC 22, 3341 (2026), Table 1, for the IP/EA analogues.
    • Each block is the sandwich ⟨i|H¯|j⟩−δijE, not the commutator form. Under Bernoulli, each block's H¯ has its N part (the ground-state amplitude residual) removed — it only vanishes at the block's own rank when that equals the amplitude rank, so keeping it would leak a spurious residual into the off-diagonal blocks.

Tests

  • tests/unit/test_mbpt_cc.cpp: Wick reduction, N/R split, $\bar H$ structure, config validation, block-truncated EOM term counts and the three rejection cases.
  • tests/integration/ucc.cpp: BCH/Bernoulli term-count pins at ranks 2–3.
  • Implementation is validated numerically against literature, see MPQC4 #792 for details.

qUCCSD Example

How to derive ground and excited state equations
#include <SeQuant/core/context.hpp>
#include <SeQuant/core/expr.hpp>
#include <SeQuant/core/op.hpp>
#include <SeQuant/core/tensor_canonicalizer.hpp>
#include <SeQuant/domain/mbpt/context.hpp>
#include <SeQuant/domain/mbpt/convention.hpp>
#include <SeQuant/domain/mbpt/models/cc.hpp>
#include <SeQuant/domain/mbpt/op.hpp>

#include <iostream>

using namespace sequant;
using namespace sequant::mbpt;

int main() {
  set_locale();
  set_default_context({.index_space_registry_shared_ptr = make_sr_spaces(),
                       .vacuum = Vacuum::SingleProduct,
                       .metric = IndexSpaceMetric::Unit,
                       .spbasis = SPBasis::Spinor});
  TensorCanonicalizer::set_cardinal_tensor_labels(cardinal_tensor_labels());
  set_default_mbpt_context({.op_registry_ptr = make_legacy_registry()});

  const CC cc(2, {.ansatz = CC::Ansatz::U,
                  .hbar_comm_rank = 2,
                  .hbar_expansion = CC::HbarExpansion::Bernoulli});


  // qUCCSD: 3 commutators in Energy, 2 in ground state amplitudes; [2,1,1,0] for excited states

  // energy
  const auto energy = cc.energy(3);
  std::wcout << "E        : " << energy->size() << " terms\n";

  // t amplitudes
  const auto t = cc.t();
  for (std::size_t p = 1; p < t.size(); ++p)
    std::wcout << "residual" << p << ": " << t[p]->size() << " terms\n";

  // EE
  const auto sigma = cc.eom_r(nₚ(2), nₕ(2), {2, 1, 1, 0});
  for (std::size_t p = 1; p < sigma.size(); ++p)
    std::wcout << "sigma" << p << "   : " << sigma[p]->size() << " terms\n";

  // IP/EA use the same block ranks, with R of unequal particle/hole rank
  std::wcout << "IP sigma1: " << cc.eom_r(nₚ(1), nₕ(2), {2, 1, 1, 0})[0]->size()
             << " terms\n";
}

Important References:

ajay-mk added 2 commits July 19, 2026 14:36
Derives the rank-r RDM as the reference expectation value of a
similarity-transformed number operator (op::N). Traditional ansatz uses
<0|(1+Λ) e^{-T} N e^{T}|0> with the linked-cluster connection constraint
enforced; unitary ansatz drops Λ and uses σ = T − T⁺ with genuine
commutators, needing no constraint.

Tests cover the traditional (rank 1 and 2) and unitary (rank 1) cases.
@ajay-mk
ajay-mk force-pushed the ajay/feat/bernoulli-v2 branch from 2b73c55 to e78e554 Compare July 27, 2026 12:08
@ajay-mk ajay-mk added the feature New feature label Jul 27, 2026
ajay-mk added 26 commits July 27, 2026 09:34
For a 1-body operator, we only need to consider up to 2 nested
commutators (in traditional coupled-cluster) and for higher operators we
use up to 4. For UCC Ansatz the user sets the commutator truncation
rank.
CC::rdm now takes an optional commutator truncation rank, mirroring
CC::energy. The traditional-ansatz default is the exact termination
point of e^{-T} N e^{T} in <0|(1+Λ) N̄|0>: T-T contractions vanish, so
each T must contract with one of the number operator's 2*rank legs, and
the T legs left over can only be closed by the single Λ of rank <= N,
giving min(2*rank, rank + N).
The unitary ansatz has no such termination and keeps using
hbar_comm_rank.
…ion rank

Lets a caller reuse an existing engine's other options while evaluating at a
different commutator rank, without mutating an engine other derivations share.
…comm_rank access

op::N was reachable but unregistered, so printing/introspecting it threw.
Register it like its sibling operators, and add CC::rdm's own guard on
hbar_comm_rank_ to match the other unitary-ansatz call sites in this class.
`N` denotes the particle-number operator by convention, but this is a
rank-r replacement operator -- a single replacer {ã^{p_1..p_r}_{p_{r+1}..p_{2r}}},
which is why the docs had to keep saying "number (replacement) operator".
Two concrete costs: `CC::rdm` had to qualify it as `op::N` to dodge the
`CC::N` member, and registering `N` in both shipped registries would make
`OpRegistry::validate_op` throw for any downstream code registering its own
`N` -- a very likely user label.

The registry label mirrors the notation the operator lowers to, so `ã`
appears at both the operator and the tensor level.

Also fold the two index loops into one and match the sibling operators
(`θ`, `t`, `λ`) by asserting the rank precondition instead of throwing.
Documentation. The formula wrote the ket indices as q_1..q_r, which match
no label in the output -- they are p_{r+1}..p_{2r}, and MPQC has to
hard-code that mapping. Point at op::ã, which now states the ordinals.
Three caveats a caller cannot infer were also missing: the reference
contribution is absent (ref_av returns 0 for a bare normal-ordered
operator, so this is the correlation part and the caller adds delta), the
result is not manifestly antisymmetric for rank >= 2, and "linked" holds
only for the traditional branch since the unitary one passes empty
connections. Record the argument for the min(2*rank, rank + N) default in
the code rather than leaving it in a commit message.

Assertions. The unitary hbar_comm_rank check was dead in three places --
the constructor already asserts it and nothing else clears it -- so drop
the copies in rdm, tʼ and eom_r and keep the constructor as the sole
guard. tʼ keeps its pertbar_comm_rank check, which the constructor does
not cover. Use the message argument form on the lines this touches.
The rdm coverage was term counts only, which drift silently with any
canonicalization or simplify change and verify none of the equations.

The load-bearing claim is that the default comm_rank, min(2*rank, rank +
N), is already exact; raising it must add nothing. Assert that directly
instead of leaving the argument in a commit message.

Spell out both 1-RDMs with EquivalentTo. Ten and eight terms are small
enough to read, and they pin the free-index convention -- p_1 is a ket
index and p_2 a bra index, since ã's creators reach the expression
through ket slots and its annihilators through bra slots -- which MPQC
reconstructs by hand. The 2-RDMs stay counts -- 94 terms is not worth
spelling out -- but the traditional one also checks its free indices
directly, since that is the shape MPQC evaluates.

Also cover what was untested: to_latex on the operator form, which threw
before op::ã was registered; the rank 0 precondition; and the unitary
rank-2 density.
CC::rdm was the one mbpt::lst() call site passing options inline rather
than lst_options(), which left use_connected_form at its false default
while every other site got the connected-product form. That contradicted
lst_options()'s own doc ("the LSTOptions this engine uses for every
mbpt::lst() call") and did redundant work: the traditional branch already
hands ref_av the {ã,t} connectivity that makes the cheaper form
equivalent, and the unitary branch gets use_connected_form == false from
lst_options() anyway, so nothing changes there.

Verified inert rather than assumed: the full to_latex serialization of
rdm(1), rdm(2), rdm(2,2), rdm(1,5), rdm(2,6) and both unitary densities
is byte-identical before and after.
Adds the bottom layer of the Bernoulli expansion of the unitary-CC
similarity-transformed Hamiltonian: a Wick reduction that retains partial
contractions, so a product of normal-ordered operators reduces to a sum of
normal-ordered operators rather than collapsing to a scalar vacuum average,
and the normal-ordered commutator built on it.

WickTheorem::use_topology is disabled explicitly rather than left alone: it
defaults to ON (wick.hpp), and its one-representative-times-multiplicity
bookkeeping is only exercised by the fully-contracted path. On this
partial-contraction path it rescales terms whose amplitude pairs are
symmetric, which leaves vacuum averages correct while corrupting projections
onto excited manifolds.

wick_commutator reindexes B's summed indices to fresh temporaries before
forming A*B, since A and B are independently constructed and may otherwise
share labels, which would fuse two independent summations.
Adds the second layer: the split of an operator O into O_N, "the non-diagonal
part containing all the excitation and de-excitation operators" (defined above
Eq. (43) of 10.1063/1.5030344), and the rank-preserving remainder O_R = O - O_N.
The Bernoulli expansion's inner commutators carry N/R subscripts, so every
nesting level needs this classification.

Classification needs definite index spaces, so expand_to_blocks first rewrites
each general index of the residual NormalOperator as a sum over the base spaces
it spans. Only the hole and particle spaces are expanded over: in the
single-reference setting the remaining base spaces are empty, so restricting to
those keeps the expansion 2-way per index instead of compounding across the
nested commutators. That makes the routine single-reference only, which the
header warns about.

The rank cutoff mirrors pdaggerq (nt_bra > bernoulli_excitation_level -> R)
rather than the paper's uncapped O_N, since that is the convention defining
qUCCSD and the one the numbers are validated against; terms above the cutoff
fall to R rather than being dropped.
Adds the top layer: hbar(N, rank, skip1) sums H̄⁰..H̄^rank of 10.1063/1.5030344
Eq. (45), each order transcribed from its equation with the published
coefficients and per-level N/R subscripts. Bernoulli numbers B₁=-1/2, B₂=1/12,
B₃=0, B₄=-1/720 (Eq. 40) enter as those coefficients; a subscript R/N on a
commutator means "form the commutator, then keep only its R/N part before the
next nesting", which is what the split from the previous commit provides.

Two cancellations from the paper are relied on and noted in place: F enters H̄
only at first order (stated just below Eq. (50)), and the higher orders carry
only R-subscripted inner commutators.

Every term is a nested commutator whose prefix is shared with other terms,
within a rank and across ranks, so nest() memoizes each prefix (keyed by the
base operator plus the tags applied so far). The nine rank-4 terms have only 3
distinct level-1 and 6 distinct level-2 nodes. Reusing a memoized ExprPtr is
safe because expression composition deep-copies its operands.

Contributions accumulate through Sum::append rather than chained operator+,
which deep-copies the whole accumulated Sum on every call and is quadratic in
the term count at high rank.
Adds CC::Options::hbar_expansion (BCH by default, Bernoulli opt-in) and
dispatches CC::hbar() to bernoulli::hbar() when it is selected. Two constructor
assertions guard the combination: the Bernoulli expansion is defined for the
unitary ansatz only, and it requires an explicit hbar_comm_rank, since CC::hbar()
otherwise falls back to rank 4 and would silently select the most expensive and
least exercised order.

CC::energy() takes the plain reference expectation value under this expansion:
the tensor-level H̄ is already fully expanded, so no operator connectivity
remains to constrain. Its comm_rank argument defaults to the amplitude rank and
is passed explicitly for the qUCCSD [2|3] split, where the energy is taken at H̄³
while the amplitudes stop at H̄².
Pins the derived equations at Bernoulli ranks 1-3 for the unitary
ansatz: term
counts for the energy and for the singles/doubles residuals, plus the
guards on
invalid configurations (Bernoulli with a non-unitary ansatz, and
Bernoulli
without an explicit hbar_comm_rank).

The rank-3 numbers (46 energy, 32 singles, 38 doubles terms) are the
ones
cross-checked term-by-term against pdaggerq, so a change here means the
derivation changed.
Corrections:
- Eq. (45) is the assembly H̄ = Σ_k H̄^k; (46)-(50) are the rank-by-rank
  operators. The file header attributed (45)-(50) to the latter.
- expand_to_blocks: the SR "o"/"g" base spaces are not empty, so the old
  justification for dropping them was wrong. They are droppable because
the
  single-reference projection annihilates those terms. The header
@warning
  said the same wrong thing.
- R_part's result is not block-resolved; it stays in compact
general-index
  form. The header claimed the opposite.
- wick_reduce leaves at most one residual NormalOperator, not exactly
one --
  fully-contracted terms carry none, which find_nop already handled.
- The memo shares level-1 nodes across ranks; it is not a prefix
relation.
- The use_topology rescaling set is {2, 1/2, 1/3, 8/3, 2/3}; the comment
said
  3 where it should have said 1/3.
… key

A character outside {A,N,R} in a nest() tag string silently read as 'A'
(no filter) and would have yielded the wrong H̄; the whole Eq. (46)-(50)
transcription lives in these strings, so assert on every tag.

Grow the memo key in place instead of deriving it from the memo iterator:
container::map is a flat_map, whose insertions invalidate iterators, so
the read-back was correct only by the accident of no insertion happening
in between.

Include <algorithm> for std::max and range/v3's primitives for
ranges::distance rather than relying on them arriving transitively.
CC::Options::screen and use_topology reach the derivation only through
CC::ref_av(); the Bernoulli path calls op::tensor::ref_av() directly and
so picks up that function's own defaults instead.
The srcc.cpp analogue for the unitary ansatz, covering both H̄
expansions. CC::t() yields the whole equation set in one derivation --
element 0 the energy, element R the residual -- and the term counts are
pinned so a change in either expansion fails ctest.

Registered variants run in seconds; the Bernoulli H̄⁴ pins are recorded
but left out of ctest, since that configuration takes ~2 minutes against
sub-second times for everything else in this directory.
Both accumulation sites in bernoulli.cpp built a Sum by append and left
every duplicate for one final simplify. Sum::append flattens nested sums
and adds up Constants but never merges like terms, so the nested
commutators -- which overlap heavily by construction, the same fact that
makes nest() memoize -- carried their duplicates all the way to the end.

hbar() now accumulates into a HashingAccumulator, which keys summands by
hash under proportional_to and merges them via Product::add_identical at
insertion. The prefactor has to be folded into each summand rather than
wrapped around the sum: appending Constant*Sum inserts the scaled sum as
one opaque summand, since append's flatten splits a Sum but not a Product
wrapping one, and nothing would collapse. Distributing it with expand()
instead is shorter but materializes an intermediate Sum and gives back
most of the gain (rank 4: 163.9 s vs 148.2 s).

expand_to_blocks_reduced's outer loop now uses transform_sum_expr, which
canonicalizes each mapped result before accumulating -- necessary here
because the block assignments carry fresh temporary indices and so cannot
hash-collide until canonical. It canonicalizes IN PLACE, hence expand_term
now clones rather than returning one of its arguments in the two early-exit
paths; that path is also parallel (std::execution::par_unseq), which is
safe because Index::next_tmp_index is a static std::atomic.

Derivation time, tests/integration/ucc 2 bernoulli <rank>, relwithdebinfo:
rank 2 0.416 -> 0.364 s, rank 3 7.584 -> 7.216 s, rank 4 170.6 -> 148.2 s.

Output is unchanged: serialize() of the rank-3 and rank-4 equations is
byte-identical to the pre-change baseline (45 543 and 292 512 bytes), and
repeat runs are byte-identical to each other despite the parallel index
minting. Term counts alone would not have been sufficient evidence -- the
use_topology bug rescaled terms while leaving counts and the VEV correct --
and to_latex() would not either, since it omits symmetry attributes.
Trim the development narrative out of bernoulli.{cpp,hpp} and keep the
reasons the code needs. The use_topology(false) comment keeps its cause
(the flag defaults to ON and silently rescales terms carrying a symmetric
amplitude pair on the partial-contraction path) and drops the stale
wick.hpp line reference. is_N_term keeps why rank > cutoff falls to R
rather than being dropped, and loses the paper quotes.

Also collapse hbar's five using-declarations into one and fix the range
notation in its error message.
CC::eom_r gains an optional block_ranks argument: a row-major K x K matrix
over the projection manifolds giving each block of the secular matrix its
own H̄ commutator truncation, instead of one uniform H̄ everywhere. The
manifolds are indexed by ASCENDING rank, so the qUCCSD ranks {2,1,1,0}
(10.1063/5.0062090 Table I, 10.1021/acs.jctc.5c01991 Table 1) serve EE, IP
and EA alike.

Each block is the sandwich <i|H̄|j> plus an explicit -E shift on the
diagonal at the block's own rank, not the commutator form <i|[H̄,r_j]|0>:
the commutator's extra -<i|r_j H̄^(k)|0> is manifold j's amplitude
residual, which vanishes only when k equals the rank the amplitudes were
converged against. Under the Bernoulli expansion each block's H̄ has its N
part removed for the same reason.

Empty block_ranks keeps the existing uniform path. That path commutes H̄
with an operator-level R, which the tensor-level Bernoulli H̄ cannot take
part in, so it now throws instead of aborting inside op.ipp. The block
shape and unitarity checks throw as well: SEQUANT_ASSERT compiles away
under SEQUANT_ASSERT_BEHAVIOR=IGNORE, and the shape check guards an
out-of-bounds read of block_ranks.
Pin the term counts of the {2,1,1,0} EE and IP sigma equations under the
Bernoulli expansion, and cover the three ways CC::eom_r rejects a
block_ranks argument: a non-square matrix, a non-unitary ansatz, and an
empty matrix under Bernoulli.
CMakeUserPresets.json is the documented per-developer companion to
CMakePresets.json, and Notes is a symlink into a personal notes repo.
… guard

Shortens the block-truncation derivation comments in cc.cpp/cc.hpp, adds
missing paper section/equation references, and switches to
std::ranges::reverse.

The previous version of this commit also dropped the explicit
Bernoulli-empty-block_ranks throw in CC::eom_r, reasoning that CC::hbar's own
ctor-time hbar_comm_rank check made it redundant. That conflated two
independent preconditions: hbar_comm_rank being set says nothing about
block_ranks being non-empty. Without the guard, CC::eom_r() under Bernoulli
falls through to the uniform path, which commutes the tensor-level Bernoulli
H̄ with an operator-level R -- exactly the mixing op.ipp's
commutes_with_atom() assert exists to catch. Restored the throw; caught by
running the existing bernoulli_quccsd_eom unit test, which this repo's
validation history (Notes/ucc/bernoulli-ucc/PR.md) never actually re-ran
after this change was first made.
@ajay-mk
ajay-mk force-pushed the ajay/feat/bernoulli-v2 branch from 65112a0 to c667779 Compare August 5, 2026 05:17
@ajay-mk
ajay-mk changed the base branch from master to ajay/feat/cc-rdm August 5, 2026 05:18
ajay-mk added 11 commits August 5, 2026 09:17
CMakeUserPresets.json and Notes are personal/local exclusions, not
project-wide ignores, and don't belong in a Bernoulli-expansion feature PR.
Use .git/info/exclude or a global excludesFile for these instead.
Trim prose that paraphrased or re-derived the paper's math in favor of
citing the equation it corresponds to. Fix a few inaccuracies found by
re-checking against 10.1063/1.5030344 and 10.1063/5.0062090 directly:
R_part was mislabeled a 'rank-preserving remainder' (it isn't, for
rank >= 2 operators), the DD EOM block was missing its bare-Fock content
(f_ij, f_ab), and the 'higher orders carry only R-subscripted inner
commutators' claim doesn't hold for the V_N-seeded terms. Also drops the
invented 'Cancellation #2' label and renames the orphaned 'Cancellation
#1' to 'the F-cancellation' throughout.
Ansatz, block_ranks shape and the Bernoulli/uniform-path mismatch are all
caller errors on an internal precondition, matching how the rest of CC
validates its configuration.
It simplified its argument in place, so the pointee the caller passed came
back canonicalized; both in-library callers cloned first to avoid that.
Clone inside instead.
The B_n listed at Eq. (40) are the paper's, i.e. B_n/n! in the textbook
normalization; say so. F-cancellation needs Brillouin, not canonicality, and
H̄⁰ does carry F. expand_to_blocks drops the other base spaces rather than
finding them empty; the projection is what makes that harmless.
Both build an operator-level similarity transform, which a tensor-level H̄
cannot enter; tʼ reached the mismatch and aborted deep inside op algebra,
rdm silently returned BCH-based RDMs.
… assert tests

Term counts are blind to the coefficients, so pin one projected equation in
full. For a single manifold the blocked path must reproduce the uniform one,
which shares no code with it. REQUIRE_THROWS_AS on a SEQUANT_ASSERT only holds
in a THROW build.
An empty block_ranks now fills every block with hbar_comm_rank instead of
being rejected: the restriction was only that the operator-level commutator
path cannot take a tensor-level H̄, which the blocked path handles. Index the
matrix with .at() so a wrong size cannot read past the end.
The hole/particle candidate list falls back to all base spaces when empty; if
that is empty too, the assignment loop indexed an empty vector and emitted a
term carrying a garbage index space.
Cut the debugging narrative, the restated code, and the per-rank node counts;
tighten the paragraphs that stay.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant