feat(3.9): tail-df consolidation - three-value df_convention knob + t(residual) defect fix (M-127) - #738
Conversation
…(residual) defect fix (M-127) Converge the tail degrees of freedom the t-distribution uses across the analytical clustered surfaces (PR C of the variance-consolidation program, after #736 inventory/D3 and #737 K_reference/M-126). Defect fix (z -> t(residual), point estimates/SEs/t-stats unchanged): - Wooldridge default-hc1 OLS arms: one knob-resolved df_fallback feeds every non-BM arm (survey df first; classical/hc2 reproduce the historical df_one_way values bit-for-bit under the default). - SunAbraham aggregates: cells and aggregates now share ONE df source per fit (the saturated regression's df_) - the D4 two-conventions-in- one-fit defect is gone. - StackedDiD pooled inference: positive-weight n_eff - k_kept. - ImputationDiD pretrends per-lead inference: n - k_kept - absorbed rank on the untreated subsample (the joint pretrend Wald F keeps its cluster-robust F(q, G-1) denominator, documented as a pairing). Three-value df_convention knob {"residual","cluster","normal"} on DiD/TWFE/MPD/LinearRegression (widened from two values; "normal" is ungated at the fallback level - clustered, unclustered, and conley) and NEW on SunAbraham, WooldridgeDiD (OLS arms), StackedDiD, ImputationDiD (pretrends), and LPDiD. LPDiD defaults to "cluster" (the existing Stata t(G-1) reference - zero movement, degenerate lanes keep literal df=None). "normal" reproduces the pre-3.9 numbers on the fixed surfaces. Survey/replicate df and hc2_bm BM DOF keep precedence under every value. set_params is atomic under unknown keys on all five standalone estimators. Shared resolver: utils.resolve_tail_df + validate_df_convention. Provenance: df_convention on five results containers; inference_df on SunAbraham/StackedDiD (None under bootstrap overrides); Wooldridge _df_one_way renamed _df_analytic_fallback with a __setstate__ pickle migration (stays live on bootstrap fits so post-fit analytical aggregate() reproduces fit-time inference). Ledger/docs: M-127 behavior row + M-128..M-131 default-flip rows (phase 5, 'residual' -> 'cluster'), matrix snapshot 105 -> 110, v4-design/DEFERRED/TODO/CHANGELOG sync; REGISTRY tail-df notes for TWFE/SA/StackedDiD/Wooldridge/ImputationDiD/LPDiD; regenerated variance-conventions inventory (defect rows converged); llms-full constructor/results updates. Tests: resolver unit branch table; per-estimator df-convention suites (cluster == hand-reconstructed t(G-1), normal == z, default != old z, survey-df precedence under every knob value, bit-identity locks); audit-matrix re-pins with measured df multisets; fixest golden p/CI now gated on both lanes (iid arms under the residual default, cluster arms under df_convention="cluster"); event-study-surface provenance flips; Wooldridge pickle-migration and bootstrap-aggregate parity tests.
Overall assessment✅ Looks good — no unmitigated P0/P1 findings. Executive summary
Methodology
No undocumented methodology mismatch, missing assumption check, or incorrect variance/SE was identified. Code QualityNo findings. The shared resolver and PerformanceNo findings. Added rank/count computations are linear and occur once per relevant fit. MaintainabilityNo findings. Parameter validation, propagation, and result provenance are consistent across the affected estimators. Tech Debt
SecurityNo findings. No secrets, unsafe execution, or new external-input risks were identified. Documentation/Tests
The added tests otherwise cover resolver branches, df precedence, estimator interactions, transactional parameter updates, and NaN-safe inference. |
…ribute entries The TwoWayFixedEffects constructor docstring still enumerated two knob values and claimed blanket unclustered/Conley inertness, which "normal" falsifies (it is deliberate z at the fallback level on every fit) - rewritten with the per-value scoping used by DifferenceInDifferences and LinearRegression. The committed docs/api/_autosummary attribute lists gain the new result fields: df_convention on the five containers, inference_df on SunAbrahamResults/StackedDiDResults (alphabetical insertion, the #715/#726 convention).
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall assessment✅ Looks good — no unmitigated P0/P1 findings. Executive summary
Methodology
No incorrect variance, SE, identification logic, or undocumented methodological deviation was found. Code QualityNo findings. Inference consistently routes through PerformanceNo findings. Added rank and cluster-count calculations occur once per relevant fit. MaintainabilityNo findings. Constructor, parameter API, wrapper, result, aggregation, and inference propagation are consistent. Tech Debt
SecurityNo findings. Documentation/TestsNo findings. The previous P2 is resolved: TWFE now documents all three conventions at Runtime tests could not be executed because the reviewer environment lacks |
Summary
df_conventionknob{"residual", "cluster", "normal"}: widened on DiD/TWFE/MultiPeriodDiD/LinearRegression ("normal"is ungated at the fallback level - clustered, unclustered, and conley) and NEW on SunAbraham, WooldridgeDiD (OLS arms), StackedDiD, ImputationDiD (pretrends), and LPDiD."normal"reproduces the pre-3.9 numbers on the fixed surfaces;"cluster"gives the fixest/Stata t(G-1) reference."cluster"(its existing Stata t(G-1) convention - zero movement, bit-identity locked; degenerate lanes keep literal df=None under every value).set_paramsis atomic under unknown keys on all five standalone estimators.df_conventionon five results containers;inference_dfon SunAbraham/StackedDiD (None under bootstrap overrides); Wooldridge_df_one_wayrenamed_df_analytic_fallbackwith a__setstate__pickle migration.'residual'->'cluster'), matrix snapshot 105 -> 110, v4-design/DEFERRED/TODO/CHANGELOG sync, REGISTRY tail-df notes for TWFE/SA/StackedDiD/Wooldridge/ImputationDiD/LPDiD, regenerated variance-conventions inventory (defect rows converged), llms-full updates.Methodology references (required if estimator / math changes)
docs/methodology/REGISTRY.md(TWFE deviation note + per-estimator tail-df notes) withdf_convention="cluster"available for reference parity now and the default flip version-gated to 4.0 (M-004..M-006 + M-128..M-131). The LPDiD RA-lane"residual"df (n_total - k0_kept - 1) is a library convention with no external anchor (Stata teffects ra reports z), stated in the LPDiD REGISTRY note.Validation
tests/test_utils.py(resolver branch table),tests/test_variance_conventions.py(audit-matrix re-pins with measured df multisets + SA-cells spy arm; regenerated byte-gated inventory),tests/test_sun_abraham.py,tests/test_wooldridge.py,tests/test_stacked_did.py,tests/test_imputation.py,tests/test_lpdid.py(per-estimator df-convention suites incl. survey-df precedence under every knob value and atomic set_params),tests/test_estimators_vcov_type.py("normal"on clustered/unclustered/conley),tests/test_fixest_did_twfe_parity.py(golden p/CI now gated: iid arms under the residual default, cluster arms underdf_convention="cluster"),tests/test_event_study_surface.py(provenance flips),tests/test_v4_matrix.py(snapshot 110)."cluster"); LPDiD audit rows and all existing goldens bit-identical under its"cluster"default.Security / privacy