Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,7 @@ Recent highlights on the Orange Pi RV2 (SpaceMiT X60, RVV): fixing an OpenBLAS `

Library-level probes — performance *and* numerical correctness:

- **[BLAS](scientific-libs/blas.html)** — OpenBLAS improvements (U74 kernel, X60 `gemv_n` fix)
- **[DGEMM](scientific-libs/dgemm.html)** — `bench_dgemm` + `difftest` performance and correctness probes
- **[BLAS](scientific-libs/blas.html)** — OpenBLAS improvements (U74 kernel, X60 `gemv_n` / TRSM fixes) and [`OpenBLAS/`](https://github.com/opensolvers/benchmarks/tree/main/OpenBLAS) verification (`bench_dgemm`, `difftest`, `verify_ctrsm`)
- **[NumPy](scientific-libs/numpy.html)** — `bench_blas.py` DGEMM and `eigvalsh` through the SciPy stack
- **[LAPACK](scientific-libs/lapack.html)** — LAPACK path via NumPy `eigvalsh`
- **[ELPA](scientific-libs/elpa.html)** — dense eigensolver (CP2K / VASP class workloads)
Expand Down
2 changes: 0 additions & 2 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,6 @@ navigation_apps:
navigation_scientific_libs:
- title: BLAS
url: /scientific-libs/blas.html
- title: DGEMM
url: /scientific-libs/dgemm.html
- title: NumPy
url: /scientific-libs/numpy.html
- title: LAPACK
Expand Down
2 changes: 1 addition & 1 deletion apps/gromacs.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ The **`Force` row confirms the A/B** — 821.35 vs 819.97 s (0.17%, noise). RVV

| Probe | Axis | Isolated speedup | Whole-app effect |
| ----- | ---- | ---------------: | ---------------- |
| [DGEMM](../scientific-libs/dgemm.html) | BLAS | ~2.3× | (pure kernel) |
| [OpenBLAS verification](../scientific-libs/blas.html#verification) | BLAS | ~2.3× | (pure kernel) |
| [QE](qe.html) (DFT SCF) | BLAS | ~1.5–2.0× on BLAS routines | ~1.2–1.3× (FFT half untouched) |
| [QE + FFTW](../scientific-libs/fftw.html) | FFT | ~1.02× on `fftw` | **~0%** (`FFTW_ESTIMATE`) |
| **GROMACS** (this page) | **FFT** | **~1.23× on `PME 3D-FFT`** | small (`Force` = 90%) |
Expand Down
6 changes: 3 additions & 3 deletions apps/qe.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Benchmark source: [opensolvers/benchmarks/qe](https://github.com/opensolvers/ben

## Why QE as a probe

[`dgemm`](../scientific-libs/dgemm.html) isolates BLAS-3; [HPL](hpl.html) and [ELPA](../scientific-libs/elpa.html) probe single HPC solvers; [FFTW](../scientific-libs/fftw.html) covers the FFT half. A full DFT SCF mixes level-3 GEMM (`calbec`, subspace rotation), dense LAPACK diagonalization, latency-bound BLAS-2, MPI, **and a large FFT fraction** — so it shows both whether a buggy vector BLAS breaks a production code, and what fraction of a real run each backend swap actually moves.
[OpenBLAS verification](../scientific-libs/blas.html#verification) isolates BLAS-3; [HPL](hpl.html) and [ELPA](../scientific-libs/elpa.html) probe single HPC solvers; [FFTW](../scientific-libs/fftw.html) covers the FFT half. A full DFT SCF mixes level-3 GEMM (`calbec`, subspace rotation), dense LAPACK diagonalization, latency-bound BLAS-2, MPI, **and a large FFT fraction** — so it shows both whether a buggy vector BLAS breaks a production code, and what fraction of a real run each backend swap actually moves.

## Correctness — stock RVV `gemv_n` breaks a real DFT SCF

Expand Down Expand Up @@ -56,7 +56,7 @@ Same X60, patched RVV vs scalar:

| Probe | Speedup | Why |
| ----- | ------: | --- |
| [DGEMM](../scientific-libs/dgemm.html) (pure level-3) | ~2.3× | all BLAS-3 |
| [OpenBLAS verification](../scientific-libs/blas.html#verification) (pure level-3) | ~2.3× | all BLAS-3 |
| [HPL](hpl.html) (Linpack) | ~1.8× | BLAS-3 + `dgemv` panel factorization |
| [ELPA](../scientific-libs/elpa.html) (eigensolver) | ~1.58× | BLAS-3 + BLAS-2 tridiagonalization |
| **QE** (full DFT SCF) | **~1.2–1.3×** | BLAS + ~40–50% FFT + MPI |
Expand All @@ -76,4 +76,4 @@ RVV_LIB=/path/to/patched/libopenblas.so ./run-qe-ab.sh si-scf.in 4
RVV_LIB=/path/to/patched/libopenblas.so ./run-perf-ab.sh si-super-64.in 4
```

**Toolchain:** QuantumESPRESSO 7.5 / foss-2025b (EESSI), FlexiBLAS 3.4.5, OpenBLAS 0.3.30. Patched vector backend = OpenBLAS 0.3.30 with the RISC-V `gemv_n` NaN fix backported — same build used by [DGEMM](../scientific-libs/dgemm.html), [HPL](hpl.html), and [ELPA](../scientific-libs/elpa.html).
**Toolchain:** QuantumESPRESSO 7.5 / foss-2025b (EESSI), FlexiBLAS 3.4.5, OpenBLAS 0.3.30. Patched vector backend = OpenBLAS 0.3.30 with the RISC-V `gemv_n` NaN fix backported — same build used by [OpenBLAS verification](../scientific-libs/blas.html#verification), [HPL](hpl.html), and [ELPA](../scientific-libs/elpa.html).
4 changes: 2 additions & 2 deletions boards/F3.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ See also the [HPL app overview](../apps/hpl.html).

| Probe | Scalar | Patched RVV | Speedup |
| ----- | ------ | ----------- | ------- |
| [DGEMM](../scientific-libs/dgemm.html) N=2048, 1 core | 1.26 GFLOP/s | 2.96 GFLOP/s | 2.35× |
| [DGEMM](../scientific-libs/dgemm.html) N=4096, 8 threads | — | 17.71 GFLOP/s | — |
| [DGEMM](../scientific-libs/blas.html#verification) N=2048, 1 core | 1.26 GFLOP/s | 2.96 GFLOP/s | 2.35× |
| [DGEMM](../scientific-libs/blas.html#verification) N=4096, 8 threads | — | 17.71 GFLOP/s | — |
| [NumPy](../scientific-libs/numpy.html) DGEMM N=4096 | 4.91 GFLOP/s | 17.51 GFLOP/s | 3.6× |
| [NumPy](../scientific-libs/numpy.html) `eigvalsh` N=2048 | 9.59 s | 5.94 s | 1.6× |
| ELPA `na=3000` | 50.42 s | **34.83 s** | 1.45× |
Expand Down
60 changes: 57 additions & 3 deletions scientific-libs/blas.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Improvements to **OpenBLAS 0.3.30** on RISC-V boards — built via EasyBuild, deployed through [EESSI](https://www.eessi.io/) with **FlexiBLAS** runtime swapping.

Verification and A/B probes: [DGEMM](dgemm.html) (`bench_dgemm`, `difftest`), [NumPy](numpy.html) (`bench_blas.py`). Source repo: [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks).
Verification microbenchmarks live in [opensolvers/benchmarks/OpenBLAS](https://github.com/opensolvers/benchmarks/tree/main/OpenBLAS) (`bench_dgemm`, `difftest`, `verify_ctrsm`). Stack probes: [NumPy](numpy.html) (`bench_blas.py`). Full repo: [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks).

**Base stack:** GCC 14.3.0, OpenBLAS 0.3.30, EESSI `2025.06-001` ([`dev.eessi.io/riscv`](https://www.eessi.io/docs/repositories/dev.eessi.io-riscv/)).

Expand All @@ -11,8 +11,8 @@ Verification and A/B probes: [DGEMM](dgemm.html) (`bench_dgemm`, `difftest`), [N
| Board / CPU | Problem (stock 0.3.30) | Fix | Result |
| ----------- | ---------------------- | --- | ------ |
| [VisionFive 2](../boards/VisionFive2.html) — SiFive **U74** (scalar) | No U74 kernel; falls back to generic `RISCV64_GENERIC` C `2×2` GEMM | New **4×4 DGEMM micro-kernel** in RV64 assembly, `TARGET=U74` | Single-core DGEMM **~1.4 → 1.77 GFLOP/s**; 4-core DGEMM **6.31 GFLOP/s**; [HPL **1.69×**](../apps/hpl.html) |
| [Orange Pi RV2](../boards/RV2.html) — SpacemiT **X60** (RVV VLEN=256) | RVV `gemv_n` zeroes an **uninitialized** vector register → `dgemv` returns NaN | Backport upstream `gemv_n` fix; `TARGET=RISCV64_ZVL256B` | [DGEMM](dgemm.html) 2.3×; [NumPy](numpy.html) DGEMM 2.4×; [HPL](../apps/hpl.html) **10.53 GFLOP/s** |
| [Banana Pi F3](../boards/F3.html) — same K1 / X60 SoC | Same RVV `gemv_n` bug as RV2 | Same [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) fix | [DGEMM](dgemm.html) 2.35×; [NumPy](numpy.html) DGEMM **3.6×**; HPL **11.52 GFLOP/s** |
| [Orange Pi RV2](../boards/RV2.html) — SpacemiT **X60** (RVV VLEN=256) | RVV `gemv_n` zeroes an **uninitialized** vector register → `dgemv` returns NaN | Backport upstream `gemv_n` fix; `TARGET=RISCV64_ZVL256B` | Verification 2.3×; [NumPy](numpy.html) DGEMM 2.4×; [HPL](../apps/hpl.html) **10.53 GFLOP/s** |
| [Banana Pi F3](../boards/F3.html) — same K1 / X60 SoC | Same RVV `gemv_n` bug as RV2 | Same [easyconfigs#26444](https://github.com/easybuilders/easybuild-easyconfigs/pull/26444) fix | Verification 2.35×; [NumPy](numpy.html) DGEMM **3.6×**; HPL **11.52 GFLOP/s** |

## Packages

Expand All @@ -23,6 +23,60 @@ Verification and A/B probes: [DGEMM](dgemm.html) (`bench_dgemm`, `difftest`), [N

Build with `eb --from-pr <num> --robot` into an **EESSI-extend** user install, then `flexiblas add` / `flexiblas default` — no downstream rebuild.

## Verification {#verification}

Small, self-contained programs in [opensolvers/benchmarks/OpenBLAS](https://github.com/opensolvers/benchmarks/tree/main/OpenBLAS) for **BLAS performance and per-routine correctness** — used to isolate broken RVV kernels before trusting downstream apps.

| File | Purpose |
| ---- | ------- |
| [`bench_dgemm.c`](https://github.com/opensolvers/benchmarks/blob/main/OpenBLAS/bench_dgemm.c) | Times square `C = A×B` (3 reps), reports GFLOP/s, prints `C[0]` |
| [`difftest.c`](https://github.com/opensolvers/benchmarks/blob/main/OpenBLAS/difftest.c) | `dlopen`s a BLAS `.so`, runs level-1/2/3 routines, reports `sum` / NaN counts |
| [`verify_ctrsm.c`](https://github.com/opensolvers/benchmarks/blob/main/OpenBLAS/verify_ctrsm.c) | Full-parameter TRSM correctness sweep (localizes [OpenBLAS#5928](https://github.com/OpenMathLib/OpenBLAS/pull/5928) VLEN bug in `_rvv_v1` kernels) |

Both switch backends at runtime via FlexiBLAS or `OPENBLAS_CORETYPE` — no recompile.

```bash
gcc -O2 bench_dgemm.c -o bench_dgemm -lflexiblas
gcc -O2 difftest.c -o difftest -ldl -lm

OPENBLAS_NUM_THREADS=8 ./bench_dgemm 4096
./difftest /path/to/libopenblas.so
```

### Correctness — `difftest` (X60, RV2 & F3)

Bit-identical on [Orange Pi RV2](../boards/RV2.html) and [Banana Pi F3](../boards/F3.html):

| Backend | `dgemv` NaN | `dgemm` NaN | `dtrsm` NaN | `dgemv` sum |
| ------- | ----------- | ----------- | ----------- | ----------- |
| Stock EESSI, default RVV | **192** | 0 | 0 | 198.94 (wrong) |
| Forced scalar | 0 | 0 | 0 | 42.06549 (reference) |
| Patched RVV (`gemv_n` fix) | 0 | 0 | 0 | 42.06549 (matches) |

Fault is in **`dgemv` only** — plain `dgemm` and `dtrsm` look fine on the broken `gemv_n` build, which is why [HPL](../apps/hpl.html) and [Quantum ESPRESSO](../apps/qe.html) can fail while a GEMM micro-benchmark passes.

A second bug — RVV `_rvv_v1` TRSM kernels not VLEN-agnostic ([OpenBLAS#5928](https://github.com/OpenMathLib/OpenBLAS/pull/5928)) — is caught by `verify_ctrsm` on `ZVL128B` builds where `GEMM_UNROLL_M ≠ VSETVL_MAX`.

### Performance — `bench_dgemm`

#### Orange Pi RV2 (1 core, N=2048)

| Backend | GFLOP/s | `C[0]` |
| ------- | ------- | ------ |
| Scalar | 1.16 | 245.24 |
| Patched RVV | 2.62 | 245.24 |

**2.3×** faster, numerically identical.

#### Banana Pi F3 (cross-board)

| Backend | GFLOP/s | `C[0]` |
| ------- | ------- | ------ |
| Scalar | 1.26 | 245.24 |
| Patched RVV | 2.96 | 245.24 |

**2.35×** at 1 core. Threaded (8 cores, N=4096): **17.71 GFLOP/s** on patched RVV.

## Notes

- **U74** — performance kernel; stock OpenBLAS works but leaves FP throughput on the table.
Expand Down
54 changes: 0 additions & 54 deletions scientific-libs/dgemm.md

This file was deleted.

4 changes: 2 additions & 2 deletions scientific-libs/lapack.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

LAPACK performance and correctness probed via NumPy **`eigvalsh`** (`dsyevd`) — the LAPACK-heavy half of the [NumPy benchmark](numpy.html).

Benchmark source: [opensolvers/benchmarks/numpy](https://github.com/opensolvers/benchmarks/tree/main/numpy) (`bench_blas.py`). For DGEMM (`A @ B`) see the [NumPy](numpy.html) or [DGEMM](dgemm.html) pages.
Benchmark source: [opensolvers/benchmarks/numpy](https://github.com/opensolvers/benchmarks/tree/main/numpy) (`bench_blas.py`). For DGEMM (`A @ B`) see the [NumPy](numpy.html) or [BLAS verification](blas.html#verification) sections.

## Orange Pi RV2 (8 threads)

Expand All @@ -12,7 +12,7 @@ Patched RVV OpenBLAS via FlexiBLAS (`SciPy-bundle` module, [easyconfigs#26444](h
| ------ | ------ | ----------- | ------- |
| EIGH (`eigvalsh`, N=2048) | 10.54 s | 6.72 s | **1.6×** |

Stock EESSI default RVV: `eigvalsh` returns NaN / fails to converge. See [DGEMM `difftest`](dgemm.html).
Stock EESSI default RVV: `eigvalsh` returns NaN / fails to converge. See [`difftest`](blas.html#verification).

## Banana Pi F3 (cross-board)

Expand Down
4 changes: 2 additions & 2 deletions scientific-libs/numpy.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ A **SciPy-stack probe** for BLAS and LAPACK through NumPy — from [opensolvers/
| DGEMM | `A @ B` | `dgemm` | GFLOP/s |
| EIGH | `np.linalg.eigvalsh(S)` | `dsyevd` | seconds |

Unlike raw [DGEMM](dgemm.html), `eigvalsh` exercises BLAS-2 tridiagonalization — a correctness gate for the same `gemv_n` bug that breaks [HPL](../apps/hpl.html) and [ELPA](elpa.html).
Unlike raw [OpenBLAS verification](blas.html#verification), `eigvalsh` exercises BLAS-2 tridiagonalization — a correctness gate for the same `gemv_n` bug that breaks [HPL](../apps/hpl.html) and [ELPA](elpa.html).

```bash
module load SciPy-bundle/2025.07-gfbf-2025b
Expand All @@ -33,6 +33,6 @@ Both patched results finite. See [LAPACK](lapack.html) for the LAPACK angle.
| DGEMM N=4096 | 4.91 GFLOP/s | **17.51 GFLOP/s** | **3.6×** |
| EIGH N=2048 | 9.59 s | 5.94 s | **1.6×** |

**Stock unpatched RVV:** `eigvalsh` raises `LinAlgError: Eigenvalues did not converge`; DGEMM alone stays finite at 10.94 GFLOP/s (only `gemv` is broken — see [DGEMM difftest](dgemm.html)).
**Stock unpatched RVV:** `eigvalsh` raises `LinAlgError: Eigenvalues did not converge`; DGEMM alone stays finite at 10.94 GFLOP/s (only `gemv` is broken — see [`difftest`](blas.html#verification)).

Eigensolver speedup (~1.6×) is lower than DGEMM (~2.4–3.6×) because `eigvalsh` mixes BLAS-3 with latency-bound BLAS-2 — same pattern as [ELPA](elpa.html).
5 changes: 3 additions & 2 deletions website.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Context file for structural and content decisions on [opensolvers.com](https://w

1. **Home** · **Videos** · **YouTube** (external)
2. **Apps** — HPL, Quantum ESPRESSO, ONNX Runtime, GROMACS
3. **Scientific libs** — BLAS, DGEMM, NumPy, LAPACK, ELPA, MLAS, FFTW, ScaLAPACK
3. **Scientific libs** — BLAS (incl. OpenBLAS verification), NumPy, LAPACK, ELPA, MLAS, FFTW, ScaLAPACK
4. **Boards** — VisionFive 2, OrangePi RV2, BananaPi F3

Nav config: `_config.yml` (`navigation`, `navigation_boards`, `navigation_apps`, `navigation_scientific_libs`). Rendered in `_includes/header.html`. Cayman theme requires `_layouts/default.html` override to include the header.
Expand All @@ -44,7 +44,7 @@ Catalog: `_data/videos.yml` (newest first). Rendered by `_includes/video-grid.ht

| Source | Used for |
|--------|----------|
| [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks) | BLAS (`dgemm/`), HPL (`hpl/`), ELPA/ScaLAPACK (`elpa/`, `scalapack/`), QE/FFTW (`qe/`, `fftw/`), GROMACS (`gromacs/`), ONNX/MLAS (`onnx/`), NumPy (`numpy/`) |
| [opensolvers/benchmarks](https://github.com/opensolvers/benchmarks) | OpenBLAS (`OpenBLAS/`), HPL (`hpl/`), ELPA/ScaLAPACK (`elpa/`, `scalapack/`), QE/FFTW (`qe/`, `fftw/`), GROMACS (`gromacs/`), ONNX/MLAS (`onnx/`), NumPy (`numpy/`) |
| [EESSI/docs#818](https://github.com/EESSI/docs/pull/818) | VisionFive 2 / U74 OpenBLAS + HPL |
| [EESSI/docs#819](https://github.com/EESSI/docs/pull/819) | Orange Pi RV2 / X60 RVV `gemv_n` fix + HPL |
| [easyconfigs#26436](https://github.com/easybuilders/easybuild-easyconfigs/pull/26436) | U74 OpenBLAS package |
Expand All @@ -60,6 +60,7 @@ Catalog: `_data/videos.yml` (newest first). Rendered by `_includes/video-grid.ht

| Date | Decision |
|------|----------|
| 2026-07-19 | Sync benchmarks PR #18: merge OpenBLAS verification into BLAS page; remove `dgemm.html`; fix GitHub links |
| 2026-07-18 | Videos page + `_data/videos.yml`; U74 YouTube link; SEO (`url`, JSON-LD, robots.txt, page descriptions) |
| 2026-07-18 | Per-board compute-backend SVGs; fix invalid UTF-8 in VisionFive 2 / K1 diagrams |
| 2026-07-18 | U74 video production moved to private repo `opensolvers/u74-video` (removed from site) |
Expand Down