Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .agents/docs/2026-07-19-compat-ffmpeg-trimmed-profiles-decision.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# compat.ffmpeg 裁剪档(decode-min 等 profile 变体)——设计决策与暂缓依据

日期:2026-07-19 · 结论:**暂缓实现,等 mcpp#242;设计已定型如下。**

## 为什么现在做不了

1. **features 是加法语义**:裁剪档要求"从全量里减",唯一表达 = 基础源集改为
min、`full` feature 补齐其余。为不破坏已发布消费者(ffmpeg 模块包 0.0.1
期望全量),包侧需 `default = ["full"]`(Feature System v2 已支持)——但
**消费端没有 default-features 关闭语法**,min 永远退不回去。已提 mcpp#242
(含 cargo 式拼写建议)。
2. **config 快照随组件集变化**:config_components.h/.asm 按 profile 不同,
generated_files 不可 feature 化。方案:包内新增 build.mcpp,内嵌两份
config 快照,按 MCPP_FEATURE_FULL(或其缺席)落盘对应一份 ——
compat.opencv 的 build.mcpp 合成模式已验证同类机制可行,这半边不需要
mcpp 改动。

## 落地顺序(#242 发版后)

1. gen_config.sh 增加 min-profile configure 变体(F0 的 decode-min 配置,
源列表推导管线复用);gen_descriptor.py 输出双快照 + feature 分段
(`full` gates 全量源集差集)+ 新增 build.mcpp。
2. `default = ["full"]`:既有消费者零感知;min 消费者
`ffmpeg = { version, default-features = false }`。
3. 成员测试:min 消费端断言 h264 decoder 存在 + 某 encoder 缺席(负向验证)。

## 与"功能完整性"的关系

ffmpeg-m 本身已是 100% 全功能(2281 TU 全组件),裁剪档是**体积优化**而非
功能补全,不阻塞 R 系列其他项。
104 changes: 104 additions & 0 deletions .agents/docs/2026-07-19-compat-opencv-unification.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# compat.opencv 合一 + v2 de-stub + FFmpeg videoio 后端 + compat.ffmpeg 瘦身

日期:2026-07-19 · 分支 feat/compat-opencv-unification(基于 workspace-only PR)

## 决策背景(用户拍板)

1. compat.opencv 只保留一份:老 install()-CMake 4.13 形态删除,`compat.opencv`
直接采用源码直编形态(原 compat.opencv5 的演进版);compat.opencv5 过渡期保留,
opencv 模块包切换依赖后删除。
2. compat.ffmpeg / compat.opencv 简洁性优化都做。
3. F2(videoio↔ffmpeg)并入本次 regen——只跑一次参考构建。

## compat.opencv v2(源码直编形态的 0.0.97 演进)

- **de-stub**:mcpp 0.0.97 修了 #233(对象路径消歧)+ #234(空格 defines 引号),
v1 的 457-TU 全量转发 stub 层撤销。sources = 真实 tarball 路径,385 个文件按
目录 glob/花括号交替(#228)压到 **19 条**;快照 dispatch stub 以
`mcpp_generated/**` 直接进 sources;flags 改真实路径 glob
(`**/modules/<mod>/**` 同时覆盖 tarball 与快照两侧)。
- **仍走 build.mcpp 的**:字体 blob2hdr、OpenCL kernel 嵌入(cl2cpp)、
libjpeg-turbo BITS_IN_JSAMPLE=12/16 重复编译 stub(43 个,同一 .c 三次编译
是构建语义,普通 sources 表达不了;stub 文件名加组前缀保证唯一 basename,
规避 mcpp#239 的绝对路径对象逃逸)。
- **FFmpeg videoio 后端**:参考构建 WITH_FFMPEG=ON(本地静态 FFmpeg prefix +
pkg-config 探测;gen_config.sh 自动构建该 prefix);cap_ffmpeg 进 videoio 组
(组 defines 带 HAVE_FFMPEG/HAVE_FFMPEG_LIBAVDEVICE);描述符
`deps = { ["compat.ffmpeg"] = "8.1.2" }` —— **compat 依赖 compat 首例**
(V0 spike 玩具实证 + 本描述符实测:传递解析、头传播、链接闭包全通)。
注意描述符键是 `deps` 不是 `dependencies`(误写会被 build 静默忽略,
`xpkg parse`/lint 会拦,mcpp#237)。
- 版本键仍 5.0.0(内容修订);111KB → 100KB。
- 参考坑:HAVE_FFMPEG 不落 cvconfig.h,按 TU `-D` 传递(校验要看
ninja-cmds.log 里的 cap_ffmpeg);gcc13 在 libswscale/output.o 也 ICE
(FFmpeg prefix 用 xlings gcc16 构建,FFMPEG_CC)。

## compat.ffmpeg 瘦身(B4)

- gen_descriptor.py 加目录 glob/花括号压缩(与 opencv 同法,FFMPEG_SRC 树上
验证"整目录编译才发目录 glob"):2281 条显式 sources → **28 条 glob**;
展开集与原列表逐文件相等(脚本断言);其余段 byte 级一致。337KB → 267KB。

## 测试

- tests/examples/opencv 成员重写:compat.opencv 5.0.0(cfg(linux) 门控),断言
imgproc/imgcodecs + **真实 mp4 编→解码回环**(CAP_FFMPEG 双向,10 帧)——
这同时是 R3 视频主线在 compat 层的验收;本地 `mcpp test -p opencv` 通过。
- tests/examples/opencv5 成员与 pkgs/c/compat.opencv5.lua 过渡期保留
(已发布 opencv 0.0.2 模块 tarball 依赖它),opencv-m v0.0.3 切换后删除。

## R4:unifont feature(随本 PR)

- `pkgs/c/compat.opencv-unifont.lua`:数据资产包 —— 上游 opencv_3rdparty 原始
`WenQuanYiMicroHei.ttf.gz`(GLOBAL 直取 raw.githubusercontent,CN 镜像
mcpp-res/opencv@unifont-1.0.0,sha256 钉,MD5 与上游 CMake pin 一致)。
裸 .gz 不是归档:安装器把它字节保真地放到 store 共享 `data/runtimedir/`
(实测),verdir 只有元数据 → 描述符带 asio 式 anchor-TU 空壳 mcpp 段。
- `compat.opencv` 新 feature `unifont`(0.0.97 features 已支持 deps/defines
门控):`deps = { ["compat.opencv-unifont"] = "1.0.0" }` +
`defines = { "HAVE_UNIFONT" }`;build.mcpp 见 `MCPP_FEATURE_UNIFONT=1` 时按
`MCPP_MANIFEST_DIR/../../../runtimedir/<fname>`(fallback:verdir 扫描)找到
字体并 blob2hdr 成 `builtin_font_uni.h`(符号 OcvBuiltinFontUni,与 cmake
ocv_blob2hdr 逐字节同构)。无 MCPP_DEP_<NAME>_DIR 契约变量是已知空洞
(待提 mcpp 增强)。
- 成员 tests/examples/opencv-unifont:长式依赖
`opencv = { version = "5.0.0", features = ["unifont"] }`,断言
`FontFace("uni")` 可用(该内建字体面只在 HAVE_UNIFONT 下存在——本身即
feature 探针)且 CJK putText 出墨;默认成员回归不受影响。

## R5:dnn feature(随本 PR,compat 层)

- 参考构建换成 BUILD_LIST+dnn(BUILD_PROTOBUF=ON、WITH_FLATBUFFERS=ON,与
非 dnn 参考构建并存);生成器带 `BASE_BLD` 交叉断言:**feature-off TU 集与
非 dnn 参考构建逐文件相等**(415 TU)。
- `features.dnn`:加法源集(309 文件 → 23 glob:modules/dnn + vendored
protobuf(.cc)+ mlas(.cpp+.S))+ `defines = { "HAVE_OPENCV_DNN" }`。
- 快照三处 profile 漂移全为良性:data_config 只差构建目录路径(rewrite 现已
中和裸路径)、opencv_modules.hpp 只差 HAVE_OPENCV_DNN 行(无 TU 消费,改走
featureDefines)、version_string.inc 纯装饰 —— 统一船 base 版,变体机制不需要。
- ISA overlay 改按(组×ISA)键控发射(dnn 的 per-ISA 定义集与基础模块不同)。
- `-isystem` 目录现纳入 include_dirs 收集(protobuf 头此前被丢)。
- 三个策划性例外(均注释在生成器里):mlas 组 `_GNU_SOURCE`;
mlas/lib/platform.cpp `-include unistd.h`(上游靠传递包含侥幸);
`MlasHGemmSupported` stub(vendored 子集声明+调用但未带定义,上游静态库
按需抽取掩盖,mcpp 全对象链接暴露 —— stub 返回 false = 语义正确回退)。
- 同源双编译第二例:mlas_threading.cpp(mlas 库版 vs dnn 版)→ tu-stub 通道,
manifest 新增 `?dnn` feature 守卫前缀语法。
- 成员 tests/examples/opencv-dnn(测试文件特意不叫 dnn.cpp —— 会与依赖包
modules/dnn/src/dnn.cpp 相撞,#233 家族在 test-scan 边亦漏,已补充至 #240)。
- 模块层(opencv-m 的 `opencv.dnn` 接口 + 模块包 feature 转发)为后续工作:
模块包把自己的 feature 转发给依赖(`opencv/dnn` 式)尚无文法,届时视需要提 mcpp 增强。

## 合并门槛(先决条件)

**mcpp#240**:#233 消歧后可执行目标的链接输入未跟随改名——依赖包与消费者源文件
同 basename(OpenCV 带 modules/{imgproc,geometry}/src/main.cpp,消费者入口几乎
都叫 src/main.cpp)即 `'obj/main.o' missing and no known rule`。成员测试文件名
不撞可以绿,但**发布后真实用户必踩**——本 PR 必须等 mcpp 修复发版 + CI pin 升级
后再合并。(2 文件最小复现已附在 issue。)

## 后续

- opencv-m v0.0.3(R1+R2)切依赖 `compat.opencv = "5.0.0"` → index 升 opencv
0.0.3 → 删 compat.opencv5.lua + opencv5 成员 + README 过渡注记。
- R4 unifont / R5 dnn 以 feature 挂在本包(gen_config 的 profile 分支)。
52 changes: 36 additions & 16 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,26 @@ on:
workflow_dispatch:

env:
# 0.0.97: default-namespace index redirect (`[indices] default = { path }`),
# which turned the public module packages (imgui/ffmpeg/opencv/tinyhttps)
# into ordinary workspace members and retired the per-package reseeding
# smoke shells + their dedicated jobs; also carries the synchronous nasm
# bootstrap (mcpp#232 — the `mcpp index update` pre-step is gone), obj-path
# disambiguation (#233), spacey-defines quoting (#234), and purview-include
# depfile tracking (#235). Older floors of note: 0.0.96 fixed the windows
# scanner symlink-escape crash (mcpp#230); 0.0.94 fixed feature-gated
# `sources` under `mcpp test` (mcpp#218); 0.0.91 added standard = "c++fly"
# to the resolver grammar, so c++fly descriptors get the lint WARN below,
# not a hard grammar-parse rejection.
MCPP_VERSION: "0.0.97"
# 0.0.98: closes the obj-path disambiguation follow-ups that gated the
# source-build compat.opencv unification — #240 (link inputs now follow
# the disambiguated object names, so a dependency + consumer sharing a
# source basename like `src/main.cpp` no longer 'obj/main.o missing') and
# #239 (absolute/`..` dep-generated source paths sanitized component-wise
# so objects stay under obj/). Also: `MCPP_DEP_<NAME>_DIR` build.mcpp
# contract (#241), consumer-side `default-features = false` (#242), and a
# loud unknown-mcpp-key warning with did-you-mean (#237, replaces the
# silent-ignore at build time). Carried from 0.0.97: default-namespace
# index redirect (`[indices] default = { path }`), which turned the public
# module packages (imgui/ffmpeg/opencv/tinyhttps) into ordinary workspace
# members and retired the per-package reseeding smoke shells + their
# dedicated jobs; synchronous nasm bootstrap (mcpp#232 — the `mcpp index
# update` pre-step is gone), obj-path disambiguation (#233), spacey-defines
# quoting (#234), purview-include depfile tracking (#235). Older floors of
# note: 0.0.96 fixed the windows scanner symlink-escape crash (mcpp#230);
# 0.0.94 fixed feature-gated `sources` under `mcpp test` (mcpp#218); 0.0.91
# added standard = "c++fly" to the resolver grammar, so c++fly descriptors
# get the lint WARN below, not a hard grammar-parse rejection.
MCPP_VERSION: "0.0.98"

jobs:
lint:
Expand Down Expand Up @@ -145,10 +153,22 @@ jobs:
# windows-only), so one command covers the matrix with no shell driver.
# The ~/.mcpp/registry cache carries the built compat packages (xpkgs) across
# runs, so repeat builds are fast.
#
# timeout-minutes is sized for the COLD build, not the cached path. compat.opencv
# is now a from-source OpenCV 5 build, and each feature variant re-keys the
# store into a full recompile, so a full run (forced whenever this workflow file
# changes — e.g. a version bump) serially builds several OpenCV variants on one
# runner: compat.opencv base + `unifont` + `dnn` feature members, plus the
# transition-window compat.opencv5 pulled by the opencv-module member (opencv-m
# still deps compat.opencv5 until its v0.0.3). That transition double-build goes
# away once opencv-m switches to compat.opencv and compat.opencv5 is retired;
# steady state is base+unifont+dnn, and the registry cache (restore-keys prefix
# below) amortizes even those across subsequent runs. 150 covers the one-time
# cold full build with headroom; it is a ceiling, not a target.
workspace:
name: workspace (${{ matrix.platform }})
runs-on: ${{ matrix.os }}
timeout-minutes: 60
timeout-minutes: 150
strategy:
fail-fast: false
matrix:
Expand All @@ -162,21 +182,21 @@ jobs:
ext: tar.gz
mcpp: bin/mcpp
xlings: registry/bin/xlings
mcpp_version: "0.0.97" # keep in sync with env.MCPP_VERSION
mcpp_version: "0.0.98" # keep in sync with env.MCPP_VERSION
- platform: macos
os: macos-15
suffix: macosx-arm64
ext: tar.gz
mcpp: bin/mcpp
xlings: registry/bin/xlings
mcpp_version: "0.0.97" # keep in sync with env.MCPP_VERSION
mcpp_version: "0.0.98" # keep in sync with env.MCPP_VERSION
- platform: windows
os: windows-latest
suffix: windows-x86_64
ext: zip
mcpp: bin/mcpp.exe
xlings: registry/bin/xlings.exe
mcpp_version: "0.0.97" # keep in sync with env.MCPP_VERSION
mcpp_version: "0.0.98" # keep in sync with env.MCPP_VERSION
env:
MCPP_EFFECTIVE: ${{ matrix.mcpp_version }}
steps:
Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,10 @@ mcpp self config --mirror CN # 切换至国内镜像,默认使用 GLOBAL 上
| 原生模块库(Form A) | [`mcpplibs.xpkg`](pkgs/x/xpkg.lua) · [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`tensorvia-cpu`](pkgs/t/tensorvia-cpu.lua) · [`ffmpeg`](pkgs/f/ffmpeg.lua)(模块层,源码经 `compat.ffmpeg` 直编) · [`opencv`](pkgs/o/opencv.lua)(模块层,源码经 `compat.opencv5` 直编) |
| C 源码 compat(含 `features`) | [`compat.cjson`](pkgs/c/compat.cjson.lua) · [`compat.zlib`](pkgs/c/compat.zlib.lua) |
| header-only(含 `features`) | [`compat.eigen`](pkgs/c/compat.eigen.lua) |
| 外部构建系统(`install()` 从源码构建) | [`compat.openblas`](pkgs/c/compat.openblas.lua)(Make) · [`compat.opencv`](pkgs/c/compat.opencv.lua)(CMake) |
| 全源码直编(config 快照 + 源列表,零外部构建系统) | [`compat.ffmpeg`](pkgs/c/compat.ffmpeg.lua)(2281 TU 含 NASM 汇编) |
| 全源码直编 + `build.mcpp` 消费端合成 | [`compat.opencv5`](pkgs/c/compat.opencv5.lua)(457 TU,SIMD dispatch 保留,字体/内核/转发 TU 由包内 build.mcpp 生成) |
| 数据资产包(feature 依赖专用) | [`compat.opencv-unifont`](pkgs/c/compat.opencv-unifont.lua)(CJK 字体,由 `compat.opencv` 的 `unifont` feature 拉取) |
| 外部构建系统(`install()` 从源码构建) | [`compat.openblas`](pkgs/c/compat.openblas.lua)(Make) |
| 全源码直编(config 快照 + 源列表,零外部构建系统) | [`compat.ffmpeg`](pkgs/c/compat.ffmpeg.lua)(2281 TU 含 NASM 汇编,28 个目录 glob 声明) |
| 全源码直编 + `build.mcpp` 消费端合成 | [`compat.opencv`](pkgs/c/compat.opencv.lua)(OpenCV 5,458 TU 真实路径直编,SIMD dispatch 保留,字体/内核/jpeg12/16 由包内 build.mcpp 生成,依赖 `compat.ffmpeg` 提供 videoio FFmpeg 后端;feature:`unifont` CJK 字体、`dnn` 深度学习模块(+309 TU 含 protobuf/mlas);旧 install() CMake 形态已移除,`compat.opencv5` 为过渡别名待删) |
| C++23 module wrapper | [`nlohmann.json`](pkgs/n/nlohmann.json.lua) · [`marzer.tomlplusplus`](pkgs/m/marzer.tomlplusplus.lua) |

### 新增一个包
Expand Down
2 changes: 2 additions & 0 deletions mcpp.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ members = [
"tests/examples/openblas",
"tests/examples/opencv",
"tests/examples/opencv5",
"tests/examples/opencv-unifont",
"tests/examples/opencv-dnn",
"tests/examples/opencv-module",
"tests/examples/spdlog",
"tests/examples/spdlog-compiled",
Expand Down
Loading
Loading