diff --git a/.agents/docs/2026-07-20-issue-253-feature-flags-and-per-os-features-design.md b/.agents/docs/2026-07-20-issue-253-feature-flags-and-per-os-features-design.md new file mode 100644 index 00000000..80882ce7 --- /dev/null +++ b/.agents/docs/2026-07-20-issue-253-feature-flags-and-per-os-features-design.md @@ -0,0 +1,187 @@ +# Issue #253 —— per-feature `flags` + per-OS `features` 设计 + +> 日期:2026-07-20 +> 基线:mcpp **0.0.100**(HEAD;`MCPP_VERSION = "0.0.100"` @ `src/toolchain/fingerprint.cppm:21`) +> 范围:GitHub issue **#253** —— xpkg 描述符 (1) `features..flags`(feature 级 per-glob 编译 flags);(2) per-OS `features`(feature 的 sources/flags 按 OS 差异化)。阻塞用例:compat.opencv 5.0.0 的 `dnn` feature off-linux 腿(macOS NEON / windows)+ base 构建的死 glob 告警。 +> 所有 file:line 锚点均在 0.0.100 HEAD 上核实;关键行为均用 HEAD 二进制(`target/x86_64-linux-gnu/44c8eb92797aa242/bin/mcpp`)实证。 +> 架构主线沿用批次总账 [2026-07-19-issues-230-243-batch-ledger-and-architecture-assessment.md](2026-07-19-issues-230-243-batch-ledger-and-architecture-assessment.md) §5.2:**一数据模型、多文法、单一收敛漏斗;同一决策不许两处推导**。 + +--- + +## 0. 结论速览 + +| # | 问题 | 裁决 | 工作量级 | +|---|------|------|----------| +| 1 | `features..flags` 不存在(HEAD 实证:`unknown mcpp-segment key 'features.foo.flags'`) | **新增第七张 feature 表 `featureFlags`,复用既有 `GlobFlags` 类型;激活时在 `apply()` 折入 `buildConfig.globFlags` 同一漏斗**,scanner/backend/fingerprint 零改动 | 小(约 4 处,每处 <30 行) | +| 2 | per-OS `features` | **文法层已经免费生效**(per-OS 段=文本拼接后同一解析循环,HEAD 实证 `linux.features.bar` 被注册)——工作 = 把 additive-append 合并语义**测试锁定 + 文档承认**,不建任何新机制 | 极小(测试+文档) | +| 2b | `features.dnn.macosx.sources`(feature 内嵌 per-OS 键)备选 | **显式否决**:会造成第二个 OS 分发点(与 `mcpp.` 拼接并存)= "同一决策两处推导"的架构债 | — | +| 3 | (顺带发现)per-OS 拼接键 = **宿主 OS** 编译期常量,与 `target_cfg` 的 resolved-target 轴不一致 | 交叉编译场景的既有缺陷,**不阻塞 #253**(native mac/windows 腿 host==target);单独开 follow-up issue | 另行 | + +死 glob 告警的消除是 #1 的自然结果:flags 规则随 feature 激活才进场,feature off 时规则不存在,告警无从发出;feature on 而 glob 仍空 → 告警照发(此时它是真问题)。 + +### 0.1 裁决记录(2026-07-20,实现启动) + +1. **传播语义分界接受**:feature `flags` 私有、per-TU、不传播(`defines`=接口开关,`flags`=构建配方细节)。e2e 以"消费者 TU 探针宏不在场"锁死。 +2. **TOML 平价文法入本 PR**:`[features].` 表格形加 `flags` 数组-of-inline-tables,与 `[build].flags` 共享解析 helper(一数据模型两文法)。 +3. **host/target 轴缺陷(§2.3)单独开 issue**,本 PR 不动拼接键;issue 引用本文 §2.3。 +4. **实现期补强(fingerprint)**:核实发现 per-package fingerprint(`canonical_package_build_metadata`,`prepare.cppm:264-317`)只序列化 cflags/cxxflags/ldflags/genfiles,**不含 globFlags**——今天靠"descriptor 随版本冻结"+"feature 开关必然改 cflags(`-DMCPP_FEATURE_*`)"间接成立。本 PR 顺手在该函数补 globFlags 全量序列化(与根侧 `:254-260` 同款),使 featureFlags 与既有 per-glob 规则的缓存语义都密不透风。`featureOrigin` 不序列化(可由 feature 集推导,而 feature 集已入 cflags)。 + +--- + +## 1. 现状核实(HEAD = 0.0.100) + +### 1.1 feature 模型:七减一张表,单一激活漏斗 + +xpkg `features` 分支(`src/manifest/xpkg.cppm:1138-1248`)与 TOML `[features]`(`src/manifest/toml.cppm:231-257`)解析到同一组 keyed-by-featureName 的表: + +``` +implies → featuresMap (types.cppm:372) +sources → buildConfig.featureSources (types.cppm:145) +defines → buildConfig.featureDefines (types.cppm:153) +requires/provides → featureRequires/featureProvides (types.cppm:381) +deps → featureDeps (types.cppm:388) +"dep/feat" → featureForwards (types.cppm:399) +``` + +全部由 `feature_closure`(`prepare.cppm:441-459`)判定激活、`apply()`(`prepare.cppm:2870-2975`)统一消费: + +- **defines**(`:2887-2916`):`-DMCPP_FEATURE_` 私有;用户 defines 进 `privateBuild.cflags/cxxflags`(→ scanner 种子进**包内每个 TU**,`scanner.cppm:998-1003`)**并** 进 `publicUsage`(接口传播,`computeUsageRequirements` `:2054-2105`,激活后 `:3120` 重跑)。 +- **sources**(`:2917-2974`):DROP 在 `!includeDevDeps` 门内、ADD 双模式都跑(0.0.94 eigen_blas 教训,注释 `:2929-2937`)。 + +未知 feature 子键**不再静默吞掉**:记入 `m.xpkgUnknownKeys`(`xpkg.cppm:1221-1224`),由收养点 `warn_unknown_xpkg_keys`(`prepare.cppm:60-73`)带 did-you-mean 报出。HEAD 实证: + +``` +$ mcpp xpkg parse probe.lua --json # probe 带 features.foo.flags +error: probe.lua: unknown mcpp-segment key 'features.foo.flags' — silently ignored at build time by this mcpp version +``` + +即 issue 第 1 半的缺口属实,且**恰好落在架构准备好的扩展点上**。 + +### 1.2 per-glob flags:类型、匹配、告警 + +- 类型 `GlobFlags { glob; cflags; cxxflags; asmflags; defines }`(`types.cppm:125-131`),**有序** vector `buildConfig.globFlags`(`:158`),声明序=应用序(GNU last-wins);显式注释为**私有、不传播**(`types.cppm:123-124`)。 +- xpkg 解析分支 `xpkg.cppm:960-1025`(entry 缺 `glob` 硬错 `:1017-1020`,未知子键硬错 `:996-1001`)。 +- 匹配在 scanner:`apply_glob_flags`(`scanner.cppm:845-864`),brace 预展开(`:841-844`),命中计数 `globFlagHits[i]++`(`:853`),flags 落到该 unit 的 `packageCflags/packageCxxflags/packageAsmflags` → ninja/compile_commands 直出(`ninja_backend.cppm:760,843,893`;`compile_commands.cppm:117`)。 +- **死 glob 告警**在 `scanner.cppm:933-939`:`"[build].flags glob '{}' matched no source file"`,故意是 warning 不是 error(cfg 门控源集在某些 target 上合法为空,`:830-834`)——但 opencv 的场景(feature off → mlas 规则死)属于"结构性必死",告警噪声由此而来。 +- glob flags 全量序列化进 fingerprint(`prepare.cppm:254-260`,`globflags:/gc:/gxx:/gas:/gd:` 标签)。 + +### 1.3 per-OS 段:文本拼接 ⇒ `features` 已经参与(实证) + +`synthesize_from_xpkg_lua` 对 per-OS 段的机制是**把命中 OS 的表体追加到 base 体后、跑同一个解析循环**(`xpkg.cppm:812-818`);非命中 OS 段被 skip-table 跳过(`:1432-1437`)。所以**mcpp 段的每个键天然支持 per-OS additive 覆盖**——`features` 也不例外。HEAD 实证(probe.lua,linux 宿主): + +``` +mcpp.features.foo = { defines={FOO_BASE}, sources={base_foo.cpp} } +mcpp.linux.features.foo = { sources={linux_foo.cpp}, defines={FOO_LINUX} } +mcpp.linux.features.bar = { defines={BAR_LINUX_ONLY} } +→ parse OK, features 2 # bar 被注册;foo 的 per-OS 子键走同一分支 +``` + +合并语义由解析分支代码直接决定(`xpkg.cppm:1158` `try_emplace` 幂等注册 + `:1209` `push_back` 追加):**per-OS 条目对同名 feature 是逐子键 append,新 feature 名直接注册**。issue 第 2 半"features 是中性顶层键、不能 per-OS"的表述,在文法层**已经不成立**——缺的是:该行为从未被测试锁定、从未写进文档、生成器(merge_opencv.lua)不知道可以用。 + +### 1.4 opencv 用例映射 + +`compat.opencv.lua`(mcpp-index origin/main,9776 行)现状: +- `features.dnn.sources` 硬编码 x86 载荷(`*/3rdparty/mlas/lib/x86_64/*.S`、avx/avx2/avx512 生成内核)在**中性**顶层段; +- mlas 专属 defines(`BUILD_MLAS_NO_ONNXRUNTIME=1` 等)只能放 `linux.flags` 基础段,glob `**/3rdparty/mlas/**` 在 dnn-off 构建必死 → 告警。 + +--- + +## 2. 架构裁决 + +### 2.1 Part 1:`featureFlags` = 第七张表,激活时折入既有 glob-flags 漏斗 + +**数据模型**:`BuildConfig` 新增 + +```cpp +// 与 featureSources/featureDefines 并列(types.cppm:145/153 一带) +std::map> featureFlags; +``` + +复用 `GlobFlags` 原类型,**不新建 per-feature flag 类型**(延续 0.0.100 "type/rule unification" 纪律)。 + +**消费(唯一新增决策点=零)**:`apply()` 内、`!includeDevDeps` 门**外**(双路径不变量,与 featureDefines 同位置,`prepare.cppm:2887-2916` 一带),对每个激活 feature `f`(按 map 序迭代 `featureFlags`、以 `active` 过滤 → 确定性顺序): + +```cpp +for (auto& gf : featureFlags[f]) bc.globFlags.push_back(gf); +``` + +此后一切照旧:scanner 匹配、per-TU 落 flags、fingerprint 序列化、死 glob 告警——**下游三个消费者一行不改**。顺序语义:base 规则在前、feature 规则按 feature 名序在后,GNU last-wins ⇒ feature 规则可覆盖 base;同一 feature 在 base 段+per-OS 段都有 `flags` 时,per-OS 条目因拼接在后而排后(OS 特化胜出)——与既有 overlay 直觉一致。 + +**传播语义分界(必须写进文档的设计决定)**:feature `defines` 是**接口开关**(包内全 TU + Public/Interface 传播,§1.1);feature `flags` 是**构建配方细节**(私有、per-TU、不传播,继承 `types.cppm:123-124` 的既有契约)。opencv 的 mlas defines 恰好就该是后者——它们本来就不该泄给消费者。glob 允许匹配包内**任意**源(不限于该 feature 自己的 sources):既有语义免费获得"feature 激活时改 base 文件 flags"的表达力(如 `platform.cpp` 的 `-include unistd.h` 场景)。 + +**告警上下文**:`GlobFlags` 加一个 `std::string featureOrigin`(空=base 规则),仅用于把告警文本改为 +`features..flags glob '...' matched no source file`——不进 fingerprint(fingerprint 逐字段显式序列化,`prepare.cppm:254-260`,新字段不加即不影响;单测锁定"加 origin 不变 fingerprint")。 + +**文法(两文法一模型)**: +1. xpkg:feature 分支(`xpkg.cppm:1160-1226`)加 `sub == "flags"` 案:把 base `flags` 分支的 entry 解析循环(`:970-1022`)抽成共享 helper `parse_glob_flags_entries(cur, dst)`,两处调用,错误文案不变。 +2. TOML:`[features]` 表格形(`toml.cppm:231-257`)加 `flags` 数组-of-inline-table,复用 TOML 侧既有 `[build]` flags 数组解析器。#243 先例:两文法只共享一个数据模型与一个解析 helper,零重复逻辑。 +3. 词表:`features..flags` 从 unknown 变 known(§1.1 的 did-you-mean 通道自动不再报);`kKnownXpkgKeys`(`xpkg.cppm:126`)如枚举 feature 子键则同步。 + +### 2.2 Part 2:per-OS `features` —— 承认、锁定、文档化,不建新机制 + +文法已生效(§1.3),裁决为: + +1. **合并语义规范化(现状即正确,测试锁死)**:per-OS 段对 feature 是**逐子键 additive append**;per-OS 段可注册新 feature(仅该 OS 存在的 feature,如 windows-only `d3d`;在其他 OS 请求它 → 既有 unknown-feature 路径报错,行为正确,文档说明即可)。 +2. **推荐写法(进 xpkg 文档 + 生成器模板)**:common/delta 模式——feature 身份与跨平台公共载荷(defines、protobuf/dnn 通用源)放中性 `features`;平台特化载荷(mlas x86 源、NEON 源、SIMD flags)放 `mcpp..features.`。与大源包批次的 common/delta 拆分方针([2026-07-19-large-source-pkg…](2026-07-19-large-source-pkg-platform-fixes-and-buildmcpp-generation-design.md) §"common/delta 拆分")同构。 +3. **`--all-os` 校验闭环**:`mcpp xpkg parse --all-os`(`cmd_xpkg.cppm:123-153`)天然逐 OS 重解析,新键自动被覆盖;mcpp-index CI 已在接入(PR#92 线)。补一条含 per-OS features + feature flags 的 fixture 进该命令的测试。 + +**显式否决的备选**:`features.dnn.macosx.sources`(feature 内嵌 OS 键)。理由:mcpp 已有**两根平台轴**——`mcpp.` 拼接(host,parse 期)与 `target_cfg`(resolved target,prepare 期,`prepare.cppm:89-108/411-428`)。在 feature 内再开第三个 OS 分发点 = 同一"这段配置在哪个平台生效"的决策三处推导,正是批次总账定性的隐性架构债(#242 的事故形态)。per-OS 段作为唯一 owner,features 只是又一个参与键。 + +同理否决"拆成 per-OS 两个包":生态负担(版本对齐、消费者写条件依赖),且 §1.3 证明根本不需要。 + +### 2.3 顺带发现:per-OS 拼接的 host/target 轴缺陷(follow-up,不入本 PR) + +拼接键 = `mcpp::platform::xpkg_platform`,**编译期宿主常量**(`platform/common.cppm:96-105`);而 `target_cfg` 按 resolved target 求值(`prepare.cppm:92-94` 仅 native 回退 host)。交叉编译(如 0.0.100 已打通的 linux→windows MinGW)时,dep 描述符的 `windows` 段**不会**被拼接——per-OS sources/flags/deps/xpm 资产全部选错。native 构建 host==target,故 #253 的实际解锁面(macOS/windows 原生腿)不受影响。修法方向:build/prepare 收养路径(`prepare.cppm:1621/:1772`)把 resolved target OS 作为 `osOverride` 传入(参数缝隙已存在,`xpkg.cppm:812`);pm/install 资产选择是否随动需单独辨析(工具类包 host、库源码包 target)。**单独开 issue,引用本节。** + +--- + +## 3. 实现清单(单 PR 可容) + +| # | 改动 | 位置 | +|---|------|------| +| 1 | `BuildConfig::featureFlags` + `GlobFlags::featureOrigin` | `types.cppm:125-158` 一带 | +| 2 | 抽 `parse_glob_flags_entries` helper;base `flags` 分支改调用;feature 分支加 `flags` 案 | `xpkg.cppm:960-1025 / 1160-1226` | +| 3 | TOML `[features].flags` 平价文法 | `toml.cppm:231-257` | +| 4 | `apply()` 激活折入(门外、双路径) | `prepare.cppm:2887-2916` 一带 | +| 5 | 告警文案按 `featureOrigin` 区分 | `scanner.cppm:933-939` | +| 6 | 文档:xpkg 参考加 `features..flags` + per-OS features 合并语义 + common/delta 推荐写法 | docs | + +**不做(非目标)**:feature flags 的接口传播(违背 §2.1 分界);per-glob `ldflags`(不在 GlobFlags 模型,链接期无 per-TU 概念);per-OS default features;host/target 轴修复(§2.3 follow-up);`.o` 注入类能力。 + +## 4. 测试矩阵 + +**单测**(`tests/unit/test_manifest.cpp` 一带): +- xpkg/TOML 各:feature `flags` 解析(顺序、缺 glob 硬错、未知子键硬错文案复用);`features..flags` 不再入 `xpkgUnknownKeys`。 +- per-OS 合并:同名 feature 子键 append、per-OS-only feature 注册、非命中 OS 段不生效(skip-table)。 +- fingerprint:激活集变化 → globFlags 序列化变;`featureOrigin` 不进 fingerprint。 + +**e2e**(host-aware,续 `tests/e2e/` 现有末号 **145** → 146 起,入 `run_all.sh`): +- 146:dep 带 `features.X.flags`(glob 命中 X 的一个源,defines 里放探针宏,TU 内 `#ifdef` 断言)。 + - 不带 feature:构建成功且 **stderr 无 "matched no source file"**(死 glob 告警消除的回归锁); + - `--features X`:探针宏在场;**双路径**同断言跑 `mcpp build` 与 `mcpp test`(0.0.94/0.0.97 不变量); + - flags 不传播:消费者 TU 断言探针宏**不在场**(与 feature defines 传播形成对照)。 +- 147:per-OS features——descriptor 在宿主 OS 段给 feature 补充源+flags,断言激活后该源被编译、flags 生效;非宿主段内容不可见。 + +**生态复验**:compat.opencv 迁移(§5)后 `mcpp build`(dnn off)stderr 零 flags 告警;`--features dnn` linux 腿字节级等价(flags 集合不变,只是搬家)。 + +## 5. 生态闭环(mcpp-index 侧,mcpp 发版后) + +1. `merge_opencv.lua`/`gen_descriptor.py`:mlas/opencl 等 feature 专属 flag 规则从 `mcpp..flags` 迁入 `mcpp..features.dnn.flags`;`features.dnn.sources` 做 common/delta 拆分——中性段留 protobuf+dnn 通用源,x86 载荷(mlas x86_64 .S、avx* 生成内核)下沉 `linux/windows.features.dnn`,macosx 段放 NEON 载荷。 +2. macOS/windows dnn 腿快照采集(snapshot-*-opencv.yml)→ 三平台 `opencv-dnn` 收官。 +3. mcpp-index CI:`--all-os` fixture 覆盖新键(接 PR#92 线)。 + +## 5.5 实现期实录(2026-07-20,随实现动态更新) + +1. **裁决 ③ 已落地**:host/target 轴缺陷开为 [issue #254](https://github.com/mcpp-community/mcpp/issues/254)(引用本文 §2.3),本 PR 不动拼接键。 +2. **TOML AOT 封闭文法守卫(#227)拦下了 inline 拼写**:`is_array_of_tables` 对"元素全为表的数组"一视同仁,inline `flags = [{...}]` 在 `features.` 下同样触发 allowlist 拒绝。修法=allowlist 引入 `*` 单段通配(`features.*.flags`),`aot_path_matches` 逐段匹配;副产品:`[[features..flags]]` AOT 拼写与 `[[build.flags]]` 一样合法(libs/toml 两拼写建同一形状,零额外分支)。文档与错误文案同步。 +3. **e2e 146 首版踩到既有 test-mode 语义**:TOML 工程若 base `src/**` 覆盖 feature 源,`mcpp test`(feature off)因 drop 是 build-only 而**无旗标编译**该源——`#error` 探针误炸。修正=fixture 采用 opencv 真实形态(base sources 不覆盖 feature 源,`[build] sources = ["src/*.cpp"]`),四象限语义随之完全一致。这也确认了 xpkg 包(feature 源只在 featureSources)不受 drop-skip 影响,mlas 无旗标编译的担忧不成立。 +4. **e2e 147 的 find 假阳性**:feature-on/off 两次构建 fingerprint 不同、目录并存,`.o` 扫描需先 `rm -rf target`。 +5. 版本 0.0.101(`fingerprint.cppm` + `mcpp.toml` 两处),CHANGELOG 已记;e2e 定号 146/147(现有末号 145)。 +6. 单测:`aot_path_matches` 经 `Manifest.FeatureFlagsTomlTableForm/FeatureFlagsTomlErrors` 间接锁定;per-OS 合并由 `SynthesizeFromXpkgLua.PerOsFeaturesAdditiveMerge` 以 linux/macosx 双 `osOverride` 腿锁定 append 序与不可见性。 + +## 6. 风险与不变量 + +- **双路径**:折入点在 `includeDevDeps` 门外——e2e 146 双模式断言锁死(0.0.94 featureSources 事故的同型防线)。 +- **顺序确定性**:map 序 + 声明序,无迭代不确定源;fingerprint 覆盖顺序(`gc:/gxx:` 序列化天然含序)。 +- **告警语义不回退**:feature 激活而 glob 空仍告警(真死 glob 不许静默);cfg 门控合法空集的既有豁免注释(`scanner.cppm:830-834`)不动。 +- **零行为漂移**:不用新键的既有描述符,解析结果与 flags 集合逐字节不变(featureFlags 空表 → `apply()` 折入为 no-op)。 diff --git a/CHANGELOG.md b/CHANGELOG.md index db08d236..170b0247 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,23 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [0.0.101] — 2026-07-20 + +> #253:feature 模型两缺口收口——per-feature per-glob `flags` + per-OS `features` 语义锁定,解锁 compat.opencv `dnn` off-linux 腿并消除 feature-off 构建的死 glob 告警。设计见 `.agents/docs/2026-07-20-issue-253-feature-flags-and-per-os-features-design.md`。 + +### 新增 + +- **`features..flags`**(#253):feature 级 per-glob 编译旗标,与 `[build].flags` 共用同一 entry 文法(`glob` 必填 + `cflags`/`cxxflags`/`asmflags`/`defines`),xpkg 与 mcpp.toml 双文法一数据模型(共享解析 helper)。激活时折入既有 globFlags 单一漏斗(scanner 匹配/per-TU 落旗标/死 glob 告警/fingerprint 四个下游零改动),追加在 base 规则之后、feature 按名序,"last flag wins" 使 feature 规则可覆盖 base;折入点在 `includeDevDeps` 门外(0.0.94 双路径不变量)。**私有 per-TU、不传播**(与接口开关 `defines` 的语义分界)。feature off 时规则不存在 → opencv mlas 一类"结构性必死 glob"的告警自然消失;feature on 而 glob 空仍告警,且文案点名归属 feature(`features..flags glob '...' matched no source file`)。TOML 侧 `[[features..flags]]` AOT 拼写同步进 #227 封闭文法 allowlist(`features.*.flags` 通配段)。 +- **per-OS `features` 语义锁定**(#253):`mcpp.` 段的文本拼接 additive overlay 本就覆盖 `features` 键——同名 feature 逐子键 append(中性段在前、OS 段在后),per-OS 段可注册 OS-only feature;现以单测(逐 OS `osOverride`)+ e2e 锁死并写入文档,配合 `features..flags` 构成 opencv `dnn` 的 common/delta 形态(中性段放跨平台公共载荷,per-OS 段放 mlas x86 / NEON 差集与其旗标)。 + +### 修复 + +- **依赖包 per-glob flags 此前不入 per-package fingerprint**:`canonical_package_build_metadata` 只序列化 cflags/cxxflags/ldflags/genfiles,globFlags 靠"descriptor 随版本冻结"间接成立;feature 折入使该向量随构建变化,现与根侧同款全量有序序列化(`globflags:/gc:/gxx:/gas:/gd:`)。 + +### 备注 + +- e2e 新增 146(feature flags 四象限 + 死 glob 告警消除/点名 + 私有不传播对照)/147(per-OS features 端到端,宿主段生效、非宿主段投毒不可见);单测新增 xpkg/TOML 解析与 per-OS 合并锁定。host/target 轴缺陷(per-OS 拼接键=宿主常量,交叉编译选段错误)另开 issue 跟踪,不入本版。 + ## [0.0.100] — 2026-07-19 > 大型源码直编包(ffmpeg/opencv 级,数千 TU)全平台化批次:#247/#248/#249 平台三修 + 增量构建修复 + build.mcpp 指令面补全(P1)。设计见 `.agents/docs/2026-07-19-large-source-pkg-platform-fixes-and-buildmcpp-generation-design.md`。 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index 566cbc91..add50f48 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -413,8 +413,8 @@ feature carry package-owned preprocessor `defines`, feature-gated source globs (`sources`, mcpp 0.0.95+ — the globs leave the default build and compile only when the feature is active, exactly like an index descriptor's `features..sources`; the highest-frequency shape for vendored libraries: *feature = a source set + a -define*), and/or capability `requires` / `provides` (see §2.8.1) alongside its -implied features: +define*), feature-gated per-glob compile flags (`flags`, mcpp 0.0.101+), and/or +capability `requires` / `provides` (see §2.8.1) alongside its implied features: ```toml [features] @@ -426,16 +426,31 @@ extra = [] mpl2only = { defines = ["EIGEN_MPL2_ONLY"] } # Table form: a define + an implied feature. fast_math = { defines = ["APP_FAST=1"], implies = ["extra"] } +# Table form: feature-gated sources + per-glob flags that co-locate with them. +simd = { sources = ["src/simd/**"], flags = [ + { glob = "src/simd/**/*.avx2.cpp", cxxflags = ["-mavx2"] } ] } ``` - `defines` are **bare** macro names (no `-D`); each desugars to `-D` on the package's own compile when the feature is active — exactly like `[targets.*] defines`. They are restricted by convention to the package's **own** namespaced - macros: a feature does **not** inject free-form `cflags`/`ldflags`, which would - break the additive feature-union model. Link flags come from a provider - dependency (§2.8.1), not from a feature. + macros: a feature does **not** inject free-form package-wide `cflags`/`ldflags`, + which would break the additive feature-union model. Link flags come from a + provider dependency (§2.8.1), not from a feature. - The automatic `-DMCPP_FEATURE_` is still defined for every active feature, so `defines` are additive to it. +- `flags` (mcpp 0.0.101+) is the same ordered array-of-inline-tables grammar as + `[build].flags` (§2.3: `glob` required, plus `cflags`/`cxxflags`/`asmflags`/ + `defines`; the `[[features..flags]]` array-of-tables spelling is accepted + too, like `[[build.flags]]`). When the feature is active the entries + are appended **after** the base `[build].flags`, features in name order, so a + feature rule wins over a broader base rule via "last flag wins"; when it is + inactive the entries do not exist (no dead-glob warning). This is how a + feature's group-specific flags co-locate with its `sources` instead of living + as base rules whose globs go dead on feature-off builds. Unlike `defines`, + feature `flags` are **private per-TU build flags** — they never propagate to + consumers (same contract as `[build].flags`), so they stay inside the additive + model: scoped by glob, deterministic order, no cross-package effect. ### 2.8.1 `provides` / `requires` — Capabilities (backend selection) diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index f900bbef..f7e0f06c 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -376,8 +376,8 @@ extra = [] 之外,携带包自有的预处理 `defines`、feature 门控的源 glob(`sources`,mcpp 0.0.95+——列出的 glob 离开默认构建,仅当 feature 激活时才编译,与 index 描述符的 `features..sources` 完全对等;这正是 vendored 大库最高频的形态:*feature = -一组源文件 + 一个 define*),以及 capability 的 `requires` / `provides` -(见 §2.8.1): +一组源文件 + 一个 define*)、feature 门控的 per-glob 编译旗标(`flags`,mcpp +0.0.101+),以及 capability 的 `requires` / `provides`(见 §2.8.1): ```toml [features] @@ -389,13 +389,25 @@ extra = [] mpl2only = { defines = ["EIGEN_MPL2_ONLY"] } # 表形式:宏 + 一个隐含 feature。 fast_math = { defines = ["APP_FAST=1"], implies = ["extra"] } +# 表形式:feature 门控源 + 与其同居的 per-glob 旗标。 +simd = { sources = ["src/simd/**"], flags = [ + { glob = "src/simd/**/*.avx2.cpp", cxxflags = ["-mavx2"] } ] } ``` - `defines` 为**裸**宏名(不带 `-D`);feature 激活时每个脱糖为 `-D`,加到该包 自己的编译上——与 `[targets.*] defines` 完全一致。按约定仅限包**自有**的带命名 - 空间宏:feature **不**注入自由的 `cflags`/`ldflags`,否则会破坏加性的 feature + 空间宏:feature **不**注入自由的包级 `cflags`/`ldflags`,否则会破坏加性的 feature 并集模型。链接旗标来自 provider 依赖(§2.8.1),而非 feature。 - 每个激活的 feature 仍会得到自动的 `-DMCPP_FEATURE_`,`defines` 与之叠加。 +- `flags`(mcpp 0.0.101+)与 `[build].flags`(§2.3)共用同一有序 inline-table 数组 + 文法(`glob` 必填,加 `cflags`/`cxxflags`/`asmflags`/`defines`;与 + `[[build.flags]]` 一样也接受 `[[features..flags]]` 拼写)。feature 激活时 + 条目追加在 base `[build].flags` **之后**(feature 按名 + 序),"last flag wins" 使 feature 规则可覆盖更宽的 base 规则;未激活时条目根本 + 不存在(不会有死 glob 告警)。这让 feature 的组内专属旗标与其 `sources` 同居, + 而不必写成 base 规则、在 feature-off 构建里留下必死的 glob。与 `defines` 不同, + feature `flags` 是**私有 per-TU 构建旗标**——永不传播给消费者(与 `[build].flags` + 同契约),因此不破坏加性模型:glob 限定作用面、顺序确定、无跨包效应。 ### 2.8.1 `provides` / `requires` —— 能力(后端选择) diff --git a/mcpp.toml b/mcpp.toml index fbb21ff8..2f3376f8 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "0.0.100" +version = "0.0.101" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index e70cebdc..70450fab 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -288,6 +288,20 @@ std::string canonical_package_build_metadata( s += " ldflag:"; s += flag; } + // Per-glob flags — same full ordered serialization as the root-side + // block above. Until #253 dependency globFlags were unfingerprinted + // (held only by "descriptor frozen per version" + "feature toggles + // always change cflags via -DMCPP_FEATURE_*"); feature-folded entries + // make the vector build-variant, so fingerprint it directly. + // featureOrigin is diagnostic-only and deliberately NOT serialized + // (the active feature set is already in cflags above). + for (auto const& gf : pkg.manifest.buildConfig.globFlags) { + s += " globflags:"; s += gf.glob; + for (auto const& f : gf.cflags) { s += " gc:"; s += f; } + for (auto const& f : gf.cxxflags) { s += " gxx:"; s += f; } + for (auto const& f : gf.asmflags) { s += " gas:"; s += f; } + for (auto const& f : gf.defines) { s += " gd:"; s += f; } + } if (pkg.usageResolved) { for (auto const& dir : pkg.privateBuild.includeDirs) { s += " private_include:"; @@ -2972,6 +2986,26 @@ prepare_build(bool print_fingerprint, } } } + // #253: per-feature per-glob flags — fold each ACTIVE feature's + // entries into the base globFlags funnel. Everything downstream + // (scanner glob match, per-TU flag landing, zero-hit warning, + // fingerprint serialization) consumes the ONE vector unchanged. + // Appended AFTER base entries, features in map (= name) order, so + // application order is deterministic and a feature rule wins over + // a broader base rule via "last flag wins". An inactive feature + // contributes nothing — its dead globs no longer exist to warn + // about. Deliberately OUTSIDE any includeDevDeps gate: like the + // sources ADD above, `mcpp build` and `mcpp test` must agree + // (0.0.94 dual-path invariant). featureOrigin tags the entry so + // the scanner's zero-hit warning can name the owning feature. + for (auto& [f, entries] : bc.featureFlags) { + if (std::ranges::find(active, f) == active.end()) continue; + for (auto const& gf : entries) { + auto tagged = gf; + tagged.featureOrigin = f; + bc.globFlags.push_back(std::move(tagged)); + } + } }; if (!packages.empty()) { auto rootReq = parse_feature_request(overrides.features); diff --git a/src/manifest/toml.cppm b/src/manifest/toml.cppm index 2d35269c..2fb583c1 100644 --- a/src/manifest/toml.cppm +++ b/src/manifest/toml.cppm @@ -52,6 +52,71 @@ bool is_array_of_tables(const t::Value& v) { return true; } +// #253: shared parser for the per-glob flags array shape +// `[{ glob = "...", cflags/cxxflags/asmflags/defines = [...] }, ...]` — +// one entry grammar for `[build].flags` and `[features]..flags`. +// `ctxLabel` names the anchoring key in error messages. Entries append to +// `dst` in declaration order (order is the override semantics). Returns an +// error message, or nullopt on success. +std::optional parse_glob_flags_value( + const t::Value& fv, std::string_view ctxLabel, std::vector& dst) +{ + if (!fv.is_array()) { + return std::format( + "{} must be an array of inline tables " + "(flags = [{{ glob = \"...\", cxxflags = [...] }}, ...])", ctxLabel); + } + for (auto& ev : fv.as_array()) { + if (!ev.is_table()) { + return std::format( + "{} entries must be inline tables with a `glob` key", ctxLabel); + } + auto& et = ev.as_table(); + GlobFlags gf; + for (auto& [k, v] : et) { + auto read_list = [&](std::vector& out) -> bool { + if (!v.is_array()) return false; + for (auto& s : v.as_array()) + if (s.is_string()) out.push_back(s.as_string()); + return true; + }; + bool ok = false; + if (k == "glob") { ok = v.is_string(); if (ok) gf.glob = v.as_string(); } + else if (k == "cflags") ok = read_list(gf.cflags); + else if (k == "cxxflags") ok = read_list(gf.cxxflags); + else if (k == "asmflags") ok = read_list(gf.asmflags); + else if (k == "defines") ok = read_list(gf.defines); + if (!ok) { + return std::format( + "{}: invalid key '{}' (expected glob = \"...\" " + "plus cflags/cxxflags/asmflags/defines arrays)", ctxLabel, k); + } + } + if (gf.glob.empty()) { + return std::format("{} entry is missing its `glob` key", ctxLabel); + } + dst.push_back(std::move(gf)); + } + return std::nullopt; +} + +// Allowlist entries are dotted paths whose segments may be the wildcard `*`, +// matching exactly one path segment — needed for #253's `features..flags`, +// whose middle segment (the feature name) is author-chosen. +bool aot_path_matches(std::string_view pattern, std::string_view path) { + while (true) { + auto pDot = pattern.find('.'); + auto sDot = path.find('.'); + auto pSeg = pattern.substr(0, pDot); + auto sSeg = path.substr(0, sDot); + if (pSeg != "*" && pSeg != sSeg) return false; + if (pDot == std::string_view::npos || sDot == std::string_view::npos) + return pDot == std::string_view::npos && sDot == std::string_view::npos; + pattern.remove_prefix(pDot + 1); + path.remove_prefix(sDot + 1); + } +} + std::optional find_disallowed_array_of_tables( const t::Table& tbl, const std::string& prefix, std::span allowlist) @@ -60,7 +125,7 @@ std::optional find_disallowed_array_of_tables( std::string path = prefix.empty() ? k : std::format("{}.{}", prefix, k); if (is_array_of_tables(v)) { bool allowed = false; - for (auto a : allowlist) if (a == path) { allowed = true; break; } + for (auto a : allowlist) if (aot_path_matches(a, path)) { allowed = true; break; } if (!allowed) return path; } else if (v.is_table()) { if (auto found = find_disallowed_array_of_tables(v.as_table(), path, allowlist)) @@ -82,11 +147,15 @@ std::expected parse_string(std::string_view content, // Closed-grammar guard: reject any array-of-tables whose dotted path // isn't explicitly allowlisted, BEFORE any section is read. See // find_disallowed_array_of_tables above. - static constexpr std::string_view kAllowedArraysOfTables[] = { "build.flags" }; + static constexpr std::string_view kAllowedArraysOfTables[] = { + "build.flags", + "features.*.flags", // #253 — the middle segment is the feature name + }; if (auto badPath = find_disallowed_array_of_tables(doc->root(), "", kAllowedArraysOfTables)) { return std::unexpected(error(origin, std::format( "[[{}]] (array-of-tables) is not allowed for section '{}'; " - "array-of-tables syntax is only supported for [[build.flags]]", + "array-of-tables syntax is only supported for [[build.flags]] " + "and [[features..flags]]", *badPath, *badPath))); } @@ -241,6 +310,21 @@ std::expected parse_string(std::string_view content, read_str_array(ft, "provides", provs); if (!reqs.empty()) m.featureRequires[fname] = std::move(reqs); if (!provs.empty()) m.featureProvides[fname] = std::move(provs); + // #253: per-feature per-glob compile flags — same entry grammar + // as [build].flags (shared parse_glob_flags_value), gated by + // this feature and folded in AFTER base globFlags at activation + // so feature rules win via "last flag wins". Both spellings + // reach here: the inline array and [[features.X.flags]] AOT + // (allowlisted via the features.*.flags pattern, mirroring + // #227's build.flags decision — libs/toml builds one shape). + if (auto it = ft.find(std::string("flags")); it != ft.end()) { + if (auto err = parse_glob_flags_value( + it->second, + std::format("[features].{}.flags", fname), + m.buildConfig.featureFlags[fname])) { + return std::unexpected(error(origin, *err)); + } + } } // #243: split `dep/feat` tokens out of `implies` into featureForwards // (raw depKey shares the `dependencies`/`featureDeps` keyspace); the @@ -782,37 +866,9 @@ std::expected parse_string(std::string_view content, "(flags = [{ glob = \"...\", cxxflags = [...] }, ...]) " "or an array of tables ([[build.flags]] glob = \"...\")")); } - for (auto& ev : fv->as_array()) { - if (!ev.is_table()) { - return std::unexpected(error(origin, - "[build].flags entries must be inline tables with a `glob` key")); - } - auto& et = ev.as_table(); - GlobFlags gf; - for (auto& [k, v] : et) { - auto read_list = [&](std::vector& out) -> bool { - if (!v.is_array()) return false; - for (auto& s : v.as_array()) - if (s.is_string()) out.push_back(s.as_string()); - return true; - }; - bool ok = false; - if (k == "glob") { ok = v.is_string(); if (ok) gf.glob = v.as_string(); } - else if (k == "cflags") ok = read_list(gf.cflags); - else if (k == "cxxflags") ok = read_list(gf.cxxflags); - else if (k == "asmflags") ok = read_list(gf.asmflags); - else if (k == "defines") ok = read_list(gf.defines); - if (!ok) { - return std::unexpected(error(origin, std::format( - "[build].flags: invalid key '{}' (expected glob = \"...\" " - "plus cflags/cxxflags/asmflags/defines arrays)", k))); - } - } - if (gf.glob.empty()) { - return std::unexpected(error(origin, - "[build].flags entry is missing its `glob` key")); - } - m.buildConfig.globFlags.push_back(std::move(gf)); + if (auto err = parse_glob_flags_value( + *fv, "[build].flags", m.buildConfig.globFlags)) { + return std::unexpected(error(origin, *err)); } } if (auto v = doc->get_string("build.c_standard")) m.buildConfig.cStandard = *v; diff --git a/src/manifest/types.cppm b/src/manifest/types.cppm index b2da3686..4d912fac 100644 --- a/src/manifest/types.cppm +++ b/src/manifest/types.cppm @@ -128,6 +128,12 @@ struct GlobFlags { std::vector cxxflags; // C++ units (.cpp/.cc/.cxx/.cppm) std::vector asmflags; // assembly units (.S/.s via cc, .asm via nasm) std::vector defines; // desugars to -D on every matched unit kind + // #253: non-empty when this entry came from `features..flags` and was + // folded in at feature activation (prepare_build). Diagnostic context only + // (names the owning feature in the zero-hit warning); deliberately NOT part + // of the fingerprint — the active feature set is already fingerprinted via + // the -DMCPP_FEATURE_* cflags. + std::string featureOrigin; }; // `[build]` section — tunables for the build backend. @@ -151,6 +157,15 @@ struct BuildConfig { // would break feature-union composition. See // .agents/docs/2026-06-29-feature-capability-model-design.md. std::map> featureDefines; + // #253: feature name → per-glob compile flags gated by that feature. Same + // ordered GlobFlags model as `globFlags` below; when the feature is active + // the entries are appended AFTER the base globFlags (prepare_build), so a + // feature rule wins over a broader base rule via "last flag wins". Lets a + // feature's group-specific flags co-locate with its sources (e.g. opencv + // dnn's mlas defines) instead of living as base rules whose globs go dead + // on feature-off builds. Private per-TU flags — never propagate (contrast + // featureDefines above, which are interface switches). + std::map> featureFlags; std::vector includeDirs; // relative to package root // #249: emitted as -idirafter (searched after system dirs) std::vector includeDirsAfter; diff --git a/src/manifest/xpkg.cppm b/src/manifest/xpkg.cppm index 4081878d..a5ab5665 100644 --- a/src/manifest/xpkg.cppm +++ b/src/manifest/xpkg.cppm @@ -793,6 +793,82 @@ list_xpkg_versions(std::string_view luaContent, std::string_view platform) { return versions; } +// Parses the `{ { glob = "...", cflags/cxxflags/asmflags/defines = {...} }, +// ... }` array-of-tables shape shared by `[build]`-level `flags` and #253's +// `features..flags` — one entry grammar, two anchoring keys. `ctxLabel` +// names the anchoring key in error messages ("flags" or "features.X.flags"). +// The cursor must sit on the opening '{' of the array; on success it is +// consumed past the closing '}'. Parsed entries are appended to `dst`. +static std::optional +parse_glob_flags_array(LuaCursor& cur, const std::filesystem::path& sourcePath, + std::string_view ctxLabel, std::vector& dst) +{ + if (!cur.consume('{')) { + return ManifestError{ + std::format("expected '{{' after `{} =`", ctxLabel), sourcePath, 0, 0}; + } + cur.skip_ws_and_comments(); + while (!cur.eof() && cur.peek() != '}') { + if (!cur.consume('{')) { + return ManifestError{ + std::format("expected '{{' opening a `{}` entry", ctxLabel), + sourcePath, 0, 0}; + } + GlobFlags gf; + cur.skip_ws_and_comments(); + while (!cur.eof() && cur.peek() != '}') { + auto sub = cur.read_key(); + if (sub.empty()) break; + if (!cur.consume('=')) { + return ManifestError{ + std::format("expected '=' in `{}` entry", ctxLabel), + sourcePath, 0, 0}; + } + cur.skip_ws_and_comments(); + if (sub == "glob") { + gf.glob = cur.read_string(); + cur.skip_ws_and_comments(); + continue; + } + std::vector* dstList = + sub == "cflags" ? &gf.cflags + : sub == "cxxflags" ? &gf.cxxflags + : sub == "asmflags" ? &gf.asmflags + : sub == "defines" ? &gf.defines + : nullptr; + if (!dstList) { + return ManifestError{ + std::format("unknown {} key '{}' (expected glob/" + "cflags/cxxflags/asmflags/defines)", ctxLabel, sub), + sourcePath, 0, 0}; + } + if (!cur.consume('{')) { + return ManifestError{ + std::format("expected '{{' after {} entry key '{}'", ctxLabel, sub), + sourcePath, 0, 0}; + } + cur.skip_ws_and_comments(); + while (!cur.eof() && cur.peek() != '}') { + auto s = cur.read_string(); + if (!s.empty()) dstList->push_back(std::move(s)); + cur.skip_ws_and_comments(); + } + cur.consume('}'); + cur.skip_ws_and_comments(); + } + cur.consume('}'); + if (gf.glob.empty()) { + return ManifestError{ + std::format("`{}` entry is missing its `glob` key", ctxLabel), + sourcePath, 0, 0}; + } + dst.push_back(std::move(gf)); + cur.skip_ws_and_comments(); + } + cur.consume('}'); + return std::nullopt; +} + std::expected synthesize_from_xpkg_lua(std::string_view luaContent, std::string_view packageName, @@ -962,66 +1038,11 @@ synthesize_from_xpkg_lua(std::string_view luaContent, // asmflags = {...}, defines = {...} }, ... }` — per-glob // compile flags (G4), same ordered data model as mcpp.toml's // [build] flags array. Lua array-of-tables preserves order. - if (!cur.consume('{')) { - return std::unexpected(ManifestError{ - "expected '{' after `flags =`", m.sourcePath, 0, 0}); - } - cur.skip_ws_and_comments(); - while (!cur.eof() && cur.peek() != '}') { - if (!cur.consume('{')) { - return std::unexpected(ManifestError{ - "expected '{' opening a `flags` entry", m.sourcePath, 0, 0}); - } - GlobFlags gf; - cur.skip_ws_and_comments(); - while (!cur.eof() && cur.peek() != '}') { - auto sub = cur.read_key(); - if (sub.empty()) break; - if (!cur.consume('=')) { - return std::unexpected(ManifestError{ - "expected '=' in `flags` entry", m.sourcePath, 0, 0}); - } - cur.skip_ws_and_comments(); - if (sub == "glob") { - gf.glob = cur.read_string(); - cur.skip_ws_and_comments(); - continue; - } - std::vector* dst = - sub == "cflags" ? &gf.cflags - : sub == "cxxflags" ? &gf.cxxflags - : sub == "asmflags" ? &gf.asmflags - : sub == "defines" ? &gf.defines - : nullptr; - if (!dst) { - return std::unexpected(ManifestError{ - std::format("unknown flags key '{}' (expected glob/" - "cflags/cxxflags/asmflags/defines)", sub), - m.sourcePath, 0, 0}); - } - if (!cur.consume('{')) { - return std::unexpected(ManifestError{ - std::format("expected '{{' after flags entry key '{}'", sub), - m.sourcePath, 0, 0}); - } - cur.skip_ws_and_comments(); - while (!cur.eof() && cur.peek() != '}') { - auto s = cur.read_string(); - if (!s.empty()) dst->push_back(std::move(s)); - cur.skip_ws_and_comments(); - } - cur.consume('}'); - cur.skip_ws_and_comments(); - } - cur.consume('}'); - if (gf.glob.empty()) { - return std::unexpected(ManifestError{ - "`flags` entry is missing its `glob` key", m.sourcePath, 0, 0}); - } - m.buildConfig.globFlags.push_back(std::move(gf)); - cur.skip_ws_and_comments(); + // Entry grammar shared with features..flags (#253). + if (auto err = parse_glob_flags_array( + cur, m.sourcePath, "flags", m.buildConfig.globFlags)) { + return std::unexpected(std::move(*err)); } - cur.consume('}'); } else if (key == "target_cfg") { // `{ ["cfg(...)"] = { cflags = {...}, cxxflags = {...}, @@ -1189,6 +1210,19 @@ synthesize_from_xpkg_lua(std::string_view luaContent, cur.skip_ws_and_comments(); } cur.consume('}'); + } else if (sub == "flags" && cur.peek() == '{') { + // #253: per-feature per-glob compile flags — the same + // entry grammar as the build-level `flags` key, gated + // by this feature. Folded into buildConfig.globFlags + // at activation (prepare_build), AFTER base entries, + // so feature rules win via "last flag wins". Private, + // per-TU, non-propagating (contrast `defines`). + if (auto err = parse_glob_flags_array( + cur, m.sourcePath, + std::format("features.{}.flags", fname), + m.buildConfig.featureFlags[fname])) { + return std::unexpected(std::move(*err)); + } } else { // Feature subfields that carry a string array. `sources` // gates source globs; `defines` carries package-owned macros diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 826050cc..11473d7b 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -932,9 +932,18 @@ void scan_one_into(ScanResult& result, } for (std::size_t i = 0; i < globFlagHits.size(); ++i) { if (globFlagHits[i] == 0) { - result.warnings.push_back(ScanError{root, 0, std::format( - "[build].flags glob '{}' matched no source file", - manifest.buildConfig.globFlags[i].glob)}); + // #253: a feature-folded entry only exists when its feature is + // active, so a zero hit here is a REAL dead glob either way — + // name the owning feature so the author knows which table to fix. + auto const& gf = manifest.buildConfig.globFlags[i]; + result.warnings.push_back(ScanError{root, 0, + gf.featureOrigin.empty() + ? std::format( + "[build].flags glob '{}' matched no source file", + gf.glob) + : std::format( + "features.{}.flags glob '{}' matched no source file", + gf.featureOrigin, gf.glob)}); } } } diff --git a/src/toolchain/fingerprint.cppm b/src/toolchain/fingerprint.cppm index 1204bc93..457f8654 100644 --- a/src/toolchain/fingerprint.cppm +++ b/src/toolchain/fingerprint.cppm @@ -18,7 +18,7 @@ import mcpp.toolchain.detect; export namespace mcpp::toolchain { -inline constexpr std::string_view MCPP_VERSION = "0.0.100"; +inline constexpr std::string_view MCPP_VERSION = "0.0.101"; struct FingerprintInputs { Toolchain toolchain; diff --git a/tests/e2e/146_feature_flags.sh b/tests/e2e/146_feature_flags.sh new file mode 100755 index 00000000..fcf86cf5 --- /dev/null +++ b/tests/e2e/146_feature_flags.sh @@ -0,0 +1,153 @@ +#!/usr/bin/env bash +# requires: gcc +# #253: [features]..flags — per-feature per-glob compile flags. +# Quadrants: feature off/on × build/test (0.0.94 dual-path invariant), plus: +# - feature OFF: the rule does not exist → NO dead-glob warning (the issue's +# opencv mlas noise), and the define is absent; +# - feature ON: the define reaches EXACTLY the glob-matched TU (not the +# package's other TUs — contrast feature `defines`, which are package-wide); +# - feature ON with a dead glob: warning fires, naming the owning feature; +# - flags do NOT propagate to a consumer (contrast feature `defines`, which +# interface-propagate) — asserted in the dep half below. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +cd "$TMP" +"$MCPP" new featflags > /dev/null +cd featflags + +mkdir -p src/simd +cat > src/simd/kernel.cpp <<'EOF' +#ifndef KERNEL_TAG +#error "feature flags did not reach the glob-matched TU" +#endif +extern "C" int kernel_tag() { return KERNEL_TAG; } +EOF + +cat > src/plain.cpp <<'EOF' +#ifdef KERNEL_TAG +#error "feature flags leaked outside their glob (must be per-TU, not package-wide)" +#endif +extern "C" int plain_ok() { return 1; } +EOF + +cat > src/main.cpp <<'EOF' +import std; +extern "C" int plain_ok(); +#ifdef MCPP_FEATURE_SIMD +extern "C" int kernel_tag(); +#endif +int main() { +#ifdef MCPP_FEATURE_SIMD + std::println("tag = {}", kernel_tag() + plain_ok()); +#else + std::println("tag = none"); +#endif + return 0; +} +EOF + +# Base sources deliberately do NOT cover src/simd/** (the opencv shape: a +# feature's sources live only under the feature) — so the gated TU is absent +# in every feature-off quadrant, including `mcpp test`, whose drop-skip only +# applies to gated globs that base sources also cover. +cat > mcpp.toml <<'EOF' +[package] +name = "featflags" +version = "0.1.0" + +[build] +sources = ["src/*.cpp"] + +[features] +default = [] +simd = { sources = ["src/simd/**"], flags = [ + { glob = "src/simd/**", defines = ["KERNEL_TAG=6"] } ] } +dead = { flags = [ { glob = "src/never/**", defines = ["NOPE"] } ] } +EOF + +# Quadrant 1: feature OFF, build — rule doesn't exist, so no dead-glob warning. +"$MCPP" build > build_off.log 2>&1 || { cat build_off.log; echo "build (off) failed"; exit 1; } +grep -q "matched no source file" build_off.log && { + cat build_off.log; echo "feature-off build must not warn about feature flag globs"; exit 1; } || true +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == "tag = none" ]] || { echo "unexpected (off): $out"; exit 1; } + +# Quadrant 2: feature ON, build — define lands on the matched TU only. +"$MCPP" build --features simd > build_on.log 2>&1 || { cat build_on.log; echo "build (on) failed"; exit 1; } +bin="$(ls -t $(find target -name 'featflags' -type f) | head -1)" +[[ -n "$bin" ]] || { echo "binary not found"; exit 1; } +out="$("./$bin" | tail -1)" +[[ "$out" == "tag = 7" ]] || { echo "unexpected (on): $out (want 6+1)"; exit 1; } + +# Active feature with a glob that matches nothing → warning naming the feature. +"$MCPP" build --features simd,dead > build_dead.log 2>&1 || { + cat build_dead.log; echo "build (dead) failed"; exit 1; } +grep -q "features.dead.flags glob 'src/never/\*\*' matched no source file" build_dead.log || { + cat build_dead.log; echo "missing feature-scoped dead-glob warning"; exit 1; } + +# Quadrants 3+4: test path — flags behavior must match build (dual-path). +mkdir -p tests +cat > tests/test_featflags.cpp <<'EOF' +#ifdef MCPP_FEATURE_SIMD +extern "C" int kernel_tag(); +int main() { return kernel_tag() == 6 ? 0 : 1; } +#else +int main() { return 0; } +#endif +EOF +"$MCPP" test > test_off.log 2>&1 || { cat test_off.log; echo "mcpp test (off) failed"; exit 1; } +grep -q "matched no source file" test_off.log && { + cat test_off.log; echo "feature-off test must not warn about feature flag globs"; exit 1; } || true +"$MCPP" test --features simd > test_on.log 2>&1 || { cat test_on.log; echo "mcpp test (on) failed"; exit 1; } + +# --- Dep half: feature flags are PRIVATE to the owning package. --- +# The dep's feature carries both an interface define (propagates) and a +# per-glob flag define (must NOT propagate to the consumer's TUs). +cd "$TMP" +mkdir -p widget/src app/src +cat > widget/mcpp.toml <<'EOF' +[package] +name = "widget" +version = "0.1.0" + +[features] +default = [] +turbo = { defines = ["WIDGET_IFACE=1"], flags = [ + { glob = "src/**", defines = ["WIDGET_PRIVATE=1"] } ] } + +[targets.widget] +kind = "lib" +EOF +cat > widget/src/widget.cppm <<'EOF' +export module widget; +#ifndef WIDGET_PRIVATE +#error "feature flags did not reach the dep's own matched TU" +#endif +export int widget_anchor() { return WIDGET_PRIVATE; } +EOF +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +widget = { path = "../widget", features = ["turbo"] } +EOF +cat > app/src/main.cpp <<'EOF' +import widget; +#ifndef WIDGET_IFACE +#error "feature defines must interface-propagate to the consumer" +#endif +#ifdef WIDGET_PRIVATE +#error "feature flags must NOT propagate to the consumer" +#endif +int main() { return widget_anchor() == 1 ? 0 : 1; } +EOF +cd app +"$MCPP" build > build_dep.log 2>&1 || { cat build_dep.log; echo "dep build failed"; exit 1; } +"$MCPP" run > /dev/null 2>&1 || { echo "dep run failed"; exit 1; } + +echo "OK" diff --git a/tests/e2e/147_per_os_features_xpkg.sh b/tests/e2e/147_per_os_features_xpkg.sh new file mode 100755 index 00000000..4c52978a --- /dev/null +++ b/tests/e2e/147_per_os_features_xpkg.sh @@ -0,0 +1,154 @@ +#!/usr/bin/env bash +# requires: gcc fresh-sandbox +# #253: per-OS `features` in xpkg descriptors — the `mcpp.` additive +# overlay applies to `features` like every other mcpp-segment key: a same-named +# feature merges per-subkey by APPEND (neutral body first, host-OS body after), +# and the non-host OS section stays invisible. This is the common/delta shape +# compat.opencv's `dnn` uses (neutral common payload + per-OS kernels/flags). +# Locked end-to-end here; per-leg merge semantics are unit-tested via +# osOverride (test_manifest.cpp PerOsFeaturesAdditiveMerge). +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +export MCPP_HOME="$TMP/mcpp-home" +source "$(dirname "$0")/_inherit_toolchain.sh" + +# Host OS key in xpkg vocabulary; the "other" OS carries a poison section that +# must never leak into this build. +case "$(uname -s)" in + Darwin) OSKEY=macosx; OTHER=linux ;; + MINGW*|MSYS*|CYGWIN*) OSKEY=windows; OTHER=linux ;; + *) OSKEY=linux; OTHER=macosx ;; +esac + +INDEX_DIR="$TMP/local-index" +mkdir -p "$INDEX_DIR/pkgs/o" +cat > "$INDEX_DIR/pkgs/o/osfeat.lua" < "$PKGSRC/base.c" <<'EOF' +int osfeat_base(void) { return 1; } +EOF +cat > "$PKGSRC/accel_common.c" <<'EOF' +#ifndef OSFEAT_ACCEL +#error "neutral feature define missing" +#endif +int osfeat_common(void) { return 2; } +EOF +cat > "$PKGSRC/accel_os.c" <<'EOF' +/* Compiled only via the host-OS features overlay; its per-OS feature flag + * must land here (and only here). */ +#ifndef OSFEAT_OS_KERNEL +#error "per-OS feature flags did not reach the per-OS feature source" +#endif +int osfeat_os_kernel(void) { return OSFEAT_OS_KERNEL; } +EOF +# Poison: if the OTHER-OS section leaks, this file compiles and breaks the build. +cat > "$PKGSRC/other_os_only.c" <<'EOF' +#error "non-host OS features section must be invisible" +EOF + +cd "$TMP/project/app" +cat > src/main.cpp <<'EOF' +extern "C" int osfeat_base(void); +extern "C" int osfeat_common(void); +extern "C" int osfeat_os_kernel(void); +int main() { + // 1 + 2 + 5 == 8: base + neutral feature payload + per-OS feature delta. + return (osfeat_base() + osfeat_common() + osfeat_os_kernel()) == 8 ? 0 : 1; +} +EOF + +cat > mcpp.toml < build.log 2>&1 || { cat build.log; echo "build failed"; exit 1; } +grep -q "matched no source file" build.log && { + cat build.log; echo "per-OS feature flags glob must match its per-OS source"; exit 1; } || true +"$MCPP" run > /dev/null 2>&1 || { echo "run failed (payload sum wrong or link failed)"; exit 1; } + +# Feature OFF: every accel source (neutral + per-OS) must stay out. +cat > src/main.cpp <<'EOF' +extern "C" int osfeat_base(void); +int main() { return osfeat_base() == 1 ? 0 : 1; } +EOF +cat > mcpp.toml < build_off.log 2>&1 || { cat build_off.log; echo "build (off) failed"; exit 1; } +find . -name 'accel_*.o' | grep -q . && { echo "accel sources compiled with feature off"; exit 1; } || true +"$MCPP" run > /dev/null 2>&1 || { echo "run (off) failed"; exit 1; } + +echo "OK" diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index ce2a1136..971d6c40 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -474,6 +474,206 @@ package = { ASSERT_TRUE(m->buildConfig.featureSources.contains("eigen_blas")); } +// #253: a [features] table-form entry may carry `flags` — per-feature per-glob +// compile flags, the same entry grammar as [build].flags. Parsed into +// buildConfig.featureFlags keyed by feature; base globFlags stay untouched at +// parse time (activation folds them in later, in prepare_build). +TEST(Manifest, FeatureFlagsTomlTableForm) { + constexpr auto src = R"( +[package] +name = "x" +version = "0.1.0" +[targets.x] +kind = "lib" +[build] +flags = [{ glob = "src/base/**", defines = ["BASE"] }] +[features] +default = [] +simd = { sources = ["src/simd/**"], flags = [ + { glob = "src/simd/**/*.avx2.cpp", cxxflags = ["-mavx2"], defines = ["HAVE_AVX2"] }, + { glob = "src/simd/**/*.neon.cpp", cxxflags = ["-mfpu=neon"] } ] } +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + + ASSERT_TRUE(m->buildConfig.featureFlags.contains("simd")); + auto& ff = m->buildConfig.featureFlags["simd"]; + ASSERT_EQ(ff.size(), 2u); // declaration order preserved + EXPECT_EQ(ff[0].glob, "src/simd/**/*.avx2.cpp"); + ASSERT_EQ(ff[0].cxxflags.size(), 1u); + EXPECT_EQ(ff[0].cxxflags[0], "-mavx2"); + ASSERT_EQ(ff[0].defines.size(), 1u); + EXPECT_EQ(ff[0].defines[0], "HAVE_AVX2"); + EXPECT_EQ(ff[1].glob, "src/simd/**/*.neon.cpp"); + // Parse-time base globFlags carry ONLY the [build].flags entry. + ASSERT_EQ(m->buildConfig.globFlags.size(), 1u); + EXPECT_EQ(m->buildConfig.globFlags[0].glob, "src/base/**"); + // Gated sources still land in featureSources alongside. + ASSERT_TRUE(m->buildConfig.featureSources.contains("simd")); +} + +// #253: feature `flags` entries share [build].flags' closed grammar — a +// missing `glob` and an unknown entry key are hard errors naming the +// feature-scoped anchoring key. +TEST(Manifest, FeatureFlagsTomlErrors) { + constexpr auto missingGlob = R"( +[package] +name = "x" +version = "0.1.0" +[targets.x] +kind = "lib" +[features] +simd = { flags = [{ cxxflags = ["-mavx2"] }] } +)"; + auto m1 = mcpp::manifest::parse_string(missingGlob); + ASSERT_FALSE(m1.has_value()); + EXPECT_NE(m1.error().message.find("[features].simd.flags"), std::string::npos) + << m1.error().message; + EXPECT_NE(m1.error().message.find("glob"), std::string::npos); + + constexpr auto badKey = R"( +[package] +name = "x" +version = "0.1.0" +[targets.x] +kind = "lib" +[features] +simd = { flags = [{ glob = "src/**", ldflags = ["-lfoo"] }] } +)"; + auto m2 = mcpp::manifest::parse_string(badKey); + ASSERT_FALSE(m2.has_value()); + EXPECT_NE(m2.error().message.find("ldflags"), std::string::npos) + << m2.error().message; +} + +// #253 Lua descriptor surface: `features..flags` parses into +// featureFlags (same entry grammar as the build-level `flags` key) and is a +// KNOWN subfield — it must not land in xpkgUnknownKeys. +TEST(SynthesizeFromXpkgLua, FeatureFlagsParse) { + constexpr auto lua = R"( +package = { + spec = "1", + name = "compat.opencv", + xpm = { linux = { ["1.0.0"] = { url = "u", sha256 = "h" } } }, + mcpp = { + sources = { "*/core/*.cpp" }, + targets = { ["opencv"] = { kind = "lib" } }, + flags = { { glob = "*/core/*.cpp", defines = { "BASE" } } }, + features = { + ["dnn"] = { + sources = { "*/3rdparty/mlas/lib/*.cpp" }, + flags = { + { glob = "**/3rdparty/mlas/**", + defines = { "BUILD_MLAS_NO_ONNXRUNTIME=1", "MLAS_GEMM_ONLY=1" } }, + { glob = "**/modules/dnn/**/*.avx2.cpp", + cxxflags = { "-mavx2" } }, + }, + }, + }, + }, +} +)"; + auto m = mcpp::manifest::synthesize_from_xpkg_lua(lua, "compat.opencv", "1.0.0"); + ASSERT_TRUE(m.has_value()) << m.error().format(); + EXPECT_TRUE(m->xpkgUnknownKeys.empty()); + + ASSERT_TRUE(m->buildConfig.featureFlags.contains("dnn")); + auto& ff = m->buildConfig.featureFlags["dnn"]; + ASSERT_EQ(ff.size(), 2u); + EXPECT_EQ(ff[0].glob, "**/3rdparty/mlas/**"); + ASSERT_EQ(ff[0].defines.size(), 2u); + EXPECT_EQ(ff[0].defines[0], "BUILD_MLAS_NO_ONNXRUNTIME=1"); + EXPECT_EQ(ff[1].glob, "**/modules/dnn/**/*.avx2.cpp"); + // Base per-glob flags untouched at parse time. + ASSERT_EQ(m->buildConfig.globFlags.size(), 1u); + EXPECT_EQ(m->buildConfig.globFlags[0].glob, "*/core/*.cpp"); + + // Unknown entry subkey is a hard error carrying the feature-scoped label. + constexpr auto bad = R"( +package = { + spec = "1", + name = "compat.opencv", + xpm = { linux = { ["1.0.0"] = { url = "u", sha256 = "h" } } }, + mcpp = { + sources = { "*/core/*.cpp" }, + features = { ["dnn"] = { flags = { { glob = "x", ldflags = { "-lfoo" } } } } }, + }, +} +)"; + auto mb = mcpp::manifest::synthesize_from_xpkg_lua(bad, "compat.opencv", "1.0.0"); + ASSERT_FALSE(mb.has_value()); + EXPECT_NE(mb.error().message.find("features.dnn.flags"), std::string::npos) + << mb.error().message; +} + +// #253: per-OS descriptor sections participate in `features` via the same +// textual-splice additive overlay every mcpp-segment key gets — same-named +// features merge per-subkey by APPEND (base body first, OS body second), and a +// per-OS section may register an OS-only feature. Locked here because the +// common/delta pattern (neutral common payload + per-OS delta) depends on it. +TEST(SynthesizeFromXpkgLua, PerOsFeaturesAdditiveMerge) { + constexpr auto lua = R"( +package = { + spec = "1", + name = "compat.opencv", + xpm = { linux = { ["1.0.0"] = { url = "u", sha256 = "h" } } }, + mcpp = { + sources = { "*/core/*.cpp" }, + features = { + ["dnn"] = { + defines = { "HAVE_OPENCV_DNN" }, + sources = { "*/modules/dnn/src/*.cpp" }, + }, + }, + linux = { + features = { + ["dnn"] = { + sources = { "*/3rdparty/mlas/lib/x86_64/*.S" }, + flags = { { glob = "**/3rdparty/mlas/**", defines = { "MLAS_X86" } } }, + }, + ["vaapi"] = { defines = { "HAVE_VAAPI" } }, + }, + }, + macosx = { + features = { + ["dnn"] = { sources = { "*/modules/dnn/neon/*.cpp" } }, + }, + }, + }, +} +)"; + // linux leg: neutral payload + linux delta, macosx section invisible. + auto ml = mcpp::manifest::synthesize_from_xpkg_lua( + lua, "compat.opencv", "1.0.0", "linux"); + ASSERT_TRUE(ml.has_value()) << ml.error().format(); + ASSERT_TRUE(ml->buildConfig.featureSources.contains("dnn")); + { + auto& srcs = ml->buildConfig.featureSources["dnn"]; + ASSERT_EQ(srcs.size(), 2u); // append order: neutral first, OS delta after + EXPECT_EQ(srcs[0], "*/modules/dnn/src/*.cpp"); + EXPECT_EQ(srcs[1], "*/3rdparty/mlas/lib/x86_64/*.S"); + } + ASSERT_TRUE(ml->buildConfig.featureFlags.contains("dnn")); + EXPECT_EQ(ml->buildConfig.featureFlags["dnn"][0].glob, "**/3rdparty/mlas/**"); + // OS-only feature registered (requestable) on this leg only. + EXPECT_TRUE(ml->featuresMap.contains("vaapi")); + ASSERT_TRUE(ml->buildConfig.featureDefines.contains("vaapi")); + EXPECT_EQ(ml->buildConfig.featureDefines["vaapi"][0], "HAVE_VAAPI"); + + // macosx leg: NEON delta instead, no mlas flags, no vaapi. + auto mm = mcpp::manifest::synthesize_from_xpkg_lua( + lua, "compat.opencv", "1.0.0", "macosx"); + ASSERT_TRUE(mm.has_value()) << mm.error().format(); + { + auto& srcs = mm->buildConfig.featureSources["dnn"]; + ASSERT_EQ(srcs.size(), 2u); + EXPECT_EQ(srcs[0], "*/modules/dnn/src/*.cpp"); + EXPECT_EQ(srcs[1], "*/modules/dnn/neon/*.cpp"); + } + EXPECT_FALSE(mm->buildConfig.featureFlags.contains("dnn")); + EXPECT_FALSE(mm->featuresMap.contains("vaapi")); +} + // Feature System v2 Stage 2a: optional deps activated by a feature. // TOML surface uses a dedicated [feature-deps.] section. TEST(Manifest, FeatureDepsTomlSection) {