!arch/arm: Use r9 as the PIC base register, and bring mknxflat in-tree

[casaroli]

Summary

ARM PIC uses r10 as the base register, but the tree has never been consistent about it. arch/arm/src/common/Toolchain.defs gives CONFIG_BUILD_PIC -mpic-register=r9 at line 594 and CONFIG_PIC -mpic-register=r10 at line 620, and arm_initialstate.c sets REG_R9 from inline assembly under one and REG_PIC under the other, with a comment that reads "Set the PIC base register (probably R10)". This settles it on r9 for all of PIC: NXFLAT, ELF PIC and CONFIG_BUILD_PIC alike.

r9 is the right register rather than an arbitrary one. It is the AAPCS platform register, the "static base", and it is what GCC itself selects for -msingle-pic-base on an EABI target; r10 is the non-EABI default.

It also fixes a configuration that cannot build today. Stack checking adds -ffixed-r10 in armv7-m/Toolchain.defs:149 and armv8-m/Toolchain.defs:168 while CONFIG_PIC adds -mpic-register=r10, and GCC rejects the pair with "unable to use 'r10' for PIC register". The comment above REG_PIC has always said the register "can be R9 if stack checking is enabled", but the definition was unconditionally REG_R10, so it would have named the wrong register even if the build had succeeded.

Why the NXFLAT thunk generator comes with it

Moving the register on its own breaks NXFLAT, silently. Import thunks are generated by mknxflat, which had the register baked into its template as add ip,ip,sl, so a module compiled -mpic-register=r9 reaches its data through r9 while its import stubs still add r10, and it branches to a wild address on its first call into the base firmware.

mknxflat lives outside this repository, in the buildroot NXFLAT toolchain, so the kernel and the tool could drift with nothing to detect it. The first commit therefore brings it in-tree as tools/nxflat/, with libbfd replaced by reading the ELF symbol table directly — libbfd was only ever used to open the file and enumerate symbols, and it is GPL, which we cannot depend on. The thunks are then generated from a single NXFLAT_PIC_REG on every build, so the two cannot drift.

That commit changes no output. Against the upstream tool, for both ARM and Thumb-2, with and without -w, over modules exercising the plain, weak and non-returning thunk paths, the generated thunk files are byte-identical.

Modules built before this change

They are refused at load rather than left to crash.

The NXFLAT header cannot carry a version: h_magic is written by ldnxflat, which is GPL, derived from elf2flt, and stays out of this repository, so it can never be changed in step with the loader. The import table can, because both of its ends are in-tree — mknxflat emits it and nxflat_bindimports() reads it — and ldnxflat passes it through untouched. Every module now imports __nxflat_abi_v2, the base firmware defines it, and a module that does not import it is refused with -ENOEXEC.

Making the marker a real exported symbol rather than a name the loader special-cases keeps it out of the build system's way: a board's symbol table picks it up exactly as it picks up printf, so mksymtab.sh and its equivalents need no change. It also diagnoses the reverse direction for free — a module built against a newer ABI than its firmware fails with Exported symbol "__nxflat_abi_v2" not found.

Impact

Warning

BREAKING CHANGE: ARM PIC moves from r10 to r9. An NXFLAT module built before this has r10 baked into its import stubs and will not run against a firmware carrying it; the loader refuses it with -ENOEXEC rather than branching to a wild address.

Quick fix: rebuild the module against this tree — its source needs no change. A board that reserved r10 by hand, or that assigned ARCHPICFLAGS or MKNXFLAT to restate a default, should drop those assignments. CONFIG_PIC without CONFIG_NXFLAT needs no action, and configurations with neither are untouched.

Most of the churn is boards restating a default. ARCHPICFLAGS is a ?= default so that a board only speaks up when it differs, and twenty-six were assigning the value the default already had; those lines are removed rather than edited. MKNXFLAT gets the same treatment: thirteen boards named the same tool, and the only thing that varies is ARM versus Thumb-2, which falls out of CONFIG_ARM_THUMB. LDNXFLAT gains a default too — it stays an out-of-tree PATH lookup, but naming it centrally fixes boards that never assigned it, where it expanded to nothing and handed make a recipe beginning -e, whose leading dash make ate as "ignore errors".

The non-ARM boards carrying -mpic-register=r10 lose it. It is an ARM-only option, reachable only through CPICFLAGS, which is only used to build NXFLAT modules, and no non-ARM board enables NXFLAT; six other RISC-V boards already omitted it.

mknxflat is built only when CONFIG_NXFLAT is set, following the CONFIG_BOARD_ETC_ROMFS_PASSWD_ENABLE precedent in tools/Unix.mk.

Licensing

tools/nxflat/mknxflat.c and the two .def templates are BSD-3-Clause, copyright Gregory Nutt and Cadenux LLC, imported from the NuttX buildroot NXFLAT toolchain with their headers intact. The containing repository's COPYING is GPL-2.0, but these particular files carry their own BSD-3 grant, name only Nutt and Cadenux as copyright holders, and descend from XFLAT's ldelflib rather than from elf2flt. @patacongo for confirmation.

ldnxflat is deliberately not imported. It is GPL-2.0-or-later and derives from elf2flt, with roughly ten copyright holders reaching back to 1996 — it cannot be relicensed and cannot live in an Apache repository. NXFLAT continues to need it from the buildroot toolchain, exactly as today.

Here is the licensing agreement mailing list archive:
https://www.mail-archive.com/dev@nuttx.apache.org/msg14863.html

How to verify this on your board

Help with this is very welcome — §1.14 asks for runtime logs from more than one real architecture, and I have only an RP2350 here.

Any board, ARM or not (two minutes). Build your usual configuration, boot it, and run ostest. That is the whole test for most boards: the only thing that touches them is the PIC flag cleanup, which is a no-op unless the board sets ARCHPICFLAGS or enables NXFLAT. Non-ARM boards lose -mpic-register=r10, which was an ARM-only option reachable only through CPICFLAGS, and CPICFLAGS is only used to build NXFLAT modules — so nothing should change at all.

If your board enables CONFIG_NXFLAT. Build and run a module, for example olimex-lpc1766stk:thttpd-nxflat or apps/examples/nxflat. mknxflat now comes from tools/nxflat and the build makes it for you; ldnxflat is still an out-of-tree tool from buildroot, unchanged.

If you have an NXFLAT module built before this change. Run it without rebuilding. It must be refused, with

nxflat_bindimports: ERROR: Module does not declare __nxflat_abi_v2: it was built by a
toolchain predating the r9 PIC base register.  Rebuild it.

and exec() failing with -ENOEXEC (-8). A crash or a wild branch instead of that message is a bug — please report it. Rebuilding the module is the fix; its source needs no change.

Build logs and ostest output from any architecture are welcome and I will collect them here.

Testing

Host: macOS 26.5.1 on arm64, Arm GNU Toolchain 14.2.Rel1, QEMU 11.0.3.

lm3s6965-ek:qemu-nxflat under QEMU, configured and built with no overrides. The nxflat example runs the errno, hello and struct modules with output identical to the same configuration built from master:

* Executing errno      -> Hello, World on stdout / stderr, errno is 2
* Executing hello      -> Hello, world!  It has been said.
* Executing struct     -> End-of-Test.. Exit-ing

The same firmware, with the modules rebuilt by the old out-of-tree generator, refuses all three:

ERROR: exec(errno) failed: 8
ERROR: exec(hello) failed: 8

Before the ABI marker that same combination locked up in a HardFault with no console output, which is what the marker is for.

mps3-an547:picostest, which is CONFIG_PIC without CONFIG_NXFLAT, builds clean and does not build the thunk generator.

tools/checkpatch.sh -f passes on every changed C file and header.

ELF PIC on hardware. Pimoroni Pico Plus 2 (RP2350, Cortex-M33), pimoroni-pico-2-plus:nsh plus CONFIG_ELF, CONFIG_PIC and apps/examples/elf. The modules link PIC with r9 as their base — hello carries 13 r9 references and 18 GOT/RELATIVE relocations — and eight of them load from the embedded ROMFS and run: errno, hello, signal, struct, mutex, pthread, task and longjmp. Heap usage returns to its exact starting value at the end of the run.

struct is the test that matters for the register. It reads GOT data, dereferences a pointer into .rodata in flash (0x10033aec), and calls through a function pointer, and every one of its checks passes.

Two things this turned up, neither introduced by this PR.

The signal test reports SIGUSR1 not received, its handler printing nothing for either raise() or kill(). This is not a PIC problem: the same test fails the same way on an ESP32-S3, where nothing in the build is PIC at all. It is a property of that test or of signal delivery to a loaded module, and is mentioned here only so the log above is not read as a regression.

A C++ ELF module cannot link under CONFIG_PIC: hello++3 fails with undefined reference to __dso_handle, which is hidden and so cannot be left unresolved by --unresolved-symbols=ignore-in-object-files. The CONFIG_PIC branch of LDELFFLAGS predates this PR and is unchanged by it. The tests above were run with CONFIG_HAVE_CXX off.

Still not covered: mps3-an547:picostest builds and boots identically to master, but NSH takes no console input in that configuration under QEMU — on master as well — so no module is executed there.

[github-actions]

MemBrowse Memory Report

No memory changes detected for:

• arduino-mega2560
• esp32-devkitc
• hifive1-revb
• mirtoo
• qemu-armv8a
• qemu-intel64
• rx65n-rsk2mb
• s698pm-dkit
• stm32-nucleo-f103rb

[casaroli]

@patacongo — a licensing question, since you are the copyright holder.

This PR imports mknxflat into the tree as tools/nxflat/. The reason is in the PR body: moving the ARM PIC base register to r9 breaks NXFLAT silently, because mknxflat bakes the register into the import thunks it generates as add ip,ip,sl, and while the tool lives outside this repository it can drift out of step with the kernel.

The three files taken are mknxflat.c and arm/dyncall_skeleton.def / thumb2/dyncall_skeleton.def from the buildroot NXFLAT toolchain. Each carries a BSD-3-Clause header naming you and Cadenux LLC, and notes that the code derives from XFLAT's ldelflib. The containing repository's COPYING is GPL-2.0, so I would like that confirmed rather than assumed: is the per-file BSD-3 grant the operative licence for those three files, and are you content for them to be imported here with their headers intact?

ldnxflat is deliberately not imported. It carries an explicit GPL-2.0-or-later notice and descends from elf2flt, with roughly ten copyright holders going back to 1996, so it stays where it is and NXFLAT continues to need it from buildroot.

The only substantive change to the imported code is dropping the libbfd dependency, which was used solely to open the file and enumerate symbols; the tool now reads the ELF symbol table directly. The generated thunk files are byte-identical to what the current tool produces, for both ARM and Thumb-2, with and without -w.

[casaroli]

Related: https://www.mail-archive.com/dev@nuttx.apache.org/msg14863.html

[cederom]

Thank you @casaroli lets just wait for Gregory Nutt states agreement on the mailing list to relicense files from BSD to Apache so we can already merge Apache :-)

[xiaoxiang781216]

@casaroli please fix the documentation ci error.

[casaroli]

@casaroli please fix the documentation ci error.

The documentation CI error is not related to this PR changes and is happening in any PR that touches documentation.

I will investigate and try to push the fix in another PR anyway.

[acassis]

@jerpelea PTAL

[cederom]

Thank you @casaroli :-)

[cederom]

Amazing piece of work @casaroli =)