Merge pull request #2254 from bettio/grow-erlang-boot-partition

ESP32: Increase Erlang boot.avm partition to 512KB

Users flashing applications to ESP32 devices with Erlang-only images must update their
flashing offset from `0x210000` to `0x250000`.

These changes are made under both the "Apache 2.0" and the "GNU Lesser General
Public License 2.1 or later" license terms (dual license).

SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later

Author: bettio

CHANGELOG.md

@@ -28,6 +28,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `erlang:float_to_binary/2` and `erlang:float_to_list/2` now accept `{decimals, 0..253}` and `{scientific, 0..249}`
 - `erlang:binary_to_float/1` and `erlang:list_to_float/1` now use locale-independent parsing and
 strict format validation
+- ESP32: the `boot.avm` partition for Erlang-only images has been increased from 256KB to 512KB,
+matching the Elixir partition layout. The `main.avm` offset is now `0x250000` for all images
+(previously `0x210000` for Erlang-only).
 
 ### Fixed
 - Fixed `erlang:cancel_timer/1` return type spec and documentation to match OTP

UPDATING.md

@@ -22,16 +22,19 @@ handling code.
 - ESP32 builds with Elixir support may be configured without making changes to git-tracked files
 using `idf.py -DATOMVM_ELIXIR_SUPPORT=on set-target ${CHIP}` instead of copying
 partitions-elixir.csv to partitions.csv. This configures the build to use partitions-elixir.csv for
-the partition table. The `main.avm` offset in the partition table will determine which flavor of
-esp32boot libraries to include for the `idf.py flash` task and the image assembled by
-`build/mkimage.sh`.
+the partition table. The `ATOMVM_ELIXIR_SUPPORT` option will determine which flavor of esp32boot
+libraries to include for the `idf.py flash` task and the image assembled by `build/mkimage.sh`.
 - ESP32 release builds may be configured with `idf.py -DATOMVM_RELEASE=on set-target ${CHIP}`
 rather than copy sdkconfig.release-defaults.in to sdkconfig.defaults.in (which still requires a
 `reconfigure` or `set-target` to be run to pick up the changes), this may also be combined with the
 `ATOMVM_ELIXIR_SUPPORT` option. For example, an Elixir-supported release build is configured using:
 `idf.py -DATOMVM_ELIXIR_SUPPORT=on -DATOMVM_RELEASE=on set-target ${CHIP}`
 - `json_encoder` module has been removed, use new (and standard) `json` module. New module uses
 standard Erlang/OTP API, that takes maps instead of proplists.
+- ESP32: the `boot.avm` partition has been increased from 256KB to 512KB for Erlang-only images.
+The `main.avm` offset is now `0x250000` for all images (previously `0x210000` for Erlang-only).
+Update flashing offsets in your tooling, build scripts, and `mix.exs` or `rebar.config` if you
+were using the `0x210000` offset.
 
 ## v0.6.4 -> v0.6.5
 

doc/src/atomvm-tooling.md

@@ -119,7 +119,7 @@ $ rebar3 atomvm esp32_flash --port /dev/ttyUSB0 --baud 921600
 ...
 ===> esptool.py --chip auto --port /dev/ttyUSB0 --baud 921600 --before default_reset
     --after hard_reset write_flash -u --flash_mode keep --flash_freq keep --flash_size detect
-    0x210000 atomvm_examples/erlang/hello_world/_build/default/lib/hello_world.avm
+    0x250000 atomvm_examples/erlang/hello_world/_build/default/lib/hello_world.avm
 ```
 
 ```{tip}
@@ -147,14 +147,14 @@ You can now use a serial console program such as [minicom](https://en.wikipedia.
         ###########################################################
 
     I (852) AtomVM: Starting AtomVM revision 0.6.0-alpha.1
-    I (862) sys: Loaded BEAM partition boot.avm at address 0x1d0000 (size=262144 bytes)
+    I (862) sys: Loaded BEAM partition boot.avm at address 0x1d0000 (size=524288 bytes)
     I (882) network_driver: Initialized network interface
     I (882) network_driver: Created default event loop
     I (902) AtomVM: Found startup beam esp32init.beam
     I (922) AtomVM: Starting esp32init.beam...
     ---
     AtomVM init.
-    I (932) sys: Loaded BEAM partition main.avm at address 0x210000 (size=1048576
+    I (932) sys: Loaded BEAM partition main.avm at address 0x250000 (size=1048576
     bytes)
     Starting application...
     Hello World

doc/src/build-instructions.md

@@ -505,9 +505,9 @@ The flash layout is roughly as follows (not to scale):
     |    (factory)    |             |
     |                 |             |
     +-----------------+             |
-    |     boot.avm    | 256-512KB   v
-    +-----------------+  ------------- 0x210000 for Erlang only images or
-    |                 |             ^  0x250000 for images with Elixir modules
+    |     boot.avm    | 512KB       v
+    +-----------------+  ------------- 0x250000
+    |                 |             ^
     |                 |             |
     |     main.avm    | 1MB+        | Erlang/Elixir
     |                 |             | Application
@@ -524,11 +524,13 @@ The following table summarizes the partitions created on the ESP32 when deployin
 | NVS | 0x9000 | 24kB | Space for non-volatile storage. |
 | PHY_INIT | 0xF000 | 4kB | Initialization data for physical layer radio signal data. |
 | AtomVM virtual machine | 0x10000 | 1.75mB | The AtomVM virtual machine (compiled from C code). |
-| boot.avm | 0x1D0000 | 256k | The AtomVM BEAM library, compiled from Erlang and Elixir files in the AtomVM source tree. |
-| main.avm | `0x210000` \| `0x250000` | 1mB | The user application.  This is where users flash their compiled Erlang/Elixir code |
+| boot.avm | 0x1D0000 | 512k | The AtomVM BEAM library, compiled from Erlang and Elixir files in the AtomVM source tree. |
+| main.avm | 0x250000 | 1mB | The user application.  This is where users flash their compiled Erlang/Elixir code |
 
-```{warning}
-There is an important difference in the partition layout between the minimal images and those build with Elixir support. To accommodate the extra Elixir modules the boot.avm partition on these images is larger, and the application offset is moved accordingly. When working with Elixir supported images it is important to always use the offset `0x250000` whether using `mix` or the `atomvm_rebar3_plugin` (possibly to test an Erlang app), otherwise part of the boot.avm partition (specifically the area where many Elixir modules are located) will be overwritten with the application, but the VM will still be trying to load from the later `0x250000` offset. This should be kept in mind reading the rest of build instructions, and [AtomVM Tooling](./atomvm-tooling.md) sections of the docs that cover the use of rebar3, for these sections an Erlang only image is assumed.
+```{note}
+Since v0.7.0-alpha.1, both Erlang-only and Elixir-supported images use the same partition layout
+and the same `main.avm` offset of `0x250000`. Previous versions used `0x210000` for Erlang-only
+images.
 ```
 
 ### The `boot.avm` and `main.avm` partitions
@@ -537,7 +539,7 @@ The `boot.avm` and `main.avm` partitions are intended to store Erlang/Elixir lib
 
 The `boot.avm` partition is intended for core Erlang/Elixir libraries that are built as part of the AtomVM build.  The release image of AtomVM (see below) includes both the AtomVM virtual machine and the `boot.avm` partition, which includes the BEAM files from the `estdlib` and `eavmlib` libraries.
 
-In contrast, the `main.avm` partition is intended for user applications.  Currently, the `main.avm` partition starts at address `0x210000` for thin images or `0x250000` for images with Elixir modules, and it is to that location to which application developers should flash their application AVM files.
+In contrast, the `main.avm` partition is intended for user applications.  Currently, the `main.avm` partition starts at address `0x250000`, and it is to that location to which application developers should flash their application AVM files.
 
 The AtomVM search path for BEAM modules starts in the `main.avm` partition and falls back to `boot.avm`.  Users should not have a need to override any functionality in the `boot.avm` partition, but if necessary, a BEAM module of the same name in the `main.avm` partition will be loaded instead of the version in the `boot.avm` partition.
 
@@ -580,7 +582,7 @@ Wrote AtomVM Virtual Machine at offset 0x10000 (65536)
 Wrote AtomVM Core BEAM Library at offset 0x1D0000 (1114112)
 ```
 
-Users can then use the `esptool.py` directly to flash the entire image to the ESP32 device, and then flash their applications to the `main.app` partition at address `0x210000`, (or `0x250000` for Elixir images)
+Users can then use the `esptool.py` directly to flash the entire image to the ESP32 device, and then flash their applications to the `main.app` partition at address `0x250000`.
 
 But first, it is a good idea to erase the flash, e.g.,
 

doc/src/getting-started-guide.md

@@ -51,7 +51,7 @@ The ES32 AtomVM virtual machine is an IDF application that runs on the ESP32 pla
 
 The AtomVM virtual machine is implemented in C, and the AtomVM binary image contains the binary object code compiled from C source files, as well as the ESP boot loader and partition map, which tells the ESP32 how the flash module is laid out.
 
-AtomVM developers will typically write their applications in Erlang or Elixir.  These source files are compiled into BEAM bytecode, which is then assembled into AtomVM "packbeam" (`.avm`) files.  This packbeam file is flashed onto the ESP32 device, starting at the data partition address `0x210000`.  When AtomVM starts, it will look in this partition for the first occurrence of a BEAM module that exports a `start/0` function.  Once that module is located, execution of the BEAM bytecode will commence at that point.
+AtomVM developers will typically write their applications in Erlang or Elixir.  These source files are compiled into BEAM bytecode, which is then assembled into AtomVM "packbeam" (`.avm`) files.  This packbeam file is flashed onto the ESP32 device, starting at the data partition address `0x250000`.  When AtomVM starts, it will look in this partition for the first occurrence of a BEAM module that exports a `start/0` function.  Once that module is located, execution of the BEAM bytecode will commence at that point.
 
 The following diagram provides a simplified overview of the layout of the AtomVM virtual machine and Erlang/Elixir applications on the ESP32 flash module.
 
@@ -66,8 +66,8 @@ The following diagram provides a simplified overview of the layout of the AtomVM
     |   Virtual     |           |
     |   Machine     |           |
     |               |           v
-    +---------------+  ----------- 0x210000 for thin images or
-    |               |           ^  0x250000 for images with Elixir modules
+    +---------------+  ----------- 0x250000
+    |               |           ^
     |               |           |
     |     data      |           | Erlang/Elixir
     |   partition   |           | Application
@@ -187,7 +187,7 @@ Instructions for building AtomVM from source are covered in the AtomVM [Build In
 
 ### Deploying an AtomVM application for ESP32
 
-An AtomVM application is a collection of BEAM files, which have been compiled using the Erlang or Elixir compiler.  These BEAM files are assembled into an AtomVM "packbeam" (`.avm`) file, which in turn is flashed to the `main` data partition on the ESP32 flash module, starting at address `0x210000` if you are using a thin image, or `0x250000` for images with Elixir support.
+An AtomVM application is a collection of BEAM files, which have been compiled using the Erlang or Elixir compiler.  These BEAM files are assembled into an AtomVM "packbeam" (`.avm`) file, which in turn is flashed to the `main` data partition on the ESP32 flash module, starting at address `0x250000`.
 
 When the AtomVM virtual machine starts, it will search for the first module that contains an exported `start/0` function in this partition, and it will begin execution of the BEAM bytecode at that function.
 

src/platforms/esp32/GetBootAVM.cmake

@@ -20,14 +20,16 @@
 
 partition_table_get_partition_info(app_offset "--partition-name main.avm" "offset")
 set(AVM_APP_OFFSET "${app_offset}")
-# 0x210000 = standard Erlang only partition layout (partitions.csv)
-if ("${app_offset}" STREQUAL "0x210000")
-    set(BOOT_LIBS "esp32boot.avm")
-    set(ATOMVM_FLAVOR "")
-# 0x250000 = Elixir-supported partition layout (partitions-elixir.csv)
-elseif ("${app_offset}" STREQUAL "0x250000")
-    set(BOOT_LIBS "elixir_esp32boot.avm")
-    set(ATOMVM_FLAVOR "-elixir")
+# Both partitions.csv and partitions-elixir.csv use 0x250000 for main.avm.
+# Use ATOMVM_ELIXIR_SUPPORT to select the boot library flavor.
+if ("${app_offset}" STREQUAL "0x250000")
+    if (ATOMVM_ELIXIR_SUPPORT)
+        set(BOOT_LIBS "elixir_esp32boot.avm")
+        set(ATOMVM_FLAVOR "-elixir")
+    else()
+        set(BOOT_LIBS "esp32boot.avm")
+        set(ATOMVM_FLAVOR "")
+    endif()
 else()
     set(BOOT_LIBS "NONE")
     set(ATOMVM_FLAVOR "")

src/platforms/esp32/partitions.csv

@@ -8,5 +8,5 @@
 nvs,      data, nvs,       0x9000,     0x6000,
 phy_init, data, phy,       0xf000,     0x1000,
 factory,  app,  factory,  0x10000,   0x1C0000,
-boot.avm,  data, phy,     0x1D0000,    0x40000,
-main.avm, data, phy,     0x210000,   0x100000
+boot.avm,  data, phy,     0x1D0000,    0x80000,
+main.avm, data, phy,     0x250000,   0x100000

tools/test/smoke-test/README.md

@@ -186,7 +186,7 @@ To run an example program, `cd` into the appropriate example directory, and buil
     ===> Verifying dependencies...
     ===> Analyzing applications...
     ===> Compiling hello_world
-    ===> esptool.py --chip auto --port /dev/tty.usbserial-0164A0E0 --baud 921600 --before default_reset --after hard_reset write_flash -u --flash_mode keep --flash_freq keep --flash_size detect 0x210000 .../atomvm_examples/erlang/hello_world/_build/default/lib/hello_world.avm
+    ===> esptool.py --chip auto --port /dev/tty.usbserial-0164A0E0 --baud 921600 --before default_reset --after hard_reset write_flash -u --flash_mode keep --flash_freq keep --flash_size detect 0x250000 .../atomvm_examples/erlang/hello_world/_build/default/lib/hello_world.avm
 
     esptool.py v3.3.4-dev
     Serial port /dev/tty.usbserial-0164A0E0
@@ -236,14 +236,14 @@ For example:
     ###########################################################
 
     I (780) AtomVM: Starting AtomVM revision 0.6.0-alpha.1
-    I (790) sys: Loaded BEAM partition boot.avm at address 0x1d0000 (size=262144 bytes)
+    I (790) sys: Loaded BEAM partition boot.avm at address 0x1d0000 (size=524288 bytes)
     I (810) network_driver: Initialized network interface
     I (810) network_driver: Created default event loop
     I (830) AtomVM: Found startup beam esp32init.beam
     I (830) AtomVM: Starting esp32init.beam...
     ---
     AtomVM init.
-    I (840) sys: Loaded BEAM partition main.avm at address 0x210000 (size=1048576 bytes)
+    I (840) sys: Loaded BEAM partition main.avm at address 0x250000 (size=1048576 bytes)
     Starting application...
     Hello World
     AtomVM finished with return value: ok