Skip to content

compiler, runtime, reflect: generate type-specific hash/equal#5359

Merged
deadprogram merged 6 commits into
tinygo-org:devfrom
jakebailey:fix-3354
May 18, 2026
Merged

compiler, runtime, reflect: generate type-specific hash/equal#5359
deadprogram merged 6 commits into
tinygo-org:devfrom
jakebailey:fix-3354

Conversation

@jakebailey
Copy link
Copy Markdown
Member

@jakebailey jakebailey commented May 1, 2026 •
edited
Loading

Fixes #3354
Updates #3794

The commits themselves have way more info; this isn't gerrit so forgive me for not repeating everything said in the commit description I've beeen tweaking for a while now...

The first commit generates type-specific hash/equal funcs for types that previously didn't have them. This leads to a significant speedup. For the benchmarks I've included running in wasmtime:

                  │ /tmp/bench_dev.txt │         /tmp/bench_opt.txt          │
                  │       sec/op       │   sec/op     vs base                │
MapStringShortGet         421.30n ± 2%   71.87n ± 4%  -82.94% (p=0.000 n=10)
MapStringLongGet           456.6n ± 2%   114.8n ± 2%  -74.85% (p=0.000 n=10)
MapCompositeGet          2427.00n ± 2%   94.30n ± 1%  -96.11% (p=0.000 n=10)
MapIntGet                  60.36n ± 1%   62.13n ± 1%   +2.93% (p=0.000 n=10)
geomean                    409.7n        83.39n       -79.65%

                  │ /tmp/bench_dev.txt │           /tmp/bench_opt.txt            │
                  │        B/op        │    B/op     vs base                     │
MapStringShortGet         8.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
MapStringLongGet          8.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
MapCompositeGet           32.00 ± 0%      0.00 ± 0%  -100.00% (p=0.000 n=10)
MapIntGet                 0.000 ± 0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
geomean                              ²               ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

                  │ /tmp/bench_dev.txt │           /tmp/bench_opt.txt            │
                  │     allocs/op      │ allocs/op   vs base                     │
MapStringShortGet         1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
MapStringLongGet          1.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
MapCompositeGet           5.000 ± 0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
MapIntGet                 0.000 ± 0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
geomean                              ²               ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

"CompositeGet" is not even a stress test, it's literally just:

type compositeKey struct {
	S string
	N int32
}

There's a net benefit here that for these types, the "interface" fallback is no longer used, so binaries can actually get smaller because they stop pulling in reflect. I haven't run sizediff in a bit, though, so I'll be checking this in CI after.

A later commit gets rid of the "interface" fallback altogether by fixing a bug in interp with pointers as map keys. They were acutally broken already and would cause a panic (which I now test). Fixing this is annoying because the "right" fix would be to return an error up the stack, but that quite literally requires 300 lines of plumbing. So, I just stuck an error onto interp.runner to sneak the error out. Then, the interp code gracefully bails out and leaves it to actual runtime code.

I can do a returned error if my trick seems too nasty, you just might not like what the plubming looks like otherwise... (It's not the worst but it does make the diff big.)

@jakebailey
Copy link
Copy Markdown
Member Author

jakebailey commented May 1, 2026

Hm, I broke the stack adding in the benchmarks

@jakebailey
Copy link
Copy Markdown
Member Author

jakebailey commented May 1, 2026

I like what I'm seeing:

Size difference with the dev branch:

Binary size difference 
 flash                          ram
 before   after   diff          before   after   diff
  28028   22492  -5536 -19.75%    7100    7052    -48  -0.68% tinygo build -size short -o ./build/test.hex -target=pico ./examples/w5500/main.go
  86864   86440   -424  -0.49%    7532    7532      0   0.00% tinygo build -size short -o ./build/test.hex -target=challenger-rp2040 ./examples/net/ntpclient/
 345916  345580   -336  -0.10%   22188   22188      0   0.00% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/mqttclient/paho/
 353056  352792   -264  -0.07%   17212   17212      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal -stack-size 8kb ./examples/net/http-get/
 313284  313020   -264  -0.08%   16240   16240      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-rp2040 -stack-size 8kb ./examples/net/websocket/dial/
 344740  344500   -240  -0.07%   22308   22308      0   0.00% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/webserver/
  74056   73832   -224  -0.30%   11092   11092      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/sdcard/console/
 396284  396060   -224  -0.06%   19248   19248      0   0.00% tinygo build -size short -o ./build/test.hex -target=matrixportal-m4 -stack-size 8kb ./examples/net/webstatic/
  67004   66796   -208  -0.31%    9356    9356      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/flash/console/qspi
  57612   57428   -184  -0.32%    3672    3672      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/sht3x/main.go
  57604   57420   -184  -0.32%    3680    3680      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/sht4x/main.go
  57612   57428   -184  -0.32%    3672    3672      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/shtc3/main.go
  62948   62764   -184  -0.29%    6296    6296      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840 ./examples/is31fl3731/main.go
  67464   67280   -184  -0.27%    7212    7212      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/ndir/main_ndir.go
  62228   62044   -184  -0.30%    3768    3768      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ndir/main_ndir.go
  67136   66952   -184  -0.27%    7180    7180      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mcp9808/main.go
  62024   61848   -176  -0.28%    6528    6528      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adt7410/main.go
  65908   65732   -176  -0.27%    6544    6544      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmi160/main.go
  64580   64404   -176  -0.27%    6568    6568      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp280/main.go
 264552  264376   -176  -0.07%   47108   47108      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/slideshow
  72036   71860   -176  -0.24%    6584    6584      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp2515/main.go
  77244   77068   -176  -0.23%    6680    6680      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/dht/main.go
  71608   71432   -176  -0.25%    6688    6688      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/alarm/
  71044   70868   -176  -0.25%    6688    6688      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/time/
  71440   71264   -176  -0.25%    6688    6688      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/timer/
  62264   62088   -176  -0.28%    8576    8576      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/i2csoft/adt7410/
  65572   65396   -176  -0.27%    6600    6600      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ndir/main_ndir.go
  62696   62520   -176  -0.28%    6332    6332      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840 ./examples/max6675/main.go
  70612   70452   -160  -0.23%    7316    7316      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/flash/console/spi
 122768  122616   -152  -0.12%    8112    8112      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/lora/lorawan/atcmd/
  76548   76436   -112  -0.15%    7788    7788      0   0.00% tinygo build -size short -o ./build/test.hex -target=p1am-100 ./examples/p1am/main.go
  71676   71580    -96  -0.13%    3640    3640      0   0.00% tinygo build -size short -o ./build/test.hex -target=pinetime     ./examples/bma42x/main.go
 120600  120512    -88  -0.07%    8428    8428      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 -stack-size 8kb ./examples/net/tcpclient/
 117100  117012    -88  -0.08%   13704   13704      0   0.00% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/webclient/
 103716  103652    -64  -0.06%   10408   10408      0   0.00% tinygo build -size short -o ./build/test.hex -target=metro-m4-airlift -stack-size 8kb ./examples/net/socket/
 167064  167032    -32  -0.02%   10400   10400      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-mkrwifi1010 -stack-size 8kb ./examples/net/tlsclient/
  17112   17112      0   0.00%    6576    6576      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/adafruit4650
   8764    8764      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adxl345/main.go
  13636   13636      0   0.00%    7136    7136      0   0.00% tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/amg88xx
   8924    8924      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/main.go
  11372   11372      0   0.00%    6928    6928      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/apds9960/proximity/main.go
   9828    9828      0   0.00%    5108    5108      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/itsybitsy-m0/main.go
   7356    7356      0   0.00%    2296    2296      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/at24cx/main.go
   8008    8008      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bh1750/main.go
   7368    7368      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/blinkm/main.go
  27664   27664      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp180/main.go
  11816   11816      0   0.00%    5152    5152      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bmp388/main.go
  23336   23336      0   0.00%    5744    5744      0   0.00% tinygo build -size short -o ./build/test.hex -target=metro-rp2350 ./examples/bno08x/i2c/main.go
   7712    7712      0   0.00%    3336    3336      0   0.00% tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/sram/main.go
  21616   21616      0   0.00%    3532    3532      0   0.00% tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/time/main.go
  29108   29108      0   0.00%    5316    5316      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/ds3231/alarms/main.go
  42980   42980      0   0.00%    5316    5316      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/ds3231/basic/main.go
   4400    4400      0   0.00%    2256    2256      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/easystepper/main.go
   7068    7068      0   0.00%    2260    2260      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/gc9a01/main.go
  67660   67660      0   0.00%    6704    6704      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/i2c/main.go
  68360   68360      0   0.00%    6852    6852      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/uart/main.go
   8420    8420      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/hcsr04/main.go
   5528    5528      0   0.00%    2256    2256      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/customchar/main.go
   5568    5568      0   0.00%    2256    2256      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/text/main.go
  10376   10376      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/hd44780i2c/main.go
  14248   14248      0   0.00%    6928    6928      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/hts221/main.go
  15980   15980      0   0.00%    2340    2340      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hub75/main.go
  10156   10156      0   0.00%    7264    7264      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/basic
  11144   11144      0   0.00%    5224    5224      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/basic
  29436   29436      0   0.00%   38424   38424      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/pyportal_boing
  10188   10188      0   0.00%    7256    7256      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/scroll
  11224   11224      0   0.00%    5216    5216      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/scroll
  10472   10472      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis3dh/main.go
  13632   13632      0   0.00%    6928    6928      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/lps22hb/main.go
  26188   26188      0   0.00%    2304    2304      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/lsm303agr/main.go
  27860   27860      0   0.00%    7168    7168      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/lsm303dlhc/main.go
  12312   12312      0   0.00%    5128    5128      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/lsm6ds3/main.go
  10020   10020      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mag3110/main.go
   8984    8984      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017/main.go
   9416    9416      0   0.00%    5128    5128      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017-multiple/main.go
   9440    9440      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp3008/main.go
  27324   27324      0   0.00%    5852    5852      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit-v2 ./examples/microbitmatrix/main.go
   7464    7464      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mma8653/main.go
   7368    7368      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mpu6050/main.go
  12312   12312      0   0.00%    5756    5756      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/pca9685/main.go
   6244    6244      0   0.00%    3268    3268      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setbuffer/main.go
   4628    4628      0   0.00%    2260    2260      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setpixel/main.go
  10604   10604      0   0.00%    5732    5732      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/seesaw/soil-sensor
  11916   11916      0   0.00%    5740    5740      0   0.00% tinygo build -size short -o ./build/test.hex -target=qtpy-rp2040 ./examples/seesaw/rotary-encoder
   3113    3113      0   0.00%     560     560      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino ./examples/servo
  13896   13896      0   0.00%    5804    5804      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico     ./examples/sgp30
   7988    7988      0   0.00%    7128    7128      0   0.00% tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/shifter/main.go
  12344   12344      0   0.00%    8712    8712      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao-ble ./examples/ssd1306/
  11728   11728      0   0.00%    5684    5684      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao-rp2040 ./examples/ssd1306/
  11816   11816      0   0.00%    5684    5684      0   0.00% tinygo build -size short -o ./build/test.hex -target=thumby ./examples/ssd1306/
   5896    5896      0   0.00%    2260    2260      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1331/main.go
   6808    6808      0   0.00%    2260    2260      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7735/main.go
   6488    6488      0   0.00%    2260    2260      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7789/main.go
  16784   16784      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/thermistor/main.go
   9640    9640      0   0.00%    4880    4880      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-bluefruit ./examples/tone
  10136   10136      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/tm1637/main.go
  10924   10924      0   0.00%    5744    5744      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/touch/capacitive
   8816    8816      0   0.00%    7128    7128      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/fourwire/main.go
  12348   12348      0   0.00%    7324    7324      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/pyportal_touchpaint/main.go
  15772   15772      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl53l1x/main.go
  14328   14328      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl6180x/main.go
  24636   24636      0   0.00%   14076   14076      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840-sense ./examples/waveshare-epd/epd1in54/main.go
   6356    6356      0   0.00%    2300    2300      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13/main.go
   5944    5944      0   0.00%    2292    2292      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13x/main.go
   6248    6248      0   0.00%    2300    2300      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd4in2/main.go
  26484   26484      0   0.00%   18816   18816      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/waveshare-epd/epd2in66b/main.go
   7096    7096      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/ws2812
   5510    5510      0   0.00%    8918    8918      0   0.00% '-xesppie' is not a recognized feature for this target (ignoring feature)
   1997    1997      0   0.00%     600     600      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino   ./examples/ws2812
   1488    1488      0   0.00%     182     182      0   0.00% tinygo build -size short -o ./build/test.hex -target=digispark ./examples/ws2812
  32068   32068      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bme280/main.go
  16392   16392      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/microphone/main.go
  11740   11740      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/buzzer/main.go
  12268   12268      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/veml6070/main.go
   6796    6796      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/simple/main.go
   8716    8716      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/speed/main.go
   6772    6772      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/simple/main.go
   9120    9120      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/speed/main.go
   7272    7272      0   0.00%    3308    3308      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-f103rb ./examples/shiftregister/main.go
   7032    7032      0   0.00%    2256    2256      0   0.00% '-xesppie' is not a recognized feature for this target (ignoring feature)
  13280   13280      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis2mdl/main.go
  11248   11248      0   0.00%    5104    5104      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/max72xx/main.go
  36252   36252      0   0.00%    6388    6388      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/pcf8523/
   7176    7176      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/clkout/
  12196   12196      0   0.00%    5708    5708      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/qmi8658c/main.go
  10876   10876      0   0.00%    5692    5692      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/pcf8591/
   8732    8732      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/ina260/main.go
  12176   12176      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/ina219/main.go
   9300    9300      0   0.00%    5232    5232      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-l432kc ./examples/aht20/main.go
  10160   10160      0   0.00%    7128    7128      0   0.00% tinygo build -size short -o ./build/test.elf -target=wioterminal ./examples/axp192/m5stack-core2-blinky/
   9020    9020      0   0.00%    5680    5680      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/xpt2046/main.go
  13128   13128      0   0.00%    4924    4924      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/sx126x/lora_rxtx/
  11300   11300      0   0.00%    6656    6656      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/irremote/main.go
  12292   12292      0   0.00%    5756    5756      0   0.00% tinygo build -size short -o ./build/test.hex -target=badger2040 ./examples/uc8151/main.go
  12296   12296      0   0.00%    6200    6200      0   0.00% tinygo build -size short -o ./build/test.hex -target=badger2040 ./examples/waveshare-epd/epd2in9v2/main.go
  10516   10516      0   0.00%    5752    5752      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/scd4x/main.go
   7984    7984      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=circuitplay-express ./examples/makeybutton/main.go
   9448    9448      0   0.00%    5104    5104      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ds18b20/main.go
  16104   16104      0   0.00%    7348    7348      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/as560x/main.go
   9912    9912      0   0.00%    5700    5700      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mpu6886/main.go
   7764    7764      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ttp229/main.go
   9364    9364      0   0.00%    5692    5692      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mpu9150/main.go
  11612   11612      0   0.00%    5728    5728      0   0.00% tinygo build -size short -o ./build/test.hex -target=macropad-rp2040 ./examples/sh1106/macropad_spi
   8508    8508      0   0.00%    6156    6156      0   0.00% tinygo build -size short -o ./build/test.hex -target=macropad-rp2040 ./examples/encoders/quadrature-interrupt
  13220   13220      0   0.00%    5688    5688      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/tmc5160/main.go
  12004   12004      0   0.00%    4900    4900      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=nicenano ./examples/sharpmem/main.go
  12856   12856      0   0.00%    5748    5748      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/ens160/main.go
  16108   16108      0   0.00%    5764    5764      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/si5351/main.go
  27444   27452      8   0.03%    3808    3808      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/microbitmatrix/main.go
  42192   42200      8   0.02%    8964    8964      0   0.00% tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/sx127x/lora_rxtx/
 154984  154992      8   0.01%    9168    9168      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-rp2040 -stack-size 8kb ./examples/net/mqttclient/natiu/
 118660  118668      8   0.01%   12192   12192      0   0.00% tinygo build -size short -o ./build/test.hex -target=elecrow-rp2350 -stack-size 8kb ./examples/net/ntpclient/
  42440   42452     12   0.03%    7536    7536      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/tmc2209/main.go
  31848   31864     16   0.05%    7200    7200      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/ssd1289/main.go
 173388  173428     40   0.02%   14168   14168      0   0.00% tinygo build -size short -o ./build/test.hex -target=elecrow-rp2040 -stack-size 8kb ./examples/net/tlsclient/
6363976 6352164 -11812  -0.00% 1040548 1040500    -48  -0.00%

@jakebailey jakebailey force-pushed the fix-3354 branch 2 times, most recently from f3ecfaf to 292f24f Compare May 1, 2026 12:33
@jakebailey
Copy link
Copy Markdown
Member Author

jakebailey commented May 1, 2026

One thing to note is that tinygo doesn't do the "indirect key/value over a cetain size" thing that BigGo does; that might also be good to do now, since the effect of this PR is to stop indirecting all non-simple keys, so large composite keys lead to large buckets.

jakebailey
jakebailey commented May 1, 2026
View reviewed changes
Comment thread tests/mapbench/mapbench_test.go
dgryski
dgryski requested changes May 1, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@dgryski dgryski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bunch of questions but really happy with this

Comment thread compiler/map.go Outdated
Comment thread compiler/map.go
Comment thread compiler/map.go
Comment thread compiler/map.go Outdated
Comment thread compiler/map.go Outdated
Comment thread compiler/map.go Outdated
Comment thread src/internal/reflectlite/value.go
@jakebailey jakebailey force-pushed the fix-3354 branch 3 times, most recently from fa036a4 to 6716279 Compare May 2, 2026 04:58
jakebailey
jakebailey commented May 2, 2026
View reviewed changes
Comment thread testdata/map_bigkey.go
dgryski
dgryski approved these changes May 4, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@dgryski dgryski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dkegel-fastly dkegel-fastly requested a review from Copilot May 5, 2026 17:08
Copilot AI reviewed May 5, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reworks map key handling across the compiler, runtime, interpreter, and reflection layers so maps can use type-specific hash/equality functions instead of falling back to interface-based hashing, while also adding coverage for pointer-key and large-key/value map cases.

Changes:

  • Switch compiler-generated map operations to use concrete key layouts plus generated hash/equality functions.
  • Extend the runtime hashmap to support generic hash/equality callbacks and indirect storage for keys/values larger than 128 bytes.
  • Add regression tests/benchmarks for reflect map keys, pointer-keyed literals/runtime maps, and large map keys/values.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
transform/maps.go Updates dead-map optimization to recognize generic hashmap constructors/setters.
tests/mapbench/mapbench_test.go Adds map performance benchmarks for string, composite, int, and large keys.
testdata/reflect.txt Updates expected reflect test output.
testdata/reflect.go Adds regression coverage for interface-typed map iteration keys.
testdata/map.txt Updates expected map runtime test output for pointer-keyed cases.
testdata/map.go Adds pointer and unsafe.Pointer map coverage, including package-level literals.
testdata/map_bigkey.txt Adds expected output for large key/value map tests.
testdata/map_bigkey.go Adds runtime tests for maps with >128-byte keys/values.
src/runtime/hashmap.go Implements generic hash/equality callbacks and indirect storage for large keys/values.
src/internal/reflectlite/value.go Reworks reflect map lookup/update/iteration to use generic hashmap paths.
main_test.go Includes the new large-key map test and skips it on AVR.
interp/memory.go Converts pointer-to-int interpreter failures into recoverable interpreter errors.
interp/interpreter.go Propagates recoverable interpreter errors after instruction execution.
interp/interp.go Stores recoverable interpreter errors on the runner.
compiler/symbol.go Adds optimizer attributes for generic hashmap runtime calls.
compiler/map.go Switches compiler map lowering to concrete-key storage and generated hash/equality functions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/internal/reflectlite/value.go
Comment thread src/internal/reflectlite/value.go
Comment thread src/internal/reflectlite/value.go
Comment thread src/internal/reflectlite/value.go
@jakebailey
Copy link
Copy Markdown
Member Author

jakebailey commented May 5, 2026

The copilot review is correct; there is still brokenness here that was not previously tested and I missed. I'll work on fixing it.

@jakebailey jakebailey requested a review from Copilot May 5, 2026 18:47
Copilot AI reviewed May 5, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/internal/reflectlite/value.go
Comment thread testdata/reflect.go
@jakebailey jakebailey requested a review from Copilot May 5, 2026 19:25
Copilot AI reviewed May 5, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread compiler/map.go
Comment thread compiler/map.go
Comment thread interp/interpreter.go
@jakebailey jakebailey requested a review from Copilot May 5, 2026 20:26
Copilot AI reviewed May 5, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/mapbench/mapbench_test.go Outdated
@jakebailey jakebailey requested a review from Copilot May 6, 2026 05:19
Copilot AI reviewed May 6, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Comment thread compiler/map.go
Comment thread compiler/map.go
@jakebailey
Copy link
Copy Markdown
Member Author

jakebailey commented May 6, 2026

Size difference with the dev branch:

Binary size difference 
 flash                          ram
 before   after   diff          before   after   diff
  28028   21420  -6608 -23.58%    7100    7052    -48  -0.68% tinygo build -size short -o ./build/test.hex -target=pico ./examples/w5500/main.go
  13220   12104  -1116  -8.44%    5688    5688      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/tmc5160/main.go
  23336   22480   -856  -3.67%    5744    5744      0   0.00% tinygo build -size short -o ./build/test.hex -target=metro-rp2350 ./examples/bno08x/i2c/main.go
  86864   86576   -288  -0.33%    7532    7532      0   0.00% tinygo build -size short -o ./build/test.hex -target=challenger-rp2040 ./examples/net/ntpclient/
  57612   57444   -168  -0.29%    3672    3672      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/sht3x/main.go
  57612   57444   -168  -0.29%    3672    3672      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/shtc3/main.go
  74056   73888   -168  -0.23%   11092   11108     16   0.14% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/sdcard/console/
  65908   65748   -160  -0.24%    6544    6544      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmi160/main.go
  57604   57444   -160  -0.28%    3680    3680      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/sht4x/main.go
  77244   77084   -160  -0.21%    6680    6680      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/dht/main.go
  71608   71448   -160  -0.22%    6688    6688      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/alarm/
  71440   71280   -160  -0.22%    6688    6688      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/timer/
  67464   67304   -160  -0.24%    7212    7212      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/ndir/main_ndir.go
  62228   62068   -160  -0.26%    3768    3768      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ndir/main_ndir.go
  65572   65412   -160  -0.24%    6600    6600      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ndir/main_ndir.go
  67136   66976   -160  -0.24%    7180    7180      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mcp9808/main.go
  67004   66848   -156  -0.23%    9356    9368     12   0.13% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/flash/console/qspi
  62024   61872   -152  -0.25%    6528    6528      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adt7410/main.go
  64580   64428   -152  -0.24%    6568    6568      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp280/main.go
 264552  264400   -152  -0.06%   47108   47108      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/slideshow
  72036   71884   -152  -0.21%    6584    6584      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp2515/main.go
  62948   62796   -152  -0.24%    6296    6296      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840 ./examples/is31fl3731/main.go
  71044   70892   -152  -0.21%    6688    6688      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/time/
  62696   62544   -152  -0.24%    6332    6332      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840 ./examples/max6675/main.go
  62264   62120   -144  -0.23%    8576    8576      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/i2csoft/adt7410/
  70612   70496   -116  -0.16%    7316    7328     12   0.16% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/flash/console/spi
  71676   71636    -40  -0.06%    3640    3640      0   0.00% tinygo build -size short -o ./build/test.hex -target=pinetime     ./examples/bma42x/main.go
  17112   17112      0   0.00%    6576    6576      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/adafruit4650
   8764    8764      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adxl345/main.go
  13636   13636      0   0.00%    7136    7136      0   0.00% tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/amg88xx
   8924    8924      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/main.go
  11372   11372      0   0.00%    6928    6928      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/apds9960/proximity/main.go
   9828    9828      0   0.00%    5108    5108      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/itsybitsy-m0/main.go
   7356    7356      0   0.00%    2296    2296      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/at24cx/main.go
   8008    8008      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bh1750/main.go
   7368    7368      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/blinkm/main.go
  27664   27664      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp180/main.go
  11816   11816      0   0.00%    5152    5152      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bmp388/main.go
   7712    7712      0   0.00%    3336    3336      0   0.00% tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/sram/main.go
  21616   21616      0   0.00%    3532    3532      0   0.00% tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/time/main.go
  29108   29108      0   0.00%    5316    5316      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/ds3231/alarms/main.go
  42980   42980      0   0.00%    5316    5316      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/ds3231/basic/main.go
   4400    4400      0   0.00%    2256    2256      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/easystepper/main.go
   7068    7068      0   0.00%    2260    2260      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/gc9a01/main.go
  67660   67660      0   0.00%    6704    6704      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/i2c/main.go
  68360   68360      0   0.00%    6852    6852      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/uart/main.go
   8420    8420      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/hcsr04/main.go
   5528    5528      0   0.00%    2256    2256      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/customchar/main.go
   5568    5568      0   0.00%    2256    2256      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/text/main.go
  10376   10376      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/hd44780i2c/main.go
  14248   14248      0   0.00%    6928    6928      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/hts221/main.go
  15980   15980      0   0.00%    2340    2340      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hub75/main.go
  10156   10156      0   0.00%    7264    7264      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/basic
  11144   11144      0   0.00%    5224    5224      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/basic
  29436   29436      0   0.00%   38424   38424      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/pyportal_boing
  10188   10188      0   0.00%    7256    7256      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/scroll
  11224   11224      0   0.00%    5216    5216      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/scroll
  10472   10472      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis3dh/main.go
  13632   13632      0   0.00%    6928    6928      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/lps22hb/main.go
  26188   26188      0   0.00%    2304    2304      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/lsm303agr/main.go
  27860   27860      0   0.00%    7168    7168      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/lsm303dlhc/main.go
  12312   12312      0   0.00%    5128    5128      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/lsm6ds3/main.go
  10020   10020      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mag3110/main.go
   8984    8984      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017/main.go
   9416    9416      0   0.00%    5128    5128      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017-multiple/main.go
   9440    9440      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp3008/main.go
   7464    7464      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mma8653/main.go
   7368    7368      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mpu6050/main.go
  12312   12312      0   0.00%    5756    5756      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/pca9685/main.go
   6244    6244      0   0.00%    3268    3268      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setbuffer/main.go
   4628    4628      0   0.00%    2260    2260      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setpixel/main.go
  10604   10604      0   0.00%    5732    5732      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/seesaw/soil-sensor
  11916   11916      0   0.00%    5740    5740      0   0.00% tinygo build -size short -o ./build/test.hex -target=qtpy-rp2040 ./examples/seesaw/rotary-encoder
   3113    3113      0   0.00%     560     560      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino ./examples/servo
  13896   13896      0   0.00%    5804    5804      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico     ./examples/sgp30
   7988    7988      0   0.00%    7128    7128      0   0.00% tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/shifter/main.go
  12344   12344      0   0.00%    8712    8712      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao-ble ./examples/ssd1306/
  11728   11728      0   0.00%    5684    5684      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao-rp2040 ./examples/ssd1306/
  11816   11816      0   0.00%    5684    5684      0   0.00% tinygo build -size short -o ./build/test.hex -target=thumby ./examples/ssd1306/
   5896    5896      0   0.00%    2260    2260      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1331/main.go
   6808    6808      0   0.00%    2260    2260      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7735/main.go
   6488    6488      0   0.00%    2260    2260      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7789/main.go
  16784   16784      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/thermistor/main.go
   9640    9640      0   0.00%    4880    4880      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-bluefruit ./examples/tone
  10136   10136      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/tm1637/main.go
  10924   10924      0   0.00%    5744    5744      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/touch/capacitive
   8816    8816      0   0.00%    7128    7128      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/fourwire/main.go
  12348   12348      0   0.00%    7324    7324      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/pyportal_touchpaint/main.go
  15772   15772      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl53l1x/main.go
  14328   14328      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl6180x/main.go
  24636   24636      0   0.00%   14076   14076      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840-sense ./examples/waveshare-epd/epd1in54/main.go
   6356    6356      0   0.00%    2300    2300      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13/main.go
   5944    5944      0   0.00%    2292    2292      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13x/main.go
   6248    6248      0   0.00%    2300    2300      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd4in2/main.go
  26484   26484      0   0.00%   18816   18816      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/waveshare-epd/epd2in66b/main.go
   7096    7096      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/ws2812
   5510    5510      0   0.00%    8918    8918      0   0.00% '-xesppie' is not a recognized feature for this target (ignoring feature)
   1997    1997      0   0.00%     600     600      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino   ./examples/ws2812
   1488    1488      0   0.00%     182     182      0   0.00% tinygo build -size short -o ./build/test.hex -target=digispark ./examples/ws2812
  32068   32068      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bme280/main.go
  16392   16392      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/microphone/main.go
  11740   11740      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/buzzer/main.go
  12268   12268      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/veml6070/main.go
   6796    6796      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/simple/main.go
   8716    8716      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/speed/main.go
   6772    6772      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/simple/main.go
   9120    9120      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/speed/main.go
   7272    7272      0   0.00%    3308    3308      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-f103rb ./examples/shiftregister/main.go
   7032    7032      0   0.00%    2256    2256      0   0.00% '-xesppie' is not a recognized feature for this target (ignoring feature)
  13280   13280      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis2mdl/main.go
  11248   11248      0   0.00%    5104    5104      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/max72xx/main.go
  36252   36252      0   0.00%    6388    6388      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/pcf8523/
   7176    7176      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/clkout/
  12196   12196      0   0.00%    5708    5708      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/qmi8658c/main.go
  10876   10876      0   0.00%    5692    5692      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/pcf8591/
   8732    8732      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/ina260/main.go
  12176   12176      0   0.00%    5120    5120      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/ina219/main.go
   9300    9300      0   0.00%    5232    5232      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-l432kc ./examples/aht20/main.go
  10160   10160      0   0.00%    7128    7128      0   0.00% tinygo build -size short -o ./build/test.elf -target=wioterminal ./examples/axp192/m5stack-core2-blinky/
   9020    9020      0   0.00%    5680    5680      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/xpt2046/main.go
  13128   13128      0   0.00%    4924    4924      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/sx126x/lora_rxtx/
  12292   12292      0   0.00%    5756    5756      0   0.00% tinygo build -size short -o ./build/test.hex -target=badger2040 ./examples/uc8151/main.go
  12296   12296      0   0.00%    6200    6200      0   0.00% tinygo build -size short -o ./build/test.hex -target=badger2040 ./examples/waveshare-epd/epd2in9v2/main.go
  10516   10516      0   0.00%    5752    5752      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/scd4x/main.go
   7984    7984      0   0.00%    5088    5088      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=circuitplay-express ./examples/makeybutton/main.go
   9448    9448      0   0.00%    5104    5104      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ds18b20/main.go
   9912    9912      0   0.00%    5700    5700      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mpu6886/main.go
   7764    7764      0   0.00%    5080    5080      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ttp229/main.go
   9364    9364      0   0.00%    5692    5692      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mpu9150/main.go
  11612   11612      0   0.00%    5728    5728      0   0.00% tinygo build -size short -o ./build/test.hex -target=macropad-rp2040 ./examples/sh1106/macropad_spi
   8508    8508      0   0.00%    6156    6156      0   0.00% tinygo build -size short -o ./build/test.hex -target=macropad-rp2040 ./examples/encoders/quadrature-interrupt
  12004   12004      0   0.00%    4900    4900      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=nicenano ./examples/sharpmem/main.go
  12856   12856      0   0.00%    5748    5748      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/ens160/main.go
  16108   16108      0   0.00%    5764    5764      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/si5351/main.go
 122768  122776      8   0.01%    8112    8112      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/lora/lorawan/atcmd/
 173388  173400     12   0.01%   14168   14300    132   0.93% tinygo build -size short -o ./build/test.hex -target=elecrow-rp2040 -stack-size 8kb ./examples/net/tlsclient/
  11300   11316     16   0.14%    6656    6672     16   0.24% tinygo build -size short -o ./build/test.hex -target=pico ./examples/irremote/main.go
 120600  120624     24   0.02%    8428    8428      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 -stack-size 8kb ./examples/net/tcpclient/
 167064  167096     32   0.02%   10400   10432     32   0.31% tinygo build -size short -o ./build/test.hex -target=arduino-mkrwifi1010 -stack-size 8kb ./examples/net/tlsclient/
 117100  117132     32   0.03%   13704   13704      0   0.00% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/webclient/
 103716  103756     40   0.04%   10408   10408      0   0.00% tinygo build -size short -o ./build/test.hex -target=metro-m4-airlift -stack-size 8kb ./examples/net/socket/
  76548   76596     48   0.06%    7788    7804     16   0.21% tinygo build -size short -o ./build/test.hex -target=p1am-100 ./examples/p1am/main.go
 154984  155096    112   0.07%    9168    9168      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-rp2040 -stack-size 8kb ./examples/net/mqttclient/natiu/
 118660  118792    132   0.11%   12192   12292    100   0.82% tinygo build -size short -o ./build/test.hex -target=elecrow-rp2350 -stack-size 8kb ./examples/net/ntpclient/
  16104   16244    140   0.87%    7348    7360     12   0.16% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/as560x/main.go
  27444   27596    152   0.55%    3808    3808      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/microbitmatrix/main.go
  42440   42596    156   0.37%    7536    7536      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/tmc2209/main.go
  31848   32008    160   0.50%    7200    7200      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/ssd1289/main.go
  27324   27508    184   0.67%    5852    5852      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit-v2 ./examples/microbitmatrix/main.go
  42192   42380    188   0.45%    8964    8964      0   0.00% tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/sx127x/lora_rxtx/
 353056  353300    244   0.07%   17212   17376    164   0.95% tinygo build -size short -o ./build/test.hex -target=pyportal -stack-size 8kb ./examples/net/http-get/
 396284  396532    248   0.06%   19248   19420    172   0.89% tinygo build -size short -o ./build/test.hex -target=matrixportal-m4 -stack-size 8kb ./examples/net/webstatic/
 344740  345084    344   0.10%   22308   22468    160   0.72% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/webserver/
 345916  346324    408   0.12%   22188   22440    252   1.14% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/mqttclient/paho/
 313284  313696    412   0.13%   16240   16456    216   1.33% tinygo build -size short -o ./build/test.hex -target=nano-rp2040 -stack-size 8kb ./examples/net/websocket/dial/
6363976 6354736  -9240  -0.00% 1040548 1041812   1264   0.00%

dgryski
dgryski approved these changes May 6, 2026
View reviewed changes
Copy link
Copy Markdown
Member

@dgryski dgryski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Any comments from @deadprogram or @aykevl

Comment thread compiler/map.go
@dgryski
Copy link
Copy Markdown
Member

dgryski commented May 11, 2026

My test corpus continues to pass on this branch.

@dgryski
Copy link
Copy Markdown
Member

dgryski commented May 13, 2026

@aykevl Did you want to review this more or are you happy to merge?

@deadprogram
Copy link
Copy Markdown
Member

deadprogram commented May 17, 2026

Last call for feedback, please!

@jakebailey do you wish to organize this PR into any specific commits, or should I be planning to just squash together on merge?

@jakebailey
Copy link
Copy Markdown
Member Author

jakebailey commented May 17, 2026 •
edited
Loading

I wouldn't mind a squash merge, but if you want me to try and reorganize I can do that when I have time (probably next weekend).

@jakebailey
compiler, runtime, reflect: generate type-specific hash/equal for com…
f0ee43c 
…posite map keys

For map keys that are not trivially binary-comparable, the compiler now
generates type-specific hash and equal functions as LLVM IR instead of
going through the interface+reflection path. This covers comparable
types: strings, floats, complex numbers, interfaces, channels, and
composites containing any mix of these.

Previously, maps with composite keys containing strings or floats
converted the key to interface{}, hashed via reflection, and compared
through interface equality. Now the compiler walks struct fields and
array elements directly, dispatching to the right runtime helper for
each field type and storing keys at their actual type.

Struct keys are always handled field-by-field so padding bytes do not
affect equality or hashing. Blank fields are ignored, matching Go
equality. Generated hash/equal function names use canonical underlying
type structure so structurally identical key types can share generated
functions. Padding zeroing before map operations is no longer needed
because structs no longer use the binary key path.

Also fix reflect map iteration for interface-keyed maps: MapIter.Key
returns an interface Value for map[interface{}] keys instead of
unpacking to the concrete key kind.
@jakebailey
Copy link
Copy Markdown
Member Author

jakebailey commented May 17, 2026

Rewrote the stack; things are now more reasonable, and each commit passes tests, though there are bugfixes and such in later commits with new tests, so it's possible that squashing is still a good idea anyway just to make bisecting better?

Up to you.

@jakebailey jakebailey requested a review from Copilot May 17, 2026 19:26
Copilot AI reviewed May 17, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

Comment thread compiler/map.go Outdated
@jakebailey jakebailey requested a review from Copilot May 17, 2026 19:36
Copilot AI reviewed May 17, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.

jakebailey added 5 commits May 17, 2026 13:33
@jakebailey
compiler: generate loops for array map key hash/equal
eca5b54 
Previously, array key hash and equal functions were unrolled at compile
time, generating one block of IR per element. For large arrays like
[1000]int inside a struct with non-binary fields, this caused code
explosion.

Now, binary-element arrays dispatch directly to hash32/memequal for the
whole array. Non-binary-element arrays generate an LLVM IR loop. The
equal loop short-circuits on the first mismatch.

Small arrays are still unrolled instead of looping, keeping the simple
cases compact.
@jakebailey
reflect: fix at-runtime map issues from review, and more found locally
c2b923b 
Maps created through reflect.MakeMap need hash/equal behavior that
matches compiler-created maps. Add hashmapMakeReflect for composite key
types, using runtime closures that reconstruct interface{} values from
raw key bytes and delegate to the interface hash and equality paths.

Interface-keyed maps are already stored as interface values, so use the
existing interface hash/equal helpers directly for those. This keeps
reflect insert, lookup, delete, and compiled lookup paths consistent.

Also fix addressable small values used as interface map keys or
interface map values. loadSmallValue puts small indirect values back in
the pointer-sized interface data field the same way valueInterfaceUnsafe
does.
@jakebailey
compiler, interp, reflect: fix pointer map literals; remove interface…
f977708 
… fallback

Package-level map literals with pointer keys (both *T and
unsafe.Pointer) crash the compiler: the interp pass panics when trying
to hash pointer data as raw bytes, because pointer values in the interp
memory model are symbolic identities that do not fit in a byte.

Fix this by setting a recoverable error flag instead of panicking. The
interp detects the error after each instruction and defers the map
insert to runtime init code, where real addresses are available for
hashing. This matches how the interp already handles other operations
it cannot evaluate at compile time.

With this fix, unsafe.Pointer can also be classified as a binary map
key, which was the last type requiring the interface-based fallback.
Since all comparable types now use either the binary or the
compiler-generated hash/equal path, remove the interface fallback from
the compiler and reflect packages.
@jakebailey
compiler, transform: always pass hash/equal function pointers to hash…
1e53fe3 
…mapMakeGeneric

The compiler now always resolves the hash and equal functions at compile
time and passes them directly to hashmapMakeGeneric, instead of passing
an algorithm enum to hashmapMake and resolving at runtime. For string
keys, the runtime hashmapStringPtrHash/hashmapStringEqual functions are
referenced directly. For binary keys, hash32/memequal are referenced.

The old hashmapMake with alg enum is retained for reflect, which still
needs runtime resolution when creating maps dynamically.

The OptimizeMaps transform pass is updated to handle both hashmapMake
and hashmapMakeGeneric, and to recognize hashmapGenericSet in addition
to hashmapBinarySet and hashmapStringSet. The now-unused
hashmapCanGenerateHashEqual helper is removed.
@jakebailey
runtime: store large map keys and values indirectly
21579fd 
When a map key or value exceeds 128 bytes, the bucket now stores a
pointer to separately allocated memory instead of the data inline. This
matches Go's MapMaxKeyBytes/MapMaxElemBytes threshold and prevents
bucket sizes from exploding for large key/value types.

For example, map[[256]byte]int previously used 2128 bytes per bucket
(16 header + 256*8 keys + 8*8 values); now it uses 144 bytes per bucket
(16 header + 8*8 pointers + 8*8 values).

The indirection is fully encapsulated in the runtime via helper
functions. Store the computed key and value slot sizes on the hashmap so
all runtime and reflect paths use the same bucket layout, including
non-indirect keys and values.

Add big-key golden coverage and benchmarks. Make the benchmark vary
enough key bytes to exercise hashing.
@jakebailey
Copy link
Copy Markdown
Member Author

jakebailey commented May 17, 2026

All green now.

@deadprogram
Copy link
Copy Markdown
Member

deadprogram commented May 18, 2026

Thank you very much for the improvements here @jakebailey and to @dgryski for getting it to land. Now squash/merging.

@deadprogram deadprogram merged commit 18033eb into tinygo-org:dev May 18, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@dgryski dgryski dgryski approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

map very slow with larger composite keys

4 participants

@jakebailey @dgryski @deadprogram