avr/bootloader
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

pureboot: the unified autobaud loader, and the hang that settled the decision

Browse Source 
Hardware testing found that a lone calibration pulse wedged the autobaud loader:
run() budgeted only the start-edge wait in measure(), and the rx() that read the
knock behind it was unbudgeted, so one stray low pulse held an unattended device
in the loader and the application never ran. Bound the whole activation — an
expired knock budget returns a byte that cannot be the knock, so control falls
back into the budgeted measure() and an idle line boots the app there.

That fix costs ~22 B, which neither version under review could absorb: the pure
one goes 508 -> 530 on the 1284P and the register one 512 -> 534, both over a
512 B slot. Their margin was never spare capacity, it was the space the missing
fix should have occupied. So the choice between them is moot; both are kept for
the record and no longer built.

pureboot_autobaud_uni.cpp replaces them at 464 B. It is pureboot 5: one read
command and one write command over named spaces (G/g, sel8, addr16, n8) instead
of four per-memory bodies, which collapses four transfer loops into one. The
selector's high nibble carries flash's bank, so the shared cursor stays 16 bits
and no command speaks word addresses. Three things fall out of the freed space:
RAM read/write — the missing feature, and with it arbitrary I/O access, since
AVR maps peripherals into the data space; host-issued SPM, so W's hardcoded
erase/write/RWW tail becomes three writes to a space and any SPM operation is
reachable; and W on the same selector-and-address decode as everything else.

Strictly pure throughout: no inline asm, no global register variable, and no
GPIOR either — the unit lives in a .noinit static, so the loader claims no chip
resource and the chips without GPIOR stop being a special case.

pureboot.py speaks both generations, keyed on the version, so the fixed-baud
path is untouched; --peek/--poke reach the new data space. pbautobaud.py adds a
RAM round-trip and a regression for the hang: a lone pulse must still let the
app boot. All 37 chips plus the 12-preset reflect spot set build and size-test
green, 444-466 B, worst case 46 B under budget. Sim suites 100%: 1284P 17/17,
328P 23/23. Only real-hardware acceptance remains (pureboot/autobaud.md).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
BlackMark
2026-07-24 00:48:54 +02:00
parent 14efef96cc
commit 2cc540c6a1
8 changed files with 869 additions and 186 deletions
Download Patch File Download Diff File Expand all files Collapse all files

367
pureboot/autobaud.md
View File

@@ -1,6 +1,6 @@
# pureboot autobaud — findings and the version decision
# pureboot autobaud — findings, and how the version question settled itself
The autobaud variant measures the host's bit timing **at runtime** from a
The autobaud loader measures the host's bit timing **at runtime** from a
calibration pulse, so the image carries no clock: one clock-agnostic binary per
chip runs at any F_CPU and locks onto whatever baud the host sends. It exists
for the software-serial deployments — the RC-oscillator parts (the tinies,
@@ -8,205 +8,244 @@ internal-oscillator megas) whose exact clock is uncertain and drifts, so today
each needs a per-clock build. Autobaud erases that axis. That is the win —
deployment, not bytes.
This document records how the fit was established, why there are **two source
files under review**, and what remains before either can ship.
This document records how the fit was established, why the two versions that
were under review are both dead, and what the loader looks like now.
## The decision to make
## The decision, settled by measurement
Two complete autobaud loaders sit side by side for review. They implement the
identical wire protocol and the identical command set; they differ in exactly
two things, and the choice between them is a single tradeoff — **the running-slot
write guard vs. strict purity.**
Two autobaud loaders were built for review, differing in one tradeoff — the
running-slot write guard against strict purity. `pureboot_autobaud_pure.cpp`
landed at **508 B** on the 1284P (4 B spare) and `pureboot_autobaud_reg.cpp` at
**512** (zero spare).
| | `pureboot_autobaud_pure.cpp` | `pureboot_autobaud_reg.cpp` |
Then hardware testing found a defect that neither could absorb.
**A single spurious calibration pulse wedged the loader.** `run()` budgeted only
the start-edge wait inside `measure()`; the `rx()` that read the knock behind it
was unbudgeted and blocked forever. One stray low pulse on an unattended device
— EMI, or a host that opens the port and never knocks — held the loader in its
activation loop and the application never ran. On a field device that is a hang,
not a hiccup, and it is exactly the deployment autobaud is for.
The fix is to bound the whole activation: an expired knock budget returns a byte
that cannot be the knock, so control falls back into the budgeted `measure()`,
and a line that stays idle boots the application there. It costs about 22 bytes.
| 1284P, with the activation fix | size | 512 B budget |
|---|---|---|
| measured unit lives in | GPIOR I/O scratch regs (RAM where absent) | one global register variable (`r4`) |
| running-slot write guard | **dropped** | **kept** |
| purity (no asm, no GRV) | **yes** | one GRV — the single break |
| ATmega1284P size | **508 B** (4 B spare) | **512 B** (exact) |
| all 37 chips | 440–508 B | 452–512 B |
| `pureboot_autobaud_pure.cpp` | 530 | **over by 18** |
| `pureboot_autobaud_reg.cpp` | 534 | **over by 22** |
| `pureboot_autobaud_uni.cpp` | **464** | **48 B spare** |
Both also take two shared, licensed simplifications (below): a slimmed info
block and a single-byte activation knock.
Both candidates were unshippable, and the margin they were competing over was
never real — it was the space the missing fix should have occupied. So the
choice is not between them. It is the third loader below, which fits with room
to spare *and* carries features neither had. The two sources stay in the tree
for the record; only the unified one is built.
**The pure version** keeps pureboot's defining constraint — *"one C++ source, no
inline assembly, no global register variables"* (README.md) — intact, and pays
for it by dropping the write guard, which the host compensates for. **The
register version** keeps the guard by pinning the measured unit to a call-saved
register, which is only expressible as a global register variable — the one
thing the purity rule forbids.
## The unified loader
Recommendation deferred to the owner. The pure version aligns with the stated
constraints (purity mandated; relaxing host-guaranteed safety licensed); the
register version keeps every feature at the cost of the purity claim and leaves
the 1284 with zero margin.
The insight that paid was the one that had already paid once: **merging command
bodies removes cost that moving them around only redistributes.** Folding `R`,
`r` and `w` into a single address-and-count path had been worth 14 B earlier.
Pushed further — one read command and one write command over *named spaces* —
it is worth far more, because four transfer loops collapse into one.
## The mechanism
`pureboot_autobaud_uni.cpp` is pureboot 5. It is strictly pure: no inline
assembly, no global register variable, and **no GPIOR either** — the measured
unit lives in a plain static, so the loader claims no chip resource an
application might want, and the GPIOR-versus-static question disappears along
with the chips that have no GPIOR.
The host sends calibration byte **0xC0** — a start bit plus six zero data bits
form a single low pulse of seven bit-times. The loader counts that pulse in a
poll loop the compiler emits at **seven cycles an iteration**, so the count is
the pulse length in cycles ÷ 7 × 7 = one bit period in cycles, and `count >> 2`
is that period in `_delay_loop_2`'s four-cycle iterations — the delay unit fed
to every rx/tx bit. Numerically sound: the per-bit delay tracks the true period
to **< 0.2 %** from 1–16 MHz at 9600 (worst seen −2.8 % at 128 kHz/1200, still
inside UART tolerance).
### The protocol
**The catch — codegen coupling.** `count >> 2` is exact only because the
calibration pulse's bit-count (7) equals the poll loop's cycles/iter (7). That
couples the on-wire calibration byte to what the compiler emits; a toolchain bump
that reshapes the loop breaks it silently. This must be pinned by a sim timing
test that fails on drift (see *What remains*).
| command | arguments | |
|---|---|---|
| `b` | — | version, then the three signature bytes |
| `J` | addr16 | ack, then jump (word address) |
| `W` | sel8, addr16, page bytes | fill the flash page buffer |
| `G` | sel8, addr16, n8 | read n bytes (0 means 256) |
| `g` | sel8, addr16, n8, then n bytes | write, each byte acked |
The activation window is a **fixed poll budget** (`PUREBOOT_AUTOBAUD_POLLS`,
default 4,000,000), not `timeout × clock` — with no clock, whole seconds cannot
be timed. A `__uint24` holds it; a `std::uint32_t`'s fourth byte would cost two
words at each countdown step.
`sel` is `space | bank << 4`. The low nibble names the space; the high nibble is
flash's third address byte, so every transfer speaks a **byte** address inside a
64 KiB bank and no command has to carry word addresses. The host must not span a
bank boundary in one transfer — it already chunks by page, so nothing it does
comes close.
## How the target was found — the hand-assembly floor
| space | | |
|---|---|---|
| 0 | flash | `lpm`/`elpm` |
| 1 | EEPROM | |
| 2 | data | SRAM — and with it the register file and every I/O register, which share the data address space on AVR |
| 3 | fuse and lock | |
| 4 | SPM | write-only: the data byte goes to SPMCSR and fires the instruction at the selected address |
Pure C++ first came out **560 B** on the 1284 (full info + guard). To learn
whether that was a hard floor or C++ overhead, a hand-optimal assembly loader was
written with the **full v4 protocol** (all commands, the full 12-byte info block,
the write guard, position independence) plus autobaud — `local/scratch/autobaud/`
in the libavr checkout, off-tree, a size probe.
Three things follow from that table that the loader never had:
**The hand-asm floor is 506 B** — it fits 512 with the *full* info block *and*
the guard, no concessions. That proved a fit was possible and gave a target. The
hand-asm achieves it two ways pure C++ cannot express directly:
- **RAM read and write**, the missing feature. In a per-command design it would
have cost a fresh dispatch arm and a fresh loop, ~30 B, on a loader with 4 B
spare. As one more space on a shared loop it is a single `ld`/`st`. It also
hands the host arbitrary **I/O register access** for free, because AVR maps
the peripherals into the same address space.
- **Host-driven SPM.** `W` used to end with a hardcoded erase, write and RWW
re-enable — 42 B. Those are now three writes to the SPM space, reusing the
store path's own address, data byte and ack. The host pays three extra
round-trips per page (18 wire bytes against 256 of data) and gains the ability
to issue *any* SPM operation, lock bits included.
- **`W` on the same footing as everything else.** It takes the same selector and
the same byte address instead of a word address of its own, which made flash
addressing uniform across the protocol *and* was 20 B cheaper than keeping its
private convention.
1. **The measured unit in a call-saved register (Y).** rx/tx/spin read it
directly — no argument, no `movw`. In C++ an *outlined* rx/tx can only receive
the unit as a parameter (a `movw` at each of ~19 call sites, +38 B) or read it
from RAM (an `lds`), unless it is a global register variable. That register is
worth ~32 B and is exactly the impurity at issue.
2. **A cheap PC-relative slot anchor** (`rcall .+0; pop; pop`, ~10 B) where the
C++ `__builtin_return_address(0)` costs ~18–26 B (GCC re-derives the stack
slot and byteswaps).
Read and write are `G` and `g` — the same letter, one bit apart — so the
transfer loop picks its direction with a one-word skip rather than a compare.
Measured on identical C++ (the same loader, unit in a register vs. RAM):
### Why the SPM space has to be one primitive
| unit home | 1284 size |
It is tempting to go further and expose a generic "poke this I/O register", from
which the host could drive SPM itself. The hardware forbids it: `out SPMCSR, x`
and the `spm` that follows must issue within four cycles, and EEPROM's
EEMPE→EEPE window is the same shape. A host cannot hit a four-cycle window
across a serial link. **Atomicity is the floor, and the atomic unit must be
resident.** That is the real limit on how low-level a bootloader's primitives
can go — not the byte count.
## Where the bytes went
The starting point was the 508 B pure build, disassembled and attributed:
| phase | bytes |
|---|---|
| global register variable | 528 |
| RAM static | 560 |
| hand-asm (register + cheap anchor) | 506 |
| `link::rx` + `link::tx` (bit-banged UART) | 102 |
| autobaud measure + knock | 62 |
| reset vector, pin init, WDRF check, jump, ack, EEPROM wait | 50 |
| command loop head + dispatch tree | 42 |
| `W` program flash page | 98 |
| `R` / `r` / `w` / `F` bodies, plus the shared address decode | 122 |
| `b` info, `J` jump | 32 |
So even *with* a GRV, C++ carries ~22 B over hand-asm; with RAM, ~54 B. The
compiler flag axis was spent (the tuned `-f` set is optimal; `-fipa-ra`,
`-fipa-icf`, `-ftree-tail-merge`, `-flto`, `-fwhole-program` all gave nothing —
GCC's AVR ABI passes arguments in fixed registers regardless).
**208 B — 41% — is physical layer and activation**, which no protocol change can
touch. The command bodies were the entire addressable surface, and they were
four copies of one idea.
## The pure lever — GPIOR
The route from a first attempt to the final loader, all on the 1284P:
The register cost is the RAM home's `lds`/`sts` (two words each) plus `.bss` and
its `__do_clear_bss`. The general-purpose I/O scratch registers (**GPIOR1:GPIOR2**,
adjacent) are reached by `in`/`out` — one word — through libavr's named register
surface: **no inline asm, no global register variable, fully pure.** Storing the
unit there sidesteps `.bss` and halves each access. Where a chip has no GPIOR
(the t13, m8, m16/32) the pure version falls back to a plain static; those chips
have ample headroom.
| step | size |
|---|---|
| unified `G`/`P`, `load`/`store` outlined, `__uint24` cursor | 600 |
| …`load`/`store` inlined; unit in `.noinit`, not `.bss` | 534 |
| …16-bit cursor with the bank in the selector; direction as a command bit | 510 |
| …erase/write/RWW moved out to the SPM space | 484 |
| …`W` sharing the selector-and-address decode | **464** |
GPIOR alone did not close the gap (the write guard's `slot_high` machinery is the
bulk). The pure version fits by combining GPIOR storage with the licensed
simplifications; the register version fits by pinning the unit and so affording
the guard.
Three of those steps are worth keeping as lessons:
### The ablation (1284, all pure except the GRV row)
- **Outlining `load`/`store` cost more than the four bodies they replaced.** As
functions they were 110 B against the 106 B of inline bodies — the AVR ABI's
argument marshalling plus prologue ate the entire saving. Inlined into the one
shared loop they cost only their own instructions. The call-site lesson cuts
both ways: *merging* call sites pays, *creating* one does not.
- **A `.bss` static drags in `__do_clear_bss`** — 18 B of startup code to zero a
variable that is always measured before it is read. `.noinit` is correct here
and free.
- **A three-byte cursor taxes every space.** Widening the shared cursor so flash
could reach past 64 KiB put an extra increment on EEPROM and RAM reads that
never need it. Moving the bank into the selector byte kept the cursor at
sixteen bits and cost nothing on the wire.
| config | GPIOR unit | GRV unit |
|---|---|---|
| full info + guard + 2-knock | 550 | 528 |
| slim info + guard + 2-knock | 540 | 518 |
| slim info + guard + 1-knock | — | **512** ✓ |
| slim info + no guard + 2-knock | 514 | — |
| slim info + no guard + 1-knock | **508** ✓ | — |
### What did not work
The pivotal cost is the write guard's `slot_high` (`__builtin_return_address`,
~26 B), removable only when the info block is slim (so nothing else needs
`slot_high`) *and* the guard is gone. That is why the pure version drops the
guard and the register version keeps it.
- **Encoding the space in the command byte** (so dispatch becomes masking rather
than a compare tree) cannot carry the bank's four bits alongside a space. The
cheap half of the idea survived as the direction bit; the rest lost to the
selector byte, which is also more extensible.
- **A generic primitive interpreter** — a loader with no logic at all, driven
entirely by the host — is not reachable on AVR. Harvard architecture means the
program counter cannot fetch from data space, so the classic "upload a flash
algorithm into RAM and jump to it" bootstrap is impossible, and on every
boot-sectioned part SPM only takes effect from the boot section anyway. What
remains is a fixed primitive set: still a protocol, still logic, only at a
different granularity. debugWIRE reaches that design point only because its
interpreter is *in silicon*; it costs the loader nothing because it is not in
the loader.
- Below 512 B the saved bytes are largely unspendable on the boot-sectioned
chips: the 328P's smallest boot section is exactly 512 B, and the 1284P's is
1024 B, of which pureboot already occupies only the top half. The margin
matters as headroom for correctness fixes — as this defect showed — not as
flash returned to the application. On the patch-vector parts, which have no
boot section, it *is* returned: on the ATtiny13 the loader is 43% of a 1 KiB
part, and every byte is real.
## Sizes — every chip, both versions
## The codegen coupling, still load-bearing
Both build and size-test green on all 37 (patched-vector budget 510, else 512):
`count >> 2` is exact only because the calibration pulse's bit-count (7, from
the 0xC0 byte) equals the poll loop's cycles per iteration (7 — `sbis` 1,
`rjmp` 2, `adiw` 2, `rjmp` 2). The loop shape survived every restructuring here,
verified in the disassembly, but a toolchain bump that reshapes it would break
the lock silently. `test/pbautobaud.py` is what pins it: a wrong unit fails the
flash verify.
| chip class | pure | register | budget |
## Sizes — every chip
Budget 510 B on the patch-vector parts, 512 elsewhere. The 1284P is no longer
the tight one: the bank nibble made far flash *cheaper* than the near-flash
arithmetic it replaced.
| size | chips | budget | spare |
|---|---|---|---|
| ATtiny13/13A | 446 | 452 | 510 |
| ATtiny25/45/85 | 440–444 | 456–460 | 510 |
| ATmega8/8A | 472 | 476 | 512 |
| ATmega16/32 | 476 | 478 | 512 |
| ATmega48 family | 440 | 456 | 510 |
| ATmega88/168/328 | 462–466 | 476–478 | 512 |
| ATmega164/324/644 | 466 | 478 | 512 |
| **ATmega1284/P** | **508** | **512** | 512 |
| 444 | ATtiny13, 13A | 510 | 66 |
| 448 | ATmega48, 48A, 48P, 48PA; ATtiny25 | 510 | 62 |
| 452 | ATtiny45, 85 | 510 | 58 |
| 460 | ATmega644, 644A, 644P, 644PA | 512 | 52 |
| 464 | **ATmega1284, 1284P**; ATmega8, 8A, 88, 88A, 88P, 88PA | 512 | 48 |
| 466 | ATmega16, 16A, 32, 32A; 164A/P/PA, 168/A/P/PA, 324A/P/PA, 328, 328P | 512 | 46 |
The 1284 is the tight one — the far-flash machinery (ELPM/RAMPZ, word-addressed
wire) it alone carries. The `pureboot_autobaud_*.size` tests run per chip in the
port's build.
All 37 chips build and size-test green, plus the 12-preset reflect spot set
(guidance rule 4 — the reflect matrix is never run in full), which matches its
generated counterpart byte for byte on every chip in the set. Worst case across
the whole set is **466 B, 46 under budget**.
## The shared simplifications, and why each is licensed
## Host tool and simulation
- **Slimmed info block.** `'b'` returns the version and the three signature bytes
— the loader's identity and the chip's — as immediates, instead of the full
12-byte block read from flash. The host derives page size, loader base, EEPROM
size and the addressing flags from the signature via its own chip database.
This drops the flash-resident table and, with the guard also gone in the pure
version, the entire `slot_high` anchor. Owner-approved: "keep version + chip
type, simplify the rest; the host derives geometry."
- **Single-byte knock.** One `'p'` activates; the calibration pulse has already
proven a host is present. (Fixed-baud pureboot needs `p`+`b` because it has no
prior proof.)
- **No running-slot write guard** *(pure version only)*. The guard stops a
*buggy* host from programming the loader's own slot; the host is guaranteed
never to do so (it knows the loader's location). Licensed by "the host
guarantees safety" (README.md). Self-update still works — it is the safety net
against host bugs that is lost, not a functional path.
## Host tool and simulation — done
Both are wired and green, for **both** variants.
- **Host tool.** `pureboot.py --autobaud` sends the 0xC0 calibration pulse and a
single knock at the host's chosen baud, then reads the slimmed info block and
**derives the full geometry from the signature** (`AUTOBAUD_GEOMETRY`, a table
over every chip pureboot targets). The rest of the tool — flash, EEPROM, fuses,
hand-over, verify — is unchanged: once connected, the derived `Info` presents
the same geometry the fixed-baud path reads off the wire. The pure version's
dropped write guard is host-transparent (the tool already never targets the
running slot).
- **Sim test** (`test/pbautobaud.py`, `pureboot.autobaud_pure` /
`pureboot.autobaud_reg`). Drives each variant over the GPIO⇄pty software-UART
- **`pureboot.py`** speaks both generations. `Info.version >= 5` selects the
unified path; everything below it keeps the four-command protocol, so the
fixed-baud loader is untouched. `--autobaud` sends the 0xC0 pulse and one
knock, then derives full geometry from the signature. New: `--peek ADDR[:N]`
and `--poke ADDR:HEX` reach the data space.
- **`test/pbautobaud.py`** drives the loader over the GPIO⇄pty software-UART
bridge through the calibration handshake, a flash + EEPROM + fuse round-trip
cross-checked against the simulator's ground-truth memory, and a hand-over to
the fixture application — then **repeats at double the F_CPU with the same
binary**, which is the clock-agnostic property autobaud exists for. Run on the
near-flash 328P and the word-addressed 1284P (both flash-addressing classes).
Because the lock fails a flash/verify if the measured unit is wrong, this test
also **pins the codegen-coupled calibration constant** (0xC0 against the
7-cycle poll loop): a toolchain bump that reshaped the loop would fail it.
cross-checked against the simulator's own memory, a RAM read/write round-trip,
and a hand-over to the fixture application — then repeats at double the F_CPU
with the same binary, which is the clock-agnostic property autobaud exists
for. Run on the near-flash 328P and the word-addressed 1284P.
- It also **pins the activation hang**: the test sends a lone calibration pulse
with no knock behind it and requires the application to boot. Against the
unfixed loader that assertion never returns.
## What remains
The decider is answered (pure C++ fits 512 on every chip) and both variants pass
in simulation. One item remains before shipping the chosen variant:
- **Real-hardware acceptance.** Autobaud exists for what a cycle-exact simulator
cannot produce: a real RC oscillator at ±10 % with drift and jitter. simavr
proves the arithmetic and the fit at *exact* clocks; only silicon proves the
feature does its job. Drive an internal-oscillator ATtiny from the host at a
fixed baud; confirm lock plus a full flash + verify (Windows environment,
hardware present — confirm first). Deferred until the variant is chosen.
- **Real-hardware acceptance.** A cycle-exact simulator cannot produce what
autobaud exists for: a real RC oscillator at ±10% with drift and jitter.
simavr proves the arithmetic and the fit at exact clocks; only silicon proves
the feature. Drive an internal-oscillator ATtiny at a fixed host baud and
confirm lock plus a full flash and verify.
- **A generic `spm::command()` in libavr.** The SPM space issues a runtime
command through `spm::detail::page_command` where RAMPZ exists, and falls back
to a dispatch over the known operations where it does not — the one
preprocessor branch in the file. A two-line library addition would make it
uniform and save a few bytes on the 36 non-RAMPZ chips, none of which are
tight.
- **Retire or revive the two dead variants.** They are kept only as the record
of the measurement; nothing builds them.
## Files
- `pureboot_autobaud_pure.cpp` — the pure version (GPIOR/RAM unit, no guard).
- `pureboot_autobaud_reg.cpp` — the register version (GRV unit, guard kept).
- `pureboot.py` — `--autobaud`: the calibration handshake, slim info, and the
signature→geometry table both variants rely on.
- `test/pbautobaud.py` — the end-to-end sim test; `CMakeLists.txt` runs it for
both variants on the 328P and 1284P, and `pureboot_add_autobaud()` size-tests
both on every chip.
- `pureboot_autobaud_uni.cpp` — the loader. pureboot 5.
- `pureboot_autobaud_pure.cpp`, `pureboot_autobaud_reg.cpp` — superseded, not
built; 530 and 534 B on the 1284P once the activation hang is fixed.
- `pureboot.py` — `--autobaud`, the unified transfer path, `--peek`/`--poke`.
- `test/pbautobaud.py` — the end-to-end sim test and the hang regression.
- `local/scratch/autobaud/floor_1284.S` (libavr checkout) — the hand-asm floor
probe, off-tree and gitignored; not sim-verified, a size reference only.
probe at 506 B, off-tree and gitignored; a size reference only. The unified
loader is 42 B under it, with features the probe never had.