Validation

Current baseline

Plan 11 step 1, 2026-10-03, commit b1a471f, Windows x64: clang-cl 23.1.2, LLVM SDK 23.1.2 (/MT, _ITERATOR_DEBUG_LEVEL=0), Lizard 1.24.0, clang-tidy 22.1.8, OTP pin 21776803ecd1 with oracle OTP 29.1.1 / ERTS 17.1.

Running the gate

From an x64 Visual Studio developer shell with C:\Program Files\LLVM\bin on PATH (automatic SDK selection reuses thirdparty/):

cmake --preset debug --fresh -DBUILD_TESTING=ON -G Ninja -DCMAKE_C_COMPILER=clang-cl -DCMAKE_CXX_COMPILER=clang-cl
cmake --build build/debug
ctest --preset debug-fast                  # during development
ctest --preset debug -j 16                 # full mode, at feature completion
cmake --build build/debug --target check-quality      # changed files + dependents
cmake --build build/debug --target check-quality-all  # whole tree

• CLAUSE_TEST_MODE=fast runs golden corpora at O0 positional plus O2 specialization-off project, runs mutations once and skips full_only tests. Full mode (default) runs all driver/policy combinations.
• check-quality, make format and make-format.bat cover files changed since HEAD plus untracked files; cmake/quality_scope.py adds translation units that include a changed header. Changes to .clang-tidy, cmake/ or production CMake select everything.
• Test programs carry no debug information (no PDBs); configure with -DCLAUSE_TEST_DEBUG_INFO=ON to rebuild them for a debugger. Nested native consumers link the parent build's runtime through ClauseRuntimeTargets.cmake instead of compiling it again; only runtime_link builds the runtime standalone.
• Thresholds and suppressions are never raised to pass the gate.

Fixtures and provenance

• Normal builds and tests need neither OTP nor its source checkout. Goldens were generated once from OTP and are committed with hashes; hash checks run before any fixture is used.
• Nineteen pattern/guard corpora hold 67,748 native expected values/errors and 106 semantic acceptance rows. Each native corpus runs positional and project drivers at O0/O2 with specialization on/off, local and remote calls.
• Committed Erlang inputs are locally authored (tests/fixtures/patternmatch/fragments/, preprocessor semantic/headers/). A 2026-10-03 audit removed all copied OTP files; 60-token window comparison against 4,150 OTP Erlang and 1,190 C/C++ files found no remaining overlap besides a generated integer tuple. fixture_sources enforces isolation.
• Six end-goal program fixtures (tests/fixtures/programs/, plan 11 step 2) hold OTP stdout/exit-status goldens and today's compile diagnostics; programs_compile checks them OTP-free and tests/compiler/programs/regenerate.py --check reproduces them under OTP (fixture map).
• Term printing goldens (tests/fixtures/printing/, plan 11 step 4) hold OTP ~w and erlang:display/1 text for 9,542 values plus OTP stdout of compiled display calls; tests/compiler/printing/regenerate.py --check reproduces them (fixture notes).
• Executable golden cases (tests/fixtures/executables/, plan 11 step 8) are source directories plus one golden.json with OTP stdout/exit status and an authored stderr pattern; tests/compiler/executables/run.py links and runs each under the policy/driver matrix (CTest executables_<case>), and regenerate.py --check reproduces them under OTP (case notes).
• Regeneration and live audits are explicit: -DCLAUSE_OTP_AUDITS=ON and tests/compiler/patternmatch/regenerate.py (instructions). Nothing refreshes goldens or the pin silently.

Test design

Behavior is tested through the real CLI, emitted objects, linked native consumers and OTP goldens. Focused unit tests remain only where source cannot reach the state, each with its purpose stated in the test:

• Injected budgets and allocation/IO faults (project_limits, project_creation_failure, codegen_limits, codegen_write_failure, runtime_lifecycle_failure, generated-call fault seams).
• Private ownership, invalid/stale handle and rollback invariants in the parser, semantic type graph and backend.
• Raw word validation and 32/64-bit term boundaries (runtime_immediate, runtime_term_tag, abi_integers).
• Diagnostic sink failures and the stable feature ID snapshot.

Filesystem capability cases (links, case aliases) report per-case skips; they never stand in for a whole-test pass.

Platform and sanitizer status

• Native generated-code execution: Windows x64 only.
• Linux, Apple Silicon and native 32-bit execution: pending (objects are only inspected).
• Compiler/frontend ASan, UBSan and LeakSanitizer: pending. The prebuilt Windows LLVM SDK conflicts with instrumented code (annotate_string 0 vs 1; earlier also duplicate rpmalloc/ASan allocator symbols). No check was disabled to bypass it.
• Runtime-only ASan passes on Windows with Release probes, /EHsc /fsanitize=address, /MT, Clang's ASan import library and static runtime thunk, and the ASan DLL on PATH.
• Historical macOS arm64 runs (2026-09-19/20) passed full Debug, C++26, ASan+UBSan, compiler-only and runtime-only builds for the parser and project stages; they predate the backend.

History

Condensed from the former per-step records (originals in Git history up to commit 2777c98). Unless noted: Windows x64, LLVM 23.1.2, pin 21776803ecd1, oracle OTP 29.1.1 / ERTS 17.1. Test counts are full CTest passes with zero skips.

PG = pattern/guard plan step (archived in .agents/00-finished.md).