Test Strategy

Overview

truecalc is measured against a large corpus of expected values taken from Google Sheets itself — not from a specification, and not from truecalc's own output. Each expression is evaluated in Google Sheets, the result is recorded, and that recorded answer becomes the reference.

Those reference values are fixed. Every change to the engine is checked against all of them, and a change that breaks one cannot ship.

Why the source of the reference matters

An engine checked against its own output can only ever agree with itself. A test suite written by reading documentation inherits whatever its author assumed the documentation meant — and the places spreadsheet engines actually diverge are rarely the places documentation describes.

Rounding at the half, text-to-number coercion, date edge cases, error propagation: these are settled by behaviour, not by prose. Capturing that behaviour directly is the only way to test against it, and it is most of the work.

How to read the results

Results are grouped by function category. A category is either enforced — a regression blocks the change — or tracked, meaning differences are recorded and reported while the work is still in progress. A category becomes enforced once it is passing and stays that way.

Cases that do not yet match are counted and published rather than removed. A conformance number that excludes its own failures is not evidence of anything, so known differences stay in the total and are tracked as bugs.

A small number of expressions depend on the surrounding workbook rather than on the formula alone — =SHEETS(), for instance, returns a value that depends on how many sheets exist. Cases like these are recorded as known differences with the reason attached, rather than quietly excluded.

Reference integrity

Two rules protect the corpus from the failure mode that would make all of this meaningless:

• Reference values are never edited by hand. They are what Google Sheets returned.
• Reference data and engine changes travel separately, and CI enforces it. If both could change in one step, a regression could be made to "pass" by rewriting the expected value to match the wrong answer. Keeping them apart means the reference is something the engine must satisfy, not something it can adjust to suit itself.

Live conformance status

For a drill-down by category, see the conformance report.