A green build is not a verification
Six apps passed type checking, bundling and compilation. Four had no working core feature and all six crashed before their first frame. Here is the gate we use instead.
A previous pass over our mobile portfolio reported all six apps "production ready". Every check had passed: tsc --noEmit was clean, expo export resolved the JS graph, xcodebuild produced a signed archive, and each app installed on a simulator without complaint.
Four of the six had no working core feature. All six crashed before rendering their first frame.
What each check actually proves
The failure was not that the checks were broken. Each one did exactly what it claims to do — the mistake was reading them as evidence of something further up the ladder.
- Type checking proves the types agree with each other. It says nothing about whether the values at runtime match those types, and nothing at all about behaviour.
- Bundling proves every import resolves. A module that throws on evaluation bundles perfectly.
- Compiling and signing proves the native project is well-formed. An app whose first screen dereferences undefined compiles and signs without a warning.
- Installing proves the package is valid for the device. It does not launch it.
Everything up to this point can pass on software that is dead on arrival, because none of it runs the app.
The two rungs that catch real defects
Two more steps change the picture entirely:
xcrun simctl launch <udid> <bundleId> # it reaches first frame
maestro test .maestro/<feature>-flow.yaml # the feature actually worksThe first catches the whole class of crash-before-first-frame bugs — in our case a missing scene configuration that broke every app on the newest OS, invisible to every check above it. The second drives the actual feature against known input.
Assert on the artefact, not the UI
Even an end-to-end run can lie if it only asserts that a success screen appeared. So the assertion is made against the thing the app produced:
- For the redaction tool: run an independent OCR pass over the exported image and fail unless it recovers none of four planted secrets.
- For the video splitter: measure the durations of the files in the camera roll and require 30.00, 30.00, 30.00 and 5.00 seconds from a 95-second source.
- For the transcriber: diff the produced transcript against a reference and require complete word recall.
Each of these has caught something a passing UI flow did not.
The rule that falls out of it
Unverified is UNKNOWN, never a pass. If a feature has not been exercised against real input with the result inspected, it is not done — regardless of how many checks are green. That rule costs time up front and it is the only reason we can say anything about our own software with a straight face.