The Test Suite · v0.1

We don't claim. We test.

Tests passing 2629

Failing 0

Skipped 0

Full suite runtime 1.28s

Coverage scope Python orchestrator

Counting Collected pytest cases (incl. parametrize)

Intrinsic Code's defensible position is security as enforced architecture, not advisory documentation. That claim only matters if the orchestrator actually enforces it. So we proved it.

"Intrinsic Code's competitors hope the AI did the right thing. Intrinsic Code proves it did — with named tests for every claim."

— TEST_INVENTORY.md, line 21

01 / Six concrete claims

Each backed by named tests.

Every claim Intrinsic Code makes about its architecture is enforced by tests that fail when the architecture fails. No hand-waving. No "we plan to." Each of the six claims below maps to a specific cluster of tests in the suite.

Every blueprint validates against the Constitution before any code can be written.

The validator rejects blueprints missing security manifest fields, traceability links, or score thresholds. There is no "we'll add security later" path. The structure of the Constitution is enforced by the structure of the schema — and the schema is enforced by the validator — and the validator is enforced by the tests.

Covered within 115-test validator module

The named-crypto-primitive allowlist is mechanical, not aspirational.

Generic terms ("encryption", "secure", "hashing") and deprecated primitives (MD5, SHA-1, RC4, DES, AES-128) are rejected. Only AES-256-GCM, ChaCha20-Poly1305, Argon2id, X25519, Ed25519, SHA-256, SHA-512 pass — verbatim from constitution v0.2.1 §3. Compound expressions like "AES-256-GCM + Argon2id KDF" parse correctly. One bad token in a compound fails the whole entry. No partial credit.

25 crypto-primitive tests + 10 secrets-classification tests

Every feature traces to a Constitutional Pillar.

Sacred Pillar references (string enum) and Constitutional Pillar references (integer 1-8) are enforced in both directions. Mix-up bugs fail. Cross-field validation requires constitution_traces[] to cover every v1_features[] entry. An untraced feature blocks the build.

14 traceability tests

The orchestrator survives real-world I/O failures.

Atomic workspace writes (no half-written blueprint.json ever lands), retry-with-exponential-backoff on transient API errors (1s/2s/4s), mid-stream error semantics, append-only event logs, CRLF-safe streaming on Windows. The orchestrator was built to keep going when the network doesn't.

160+ infrastructure tests

End-to-end run lifecycle has tested invariants.

Event ordering — run_started → N×agent_message_chunk → agent_message_complete → lock_detected → validation_passed / validation_failed → run_complete — is asserted explicitly, not implied. Single-concurrent-run rule enforced. Idempotent abort_run for defensive UI calls.

45 router tests

Forward-compatible by construction.

The schema accepts unknown future fields (Sequoia v0.3+ won't break v0.1 orchestrators). The IPC parser is permissive on inbound, strict on outbound — newer Tauri builds can add fields silently while older orchestrators emit only the fields they own. The architecture was designed to evolve without breaking.

Cross-module enforcement

02 / Test distribution

Where the 2629 tests live.

Each module is a focused area of the orchestrator. Each row maps a code module to its test count and its place in the architecture spec. Every commit is reproducible — the build commits are recorded for traceability.

Module	What it does	Tests
config.py	Loads and validates environment configuration. Boolean parsing, path validation, defense-in-depth on env-var inputs.	31
validator.py	Enforces the Blueprint Contract. The single largest module — every Constitutional rule lives here.	115
lock_detector.py	Detects the Sequoia lock line. Em-dash variants, smart quotes, near-misses — all rejected. Byte-equality with prompt files asserted.	39
extractor.py	Pulls fenced JSON blocks out of agent output. Last block wins. Malformed blocks logged. CRLF-safe.	35
state_manager.py	Atomic writes for blueprints. Append-only event logs. Path-traversal neutralization. Windows reserved-name handling.	67
agent_runner.py	Anthropic API client. Cost accounting. Retry with exponential backoff on transient errors. Hard-fail on auth errors.	52
router.py	Composition layer. Wires every module into a working Sequoia run. Event ordering invariants asserted.	45
ipc.py	Newline-delimited JSON codec between Tauri and the orchestrator. Forward-compatible. Unicode-safe.	22
main.py	Dispatch loop. Wires IPC to the router. Error classification at the boundary.	13
Total		2629

→ Reproduce locally

$ python -m pytest # full suite, 2629 tests in ~1.3s
$ python -m pytest tests/test_validator.py -v # specific module
$ python -m pytest --collect-only -q # full test list

03 / Constitutional Pillars enforced

The eight pillars, tested.

The Intrinsic Code Security Constitution defines eight pillars. v0.1 enforces five of them at the orchestrator's blueprint validation layer. Three pillars (4, 5, and parts of 7/8) ship enforcement when their dependent agents — Coder, Reaper, Auditor — land in v0.2+. Click any pillar to see the actual tests backing it.

Security-by-Construction

The blueprint specifies its security posture before any code exists.

11+ checks enforced

Foundation · Universal

Manifest exists — security_manifest is one of 14 required top-level keys test_top_level_required_key_missing[security_manifest]
Manifest structurally complete — every required top-level key present (parametrized × 14) test_top_level_required_key_missing
Constitution version pinned at 0.2.1 at top level AND inside manifest, AND the two must match test_constitution_version_must_be_0_2_1, test_manifest_constitution_version_must_match_top_level
Tier declared and bounded — app_tier ∈ {1, 2, 3, 4}; strings, floats, zero, negatives all fail test_app_tier_outside_allowed_range (× 6), test_app_tier_accepts_1_through_4 (× 4)
Score gate enforced — security_score ≥ 7 (Phase 1 floor); scores 6, 5, 0, -1 fail; 11+ fails (out of range) test_security_score_seven_passes_boundary, test_security_score_below_seven_fails (× 4)
Build paths enumerated — structural_path ∈ {WALK, DRIVE, FLIGHT}; build_path ∈ {pure_native, kmp_shared} test_structural_path_invalid_enum (× 4), test_build_path_invalid_enum (× 4)
Sacred Pillar bounds — non_negotiables must have 2-4 entries (1 too thin, 5+ dilutes) test_sacred_pillars_non_negotiables_min_two/max_four/within_bounds
No-bypass on lock detection — invalid blueprints in audit mode set locked_invalid status; conversation persists, blueprint does NOT test_lock_audit_invalid_blueprint_sets_locked_invalid
All errors surface — 5 simultaneous violations return all 5 errors, not the first failure. No error-swallowing. test_multiple_simultaneous_failures_all_returned
Field paths in errors — every error includes dot-notation path with array index notation test_error_messages_include_dot_notation_paths
Garbage input doesn't crash — None, strings, integers, lists all fail cleanly without raising test_non_dict_input_fails_cleanly (× 5)

Data Minimization & Privacy by Design

Cloud is the exception. Every data element captured earns its place.

2 checks (structural)

Tier 2+ · Cross-field in v0.2

Retention policies declared — data_retention_policies required at top level test_top_level_required_key_missing[data_retention_policies]
External processors first-class — external_processor_retention[] required at top level test_top_level_required_key_missing[external_processor_retention]

Honest scope: v0.1 enforces structural presence of retention/processor sections. Content-level cross-field validation (every external_processor_retention entry must justify the processor and link to a data_retention_policies entry) is deferred to v0.2 cross-field rules. The Sequoia prompt elicits the content; v0.2 mechanically enforces the linkage.

Cryptography & Dependency Standards

Modern primitives only. Generic terms rejected. No partial credit on compounds.

25 crypto-primitive + 10 secrets tests

Basic Tier 1 · Max Tier 4

Named-primitive allowlist enforced — only the 7 primitives in constitution §3 pass (parametrized × 7) test_crypto_primitive_each_allowlist_entry_passes
Compound expressions parsed correctly — "AES-256-GCM + Argon2id KDF" passes; validator splits on +/, strips role suffixes (KDF, AEAD, HKDF, HMAC, MAC), verifies each token test_crypto_primitive_compound_expressions_pass (× 6)
Generic crypto terms rejected — "encryption", "encrypted", "secure", "hashing" all fail. Constitution §3 forbids paraphrased crypto language. test_crypto_primitive_generics_and_deprecated_rejected (generics)
Deprecated primitives forbidden — MD5, SHA-1, RC4, DES, AES-128, AES-128-CBC all rejected. Plus malformed compounds. test_crypto_primitive_generics_and_deprecated_rejected (× 11)
One bad token in compound fails the whole entry — "AES-256-GCM + MD5" fails because MD5 is forbidden, even though AES-256-GCM is allowed. No partial credit. test_crypto_primitive_compound_with_one_bad_token_fails
Secrets classified — every secret entry must have classification ∈ {server-only, client-buildtime, client-runtime, public}. Unclassified secrets rejected to force explicit thinking about exposure. test_secrets_classification_each_allowed_value_passes (× 4), test_secrets_classification_invalid_rejected (× 6)

Honest scope: v0.1 enforces the primitive allowlist and secrets classification at validation time. Version-pinning enforcement (crypto_inventory[].version_pin for third-party libraries), reachability discipline, and platform-API mapping (iOS Keychain, Android Keystore) are Coder/Reaper concerns and ship in v0.2+.

Authentication & Identity

Phishing-resistant by default when authentication is required.

v0.2+ enforcement

Tier 3+ scope

Honest scope: Pillar 4 applies only at Tier 3+ (apps that authenticate users). The minimal valid blueprint fixture is a Tier 1 Pomodoro-class app with no auth, so v0.1 has no direct Pillar 4 enforcement tests. When the Tier 3 reference fixture (identity-asserting social app) lands in Step 14, Pillar 4 controls — passkey-or-justification, named identity provider, liveness checks if identity is claimed — get their own validation pass.

Secure Architecture & Runtime

Input validated, output encoded. No hardcoded secrets ever.

v0.2+ enforcement

Coder + Reaper scope

Honest scope: Runtime controls are enforced by the Coder Agent during code generation and the Reaper Agent during security audit — not at the orchestrator's blueprint validation layer. v0.1 ships Sequoia (Blueprint Agent) only; Coder + Reaper integration ships in v0.2+. Pillar 5 enforcement tests will land alongside those agents.

Traceability

Every feature carries a constitution_trace. No silent drift.

14 traceability tests

Foundation · Universal

Every feature traces to a Sacred Pillar — v1_features[].traces_to_pillar required and must be one of {core_purpose, target_user, non_negotiables} test_v1_features_missing_traces_to_pillar, test_v1_features_traces_to_pillar_wrong_string_value
At least one v1 feature — v1_features[] must have at least one entry test_v1_features_minimum_one
Features have names — every v1_features[] entry must have a name field test_v1_features_missing_name
Sacred-vs-Constitutional Pillar mix-up rejected (forward) — feature with traces_to_pillar: 3 (integer) instead of "core_purpose" (string) fails test_traces_to_pillar_rejects_integer_constitutional_form
Sacred-vs-Constitutional Pillar mix-up rejected (reverse) — constitution_traces[] entry with pillar: "core_purpose" (string) instead of 1-8 (integer) fails test_constitution_traces_pillar_rejects_string_sacred_form
Constitutional Pillar bounded 1-8 — pillar 0, 9, -1, 100 all fail test_constitution_traces_pillar_outside_1_to_8 (× 4)
Constitution traces cover every feature (cross-field) — every v1_features[] entry must be covered by at least one constitution_traces[] entry test_constitution_traces_must_cover_every_v1_feature
Multiple traces per feature allowed — one feature can be traced under multiple Pillars simultaneously test_traces_can_reference_one_feature_multiple_times
Trace lifecycle enumerated — lifecycle_status ∈ {active, pending, orphaned, retired}; mis-cased values, synonyms, and empty strings all fail test_lifecycle_status_each_allowed_value_passes (× 4), test_lifecycle_status_invalid_rejected (× 5)

Last-Mile Compliance

App Store and Play Store readiness represented at the blueprint level, with gate-content enforcement delegated to the Scaffolder Agent.

2 checks (gate status)

Universal · Scaffolder enforces content

Gates status enforced — last_mile_gates.status must equal "READY" before lock. "PENDING" blocks the lock. test_last_mile_gates_status_pending_blocks_lock
Gate status enum — any value other than READY/PENDING fails test_last_mile_gates_status_invalid_enum_rejected

Honest scope: v0.1 validates the gate status field. The actual content of last_mile_gates.ios[] and last_mile_gates.android[] (privacy nutrition labels, Sign in with Apple, Restore Purchases, etc.) is enforced by the Scaffolder Agent during template generation, not by the orchestrator's validator. The Pomodoro fixture carries a Pillar 7 trace for the privacy nutrition label gate even at Tier 1, modeling the "even an app that collects nothing must still file a privacy label" rule.

Incident Response & Disclosure

Prevention is most of security. Recovery is the rest.

1 check (structural)

Foundation · Sequoia elicits content

Incident response plan declared — incident_response_plan is one of 14 required top-level keys test_top_level_required_key_missing[incident_response_plan]

Honest scope: v0.1 enforces structural presence of the incident response plan at the top level. Deeper validation of the plan's content (severity classes defined, time-to-disclose set, disclosure channels named, internal routing recorded) is enforced by Sequoia's elicitation rather than the validator's mechanical pass. Tier 4 regulator-notification timeline enforcement, including HIPAA breach-notification rules (45 CFR §164.400-414) and GDPR Article 33, ships when the Tier 4 reference fixture lands in Step 14.

→ Honest scope

What v0.1 does not yet test.

The honest list of gaps. Investors who've seen too many "100% secure" claims trust this section more than the rest of the document. Every gap below has a named landing point in the build order.

Live Anthropic API end-to-end. Step 13 in the build order. v0.1 mocks the Anthropic client realistically (mimicking streaming and error classes), but no replacement for a real API round-trip.
Three reference-app fixtures. Step 14: Pomodoro Tier 1 (have it), identity-asserting social Tier 3, banking-for-kids Tier 4. Tier 3/4 fixtures will exercise pillars currently untested at v0.1.
Tauri frontend. Steps 10-12. Python orchestrator is fully tested at the IPC boundary; the Rust+JS Tauri shell that wraps it ships next.
Phase 2 cross-family verification gates. Per architecture spec §9: Gemini Blueprint Sanity Check, Grok Reaper's Shadow, GPT Pre-Export Verification. All Phase 2 work, not v0.1.
Long-line defensive limits on inbound IPC. A pathological 100MB inbound line could OOM the orchestrator. Tracked as v0.2 hardening if it ever matters in practice.
Concurrent run support. Architecture spec §11 lists "Single concurrent run only" as an acceptable v0.1 limitation. Multi-tab support arrives in v0.2+.
Conversation resume after interrupt. Closing the window mid-conversation loses state. Append-only run_log preserves the audit trail but no replay UX in v0.1.