Test layout

Purpose

A recipe's Tests section names files a verifier can run without the author, and this foundation says where those files live, what they may depend on, and how each kind proves its claim: a Python test against a real Postgres in a transaction that is rolled back, a pure test that needs no service at all, a browser test that walks a page, a Node test for a helper with no framework. The invariant that makes it trustworthy is that a test's dependencies are visible from its path and its first lines: a file under tests/ either runs hermetically or says which variable turns it on and skips when that variable is unset, so a green run means what it says and a red run points at one thing.

For a frontend recipe this foundation is what "vitest colocated tests, jsdom render-check with fetch stubbed, the Playwright gate spec, the node --test for pure helpers" resolves to; for a backend recipe it is what "S9 file names, known-answer vectors, hermetic vs live" resolves to.

When to use

Include it from every recipe, backend and frontend; the Tests section of each is written against the file names here. Read it before adding a test file (where does it go, which conftest does it inherit), before adding a fixture (does it belong to the shared conftest or to a seam folder), before writing a test that talks to a database, a signing key or a clock, and before reading a red CI run (which of the four layouts failed, and is the failure in the baseline). Trigger phrases: "where do I put the test", "the test needs a database", "the suite hangs", "relation does not exist in tests", "known-answer vector", "the Playwright spec fails under vitest", "run the live tests".

Do not use it for the migration round trip itself (_shared/alembic-migrations, S8, owns upgrade, downgrade -1, upgrade), for the rendering harness (modules/AUTHORING.md, "Renderings": a screenshot is evidence for a recipe, not a test of the app), or for Argus's own checks against running hosts (monitoring-ops recipes).

Stack assumptions

Four layouts, measured 2026-09-11:

Layout	Where	Runner	Pins
Labs backend	alchemy_labs/backend/tests/	pytest from backend/, pytest.ini	Python 3.13.7 in the backend virtualenv (CI installs 3.11), pytest 7.4.3, SQLAlchemy 2.0.43, fastapi.testclient, moto for the S3 and Cognito fakes, the postgres:14 service postgres-test on host port 5434 (docker-compose.test.yml)
Prima backend	prima/tests/	pytest from the repo root, pyproject.toml [tool.pytest.ini_options]	pytest 8.3.4, SQLAlchemy 2.0.36; no service of any kind for the hermetic suite; postgres:16-alpine on 5433 (docker-compose.yml) or any scratch Postgres for the live files
Labs frontend	alchemy_labs/frontend/src/** (vitest) and frontend/e2e/ (Playwright)	vitest run, playwright test	vitest ^2.1.5, jsdom ^25, @testing-library/react ^16.3.0, @testing-library/jest-dom ^6.8.0, @playwright/test ^1.42.1 (frontend/package.json); coverage provider v8, thresholds 50 on statements, branches, functions and lines (vite.config.ts lines 79 to 96)
Plain Node packages	prima/packages/ui/test/, prima-developer/src/**/*.test.ts, prima-chat/server/*.test.ts, alchemy_labs/backend/tests/js/*.mjs	node --test, node --test --experimental-strip-types, tsx --test, plain node	Node 22 (v22.22.2 on the measuring machine); no test framework installed

Foundations assumed: S8 for how the live-test schema is built (always alembic upgrade head, never create_all(), on Prima; create_all() under pytest only, on Labs); _shared/fastapi-project-layout (S4) for app_fastapi.app, api.dependencies.get_db and core.database.get_session, the three names the client fixtures override; _shared/react-app-skeleton (S2) for the provider tree renderWithProviders mounts; _shared/hmac-seam-pattern (S10) for the seams whose vectors are pinned twice.

Environment names and fail behaviour:

• Labs: TEST_DATABASE_URL, default postgresql://test_user:test_password@localhost:5434/test_db (tests/conftest.py lines 44 to 47). The three values are the postgres-test service in docker-compose.test.yml (lines 114 to 122): image: postgres:14, POSTGRES_DB: test_db, POSTGRES_USER: test_user, POSTGRES_PASSWORD: test_password, ports: "5434:5432". They are fixture values the compose file publishes, not credentials to protect, and they are written out here because a checkout of this kit without that compose file cannot otherwise reach the :5434 server the checks name (the 2026-09-12 rebuild from the recipes had to run its own container on another port for exactly that reason); the conftest copies it into DATABASE_URL and SYNC_DATABASE_URL (48, 49) before any application module is imported, so the app under test can only ever see the test database. PRIMA_CREDITS_TEST_DATABASE_URL and SPARKS_TEST_DATABASE_URL switch on the two self-contained seam folders; unset, every database test in them skips with a reason and the pure tests run.
• Prima: none for the hermetic suite; PRIMA_TEST_DATABASE_URL for the four *_live.py files (skip with set PRIMA_TEST_DATABASE_URL to run the live-database tests when unset), PRIMA_LIVE_UPSTREAM_TEST=1 plus the upstream base URL and token for tests/gateway/test_live_infomaniak.py (6 tests, skipped otherwise). tests/conftest.py deletes every variable named in VALID_ENV and every banned one from the process before each test (_clean_process_state, lines 47 to 64), so a developer's shell cannot leak into a result.
• Frontend: PLAYWRIGHT_BASE_URL (default http://localhost:3000); a remote target skips the webServer block and smoke-tests a deployed app; CI set makes forbidOnly true, retries 2, workers 1 (playwright.config.ts).

Data model

The layout is the data. Labs backend, 164 test_*.py files and 7 conftest.py at fd8558f:

backend/
  pytest.ini                          testpaths = tests; -v -ra --cov=. -W default --strict-markers -p no:warnings;
                                      markers unit, integration, e2e, slow, auth, database, api, payment, ai,
                                      file, security, critical; coverage fail_under = 75
  tests/
    conftest.py                       the shared fixtures (below); 72 flat test_<area>_*.py files beside it
    desk_support.py                   factories make_user / make_workspace for the service-desk files
    unit/                             pure: test_auth, test_security, test_ssrf_guard, test_primaapi_error_shape (legacy per D3), ...
      llm/  primalayered/
    integration/                      whole-journey TestClient files (one has a stale import, Gotchas)
    v2/
      conftest.py                     second_user, second_workspace, cross_tenant_client
      test_f1_scribe_leak.py, test_m1_alembic_chain.py, test_scheduler_singleton.py, ...   the V2 findings by id
      coverage/
        conftest.py                   `import app_fastapi` so create_all() sees every table
        test_<area>_g<N>_<what>.py    31 service-level files: booking, forms, voila
        voila_g1/  voila_g8/          two sub-folders with their own conftest
    prima_credits/                    self-contained seam folder: conftest.py + 2 files
    sparks/                           self-contained seam folder: conftest.py + 9 files
    prima_keys/                       test_prima_key_sync.py (pure, the Labs half of a vector)
    accounting/                       test_engine, test_vat, test_qr_bill, test_reports, test_capital
    js/                               test_scribe_identity.mjs, test_booking_questions.mjs (plain node + jsdom)

The shared fixtures (tests/conftest.py):

Fixture	Scope	What it is
module top (lines 27 to 49)	process	sets ENVIRONMENT=development, REDIS_ENABLED=false, REQUIRE_EMAIL_VERIFICATION=false, placeholder values for every third-party key, and the database URL, before the first application import
test_engine (57 to 109)	session	imports the model modules the fixture lists, services.workspace.models, then app_fastapi so Base.metadata is complete; drop_all() then create_all() on the test database; drop_all() at the end
db_session (113 to 141)	function	one connection, one outer transaction, a SAVEPOINT, and an after_transaction_end listener that reopens the SAVEPOINT after every commit() inside a route; the outer transaction is rolled back after the test. That is the SQLAlchemy 1.x recipe, still what the Labs file does on 2.0.43; the form a new repository copies is the 2.0 one, Session(bind=connection, join_transaction_mode="create_savepoint") with no listener, which gives the same behaviour (a route's commit() releases a savepoint, the outer transaction is rolled back after the test; used by the 2026-09-12 rebuild, where a route's commit stayed inside the rollback)
client (146 to 167)	function	TestClient(app, raise_server_exceptions=False) with get_db, get_session and get_db_with_rls overridden to yield db_session; dependency_overrides.clear() after
authenticated_client (225 to 254), admin_client (258 to 289)	function	the same, plus get_current_user, get_current_active_user, get_current_verified_user overridden to return test_user or test_admin_user; no JWT is minted
test_user (171), test_admin_user (198)	function	real User rows flushed on db_session
test_workspace_id (293 to 323)	function	a real Workspace plus an owner WorkspaceUser row, returned as a string
reset_environment (532 to 546)	function, autouse	snapshots os.environ, clears the in-memory rate-limit store, restores both after
mock_cognito, mock_s3_storage, mock_bedrock, mock_stripe, the *_factory fixtures	function	moto and MagicMock stand-ins for the third parties

tests/v2/conftest.py adds the second tenant (second_user, second_workspace, cross_tenant_client, lines 24 to 84) so a tenancy test has two concrete identities; tests/v2/coverage/conftest.py is one line of code (import app_fastapi) and a docstring that says why (a pure-service suite must register the full model graph before test_engine runs or create_all() builds a partial schema).

The seam folders (tests/prima_credits/, tests/sparks/) carry their own conftest.py and never import the shared one: they build an engine from their own variable, skip when it is unset, skip again when the scratch database is not migrated (SELECT to_regclass('public.prima_credit_grants') IS NULL at prima_credits/conftest.py line 78, 'public.sparks_grants' at sparks/conftest.py line 55), and create throwaway users and workspaces by raw SQL. Their docstrings give the exact three commands: createdb, alembic upgrade head with WEBAUTHN_RP_ID_BACKFILL=localhost, then pytest tests/<folder> -p no:cacheprovider --confcutdir=tests/<folder>. --confcutdir is what stops pytest from loading tests/conftest.py above them, and with it the fixture that drops every table in whatever database it is pointed at.

Prima, 36 test_*.py files and 3 conftest.py at cd2c248:

pyproject.toml         testpaths = ["tests"]; addopts = "-q --strict-markers --strict-config";
                       filterwarnings = ["error::DeprecationWarning:prima.*"]
tests/
  conftest.py          VALID_ENV (the one complete environment), _clean_process_state (autouse),
                       valid_env, settings, client (TestClient with db.init_engine / ping / dispose stubbed), repo_root
  test_<area>.py       12 flat hermetic files (15 with the three live ones): test_app, test_config, test_credits, test_metering_chain, test_migration_chain,
                       test_schema_guard, test_repo_invariants, test_seams_are_declared_not_built, ...
  test_*_live.py       test_credits_live, test_credit_grant_live, test_key_sync_live: opt-in, migrated schema only
  auth/                conftest.py + 13 files (test_jwt_ed25519, test_jwks, test_claims_and_surfaces, test_schema_partition,
                       test_import_graph, test_repository_live, ...)
  gateway/             conftest.py + test_aliases, test_gateway, test_streaming, test_live_infomaniak (opt-in)
  runtime/             test_cluster, test_job_ids, test_poll_crosses_replicas, test_port_is_faithful

Labs frontend at fd8558f: 11 vitest files under src/ (src/App.test.tsx, src/config/authConfig.test.ts, src/lib/deskLogin.test.ts, src/lib/deskFormat.test.ts, src/components/desk/ActingEmailBadge.test.tsx, src/components/desk/deskRendering.test.ts, src/components/config/routingModel.test.ts, and four under __tests__/ folders: src/components/__tests__/nav.test.ts, src/services/__tests__/contentReviewService.test.ts, src/pages/__tests__/ClientOnboarding.skonto.test.ts, src/__tests__/integration/auth.integration.test.tsx); src/setupTests.ts (one import, @testing-library/jest-dom/vitest); src/test-utils/index.tsx (MockAuthProvider, AllProviders, renderWithProviders, renderWithAuth, renderWithoutAuth, the mockUser, mockConnection, mockQuery, mockDashboard fixtures, mockApiSuccess); 8 Playwright specs in e2e/ with 50 test( calls between them.

Plain Node: prima/packages/ui/test/core.test.cjs with test/fixtures/; prima-developer/src/lib/billing.test.ts and gateway-key.test.ts (w6-konto); prima-chat/server/*.test.ts (4 files); alchemy_labs/backend/tests/js/*.mjs (2 files, a hand-rolled check(name, fn) harness over jsdom resolved from frontend/node_modules).

What is NOT stored: no fixture data files for the Python suites (rows are built in fixtures and rolled back); no snapshot files; no recorded HTTP cassettes (third parties are stubbed at the module boundary or the test is opt-in live).

API surface

No HTTP route of its own. The contracts a test file consumes:

• pytest fixtures by name, as above. A test that needs the app calls client, authenticated_client or admin_client; one that needs rows without HTTP calls db_session; one that needs two tenants imports nothing and asks for cross_tenant_client.
• FastAPI dependency_overrides for the three database dependencies and the three user dependencies. A recipe that adds a fourth dependency (a workspace resolver, a surface gate) must add it to the override list in the client fixtures, or the test hits the real one with no request context. An override reaches only a session the dependency takes as a parameter: the Labs surface gate opens get_db_context() itself (api/surface_gate.py line 154), no override reaches a session opened that way, and tests/test_surface_scoped_access.py therefore commits its rows on a plain SessionLocal() outside db_session (lines 65 to 72 and 129; Gotchas). A new service's gate declares db: Session = Depends(get_db) (S4 Data model), so the one get_db override covers it and nothing is added to the list.
• The application's own envelope on the wire: a TestClient test asserts status_code and the parsed body shape ({detail}, {error, code, request_id, timestamp}, the OpenAI shape under /api/prima/v1, which on Labs is the legacy copy per D3; _shared/error-envelope, S7), never a substring of a log line.
• Known-answer vectors, each pinned in two repositories that share no code: the credit-grant signature (alchemy_labs/backend/tests/prima_credits/test_credit_grant_client.py lines 40 to 43 and prima/tests/test_credit_grant_seam.py::test_the_known_answer_vector), the key-sync signature (alchemy_labs/backend/tests/prima_keys/test_prima_key_sync.py::test_the_signature_matches_the_prima_vector and prima/tests/auth/test_key_sync_seam.py::test_the_canonical_string_matches_the_labs_vector), the inbound Sparks signature (alchemy_labs/backend/tests/sparks/test_prima_inbound_sig.py), the meter's row digest (prima/tests/test_metering_chain.py::test_reproduces_a_digest_labs_actually_wrote, parametrised over digests Labs wrote), and the Ed25519 token interop (alchemy_labs/backend/tests/test_prima_ed25519_interop.py, prima/tests/auth/test_jwt_ed25519.py). The vector's constants carry a comment naming the twin file: "IF YOU CHANGE THESE NUMBERS, CHANGE THEM IN PRIMA TOO".
• vitest: renderWithProviders(ui, { authValue, initialRoute }) from src/test-utils; vi.mock for the service module a page imports; screen.getByText on the copy a state renders.
• Playwright: test, expect, page, request from @playwright/test; baseURL from the config; five projects (chromium, firefox, webkit, Mobile Chrome, Mobile Safari).
• node --test: describe, it, assert from node:test and node:assert; TypeScript sources run through --experimental-strip-types (Node 22) or tsx.

Application logic

A new repository, before its first test. The steps below assume a suite exists; this block is the minimum that makes one. Backend, in creation order: (1) pyproject.toml with Prima's [tool.pytest.ini_options] shape (testpaths, addopts = "-q --strict-markers --strict-config", filterwarnings = ["error::DeprecationWarning:<package>.*"]; not Labs' pytest.ini, whose coverage-in-addopts and -W default are the Gotchas); (2) tests/conftest.py whose module top sets the test environment before the first application import, then in fixture order test_engine (session; imports the model modules and the app, create_all() under pytest only, S8), db_session (function; one connection, one outer transaction, join_transaction_mode="create_savepoint"), client (function; TestClient(app, raise_server_exceptions=False) with get_db, get_session and get_db_with_rls overridden to yield db_session), copied from the Labs reference lines in the Data model table; (3) docker-compose.test.yml with one postgres-test service on host port 5434 and a TEST_DATABASE_URL default naming it; (4) one flat tests/test_app.py that requests client and asserts the health route; (5) a tests/<seam>/conftest.py only when a seam folder exists, with its own variable and the to_regclass skip; (6) a *_live.py file only when a claim needs the migrated schema, with the opt-in variable in its header. Frontend, in creation order: (1) the test block in vite.config.ts with environment: 'jsdom', setupFiles: './src/setupTests.ts' and exclude: ['e2e/**', 'node_modules/**'] from the first commit; (2) src/setupTests.ts with the one import; (3) src/test-utils/index.tsx with renderWithProviders over the S2 provider tree; (4) playwright.config.ts with testDir: './e2e', baseURL from PLAYWRIGHT_BASE_URL, webServer for local targets only. A plain Node package needs one line, "test": "node --test --experimental-strip-types \"src/**/*.test.ts\"" in package.json, and nothing installed (Tests, the prima-developer row). The wave-1 gate in the wiki inventory ("a new FastAPI service boots from S4 with one passing test on :5434") is backend steps 1 to 4.

Adding a test to a recipe, in order:

1. Decide the layer by what the claim needs. Nothing but the function: tests/unit/ (Labs), a flat tests/test_<area>.py (Prima), node --test (a TypeScript helper). Rows and constraints: db_session. A route: client or authenticated_client. A second tenant: cross_tenant_client. A constraint that only the migrated schema has: a *_live.py file with the opt-in variable and the to_regclass skip. A page: a colocated vitest file with the fetch layer mocked. A user's whole flow: one Playwright spec.
2. Name the file test_<area>_<what>.py next to its neighbours (Labs flat files: test_desk_sla.py, test_commerce_images.py; V2 findings: tests/v2/test_<finding id>_<what>.py; coverage: tests/v2/coverage/test_<area>_g<N>_<what>.py). A frontend test is <Component>.test.tsx beside the component or __tests__/<name>.test.ts beside the module; a Playwright spec is e2e/<flow>.spec.ts.
3. Build rows through fixtures and flush(), never commit(); the route under test may commit and the savepoint (the listener on Labs, join_transaction_mode on a new repository) keeps the outer transaction open. Assert on the database with a second query in the same db_session, or with a hand-written statement when the claim is about a constraint (prima/tests/test_credits_live.py::test_check_constraint_rejects_a_negative_balance issues an UPDATE that bypasses every function in prima/credits.py, because the design promise is that the guarantee survives careless callers).
4. Inject time. Prima's functions take now: or clock= (prima/auth/claims.py::parse_claims(..., now: int), prima/auth/jwks.py::JwksStore(cache_path, clock=lambda: clock["t"]), prima/auth/sessions.py::feed_health(state, now)), and the tests advance a dict (tests/auth/test_jwks.py lines 116 to 124). A test that reads the wall clock rots on a date; the two failures recorded in Gotchas are exactly that.
5. Pin the wire shape with a vector when the claim crosses a repository boundary: constants for the key, timestamp, nonce and body, the expected canonical string, the expected signature, and the twin's path in a comment. Change both or neither.
6. Stub third parties at the module boundary (moto for AWS, MagicMock for Stripe, monkeypatch.setattr(db, "init_engine", ...) on Prima) and never with a network. A test that must talk to a real upstream is opt-in by variable and skips otherwise.
7. Run the file alone first (pytest tests/test_<area>_<what>.py -p no:cacheprovider), then the folder, then the suite, and compare the failure list with the baseline under Tests before reading a red as yours.
8. For a frontend recipe: one vitest file per component state (loading, empty, error, populated) with fetch or the service module stubbed and the copy asserted; one Playwright spec that signs in, walks the page's states and asserts the visible outcome of one write; npm run build (tsc --noEmit && vite build) as the type gate.

Invariants

• A test under tests/ runs without a service, or names the variable that turns it on and skips with a reason when it is unset. Prima's whole hermetic suite is the proof: 704 passed, 85 skipped in 70.97s on 2026-09-11 with no Postgres, no upstream and no network; the 85 skips are 79 live-database tests and 6 live-upstream tests, each with its variable in the skip reason.
• The shared Postgres is localhost:5434 and nothing else. Labs' conftest writes the URL into DATABASE_URL and SYNC_DATABASE_URL before the first application import; the seam folders refuse to run against a default at all.
• Every test that touches rows runs inside db_session's transaction and leaves nothing behind; the savepoint makes a route's commit() harmless.
• The schema for the hermetic Labs suite is built once per process, by test_engine, before any TestClient exists; init_db() under pytest is a no-op after the first call (S8, and the docstring at core/database.py lines 289 to 316 on the relation-lock hang this prevents).
• The schema for a live test is the migrated one, never create_all() (S8; prima/tests/test_credits_live.py lines 51 to 53).
• A wire shape shared by two services has one known-answer vector in each, with the same constants.
• Time is injected where a function reads it; a hard-coded calendar date in a test is a defect from the day it passes.
• -W error::DeprecationWarning is on for the code the suite owns: Prima (error::DeprecationWarning:prima.*, pyproject.toml line 22) and Argus (error::DeprecationWarning, argus/pyproject.toml line 27). Labs runs -W default (pytest.ini) and is the exception, recorded in Gotchas.
• Playwright specs live in e2e/ and vitest files under src/; a runner that collects the other's files is a configuration defect (Gotchas: at fd8558f vitest collects e2e/*.spec.ts).
• The baseline is written down. A recipe's Tests section quotes the count that passed on its measured_on commit, so a verifier can tell a regression from the standing failures.

Frontend

The frontend half of this foundation is the vitest and Playwright layout of S2, restated as files:

frontend/
  vite.config.ts                 test: { globals, environment: 'jsdom', setupFiles: './src/setupTests.ts',
                                 coverage: { provider: 'v8', thresholds: 50 } }; no include/exclude (Gotchas)
  src/setupTests.ts              import '@testing-library/jest-dom/vitest'
  src/test-utils/index.tsx       renderWithProviders / renderWithAuth / renderWithoutAuth, MockAuthProvider, fixtures
  src/components/__tests__/nav.test.ts          64 table-driven route cases (S2)
  src/<area>/<Component>.test.tsx               colocated: ActingEmailBadge, deskRendering, routingModel, deskLogin, deskFormat
  src/services/__tests__/<service>.test.ts      envelope parsing per service
  e2e/                           auth, critical-flows, forms-gate, commerce-gate, onboarding, query-execution,
                                 staging-smoke, prod-client-booking (8 specs, 50 tests)
  playwright.config.ts           testDir ./e2e; baseURL from PLAYWRIGHT_BASE_URL; webServer `npm run dev` for local
                                 targets only; html + json + junit reporters; trace on first retry

A jsdom render check mounts the component through renderWithProviders with vi.mock on the service module, renders one state per test and asserts the copy the user would read (the German du-Form string on a merchant page, the English string on an admin page). It does not assert class names; the design system is S1's concern and its check is the screenshot.

The Playwright gate for a recipe is one spec that opens the page at baseURL, signs in through the real form (or the harness's session file when the spec runs against the local stack), walks every state the recipe's Frontend section lists, performs one write, and asserts the visible outcome. staging-smoke.spec.ts (10 tests) is the shape for a deploy check: request.get on the health and readiness routes, then the shell renders.

Minimum debug surface: the vitest and Playwright HTML reports (playwright-report/), and pytest -ra (in pytest.ini's addopts) so the summary lists every skip and failure with its reason.

Security & tenancy

• Test credentials are placeholders and are typed in the conftest so nobody reaches for a real one: sk_test_fake, whsec_test_fake, test-pool-id (tests/conftest.py lines 27 to 41); Prima's VALID_ENV uses "x" * 32 and "y" * 32 for the two 32-character minimums and example hosts for the URLs. A test that needs a real third-party value is opt-in (PRIMA_LIVE_UPSTREAM_TEST) and reads it from the environment at run time, never from a file in the repo; prima/tests/test_repo_invariants.py::test_no_env_file_is_committed fails the suite if a .env lands in git.
• Tenancy is tested with two identities, never one: cross_tenant_client is a TestClient authenticated as a user of workspace B, and the assertion is a 404 (not 403) for workspace A's ids (S6). tests/v2/test_f1_scribe_leak.py, tests/test_contact_cross_tenant.py, tests/integration/test_tracking_multi_tenant.py and test_workspace_isolation.py are the files to copy.
• The authenticated fixtures bypass token verification by overriding the user dependency. That is the right level for a route test and the wrong level for an auth test: tests/test_login_verification.py, test_refresh_verification.py, test_register_verification.py and test_revoke_unverified_sessions.py use the unauthenticated client and go through the real login, and test_register_verification.py flips REQUIRE_EMAIL_VERIFICATION back on per test with monkeypatch to cover the production default (the conftest comment at lines 37 to 41).
• The shared test_engine fixture drops every table in the database it is pointed at. That is why the seam folders refuse a default URL, why --confcutdir exists in their run lines, and why no Labs conftest ever resolves a URL from DB_HOST and friends: a test suite that can be pointed at a real database by a stray variable is a data-loss path.
• Fail direction: a missing variable skips (live tests) or falls back to the local test URL (the shared conftest); it never falls back to a production or staging URL. Rate limits: the in-memory fallback store is cleared after every test by reset_environment, so a rate-limit test cannot be poisoned by the previous one.
• Attacker-controlled input in tests is the point of several files (tests/unit/test_ssrf_guard.py, tests/v2/test_f5_html_sanitize.py, tests/v2/test_f3_credential_logging.py); they assert the guard, not the payload.

Tests

The layout is tested by running it. Baselines at the measured commits, 2026-09-11, on the measuring laptop:

• Labs backend, one invocation of the whole suite against the shared postgres:14 on 5434 (pytest tests/ -q --no-cov -p no:cacheprovider --continue-on-collection-errors -o addopts=""): 144 failed, 3710 passed, 93 skipped, 5 xfailed, 58 errors in 601.37s, plus one collection error (tests/integration/test_client_onboarding_api.py imports backend.app_fastapi, a path that does not exist). The schema state of test_db before that run was not recorded, and the count is not exact across invocations: two independent reruns on the same day, same commit, same command and Postgres, gave 146 failed, 3723 passed, 93 skipped, 5 xfailed, 43 errors and 174 failed, 3503 passed, 88 skipped, 5 xfailed, 227 errors, the spread being the schema round trip the Gotchas describe (relation "users" does not exist 256 times in the worst run). Read the figure as a band, and reset the public schema of test_db before a run you mean to compare. The failing groups that held across all three runs, by file: test_commerce_subscriptions.py (22), test_api_endpoints.py (21), v2/coverage/test_forms_g11_* (12), test_quentn_sync.py (11), test_critical_endpoints.py (10), and the collection error. test_primaapi_metering.py (16 in the first run, the legacy in-Labs PrimaAPI copy per D3), integration/test_full_user_journey.py (16, then 11) and the v2/coverage/test_booking_g4_* errors (24 of 58, then 0) moved between runs. This is the baseline a recipe compares against, not a target: a recipe whose tests are in the failing set says so in its Tests section.
• Labs backend, the static chain test alone: pytest tests/v2/test_m1_alembic_chain.py -p no:cacheprovider --no-cov -q prints 7 passed.
• Prima, hermetic: .venv/bin/python -m pytest prints 704 passed, 85 skipped in 70.97s. Live, against a scratch database migrated with the chain (S8): pytest tests/test_credits_live.py tests/test_credit_grant_live.py tests/test_key_sync_live.py tests/auth/test_repository_live.py prints 2 failed, 77 passed in 15.73s; the two failures are test_credit_grant_live.py::test_an_allowance_grant_lands_in_allowance_micro_only and ::test_a_second_allowance_grant_resets_the_cycle_rather_than_stacking (Gotchas).
• Labs frontend, npx vitest run: Test Files 10 failed | 9 passed (19), Tests 1 failed | 160 passed (161). The 10 failed files are the 8 e2e/*.spec.ts Playwright specs collected by vitest, src/__tests__/integration/auth.integration.test.tsx (written against jest.mock), and src/App.test.tsx (one test, renders learn react link, a scaffold leftover). The 160 that pass are the baseline for a frontend recipe.
• @prima/ui: npm test in prima/packages/ui compiles the tests with tsc -p tsconfig.test.json and runs node --test test/core.test.cjs: # tests 19, # pass 19.
• Developer portal (prima-developer w6-konto d5f9aa8, unmerged): npm test is node --test --experimental-strip-types "src/**/*.test.ts": # tests 119, # pass 119 in 152 ms. This is the reference for the node --test layout the wiki's S9 row names; at the portal's main (4428767) there is no test script.
• prima-chat (e606308): npm test is tsx --test server/*.test.ts over 4 files; test:unit runs server/untrusted.test.ts alone. Not run for this recipe.
• backend/tests/js/test_scribe_identity.mjs: node backend/tests/js/test_scribe_identity.mjs from the Labs root. Started for this recipe and did not finish within three minutes; recorded as not measured.
• Argus (e42e467): pytest with filterwarnings = ["error::DeprecationWarning"]. Not run for this recipe.
• prima.li (w8b-content-source 03f4205, unmerged): npm run check:docs syncs the content repo and runs scripts/check-module-cards.mjs (366 lines: every field present, banned words, em dashes, source paths that exist, retired claims). On main (9c01b14) package.json has dev, build, preview, one and no check or test script.

Deploy & ops

Config keys: TEST_DATABASE_URL for a Labs run against a Postgres that is not on 5434 (the conftest reads it; nothing else does, see Gotchas on CI); PRIMA_CREDITS_TEST_DATABASE_URL, SPARKS_TEST_DATABASE_URL, PRIMA_TEST_DATABASE_URL for the live folders and files, always a scratch database created for the run; WEBAUTHN_RP_ID_BACKFILL=localhost when migrating that scratch database (S8); PLAYWRIGHT_BASE_URL for a Playwright run against a deployed target; CI to turn on retries and single-worker mode.

Local services: docker compose -f docker-compose.test.yml up -d postgres-test gives the 5434 Postgres (the rendering harness does this for you, tooling/render/labs-local.sh); make up in prima/ gives the 5433 Postgres for Prima's live files (or point PRIMA_TEST_DATABASE_URL at a database on the 5434 server, which is what the acceptance checks do).

CI (.github/workflows/ci-cd.yml): backend-tests runs pytest tests/ -v --cov=. --cov-report=xml --cov-report=term with DB_* variables pointing at a postgres:15 service on 5432 (the env: block at lines 94 to 104, the pytest line at 107); critical-coverage runs pytest tests/ -v -m "critical" --cov-fail-under=75 (line 212; 295 tests in 29 files carry the marker); frontend-tests runs the vitest and build; migration-check is S8's. Both pytest jobs export DB_HOST and never TEST_DATABASE_URL, and the conftest ignores DB_* (Gotchas).

Ordering on a deploy: the pre-deploy gate is the wiki's /labs-deploy runbook (a staging deploy needs no test gate; a production deploy runs the layers including the Playwright gates and a live smoke against the deployed frontend, e2e/staging-smoke.spec.ts with PLAYWRIGHT_BASE_URL set). This recipe does not run anything against a deployed host; S13 owns that step.

Scheduler jobs, Argus checks: none for the test layout. Rollback: not applicable; a test run changes no deployed state, and a local run changes only the scratch database it was pointed at.

Gotchas

• 2026-09-11 (measured from the files, fd8558f): CI's two pytest jobs cannot reach their database. .github/workflows/ci-cd.yml starts postgres:15 on 5432:5432 and exports DB_HOST=localhost, DB_PORT=5432 (lines 95 to 96 and 201 to 202); backend/tests/conftest.py lines 44 to 49 ignore DB_*, default TEST_DATABASE_URL to port 5434, and write that into DATABASE_URL and SYNC_DATABASE_URL before the app imports. No step in the workflow sets TEST_DATABASE_URL, and the 5434 default has been in the conftest since 2026-02-18 (commit c0e208e). The fix is one env: line per job (TEST_DATABASE_URL: postgresql://test_user:test_password@localhost:5432/test_db); until it lands, the laptop run above is the only measured baseline and CI's result for these jobs is not evidence either way.
• 2026-09-11 (measured, fd8558f): vitest collects the Playwright specs. vite.config.ts's test block has no include or exclude, so vitest run picks up e2e/*.spec.ts, fails all 8 on @playwright/test being outside a Playwright runner, and reports 10 failed | 9 passed files on a tree whose vitest tests pass. Add exclude: ['e2e/**', 'node_modules/**'] (or include: ['src/**/*.{test,spec}.{ts,tsx}']) and the baseline becomes 1 failed | 160 passed tests in 11 files, the one being App.test.tsx. S2's Tests section describes the intended scope (src/**); at fd8558f the config does not enforce it, and S2 is corrected in its own round.
• 2026-09-11 (measured, fd8558f): src/App.test.tsx is the scaffold's renders learn react link and fails; src/__tests__/integration/auth.integration.test.tsx uses jest.mock and has its assertions commented out (S2 says the same). Delete the first, rewrite or delete the second; neither is a gate.
• 2026-09-11 (measured, fd8558f): the shared test_engine drops and recreates the whole schema per session, and the v2/coverage/conftest.py docstring records that over this model graph (circular foreign keys, shared enum types) the round trip does not always complete between separate invocations. The full-suite run above shows the symptom: errors reading relation "system_logs" does not exist, relation "connections" does not exist, relation "user_surface_grants" does not exist mid-run. When a run starts with those, reset the public schema of test_db and run once, as the docstring says.
• 2026-09-11 (measured, fd8558f): tests/integration/test_client_onboarding_api.py imports backend.app_fastapi and fails at collection; with pytest.ini's defaults a collection error stops the run before the first test (-x was not set, but a collection error is fatal without --continue-on-collection-errors). Fix the import or delete the file; the baseline run used the flag.
• 2026-09-11 (measured, fd8558f): pytest.ini's addopts turns coverage on for every invocation (--cov=. with three reporters) and sets -W default and -p no:warnings. Running one file therefore writes htmlcov/ and coverage.xml into backend/ and takes seconds longer; the acceptance checks pass --no-cov (or -o addopts=""). fail_under = 75 applies only when the coverage plugin runs and only CI's critical-coverage job asks for it with --cov-fail-under.
• 2026-09-11 (measured, cd2c248): two Prima live tests carry a calendar date. tests/test_credit_grant_live.py line 330 posts "allowance_expires_at": "2026-09-08T00:00:00Z" as the allowance's expiry; the file was green before that date and reads allowance_micro == 0 after it, because the allowance it granted is already expired. Both test_an_allowance_grant_lands_in_allowance_micro_only and test_a_second_allowance_grant_resets_the_cycle_rather_than_stacking fail from 2026-09-08 on. The fix is a relative timestamp (now + 30 days) or the injected clock the rest of Prima's auth tests use. Nothing about the schema or the grant path is wrong.
• 2026-09-11 (measured, fd8558f): the wiki's S9 row (operations/prima-modules-inventory.md §1.1) lists pytest -W error::DeprecationWarning as the convention and names Argus. Argus and Prima do it in their config; Labs runs -W default and prints 12237 warnings on a full run. The code wins per repo: Labs is not on that rule and this recipe says so rather than claiming it.
• 2026-09-11 (measured, fd8558f): the wiki's S9 row names test_user/test_db as the shared Postgres identity, which the compose file and the conftest confirm; it also says node --test --experimental-strip-types for plain-Node packages, which at the measured commits is true of one repo (prima-developer, on an unmerged branch) while @prima/ui compiles first and runs node --test on the emitted .cjs, and prima-chat uses tsx --test. Three spellings of one idea; a new package picks the first.
• 2026-09-11 (measured, fd8558f): the Labs seam folders skip their database tests with a visible reason when their variable is unset, but the flat tests/sparks/test_sparks_grants_db.py and friends still import the shared tests/conftest.py unless --confcutdir is passed; a plain pytest tests/sparks from backend/ loads both conftests, and the shared one's test_engine fixture is then available to be requested by mistake. Always run a seam folder with the exact line from its docstring.
• 2026-09-11 (measured, 9c01b14 and 03f4205): prima.li's main has no check or test script; the card gate lives on the unmerged w8b-content-source branch. S3's Tests section cites npm run check:docs from that branch; a verifier on main finds nothing to run.
• 2026-09-11 (measured, fd8558f): backend/tests/js/test_scribe_identity.mjs did not finish within three minutes when started from the Labs root for this recipe. Not diagnosed; recorded so nobody quotes it as passing.
• 2026-09-12 (measured at fd8558f, from a rebuild of this layout): the transactional db_session does not reach the surface gate. SurfaceGate.__call__ opens its own get_db_context() (api/surface_gate.py line 154), so the get_db override in client covers the handlers and not the gate, and a grant row flushed on db_session is invisible to it; tests/test_surface_scoped_access.py works around that with a module-scoped engine, a plain SessionLocal() and db.commit() (lines 47 to 72 and 129). A new service's gate takes db: Session = Depends(get_db) as a parameter (S4 Data model), the one override covers it, and the workaround is not copied.
• 2026-09-05 (commit 3a26c72, in the test_engine docstring): a TestClient entering the lifespan used to run create_all() while the test held row locks, and the DDL waited on a relation lock only the test could release: a hang with no timeout. The fix is the import app_fastapi inside test_engine plus the once-per-process guard in init_db() (S8). If a suite hangs on the first request, this is the first thing to check.

Reference implementation

Paths are relative to each repo's root; commits as in measured_on and also_measured_on.

• alchemy_labs (d1a-token-source, fd8558f; backend and frontend test files unchanged from feat/service-desk b1089b6):
  • backend/pytest.ini: addopts (16 to 36), markers (38 to 50), fail_under = 75 (64).
  • backend/tests/conftest.py: environment (27 to 49), test_engine (57 to 109), db_session (113 to 141), client (146 to 167), test_user (171), test_admin_user (198), authenticated_client (225 to 254), admin_client (258 to 289), test_workspace_id (293 to 323), reset_environment (532 to 546).
  • backend/tests/v2/conftest.py: second_user (24 to 52), second_workspace (54 to 81), cross_tenant_client (83 onward). backend/tests/v2/coverage/conftest.py: the docstring and its one import.
  • backend/tests/prima_credits/conftest.py, backend/tests/sparks/conftest.py: the self-contained pattern (docstring, SCRATCH_URL, requires_db, engine with the to_regclass skip, orm, owner_user_id, tenant, merchant).
  • backend/tests/prima_credits/test_credit_grant_client.py (the vector, lines 40 to 43), backend/tests/prima_keys/test_prima_key_sync.py, backend/tests/sparks/test_prima_inbound_sig.py, backend/tests/test_prima_ed25519_interop.py.
  • backend/tests/v2/test_m1_alembic_chain.py, backend/tests/test_desk_domain_login_schema.py (S8), backend/tests/v2/test_f1_scribe_leak.py, backend/tests/test_contact_cross_tenant.py (tenancy shape).
  • backend/tests/js/test_scribe_identity.mjs, test_booking_questions.mjs.
  • docker-compose.test.yml (postgres-test, lines 114 to 124); .github/workflows/ci-cd.yml (backend-tests 28 to 118, critical-coverage 163 to 223, migration-check 253 to 303).
  • frontend/vite.config.ts lines 79 to 96 (the test block), frontend/src/setupTests.ts, frontend/src/test-utils/index.tsx, frontend/src/components/__tests__/nav.test.ts, frontend/playwright.config.ts, frontend/e2e/staging-smoke.spec.ts, frontend/package.json scripts test, test:ci, test:coverage, test:e2e*.
• prima (feature/per-model-pricing, cd2c248): pyproject.toml lines 19 to 22; tests/conftest.py (VALID_ENV 29 to 44, _clean_process_state 47 to 64, client 91 to 109); tests/test_credits_live.py (the opt-in header, 1 to 26; engine 51 to 61); tests/test_credit_grant_seam.py (the vector, from line 375); tests/test_metering_chain.py; tests/auth/test_jwks.py lines 116 to 145 (the injected clock); tests/test_repo_invariants.py; tests/gateway/test_live_infomaniak.py line 34; packages/ui/package.json line 35 and packages/ui/test/core.test.cjs.
• prima-developer (w6-konto, d5f9aa8, unmerged): package.json line 11, src/lib/billing.test.ts, src/lib/gateway-key.test.ts.
• prima-chat (integration/redteam-fixes, the security-review branch, e606308): package.json lines 12 and 13, server/*.test.ts.
• prima.li (w8b-content-source, 03f4205, unmerged): scripts/check-module-cards.mjs, package.json check:docs.
• argus (fix/collapse-host-blindness, e42e467; main is f0c2170, three commits behind, with the same three lines): pyproject.toml lines 25 to 27.
• Wiki philosophers_stone (d4f5b95): operations/prima-modules-inventory.md §1.1 row S9 and §2 ("Dated measurements over prose").

Deviations of the references from this recipe, all dated 2026-09-11: CI's pytest jobs point at a port the conftest does not read; vitest collects the Playwright specs; two scaffold or jest-style frontend tests fail; the shared schema round trip is unstable across invocations; one integration module fails at collection; Labs runs with -W default; two Prima live tests carry a fixed date; the Node test layout is spelled three ways; prima.li's main has no check script; one Node test did not finish.

Acceptance checks

Run without the author at the commits named above. LABS, PRIMA, PY as in S8; the Postgres is the S9 one on 127.0.0.1:5434. Expected observations are exact.

1. cd $LABS/backend && grep -n "5434" tests/conftest.py prints one line (line 46, the default URL); grep -c "TEST_DATABASE_URL" ../.github/workflows/ci-cd.yml prints 0 (the CI gap; 2 or more once fixed, and this recipe's version moves).
2. cd $LABS/backend && find tests -name "test_*.py" | wc -l prints 164; find tests -name conftest.py | wc -l prints 7; ls tests/v2/coverage/test_*.py | wc -l prints 31 and ls tests/v2/coverage/ lists voila_g1 and voila_g8 beside them.
3. cd $LABS/backend && $PY -m pytest tests/v2/test_m1_alembic_chain.py tests/unit/test_primaapi_error_shape.py -p no:cacheprovider --no-cov -q ends 10 passed; the two files need no database (the second tests the legacy in-Labs PrimaAPI copy, D3; the PrimaAPI patterns to copy are Prima's tests/gateway/).
4. cd $LABS/backend && $PY -m pytest tests/prima_credits -p no:cacheprovider --no-cov --confcutdir=tests/prima_credits -q -rs with PRIMA_CREDITS_TEST_DATABASE_URL unset: ends 18 passed, 20 skipped, every skip reason reading PRIMA_CREDITS_TEST_DATABASE_URL is not set; the 18 are test_credit_grant_client.py's pure tests.
5. cd $LABS/backend && grep -c "^KAT_" tests/prima_credits/test_credit_grant_client.py prints 4 (key, timestamp, nonce, body); grep -c "^KAT_" $PRIMA/tests/test_credit_grant_seam.py prints 5 (the same four plus the expected signature); grep -c '^KAT_TIMESTAMP = "1785500000"' prints 1 on each (a bare grep -c 1785500000 prints 2 on Labs, because the body-is-hashed test embeds the timestamp in its sneaky payload at line 79).
6. cd $PRIMA && .venv/bin/python -m pytest ends 704 passed, 85 skipped (time varies; -q is already in addopts, and a second one silences the summary line); with -rs every skip reason names PRIMA_TEST_DATABASE_URL or PRIMA_LIVE_UPSTREAM_TEST. Run it from a shell with no PRIMA_* variable set; the autouse fixture removes the ones it knows about, not arbitrary others.
7. cd $PRIMA && PRIMA_TEST_DATABASE_URL=<the migrated scratch database from S8 check 8> .venv/bin/python -m pytest tests/test_credits_live.py tests/test_key_sync_live.py tests/auth/test_repository_live.py -p no:cacheprovider ends 58 passed; adding tests/test_credit_grant_live.py ends 2 failed, 77 passed with the two names from Gotchas (until the fixed date is replaced, after which 79 passed).
8. cd $LABS/frontend && npx vitest run 2>&1 | grep -E "Test Files|Tests " prints Test Files 10 failed | 9 passed (19) and Tests 1 failed | 160 passed (161); grep -c "exclude" vite.config.ts prints 0 (the vitest gap; the file counts change to 1 failed | 10 passed (11) once an exclude lands).
9. cd $LABS/frontend && ls e2e/*.spec.ts | wc -l prints 8; grep -c "name: '" playwright.config.ts prints 5; grep -n "isRemoteTarget" playwright.config.ts | wc -l prints 2.
10. cd $PRIMA/packages/ui && npm test 2>&1 | grep -E "^# (tests|pass|fail)" prints # tests 19, # pass 19, # fail 0.
11. cd <prima-developer w6-konto> && npm test 2>&1 | grep -E "^# (tests|pass|fail)" prints # tests 119, # pass 119, # fail 0.
12. grep -n "DeprecationWarning" $PRIMA/pyproject.toml <argus>/pyproject.toml prints one line each with error::; grep -n "^ -W" $LABS/backend/pytest.ini prints -W default.
13. npm run check in this repo prints 0 problems.