TLDR

• AI*Right is Scale's enterprise-by-default control layer for AI agents. Every model and tool call runs through governed gateways, and a named human approves every write.
• Where we are: the Day 1 build is done: approver-screen prototype with red-flag triage (35/35 tests), pilot exit criteria EX-1 to EX-4 drafted, and the pilot-minimum connector fixes built (25/25 tests). Assess starts Tue Oct 20 and the Pilot starts Nov 3.
• Today's wins: a multi-tenant governed prototype, built and independently re-tested green. Every AI write waits for my approval and is sealed to exactly what I approved. Our Clef judge made its first live decision on Cloudflare, and the write still waited for a human.
• Schedule: full scope fits Nov 3–30 if the people-dependent steps clear by Tue Oct 27 EOD. Ed's build time is about 36–70 hours total, so build is not the constraint.
• Main dependency: Entra (Microsoft sign-in) and admin consent. Next: the red-team dry run on Thu Oct 22 and the first weekly status on Mon Oct 26.
• From you: the Your input needed box below, by Fri Oct 23.

Who's who: Ed = Scale's AI engineering agent (a bot that builds and tests). Pam = Scale's AI project-management agent (a bot that plans and witnesses test runs). Oz = Scale's AI coordinator agent. Sal = Scale's AI sales-scoreboard agent. Jon = the only human approver of writes.

Team,

v2 (EOD): updated with this afternoon's and end-of-day changes. The plan, dates and cadence below are unchanged from v1.

We're kicking off AI*Right, the first major project run by our Grok bot team. Every gate is approved by a named human.

SHIPPED TODAY (MON OCT 19)

• v0.4.3 frozen at 15:07 EDT. 108/108 unit tests and 44/44 spike; the Clef spend ledger is reconciled. Waiting on Pam's witness.
• Approver prototype, demo-able now (localhost, Ed). Pending list; the request hash is recomputed server-side and shows MATCH; CSRF protection; sign-in must be within 300 s; single-use approvals; 40/40 audit events valid; 13/13 tests. Sign-in is a mock that admits only me.
• Approval-screen triage built into the approver prototype (synthetic, unwitnessed, outside the freeze). Summary line at the top (tool, doc, tenant, doc type, judge decision); up to 3 red-flag snippets with offsets; a red injection banner even when the judge says proceed; an isolation banner that is never green by default; isolation proof via a context-set hash + tenant_id bound into the approve form (missing or mismatched = 409, fail closed); an "I did not pre-approve this" kill button that halts the session (423); an 8 s server-enforced approve delay; an 8-per-session cap with a 15-min lockout; and a trap item (SYN-T01) that locks the approver if approved. 35/35 tests (13 base + 8 Entra + 14 HITL). All timings and caps are EXAMPLE.
• G45 per-client approver design done (design only, not built, not in the freeze).
• Pilot exit criteria EX-1 to EX-4 drafted: EX-1 a complete audit trail from intake via connector to approval to write; EX-2 tenant isolation; EX-3 wrong-write undo (a compensating write I approve; no auto-apply); EX-4 judge catch on the 49-attack set. Thresholds are UNKNOWN until Thursday's run.
• Connector governance check: 0 MET / 5 PARTIAL / 1 MISSING (nothing reads from a real source yet). New gaps G46–G52 logged.
• Pilot-minimum connector fixes G46/G49/G50 built in connector-proto/ (25/25 tests, synthetic, unwitnessed, no live calls): a delegated-only OBO hop whose results are the intersection of what the user can see and the SharePoint Sites.Selected grant (broad, wildcard and write scopes rejected; cross-tenant denied); app-only access limited to metadata and always flagged in the audit; an intake audit event that comes first and starts the clock, with every read linked to it. Not yet wired into the harness write gate or the approver screen.
• Freeze intact: v0.4.3 verifies 50/50 and v0.4.2 verifies 37/37. Nothing above touched the frozen releases.
• R0 40-doc red-team set ready for Thu Oct 22. Offline run: 39/40 pass, 30/30 attacks contained, 10/10 judge failures fail closed. 24 of the 30 attacks reach the approver screen, 22 with no harness flag, so the human is the last line of defense when the judge is wrong. Four findings, owner Ed, fix in v0.4.4: F-R0-1 a null-text doc crashes before audit (no write, but the refusal is unaudited); F-R0-2 zero-width characters beat the mock keyword check; F-R0-3 no review_flag for leaked credentials; F-R0-4 mock flag detection is 0/8.
• R0 v1.1 and live Clef calibration (unwitnessed, outside the freeze). R0 v1.1 is 62 synthetic docs (13 clean, 49 attacks) with Heavy's ideas folded in; SHA256SUMS 71 OK. Live Clef calibration: 183/183 HTTP 200, 0 writes, $0.058 of the $1 cap, p50 365 ms / p95 1,318 ms, AUC 0.987. At 0.80/0.30: 0 false-allows but only 57% clean-allow. At the proposed 0.70/0.30: 0 false-allows and 71% clean-allow. The closest attack (a fake Teams approval) scores 0.54. Proposed for v0.4.4: separate allow/deny thresholds and a provenance banner (red disables Approve). The threshold change comes to me for signature after Pam witnesses Thursday's run.
• Independent red-team review. An independent model (Grok Heavy) red-teamed our Entra design and attack set; its ideas are being folded into R0 v1.1.
• v0.4.2 frozen (14:46:16 EDT), witnessed. This is a git-less freeze: 35 files pinned by SHA-256 in freeze-v0.4.2-manifest.md. It bundles the G44 approver fix, review_flags, the G40 tenant-claim resolver and the Clef cap and guards. Results: 104/104 unit (test_harness 72, test_judge_gate 32), 44/44 spike (4 sink rows), 79/79 demo and 27/27 spike audit events schema-valid, and 27/27 negatives failing as expected. On the pilot tenant, I'm the only approver, enforced across the approval record, gateway and sink. 17 review_flag types are in the request hash. Pam re-ran everything in a scratch copy from 14:47 to 14:52 ET: all MATCH, no source hash changed, signed by me at 2:56 PM ET.
• Test rig (Ed), all localhost, synthetic only:
  • mock_mcp_server.py: a localhost MCP server on the Python MCP SDK 2.x (MCPServer, the renamed FastMCP), served over streamable HTTP on 127.0.0.1 for one tenant. It exposes get_document (READ, untrusted content) and write_classification (WRITE). The write runs only if a gateway assertion verifies against the tenant's derived key and matches tenant, aud, tool and server_version. It recomputes request_hash instead of trusting it, then re-checks the approval record (tenant, approved, single-use, unexpired, hash match, human approver). Finally it consumes the approval atomically with the write in SQLite. Every refusal is logged sink-side to JSONL.
  • airight_gw_auth.py: the shared gateway-identity library. It mints HMAC-SHA256 gateway assertions (30 s TTL, EXAMPLE) from per-tenant derived keys, computes the tenant-bound request hash, does the sink's SQLite single-use bookkeeping, and holds the approver list (Jon only, human_user). The production mechanism (mTLS or JWT, Vault) is UNKNOWN.
  • approval_hold_guardrail.py: a LiteLLM 1.106.0 custom guardrail at pre_mcp_call. It holds every write until a valid approval exists, then injects a short-lived assertion bound to that exact write. It takes the tenant from a header (a spike stand-in for the token claim). Any exception becomes an audited error and an HTTP 403.
  • run_spike.py: drives LiteLLM's MCP gateway (/mcp JSON-RPC streamable HTTP and /mcp-rest) on localhost and prints PASS/FAIL for each check.
  • judge_gate.py: a stdlib-only Clef adapter plus BoundedJudgmentGate. It builds the documented Workers AI REST request and fails closed on every failure path. The v0.4.2 bundle adds the $10 spend-cap ledger, secret redaction and a 32-hex account-ID guard.
  • validate_audit.py + audit-log-schema.json 0.4.2 (JSON Schema 2020-12): validates every harness and spike audit event, plus 27 mutated negatives that must fail (bot, Pam or service-principal approver; unknown or duplicate review_flags; old schema versions).
  • mcp-server-contract.sample.yaml (0.4.2-example): the declarative contract the Registry lists servers from. It denies unlisted tools and servers, turns off dynamic tool discovery, maps the 7-step lifecycle and carries the Entra OBO identity block and approvers.
  • Freeze kit: run-freeze-v0.4.2.sh is a single runner (purges .pyc, then runs the demo, validator, spike on 127.0.0.1 and unittest, with no live Clef call). freeze-v0.4.2-manifest.md holds 35 SHA-256 rows, the exact commands and the claimed counts. rerun-g44-manifest.sha256 covers 22 files and runs with sha256sum -c.
• Earlier today: 74/74 green (v0.4.0), signed by me at 14:19 ET. unittest discover -s tests: harness 52 (T1–T11) + judge 22 (J1–J8), re-run by Pam at 14:15 ET against hashed code. The harness half covers no execute before approval; forged, expired, replayed, tampered and self/agent approvals denied; poisoned-doc tool injection stripped; ceiling, pin and digest enforcement; expired OBO; Authorize before Choose Brain; 11 bypass and 14 tenant tests. The judge half covers fail-closed on exception, timeout and malformed output (16 crafted + 300 random garbage responses never allow), no token means no call and deny, and low confidence escalates. Jev can't be primary.
• Also witnessed at 14:15, signed by me at 2:46 PM ET: the v0.3.1 bypass harness, 38/38, and the live LiteLLM 1.106.0 spike, 31/31 (localhost, mock model). That includes tenant checks C8a–e and fault injection C9a–d: a corrupt approvals store or an exception inside the hook means the write is denied, and the approved write then succeeds exactly once.
• First live Clef decision (G33), 14:51:51 EDT. Cloudflare Workers AI @cf/cloudflare/clef (27B multimodal decision model, typed questions in and per-option probabilities out, 65,536-token context, $0.24 per M input tokens, no output charge) ran on SYN-01 through the full harness and JudgeGate. HTTP 200. The gate scored proceed 0.8707, escalate 0.0924, block 0.0369 (margin 0.78), above the uncalibrated 0.80 / 0.30 thresholds. The side questions were pricing visible = no at 0.9865 and risk = low at 0.5971. It took 1,410 ms and 728 input tokens and cost $0.00017472; cumulative spend is $0.00072936 of the $10 cap. The gate allowed, and the write was held at AWAIT_HUMAN_APPROVAL with 0 commits. 10 audit events, 0 invalid. The run used a wrapper outside the v0.4.2 freeze and is unwitnessed. The 14:38 first attempt hit HTTP 404 (code 7003, bad account path) and failed closed with 0 writes, so a 32-hex account-ID guard now runs before any send. Jev is benchmark-only. Thresholds get tuned in Assess on the R0 set.
• G30 multi-tenancy: CLOSED. tenant_id is resolved from the Entra tid claim by the TenantRegistry, never from caller input, and copied through OBO (mock GUIDs; the real Entra OBO path isn't exercised yet). It's required on every audit event. Each tenant has its own allowlist and exact pins, policy overlay, classification ceiling, approvers and caps (requests, rate, tokens, budget). tenant_id is in request_sha256, each tenant has a derived signing key and its own sink, and a cross-tenant read returns not-found before any model call.
• Synthetic Pilot 1 eval set synthetic-v1.1 (pilot-data/synthetic-v1/): 20 docs (6 quotes, 6 invoices, 6 other, 2 that the gateway must block) with 20 seeded issues across 17 types, including price leaks, prompt injection, a lookalike-domain bank change, a duplicate invoice, a total mismatch and PII. The 2 blocked docs are one above the classification ceiling and one cross-tenant. It ships with a ground-truth key and score.py with 19 unit tests. Pam independently checked 20/20 labels (0 errors), 28/28 hashes and 19/19 tests. The gateway rejected both cross-boundary docs on first load. The model was a placeholder stub, so there's no accuracy baseline yet.
• review_flags (approved 14:32 ET). write_classification carries a closed enum of the 17 seeded issue types, sorted into request_sha256, so my approval binds the findings I saw. Any change after approval blocks the write, and unknown or duplicate values are refused at the gateway and the sink. It's kept separate from the audit security flags. Tested in T13 and C11a–e.
• G31 and program hygiene. The Pilot gets a minimal approver screen. Phase 3 has no UI and the Registry stays config-as-code; tenant admin comes in P4, the console in P5 and evidence export in P6. The gap register runs G1–G44, with G30, G32, G44 and G39 (LiteLLM side) closed. All 10 pressure-test decisions are approved. Phases are dated: Assess Oct 20–Nov 2, Pilot Nov 3–30, Core Control Plane Dec 1–18. Pam's Assess week-1 plan is written (R0 on Thu Oct 22 at 2:00 PM ET; status #1 on Mon Oct 26 by 9:00 AM ET). A v2 customer-deck draft swaps Jev for Clef and is pending my approval.

ENTRA ROADBLOCKS (MICROSOFT SIGN-IN)

• Go-live tenant + Entra confirmation (tracker item 13). Blocks rehearsal set C, the Entra half of 7c (real OBO and idtyp tests) and the tenant-specific parts of the 7a sink network lock. Today Entra is only assumed, and every harness/spike token is a mock (claims checked; no real issuer/JWKS signature validation). Unblock: me (IdP owner, any spend); Ed builds. Pam's week-1 plan has the decision on Tue Oct 20; tracker due Mon Nov 2.
• Approver app registration + admin consent. The approver page needs an Entra app registration in Scale's tenant, admin consent, the optional idtyp claim with include_user_token, and my oid for the allowlist pin. Without idtyp, approver_check refuses every approver: fail closed, but the Pilot stops. Who holds Global Admin to grant consent: UNKNOWN. Licensing is covered: Scale has Entra ID P2 (15 seats, 14 assigned) plus Business Premium and E3, so Conditional Access with MFA and risk-based sign-in policies are licensed. Whether my own account holds a P2 seat is UNKNOWN; Ed verifies during setup. Verify in Assess (by Nov 2).
• Approver page: prototype done; production path re-estimated (see Schedule). Still needed from Entra: app registration, admin consent, the idtyp claim, my oid, and TLS. LiteLLM's admin UI stays ruled out.
• Hosting on Scale-owned servers (details Tue); builds on Harold's MCP MVP work.
• Entra hardening: Grok Heavy's independent Entra review, checked by Ed against Microsoft Learn. 8 claims, 0 disagreements (details in ai-right/heavy-entra-review-ed-assessment.md). Now built into the approver prototype: a (tid, oid) pair allow-list, an exact iss match, amr must include mfa, scp required, an email/upn match is refused, agent user accounts are rejected (xms_sub_fct=13), and claims are re-checked at approve/reject. 21/21 tests pass (8 new), and the v0.4.3 freeze still verifies 50/50. Still mocked: signature/JWKS, aud, azp and real tid/oid values. Agreed but not built: a Conditional Access MFA policy (P2 licensed) and per-client go-live sign-off before client #2.
• Approvers (G45, locked by me 3:27 PM ET). The target is per-client named approvers in each client's Entra tenant, with a (tid, oid) allow-list and dual control for high-risk writes. I'm the sole approver for the Pilot only, through Nov 30.
• Client tenant mapping (G40). Ed recommends per-client Entra tenants via a multitenant app (iss + tid → registry), B2B guest + exactly one tenant app role as fallback, Okta only on request. Pending my decision. Each client's admin consent, per-client onboarding steps, cross-tenant OBO behaviour, B2B pricing and the Okta API Access Management licence are all UNKNOWN. Decide in Assess; team flag by Fri Oct 23.
• Per-client go-live (G43). Go-live and Registry status are still global, so one sign-off covers every tenant. Per-tenant go-live is my design call; not built; date UNKNOWN.

SCHEDULE (ED'S RE-ESTIMATE)

• Build time: about 36–70 hours total. The full scope fits Nov 3–30 if the human waits clear by Tue Oct 27 EOD. The slow parts are live-tenant debugging, live hosting setup, and the re-freeze, witness and sign cycles.
• Critical path: tenant + sources (Oct 20) → admin consent → SharePoint Sites.Selected grant + shared intake mailbox + hosting → live integration (~2–3 days) → re-freeze + Pam's go-live witness (Oct 29–Nov 2) → I sign by Nov 2 → first real write Nov 3.
• If the waits slip past Oct 27: real writes on Nov 3 go first (the pilot becomes read-and-classify at intake only, or slips day for day).

WHAT IT IS
AI*Right is Scale's enterprise-by-default control layer for AI agents: Agent = Harness + Model (through an AI Gateway) + Governed MCP (MCP Gateway + Registry). Every model call goes through the AI Gateway and every tool call through the MCP Gateway. Nothing connects directly. We deliver it as Assess, Architect, Deliver, Manage, and it complements Cloud iQ.

PROJECT PLAN

THE WIREFRAME: HOW A GOVERNED REQUEST FLOWS

1. Request: a user or agent asks for work.
2. Authorize: identity and on-behalf-of scope are checked first.
3. Choose Brain: the AI Gateway routes to the right model.
4. Govern & Route: the MCP Gateway applies policy, limits and PII rules. Any write waits here for human approval.
5. Resolve: the Registry returns only approved, healthy, version-pinned servers.
6. Execute: the MCP server performs one scoped action.
7. Observe: audit, FinOps and feedback, with every step logged.

Five pillars: Agents; Security & Identity; Core Control Plane; Data & Memory; Operations & Alignment.

WHERE WE ARE (Oct 19, 2026)

• All 9 roles assigned. Ed: Framework Architect, Agent & Harness Designer, MCP & Integration, Security & Identity (interim). Pam: Governance & Compliance Lead, Delivery Program Manager. Oz: KB Curator, Gap & Risk Analyst, Observability & FinOps (interim).
• Pressure test held today; decisions logged.
• Pilot 1: internal-only governed document review on Scale's own non-sensitive documents. Synthetic test documents first, no client data, and a human approves before any write.
• Pilot gateway: self-hosted LiteLLM, with Kong targeted for Phase 3.
• Independence: bots witness and log the tests; I sign gate and Registry go-live. SCOUT can be asked for an outside review.

WHAT CHANGED TODAY (Oct 19, afternoon)

• Multi-tenancy designed in (G30). Every audit event, Registry allowlist, policy, approval and cap now carries a tenant ID; cross-tenant reads, writes and approval reuse are blocked in the prototype (mock tenants, synthetic data).
• Gateway bypass fixed (G32). The results store now accepts only a gateway-signed, single-use credential tied to an approved request.
• Tests independently re-run and signed. Pam re-ran the full suite herself: 74/74 pass, no mismatches. I signed it. G30, G32 and the gateway-side fail-closed fix are closed.
• Console decided (G31), per Pam's plan. The Pilot gets only a minimal approver screen. No UI in Phase 3; the Registry stays config-as-code. The full console comes in Phase 5, tenant admin in Phase 4, evidence-pack export in Phase 6. UI stays minimal; there is no front-end owner.
• I am the sole Pilot write approver. No bot approves a write; Pam witnesses and logs only. Ed is updating the files and will re-run the tests.
• Clef replaces Jev (G33). Cloudflare Workers AI Clef is the bounded-judgment check at the Guard step; Jev stays benchmark-only. It never replaces human approval for writes. The live Clef test ran at 2:51 PM: HTTP 200, proceed at 0.87, and the write was held for human approval, with 0 writes. Spend is under $0.001 of the $10 cap.
• Approvers decided (G45, option A). Per-client approvers in each client's own Entra tenant. I'm the sole approver for the Pilot only, through Nov 30.
• Isolation that can't be proven = hard block. No two-approver override; if the proof fails on a legitimate document, we fix the proof, not the rule.
• No second-approver dollar thresholds in the Pilot.
• I'm the EX-1 sign-off for the pilot exit criteria.
• The Pilot classifies at intake via a governed connector, not chat paste.

ALSO OPEN: FLAG BY FRI OCT 23

Full list: open-items-for-team.md in the AI*Right workspace.

Attached: open-gaps-team.pdf, a plain-English list of every open gap the team can act on, with owners and due dates.

HOW WE'LL COMMUNICATE

• Weekly status email every Monday by 9:00 AM ET: progress against plan, red/amber/green, blockers, next week's focus, updated dates.
• Phase gate review at the end of each phase: a one-page summary plus the decision needed.
• Ad hoc messages only for high-risk blockers or decisions.

WHAT I NEED FROM YOU
Help clear the blockers in the Your input needed box, flag any concern with the dates, and answer the table above, by Fri Oct 23. We're moving fast; the people-dependent steps need to clear by Tue Oct 27.

Jon