# AI*Right v0.4.2 freeze run 2026-10-19 14:45:56 EDT; PYTHONDONTWRITEBYTECODE=1; python -B; venv /workspace/.venv-airight/bin/python (Python 3.13.5)
$ /workspace/.venv-airight/bin/python -B harness-skeleton.py --demo --audit-file audit-demo.jsonl
[1] doc-001 after plan: AWAIT_HUMAN_APPROVAL
[1] execute before approval -> BLOCKED: HITL gate: EXECUTE blocked until an approval record exists
[1] forged approval_id at gateway -> BLOCKED: deny: approval record not found (forged?)
[v0.4.1] approval by Pam (witness only) -> REFUSED: approver pam.pm@scaletg.com (mock) can never approve writes (go-live witness / agent / designer)
[v0.4.1] approval by a service principal presenting as Jon -> REFUSED: approver token says principal_type=service_principal; only a human Entra user may approve
[v0.4.1] approval by the agent -> REFUSED: approver airight-pilot1-doc-reviewer can never approve writes (go-live witness / agent / designer)
[v0.4.1] approval by Jon (human_user, oid pinned) -> recorded, principal_type=human_user
[G32] direct call to results-sink, approved request, NO gateway credential -> BLOCKED: sink: no gateway credential (direct call?)
[G32] direct call with FORGED gateway credential -> BLOCKED: sink: gateway credential signature invalid (forged?)
[G32] writes after direct-bypass attempts: 0 (was 1 in Day-1 spike caveat 1)
[1] after approval -> DONE, write={'written': True, 'record_id': 'rec-1'}
[1] replay of used approval -> BLOCKED: deny: approval already used
[G32] direct replay with VALID gateway credential after consumption -> BLOCKED at sink: sink: approval already consumed
[2] poisoned doc-002: AWAIT_HUMAN_APPROVAL, flags=['possible_prompt_injection']
[2] after human reject -> REJECTED
[3] client-confidential doc-003 -> REJECTED (classification ceiling)
[G30] northwind nw-001 approved by its own approver -> DONE, write={'written': True, 'record_id': 'rec-2'}
[G30] northwind asks for scale-internal doc-001 (cross-tenant read) -> REJECTED
[G30] Scale approver (Jon) approving a Northwind write -> REFUSED: approver jon@scaletg.com not on northwind's approver list
[G30] scale-internal token + approval replayed against Northwind's sink -> BLOCKED: deny: token tenant scale-internal / run tenant northwind != target tenant northwind
[G30] request with no tenant claim -> REJECTED
[G30] results visible to northwind: 1 | to scale-internal: 1
[G30] FinOps northwind: requests=3 model_calls=2 tokens=213 cost_usd(synthetic)=0.000426 writes=1 denied=3
[G30] FinOps scale-internal: requests=3 model_calls=2 tokens=226 cost_usd(synthetic)=0.000452 writes=1 denied=7
[G30] FinOps unresolved: requests=1 model_calls=0 tokens=0 cost_usd(synthetic)=0.0 writes=0 denied=1
writes committed: 2 | audit events: 79 -> audit-demo.jsonl
exit 0
$ /workspace/.venv-airight/bin/python -B -c <extract northwind write_classification tool_call success -> audit-event.sample.json>
sample event: 553d69c6-7391-45fd-a08a-14c8cc240904 northwind write_classification approver_principal_type=human_user review_flags= []
exit 0
$ /workspace/.venv-airight/bin/python -B validate_audit.py audit-demo.jsonl ../spikes/litellm-day1/spike-audit.jsonl
PASS audit-event.sample.json
negative: missing correlation: FAIL as expected
negative: bad approval decision: FAIL as expected
negative: successful write with no approval: FAIL as expected
negative: approval missing request_sha256: FAIL as expected
negative: identity missing token expiry: FAIL as expected
negative: trace_id not 32-hex: FAIL as expected
negative: unknown classification: FAIL as expected
negative: raw document text field smuggled in: FAIL as expected
negative: old v0.2 step label 3-Authorize: FAIL as expected
negative: old schema_version 0.2.0: FAIL as expected
negative: old schema_version 0.3.0: FAIL as expected
negative: old schema_version 0.3.1: FAIL as expected
negative: old schema_version 0.4.0: FAIL as expected
negative: old schema_version 0.4.1: FAIL as expected
negative: approval missing approver_principal_type (v0.4.1): FAIL as expected
negative: approval by a service principal (v0.4.1): FAIL as expected
negative: approval by a bot (v0.4.1): FAIL as expected
negative: approval by Pam, witness only (v0.4.1): FAIL as expected
negative: unknown review_flag: FAIL as expected
negative: security flag smuggled into review_flags: FAIL as expected
negative: review flag smuggled into security flags: FAIL as expected
negative: duplicate review_flags: FAIL as expected
negative: missing tenant_id (G30): FAIL as expected
negative: bad tenant_id format: FAIL as expected
negative: write tool_call with unresolved tenant: FAIL as expected
negative: approval missing tenant_id: FAIL as expected
negative: judge failure that allowed (fail-open): FAIL as expected
PASS audit-demo.jsonl#1 state_transition 1-Request success
PASS audit-demo.jsonl#2 state_transition 2-Authorize success
PASS audit-demo.jsonl#3 policy_decision 2-Authorize success
PASS audit-demo.jsonl#4 policy_decision 4-GovernRoute success
PASS audit-demo.jsonl#5 policy_decision 5-Resolve success
PASS audit-demo.jsonl#6 tool_call 6-Execute success
PASS audit-demo.jsonl#7 state_transition 3-ChooseBrain success
PASS audit-demo.jsonl#8 model_call 3-ChooseBrain success
PASS audit-demo.jsonl#9 state_transition 4-GovernRoute blocked_awaiting_approval
PASS audit-demo.jsonl#10 policy_decision 4-GovernRoute blocked_awaiting_approval
PASS audit-demo.jsonl#11 policy_decision 4-GovernRoute denied
PASS audit-demo.jsonl#12 policy_decision 4-GovernRoute denied
PASS audit-demo.jsonl#13 policy_decision 4-GovernRoute denied
PASS audit-demo.jsonl#14 policy_decision 4-GovernRoute denied
PASS audit-demo.jsonl#15 approval 7-Observe success
PASS audit-demo.jsonl#16 state_transition 6-Execute success
PASS audit-demo.jsonl#17 policy_decision 4-GovernRoute success
PASS audit-demo.jsonl#18 policy_decision 5-Resolve success
PASS audit-demo.jsonl#19 tool_call 6-Execute success
PASS audit-demo.jsonl#20 state_transition 7-Observe success
PASS audit-demo.jsonl#21 state_transition 7-Observe success
PASS audit-demo.jsonl#22 policy_decision 4-GovernRoute denied
PASS audit-demo.jsonl#23 state_transition 1-Request success
PASS audit-demo.jsonl#24 state_transition 2-Authorize success
PASS audit-demo.jsonl#25 policy_decision 2-Authorize success
PASS audit-demo.jsonl#26 policy_decision 4-GovernRoute success
PASS audit-demo.jsonl#27 policy_decision 5-Resolve success
PASS audit-demo.jsonl#28 tool_call 6-Execute success
PASS audit-demo.jsonl#29 state_transition 3-ChooseBrain success
PASS audit-demo.jsonl#30 model_call 3-ChooseBrain success
PASS audit-demo.jsonl#31 policy_decision 4-GovernRoute denied
PASS audit-demo.jsonl#32 state_transition 4-GovernRoute blocked_awaiting_approval
PASS audit-demo.jsonl#33 approval 7-Observe rejected
PASS audit-demo.jsonl#34 state_transition 7-Observe rejected
PASS audit-demo.jsonl#35 state_transition 1-Request success
PASS audit-demo.jsonl#36 state_transition 2-Authorize success
PASS audit-demo.jsonl#37 policy_decision 2-Authorize success
PASS audit-demo.jsonl#38 policy_decision 4-GovernRoute success
PASS audit-demo.jsonl#39 policy_decision 5-Resolve success
PASS audit-demo.jsonl#40 policy_decision 4-GovernRoute denied
PASS audit-demo.jsonl#41 state_transition 7-Observe denied
PASS audit-demo.jsonl#42 state_transition 1-Request success
PASS audit-demo.jsonl#43 state_transition 2-Authorize success
PASS audit-demo.jsonl#44 policy_decision 2-Authorize success
PASS audit-demo.jsonl#45 policy_decision 4-GovernRoute success
PASS audit-demo.jsonl#46 policy_decision 5-Resolve success
PASS audit-demo.jsonl#47 tool_call 6-Execute success
PASS audit-demo.jsonl#48 state_transition 3-ChooseBrain success
PASS audit-demo.jsonl#49 model_call 3-ChooseBrain success
PASS audit-demo.jsonl#50 state_transition 4-GovernRoute blocked_awaiting_approval
PASS audit-demo.jsonl#51 approval 7-Observe success
PASS audit-demo.jsonl#52 state_transition 6-Execute success
PASS audit-demo.jsonl#53 policy_decision 4-GovernRoute success
PASS audit-demo.jsonl#54 policy_decision 5-Resolve success
PASS audit-demo.jsonl#55 tool_call 6-Execute success
PASS audit-demo.jsonl#56 state_transition 7-Observe success
PASS audit-demo.jsonl#57 state_transition 7-Observe success
PASS audit-demo.jsonl#58 state_transition 1-Request success
PASS audit-demo.jsonl#59 state_transition 2-Authorize success
PASS audit-demo.jsonl#60 policy_decision 2-Authorize success
PASS audit-demo.jsonl#61 policy_decision 4-GovernRoute success
PASS audit-demo.jsonl#62 policy_decision 5-Resolve success
PASS audit-demo.jsonl#63 policy_decision 4-GovernRoute denied
PASS audit-demo.jsonl#64 state_transition 7-Observe denied
PASS audit-demo.jsonl#65 state_transition 1-Request success
PASS audit-demo.jsonl#66 state_transition 2-Authorize success
PASS audit-demo.jsonl#67 policy_decision 2-Authorize success
PASS audit-demo.jsonl#68 policy_decision 4-GovernRoute success
PASS audit-demo.jsonl#69 policy_decision 5-Resolve success
PASS audit-demo.jsonl#70 tool_call 6-Execute success
PASS audit-demo.jsonl#71 state_transition 3-ChooseBrain success
PASS audit-demo.jsonl#72 model_call 3-ChooseBrain success
PASS audit-demo.jsonl#73 state_transition 4-GovernRoute blocked_awaiting_approval
PASS audit-demo.jsonl#74 policy_decision 4-GovernRoute denied
PASS audit-demo.jsonl#75 policy_decision 4-GovernRoute denied
PASS audit-demo.jsonl#76 state_transition 1-Request success
PASS audit-demo.jsonl#77 state_transition 2-Authorize success
PASS audit-demo.jsonl#78 policy_decision 2-Authorize denied
PASS audit-demo.jsonl#79 state_transition 7-Observe denied
audit-demo.jsonl: 79/79 valid
PASS ../spikes/litellm-day1/spike-audit.jsonl#1 policy_decision 4-GovernRoute success
PASS ../spikes/litellm-day1/spike-audit.jsonl#2 policy_decision 4-GovernRoute blocked_awaiting_approval
PASS ../spikes/litellm-day1/spike-audit.jsonl#3 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#4 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#5 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#6 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#7 policy_decision 4-GovernRoute success
PASS ../spikes/litellm-day1/spike-audit.jsonl#8 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#9 policy_decision 2-Authorize denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#10 policy_decision 2-Authorize denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#11 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#12 policy_decision 4-GovernRoute success
PASS ../spikes/litellm-day1/spike-audit.jsonl#13 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#14 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#15 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#16 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#17 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#18 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#19 error 4-GovernRoute error
PASS ../spikes/litellm-day1/spike-audit.jsonl#20 error 7-Observe error
PASS ../spikes/litellm-day1/spike-audit.jsonl#21 policy_decision 4-GovernRoute success
PASS ../spikes/litellm-day1/spike-audit.jsonl#22 policy_decision 2-Authorize denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#23 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#24 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#25 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#26 policy_decision 4-GovernRoute denied
PASS ../spikes/litellm-day1/spike-audit.jsonl#27 policy_decision 4-GovernRoute success
../spikes/litellm-day1/spike-audit.jsonl: 27/27 valid
exit 0
$ /workspace/.venv-airight/bin/python -B -c <validate ../spikes/clef-live-audit-2026-10-19.jsonl against audit-log-schema.json.v0.4.1 (schema in force at the 14:38 live call)>
clef-live audit vs schema 0.4.1: 10/10 valid
exit 0
$ /workspace/.venv-airight/bin/python -B -m unittest discover -s tests -v
test_approval_for_different_request_rejected (test_harness.T10_GatewayBypassG32.test_approval_for_different_request_rejected) ... ok
test_consumption_is_atomic_under_concurrency (test_harness.T10_GatewayBypassG32.test_consumption_is_atomic_under_concurrency) ... ok
test_direct_call_without_gateway_credential_rejected (test_harness.T10_GatewayBypassG32.test_direct_call_without_gateway_credential_rejected) ... ok
test_expired_or_wrong_audience_credential_rejected (test_harness.T10_GatewayBypassG32.test_expired_or_wrong_audience_credential_rejected) ... ok
test_forged_gateway_credential_rejected (test_harness.T10_GatewayBypassG32.test_forged_gateway_credential_rejected) ... ok
test_invalid_approver_or_expired_approval_rejected_at_sink (test_harness.T10_GatewayBypassG32.test_invalid_approver_or_expired_approval_rejected_at_sink) ... ok
test_legitimate_path_still_works_and_is_logged (test_harness.T10_GatewayBypassG32.test_legitimate_path_still_works_and_is_logged) ... ok
test_replay_after_consumption_rejected (test_harness.T10_GatewayBypassG32.test_replay_after_consumption_rejected) ... ok
test_reused_gateway_assertion_rejected (test_harness.T10_GatewayBypassG32.test_reused_gateway_assertion_rejected) ... ok
test_sink_denial_via_gateway_is_audited_as_mcp_server (test_harness.T10_GatewayBypassG32.test_sink_denial_via_gateway_is_audited_as_mcp_server) ... ok
test_valid_gateway_credential_without_approval_rejected (test_harness.T10_GatewayBypassG32.test_valid_gateway_credential_without_approval_rejected)
Simulates a gateway bug or compromised gateway signing a write that has no valid approval: the sink re-checks. ... ok
test_approver_must_be_on_tenant_list (test_harness.T11_MultiTenancyG30.test_approver_must_be_on_tenant_list) ... ok
test_budget_cap_per_tenant (test_harness.T11_MultiTenancyG30.test_budget_cap_per_tenant) ... ok
test_cross_tenant_approval_reuse_blocked (test_harness.T11_MultiTenancyG30.test_cross_tenant_approval_reuse_blocked) ... ok
test_cross_tenant_document_read_blocked (test_harness.T11_MultiTenancyG30.test_cross_tenant_document_read_blocked) ... ok
test_cross_tenant_write_blocked_at_gateway_and_sink (test_harness.T11_MultiTenancyG30.test_cross_tenant_write_blocked_at_gateway_and_sink) ... ok
test_every_event_carries_tenant_and_finops_rollup_per_tenant (test_harness.T11_MultiTenancyG30.test_every_event_carries_tenant_and_finops_rollup_per_tenant) ... ok
test_gateway_rejects_token_without_tenant (test_harness.T11_MultiTenancyG30.test_gateway_rejects_token_without_tenant) ... ok
test_missing_or_unknown_tenant_rejected_before_model (test_harness.T11_MultiTenancyG30.test_missing_or_unknown_tenant_rejected_before_model) ... ok
test_per_tenant_server_allowlist_pin_and_ceiling (test_harness.T11_MultiTenancyG30.test_per_tenant_server_allowlist_pin_and_ceiling) ... ok
test_rate_cap_per_tenant_and_window_resets (test_harness.T11_MultiTenancyG30.test_rate_cap_per_tenant_and_window_resets) ... ok
test_request_cap_per_tenant (test_harness.T11_MultiTenancyG30.test_request_cap_per_tenant) ... ok
test_results_store_rows_tenant_keyed_no_cross_tenant_read (test_harness.T11_MultiTenancyG30.test_results_store_rows_tenant_keyed_no_cross_tenant_read) ... ok
test_token_cap_per_tenant_blocks_before_model_call (test_harness.T11_MultiTenancyG30.test_token_cap_per_tenant_blocks_before_model_call) ... ok
test_token_for_tenant_a_against_tenant_b_server_blocked (test_harness.T11_MultiTenancyG30.test_token_for_tenant_a_against_tenant_b_server_blocked) ... ok
test_bot_agent_service_principal_unknown_tokens_rejected (test_harness.T12_HumanOnlyApproversJon.test_bot_agent_service_principal_unknown_tokens_rejected) ... ok
test_jon_happy_path_record_carries_principal_type (test_harness.T12_HumanOnlyApproversJon.test_jon_happy_path_record_carries_principal_type) ... ok
test_non_human_allowlist_entries_rejected_at_load (test_harness.T12_HumanOnlyApproversJon.test_non_human_allowlist_entries_rejected_at_load) ... ok
test_oid_mismatch_wrong_tenant_and_expired_approver_token_rejected (test_harness.T12_HumanOnlyApproversJon.test_oid_mismatch_wrong_tenant_and_expired_approver_token_rejected) ... ok
test_pam_cannot_approve (test_harness.T12_HumanOnlyApproversJon.test_pam_cannot_approve) ... ok
test_pam_cannot_be_put_on_any_approver_list (test_harness.T12_HumanOnlyApproversJon.test_pam_cannot_be_put_on_any_approver_list) ... ok
test_pilot_list_is_exactly_jon_and_every_approver_is_human (test_harness.T12_HumanOnlyApproversJon.test_pilot_list_is_exactly_jon_and_every_approver_is_human) ... ok
test_runtime_defense_if_allowlist_is_tampered_after_load (test_harness.T12_HumanOnlyApproversJon.test_runtime_defense_if_allowlist_is_tampered_after_load) ... ok
test_sink_rejects_stored_records_naming_pam_or_non_humans (test_harness.T12_HumanOnlyApproversJon.test_sink_rejects_stored_records_naming_pam_or_non_humans) ... ok
test_change_add_remove_after_approval_changes_hash_and_blocks (test_harness.T13_ReviewFlags.test_change_add_remove_after_approval_changes_hash_and_blocks) ... ok
test_enum_matches_answer_key_exactly (test_harness.T13_ReviewFlags.test_enum_matches_answer_key_exactly) ... ok
test_review_flags_distinct_from_security_flags (test_harness.T13_ReviewFlags.test_review_flags_distinct_from_security_flags) ... ok
test_spike_request_hash_identical_to_harness (test_harness.T13_ReviewFlags.test_spike_request_hash_identical_to_harness) ... ok
test_unknown_duplicate_or_malformed_flag_rejected (test_harness.T13_ReviewFlags.test_unknown_duplicate_or_malformed_flag_rejected) ... ok
test_guest_in_two_tenant_groups_is_ambiguous (test_harness.T14_TenantClaimResolverG40.test_guest_in_two_tenant_groups_is_ambiguous) ... ok
test_guest_without_tenant_group_rejected (test_harness.T14_TenantClaimResolverG40.test_guest_without_tenant_group_rejected) ... ok
test_happy_path_each_option (test_harness.T14_TenantClaimResolverG40.test_happy_path_each_option) ... ok
test_mapped_token_cannot_switch_tenant (test_harness.T14_TenantClaimResolverG40.test_mapped_token_cannot_switch_tenant) ... ok
test_unknown_tid_rejected (test_harness.T14_TenantClaimResolverG40.test_unknown_tid_rejected) ... ok
test_wrong_issuer_rejected (test_harness.T14_TenantClaimResolverG40.test_wrong_issuer_rejected) ... ok
test_gateway_holds_write_with_no_approval_id (test_harness.T1_PreApprovalExecutionBlocked.test_gateway_holds_write_with_no_approval_id) ... ok
test_harness_blocks_execute_before_approval (test_harness.T1_PreApprovalExecutionBlocked.test_harness_blocks_execute_before_approval) ... ok
test_expired_approval (test_harness.T2_ForgedWriteBlocked.test_expired_approval) ... ok
test_forged_approval_id (test_harness.T2_ForgedWriteBlocked.test_forged_approval_id) ... ok
test_rejected_approval_cannot_write (test_harness.T2_ForgedWriteBlocked.test_rejected_approval_cannot_write) ... ok
test_replay_of_used_approval (test_harness.T2_ForgedWriteBlocked.test_replay_of_used_approval) ... ok
test_self_approval_and_agent_approval_denied (test_harness.T2_ForgedWriteBlocked.test_self_approval_and_agent_approval_denied) ... ok
test_tampered_args_after_approval (test_harness.T2_ForgedWriteBlocked.test_tampered_args_after_approval) ... ok
test_classification_above_ceiling_never_reaches_model (test_harness.T3_PoisonedDocToolRequestBlocked.test_classification_above_ceiling_never_reaches_model) ... ok
test_gateway_denies_injected_tool_even_if_harness_forwards_it (test_harness.T3_PoisonedDocToolRequestBlocked.test_gateway_denies_injected_tool_even_if_harness_forwards_it) ... ok
test_injected_tools_stripped_and_audited (test_harness.T3_PoisonedDocToolRequestBlocked.test_injected_tools_stripped_and_audited) ... ok
test_read_token_cannot_write (test_harness.T3_PoisonedDocToolRequestBlocked.test_read_token_cannot_write) ... ok
test_digest_mismatch (test_harness.T4_UnregisteredOrUnpinnedServerRejected.test_digest_mismatch) ... ok
test_inactive_or_unhealthy_server (test_harness.T4_UnregisteredOrUnpinnedServerRejected.test_inactive_or_unhealthy_server) ... ok
test_unpinned_version_on_write_blocked (test_harness.T4_UnregisteredOrUnpinnedServerRejected.test_unpinned_version_on_write_blocked)
Write path: approval is bound to server_version via request_sha256, so a version swap dies at step 4 ... ok
test_unpinned_versions_rejected_at_registry (test_harness.T4_UnregisteredOrUnpinnedServerRejected.test_unpinned_versions_rejected_at_registry)
Read path (no approval involved): Registry step 5 must reject every non-exact pin. ... ok
test_unregistered_server (test_harness.T4_UnregisteredOrUnpinnedServerRejected.test_unregistered_server) ... ok
test_expired_human_session_rejected_at_authorize (test_harness.T5_ExpiredCredentialRejected.test_expired_human_session_rejected_at_authorize) ... ok
test_expired_obo_token_at_gateway (test_harness.T5_ExpiredCredentialRejected.test_expired_obo_token_at_gateway) ... ok
test_happy_path (test_harness.T6_ApprovedWriteAllowedAndLogged.test_happy_path) ... ok
test_contract_and_registry_agree (test_harness.T7_ContractMatchesHarnessRegistry.test_contract_and_registry_agree) ... ok
test_authorize_precedes_choose_brain (test_harness.T8_LifecycleOrderG16.test_authorize_precedes_choose_brain) ... ok
test_draft_server_cannot_serve_before_go_live (test_harness.T9_GoLiveGateG29.test_draft_server_cannot_serve_before_go_live) ... ok
test_ed_oz_agent_or_pam_cannot_sign (test_harness.T9_GoLiveGateG29.test_ed_oz_agent_or_pam_cannot_sign) ... ok
test_jon_signs_pam_witnesses_goes_active (test_harness.T9_GoLiveGateG29.test_jon_signs_pam_witnesses_goes_active) ... ok
test_optional_scout_review (test_harness.T9_GoLiveGateG29.test_optional_scout_review) ... ok
test_witness_must_be_pam_and_evidence_logged (test_harness.T9_GoLiveGateG29.test_witness_must_be_pam_and_evidence_logged) ... ok
test_confident_proceed_allows (test_judge_gate.J1_HappyPath.test_confident_proceed_allows) ... ok
test_noul_and_score_questions (test_judge_gate.J1_HappyPath.test_noul_and_score_questions) ... ok
test_exception_can_escalate_but_never_allow (test_judge_gate.J2_ErrorFailsClosed.test_exception_can_escalate_but_never_allow) ... ok
test_exception_denies (test_judge_gate.J2_ErrorFailsClosed.test_exception_denies) ... ok
test_slow_judge_denied (test_judge_gate.J3_TimeoutFailsClosed.test_slow_judge_denied) ... ok
test_each_malformed_response_denied (test_judge_gate.J4_MalformedFailsClosed.test_each_malformed_response_denied) ... ok
test_random_garbage_never_allows (test_judge_gate.J4_MalformedFailsClosed.test_random_garbage_never_allows) ... ok
test_below_threshold_escalates (test_judge_gate.J5_LowConfidence.test_below_threshold_escalates) ... ok
test_confident_block_denies_and_confident_escalate_escalates (test_judge_gate.J5_LowConfidence.test_confident_block_denies_and_confident_escalate_escalates) ... ok
test_small_margin_escalates_even_above_threshold (test_judge_gate.J5_LowConfidence.test_small_margin_escalates_even_above_threshold) ... ok
test_uncertain_can_deny (test_judge_gate.J5_LowConfidence.test_uncertain_can_deny) ... ok
test_documented_example_answers_parse (test_judge_gate.J6_ClefAdapterMatchesDocs.test_documented_example_answers_parse) ... ok
test_no_token_means_not_called_and_denied (test_judge_gate.J6_ClefAdapterMatchesDocs.test_no_token_means_not_called_and_denied) ... ok
test_pricing_constant_from_docs (test_judge_gate.J6_ClefAdapterMatchesDocs.test_pricing_constant_from_docs) ... ok
test_request_shape_url_and_auth (test_judge_gate.J6_ClefAdapterMatchesDocs.test_request_shape_url_and_auth) ... ok
test_rest_envelope_accepted_and_error_envelope_denied (test_judge_gate.J6_ClefAdapterMatchesDocs.test_rest_envelope_accepted_and_error_envelope_denied) ... ok
test_jev_cannot_be_primary (test_judge_gate.J7_JevBenchmarkOnly.test_jev_cannot_be_primary) ... ok
test_side_by_side_records_but_never_decides (test_judge_gate.J7_JevBenchmarkOnly.test_side_by_side_records_but_never_decides) ... ok
test_allow_still_requires_human_approval (test_judge_gate.J8_HarnessIntegration.test_allow_still_requires_human_approval) ... ok
test_block_rejects_before_approval (test_judge_gate.J8_HarnessIntegration.test_block_rejects_before_approval) ... ok
test_error_timeout_malformed_reject_run (test_judge_gate.J8_HarnessIntegration.test_error_timeout_malformed_reject_run) ... ok
test_low_confidence_escalates_flagged_for_human (test_judge_gate.J8_HarnessIntegration.test_low_confidence_escalates_flagged_for_human) ... ok
test_cap_refusal_when_estimate_plus_spend_exceeds_cap (test_judge_gate.J9_SpendCapLedgerRedaction.test_cap_refusal_when_estimate_plus_spend_exceeds_cap) ... ok
test_corrupt_missing_or_tampered_ledger_fails_closed (test_judge_gate.J9_SpendCapLedgerRedaction.test_corrupt_missing_or_tampered_ledger_fails_closed) ... /workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:167: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpque7sths.json' mode='r' encoding='UTF-8'>
  L = _ledger(); raw = json.load(open(L.path)); r = mut(L, raw)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:170: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpque7sths.json' mode='w' encoding='UTF-8'>
  cases = {"garbage": lambda L, d: open(L.path, "w").write("{not json") and "skip",
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:167: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmp4numgfvf.json' mode='r' encoding='UTF-8'>
  L = _ledger(); raw = json.load(open(L.path)); r = mut(L, raw)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:171: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmp4numgfvf.json' mode='w' encoding='UTF-8'>
  "truncated": lambda L, d: open(L.path, "w").write(json.dumps(d)[:25]) and "skip",
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:167: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpskt5lz37.json' mode='r' encoding='UTF-8'>
  L = _ledger(); raw = json.load(open(L.path)); r = mut(L, raw)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:167: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmptybh197w.json' mode='r' encoding='UTF-8'>
  L = _ledger(); raw = json.load(open(L.path)); r = mut(L, raw)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:168: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmptybh197w.json' mode='w' encoding='UTF-8'>
  if r != "skip": json.dump(raw, open(L.path, "w"))
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:167: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpokutsjg7.json' mode='r' encoding='UTF-8'>
  L = _ledger(); raw = json.load(open(L.path)); r = mut(L, raw)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:168: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpokutsjg7.json' mode='w' encoding='UTF-8'>
  if r != "skip": json.dump(raw, open(L.path, "w"))
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:167: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpvcs9gxoq.json' mode='r' encoding='UTF-8'>
  L = _ledger(); raw = json.load(open(L.path)); r = mut(L, raw)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:168: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpvcs9gxoq.json' mode='w' encoding='UTF-8'>
  if r != "skip": json.dump(raw, open(L.path, "w"))
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:167: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmphmn8a2m7.json' mode='r' encoding='UTF-8'>
  L = _ledger(); raw = json.load(open(L.path)); r = mut(L, raw)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:168: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmphmn8a2m7.json' mode='w' encoding='UTF-8'>
  if r != "skip": json.dump(raw, open(L.path, "w"))
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:167: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpj35i0o1o.json' mode='r' encoding='UTF-8'>
  L = _ledger(); raw = json.load(open(L.path)); r = mut(L, raw)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:168: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpj35i0o1o.json' mode='w' encoding='UTF-8'>
  if r != "skip": json.dump(raw, open(L.path, "w"))
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:167: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpi8rrsuq_.json' mode='r' encoding='UTF-8'>
  L = _ledger(); raw = json.load(open(L.path)); r = mut(L, raw)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:168: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpi8rrsuq_.json' mode='w' encoding='UTF-8'>
  if r != "skip": json.dump(raw, open(L.path, "w"))
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:167: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpikxpuw1m.json' mode='r' encoding='UTF-8'>
  L = _ledger(); raw = json.load(open(L.path)); r = mut(L, raw)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:179: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpikxpuw1m.json' mode='w' encoding='UTF-8'>
  "not an object": lambda L, d: open(L.path, "w").write("[]") and "skip"}
ResourceWarning: Enable tracemalloc to get the object allocation traceback
ok
test_cost_math_and_cap_bounds (test_judge_gate.J9_SpendCapLedgerRedaction.test_cost_math_and_cap_bounds) ... ok
test_cumulative_cap_never_exceeded_across_calls (test_judge_gate.J9_SpendCapLedgerRedaction.test_cumulative_cap_never_exceeded_across_calls) ... ok
test_env_values_never_in_errors_audit_ledger_or_repr (test_judge_gate.J9_SpendCapLedgerRedaction.test_env_values_never_in_errors_audit_ledger_or_repr) ... /workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:201: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/clef-ledger-tests-9svs0hd3/tmpse47oko8.json' mode='r' encoding='UTF-8'>
  blob = json.dumps({"reason": dec.reason, "audit": dec.audit_block(), "flags": dec.flags}) + repr(j) + open(L.path).read()
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/workspace/oz-docs/ai-right/artifacts/tests/test_judge_gate.py:209: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmpdt2e_jdk.jsonl' mode='r' encoding='UTF-8'>
  self.assertEqual(h.start("doc-001")["state"], H.State.REJECTED); log = open(path).read(); gw.store.db.close(); os.unlink(path)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
ok
test_extra_questions_recorded_never_decide (test_judge_gate.J9_SpendCapLedgerRedaction.test_extra_questions_recorded_never_decide) ... ok
test_live_smoke_script_exits_cleanly_without_token (test_judge_gate.J9_SpendCapLedgerRedaction.test_live_smoke_script_exits_cleanly_without_token) ... ok
test_malformed_account_id_fails_closed_without_network (test_judge_gate.J9_SpendCapLedgerRedaction.test_malformed_account_id_fails_closed_without_network) ... ok
test_missing_usage_keeps_estimate_and_transport_error_still_counted (test_judge_gate.J9_SpendCapLedgerRedaction.test_missing_usage_keeps_estimate_and_transport_error_still_counted) ... ok
test_reserve_before_send_then_settle_to_actual (test_judge_gate.J9_SpendCapLedgerRedaction.test_reserve_before_send_then_settle_to_actual) ... ok

----------------------------------------------------------------------
Ran 104 tests in 0.756s

OK
exit 0
pyc after run: 0
