Harness Engineering2026-09-2112 min read
AI Approval Stale Data: Why Snapshots Expire
A human approval binds to one data snapshot. Learn why AI approval stale data checks must recheck inputs before acting, not just before asking.
Rod Rivera
Topic Hub
Filed under Harness Engineering
41 entries filed under this tag.
Harness Engineering
Harness Engineering11 min read
AI Tool Error Handling: Don't Fake an Empty Result
AI tool error handling must distinguish an empty catalog from unavailable data. Use separate result types and a mutation test that catches a swallowed timeout.
Harness Engineering13 min read
AI Agent Tool Design: One Job Per Tool Boundary
AI agent tool design should expose effects and permission boundaries. Compare an overloaded inventory function with explicit read and draft tools.
Harness Engineering11 min read
AI Agent Latency Measurement: Find the Real Delay
AI agent latency measurement means tracing the whole user-visible path, not just timing the model call that feels easiest to optimize.
Harness Engineering11 min read
Agent Evaluation Variance: Did A Really Beat B?
Agent evaluation variance can make one prompt look better by chance. Design paired repeated trials, report critical failures and inspect discordant outcomes.
Harness Engineering13 min read
Bounded LLM Repair Loop: Three Independent Limits
A bounded LLM repair loop needs separate attempt, deadline and work limits. Test late responses, exhausted repairs and forbidden writes with a fake clock.
Harness Engineering12 min read
AI Cache Authorization: When Answers Outlive Access
AI cache authorization needs current permissions, revision-aware keys and a defined release point. Test revoked access before and during an answer request.
Harness Engineering13 min read
Agent Prompt Injection Boundary: A Worked Case
A retrieved note claims approval for a purchase. See why the agent prompt injection boundary must still deny the write.
Harness Engineering12 min read
LLM Validation Error Feedback That Won't Grant Policy Control
Design LLM validation error feedback that identifies malformed fields, limits repair attempts and keeps budget decisions under application control.
Harness Engineering6 min read
Who Can Change a Dark Software Factory’s Tests?
A dark software factory can ask agents to improve its tests. Learn how to preserve acceptance authority, inspect proposed changes and challenge the checker with a faulty candidate.
Harness Engineering12 min read
AI Agent Trusted Configuration: Who Owns the Budget?
AI agent trusted configuration keeps budgets and prices outside generated proposals. Test top-level and nested overrides against a closed Python schema.
Harness Engineering6 min read
Dark Software Factories Still Need a Task Owner
Build a dark software factory around a concrete change request. Separate implementation from acceptance, preserve evidence and decide which decisions remain human.
Harness Engineering12 min read
Notebook to AI Application: Build a Callable Contract
Turn a working notebook cell into a callable contract with declared inputs, rejected cases and stable outputs before another service depends on it.
Harness Engineering13 min read
AI Agent Approval Boundary: Draft Accepted Isn't Purchase Made
An AI agent approval boundary means accepted_draft carries no supplier receipt. Learn to separate validation from execution.
Harness Engineering12 min read
MCP Token Audience Validation: A Worked Denial Case
Validate an MCP token for the receiving service as well as its issuer, expiry and scope. Test wrong-audience denial before protected work executes.
Harness Engineering10 min read
Agent Evaluation Environment: Freeze the Task Contract
An agent evaluation environment must freeze inputs, tools and an independent oracle, or the same message silently grades against two different answers.
Harness Engineering12 min read
AI Judge Calibration: Read the Confusion Matrix First
AI judge calibration means checking a trace evaluator against blind human labels before routing any case on its verdict alone.
Harness Engineering11 min read
LLM Duplicate Key Validation Before Dictionary Conversion
A dictionary of order lines can hide duplicate rows. Check the list for repeats before conversion, not after, to preserve evidence.
Harness Engineering11 min read
RAG Access Control: Stop Leaks Before the Prompt Builds
RAG access control means filtering by permission before retrieval, not after the answer is written. A worked two-user case shows where checks belong.
Harness Engineering11 min read
AI Reviewer Context Isolation: What Actually Contaminates
AI reviewer context isolation starts with the actual request payload. Build a tested allowlist and separate primary evidence from prior verdicts.
Harness Engineering9 min read
Scheduled publication timezone: the worked case
Why a date-only release rule needs one clock across build, request and cache before a scheduled publication timezone decision is trustworthy.
Harness Engineering11 min read
Design a Cache Experiment That Measures the Right Thing
Prompt caching measurement means separating context size, billed usage and latency instead of reading one counter as proof of all three.
Harness Engineering12 min read
AI Human Review Cost: Find the Review Queue Break-Even
Faster drafting can raise total cost if acceptance drops. Work the break-even arithmetic before you trust a lower per-call price.
Harness Engineering12 min read
Pydantic Strict Validation: Choosing What an Input May Become
Learn Pydantic strict validation to decide which conversions a quantity field should accept, and which ones deserve a clear rejection.
Harness Engineering12 min read
AI Agent Retry Policy: When to Retry, Repair, or Stop
An AI agent retry policy must treat timeouts, auth errors, schema faults, and policy refusals as different problems, not one loop.
Harness Engineering12 min read
Agent Tool Idempotency: Reconcile an Uncertain Write
A timed-out tool call leaves you unsure a write happened. Choose an idempotency key, and reconcile the result before retrying anything.
Harness Engineering11 min read
AI Workflow Baseline: What Time Saved Actually Means
Build an AI workflow baseline that counts setup, review and repair, separates elapsed time from human effort, and keeps the task mix comparable.
Harness Engineering13 min read
Durable Agent Execution: Resume Without Repeating Effects
A checkpoint that saves state is not the same as a receipt from the outside world. Learn what must persist before an actor is replaced.
Harness Engineering11 min read
Agent Memory Invalidation: Retire Stale Facts Correctly
A changed source revision should retire a stored fact without deleting its history. Here is how to tell which memories must go.
Harness Engineering11 min read
LLM Model Routing Evaluation From Accepted Work Cases
Evaluate model routes on ordinary tasks, missing data and policy conflicts, then set acceptance thresholds before comparing cost savings.
Harness Engineering11 min read
AI Agent Per User Credentials: Who Owns the Key?
AI agent per user credentials means separating who invokes an agent from which account its tools use, so one caller never inherits another's access.
Harness Engineering10 min read
Stateless MCP Application State: Where Identity Really Lives
Keep MCP job state durable while connections change. Check authenticated identity and job permissions on each protected request before resuming work.
Harness Engineering11 min read
AI Agent Preview Deployment: Test Without Real Side Effects
A new hostname doesn't mean new authority. Learn why AI agent preview deployment needs credential isolation, not just a branch deploy.
Harness Engineering11 min read
Translate a Coding Agent Benchmark Into a Release Decision
A coding agent benchmark evaluation measures one task distribution. Learn to check whether it matches your actual release decision.
Harness Engineering12 min read
Agent Regression Dataset: Save the Failure, Not the Fix
Learn to turn one failed agent run into an agent regression dataset that tests recovery without pretending to know the original cause.
Harness Engineering13 min read
LLM Business Rule Validation: JSON Parses, Now What?
Check an LLM proposal against trusted stock, prices and budget after its fields validate, with explicit errors for wrong quantities and stale data.
Harness Engineering13 min read
Write an AI Agent Handoff Checklist That Actually Verifies
Learn to write an AI agent handoff checklist that lets a new engineer verify state and evidence instead of trusting a summary.
Harness Engineering18 min read
Your Claude Code Permissions Are Your Org Chart
Anthropic measured it: users approve about 93% of permission prompts. A gate that opens 93% of the time is a doorbell. Here is what the permission system actually enforces, what it cannot, and the standing rules worth adopting before you delegate anything that matters.
Harness Engineering17 min read
What Claude Code Actually Costs — and Which Dials Move It
Your lunch break has a token price. The model remembers nothing between turns, so every request re-sends the whole conversation — which means cost scales with context carried, not work requested. Here is the cost function, the dials that matter, and the ledger line worth keeping.
Harness Engineering10 min read
When Can You Actually Walk Away?
Choose an agent task you can verify, bound its effects, and plan the handover. A practical delegation grid, six operating modes, and checks that can miss the real failure.
Harness Engineering10 min read
Your Chat History Is Not a System of Record
Make agent work resumable from records: capture decisions, source revisions, test evidence and unresolved effects, then check whether another session can continue safely.
