AgentSmyth published Engineering Note 1 and a companion Verdict rules v1 document setting out how it decides its research agents are ready to ship. Each run must clear nine minimum checks, each agent eval case is run three times with the worst run deciding the release gate, and an unproven check never counts as a pass. The rules also define what every evaluation run must record (code version, model and provider, instruction and tool versions, data snapshot, sources fetched) and state that a client's own rules can add requirements but never remove the house standard.
Our readA bank or broker validating this platform under model risk rules can read the vendor's written release criteria and evaluation record format instead of relying on a demo. It gives a validation team concrete controls to test against, such as worst of three gating and the per run evidence record.