Fiddler AI

The strongest centrality case of the three governance vendors in this pocket, and still B. Unlike a pure governance system of record, this vendor ships its own inference as product: named evaluator model families that score hallucination, toxicity and personal data leakage, plus explanation computation using Shapley values.

But the platform underneath is observability infrastructure, and stripping the evaluator models leaves drift statistics, performance monitoring, alerting and audit capture, which is a working product and historically was the whole product. B for the same structural reason recorded on the two comparable platforms: the closer a vendor product sits to genuine oversight of other people models, the less of its own value comes from inference.

Held at B on a precedent that decides it cleanly. The mechanism is well placed and well described for once: guardrails sit in the execution path rather than beside it, moderating, intervening and blocking agent conversations in real time, detecting personal data leakage before exposure, preventing deviation from approved scripts, and enforcing human approval on high value loans. That is better positioned than most claims in this index.

But the enforcement is described as blocking regulatory violations and policy breaches without naming one rule, and the human approval threshold is left to the customer with no default and no floor. That is precisely the failure recorded against the collections agent elsewhere in this index, and the A bar set by the consumer calling vendor requires a control that cites the rule it enforces. Same reasoning, same grade.

Dense and specific for what the product does: explanation by Shapley values and a proprietary variant at local and global level, what if analysis on prediction outcomes, drift root cause analysis down to contributing features, segment level analysis covering feature impact, correlation and distribution, and generative metrics named individually as faithfulness, context relevance, coherence and consistency. Model risk and governance reporting workflows are supported explicitly.

Held off A on the distinction that governs this pocket: every one of those mechanisms is applied to the customer models, and nothing documents the vendor own evaluator models, their validation, their error rates or their drift policy. A banked check worth running: a third party directory reports documented alignment to United States supervisory model risk guidance with reports generated for periodic examination, which if confirmed in the vendor own material would likely move this to A.

One named customer with a named executive quoted on production monitoring, which meets the B bar, though that customer is outside financial services. The financial services evidence is real but anonymised: a Fortune 100 institution across wealth, brokerage and asset management, an investment grade institution deploying evaluator models inside its own virtual private cloud, and a consumer lending platform reporting a large annual hours saving.

The hours figure was located in a third party directory rather than the vendor own material and is therefore not credited. Held at B: the named reference and the quantified outcomes never join, which is the standing shape on this axis.

The pooled corpus question is answered by architecture rather than by promise, which is the better way to answer it. Evaluation runs on the vendor own model families executing inside the customer estate, with no external model interface call, so customer prompts and outputs are not transmitted to a third party for scoring. That is a structural answer to the concern most vendors address with a marketing sentence.

Held off A against the reference bar set elsewhere in this index, which additionally requires an explicit statement that customer data never influences another customer results, a stated model hosting policy, supplier retention posture and concrete lifecycle terms. None of those is published here.

No privacy posture, processing terms, retention schedule or subprocessor list located. Personal and health data leakage detection is shipped as a product feature, which is a capability sold rather than a posture disclosed, and the in environment architecture is recorded on the deployment row where it belongs. Deliberately not double counted: one architectural fact should move one grade, and it moved deployment.

No certification, attestation or trust portal located in the vendor own swept material during this pass, so nothing is credited. Banked check and a likely miss rather than a genuine absence: a platform selling into a Fortune 100 financial institution and deploying inside customer cloud estates will have passed enterprise security review repeatedly and will hold attestations. Look for a trust or security page and for a documentation portal, which for a developer facing product is where this material usually sits.

No licence, no supervised test of the product by a financial regulator and no programme enrolment. Producing reports intended for supervisory examination is a product feature bought by a supervised institution, not standing held by the vendor, and the distinction is the same one applied to the two comparable governance platforms in this pocket.

The third instance of one pattern in a single pocket, and together they make a finding. This vendor ships the most concrete fairness capability located anywhere in this index, four named metrics available out of the box, disparate impact, group benefit, equal opportunity and demographic parity, explicitly aimed at credit card approval and lending decisions.

It publishes nothing about fairness testing of its own evaluator models, which decide what counts as toxic, what counts as a policy breach and which agent conversations get blocked. Those are consequential judgements applied to customer communications, and a model that moderates language has well documented potential for uneven performance across dialect and demographic group. Selling four fairness metrics and publishing none about yourself is the sharpest version of the gap this axis exists to record.

No published position on responsibility when a guardrail fails to block, when an evaluator wrongly flags, or when a governed agent causes harm the platform did not surface, and no route for an affected individual, who is two steps removed from this vendor. Worth recording that the exposure is real rather than theoretical here: a product whose function is to block and escalate is one whose failures are silent by construction, since a violation that was not detected leaves no artefact.

The vendor names its own evaluator model families and states that customers may instead use custom evaluators or external judging models, which tells a buyer where inference happens and who operates it. It names no base model, provider, version, architecture or training provenance for any of them. Recording the distinction because it is easy to miss and will recur: this is a deployment disclosure that reads like a supply chain disclosure. Saying the models run in your environment answers where, and says nothing at all about what.

Integrates at the layer that matters for this product class, governing agents through the gateway the enterprise already runs rather than requiring a parallel path, with a published interface, model pipeline integration and reporting that feeds existing model risk and governance workflows. Coverage extends to first party, third party and coding agents, so it reaches software the customer did not build, which is the harder half.

Held off A because no core banking, loan origination, decisioning or model operations platform is named as a certified connector, so integration is described by architectural position rather than by counterparty.

Genuinely earned and rare on this axis, which sits at C for most of the index. The evaluator models run inside the customer virtual private cloud, so generative agents can be governed with no data sharing and no call to an external model interface, and the vendor states this as an architectural property rather than a configuration option. That answers the two questions a regulated buyer actually asks, where the data goes and whether it leaves the estate. Held off A because no region list, tenancy model, residency guarantee or subprocessor detail is published.

No price, tier, unit or pricing basis located. One adjacent commercial claim is published and is unusual enough to record: the vendor positions its own in environment evaluator models as removing external model interface costs and fees, which is a cost argument about a competitor architecture rather than a disclosure of its own.

Within financial services the span is wide and specifically addressed: banks, credit card organisations, consumer lending platforms, fintechs, and a Fortune 100 institution running wealth management, brokerage and asset management divisions. Use cases reach credit, underwriting, automated lending, fraud, payment default, collections, robo advisory and trading.

Held off A because financial services is one vertical of a horizontal platform that also sells to technology, healthcare and government, and because the institutions evidencing the breadth are described by size and type rather than named.