Check every vendor claim before you rely on it.

Enterprise-ready. Private. Accurate. Responsible. These descriptions may start a useful conversation about an AI product. They cannot finish a purchasing decision.

The question is not whether the vendor has supplied enough documents. It is whether those documents support the claim your organization is about to rely on. A useful review connects a specific claim to an exact product, version, service arrangement and intended use. It also keeps the unanswered questions visible.

Consider a customer-service scenario; the team and project are hypothetical. A supplier says its assistant can resolve requests and protect customer information. Before judging that statement, separate the work involved. Drafting a reply, retrieving an approved answer and issuing a refund are different tasks. Each needs different evidence and different permissions.

Ask the supplier to make the claim precise. Which request types were tested? What counts as resolved? Was a person checking the answers? Which information was sent to the model? Does the privacy statement apply to the particular API, subscription and contract under consideration?

The same discipline applies to security language. “Enterprise secure” leaves too much unspecified. A documented access control or a clearly scoped retention commitment is something a buyer can examine. It is still necessary to establish whether it applies to the proposed service.

Provider documentation is often the direct source for a product feature, published limit or announced change. An external evaluation answers a different question under its own method. An observation from your deployment answers another.

These sources can complement one another without becoming interchangeable. A vendor can describe an available configuration without proving that your organization enabled it. A benchmark can report a result without establishing your integration, workflow or data quality.

NIST describes its AI Risk Management Framework as voluntary guidance for considering trustworthiness throughout AI design, development, use and evaluation. That is a framework for managing risk, not evidence of a particular deployment's performance. [1]

Source context: NIST AI Risk Management Framework ↗

ISO/IEC 27001 concerns an organization's information-security management system. Implementation and certification are distinct. A buyer should examine a certificate's entity, scope and period rather than treat it as proof that an AI model produces accurate answers. [2]

An evaluation has a date. So does a source edition, a contract and the review that relied on them. These dates need not be the same.

A retained source can show what was documented when it was reviewed. It does not automatically establish what is true today. A new visit to a webpage also does not make an old evaluation newly performed.

At ETHORITY, reviewed, last-confirmed and current are not interchangeable assurances. The useful question is what was checked, against which retained source and for which claim. Preserve the product version, deployment route, evidence date and conditions that mattered. A material change is a reason for a reasoned check, not an automatic reversal of the decision.

A limitation is not an inconvenient footnote. It tells the buyer where additional work is needed.

In the customer-service example, an evaluation of answer quality would not establish permission to refund a payment. A successful demonstration would not establish performance across every customer request. A stated hosting option would not establish the actual account configuration.

A practical record can be simple: the decision question, the claim, its source and scope, the known limitation, and the next check. Assign that check to someone. “Confirm deletion behavior for our chosen service before providing customer records” is more useful than a generic red or green vendor label.

A provider that clearly describes a limitation may help you rule out an unsuitable use. That is valuable transparency, even when it leads to a decision not to buy. Missing documentation, conversely, is not proof of misconduct. It means a relevant point has not yet been established.

Our proposed transparency research asks which decision-relevant fields are documented, which are independently supported, where evidence is old and where sources conflict. It does not assume those observations can become a universal trust percentage. Planned research is not a published finding.

Procurement should leave an understandable record of why the option was selected, which assumptions mattered and what would trigger another review. Keep that record with the project, not only with purchase paperwork.

For the service-assistant scenario, triggers might include a different model version, permission to take a new action, a new hosting route or repeated incorrect answers. Their significance depends on the actual workflow. A change notice is a prompt to examine relevance, not automatic proof that the system is unsafe.

Write the five statements that would change your buying decision. For a document assistant, these might concern permitted data, the location of processing, the use of submitted material for training, deletion and the price of the expected workload. Avoid filling the sheet with every available feature. Its purpose is to reveal the assumptions on which the business is about to depend.

For each statement, record the exact product and plan, the source, the source date, the relevant condition and the person who must resolve a gap. A general company privacy page may not describe the API contract you are considering. A published enterprise feature may require an additional agreement. Preserve those conditions in the comparison so that a short card does not turn a conditional option into an unconditional promise.

The supplier meeting then becomes more productive. Instead of asking whether the product is secure, ask which entity operates the selected service, which subprocessors receive the relevant data and which contractual document governs deletion. The answer may require a written follow-up. Record that dependency without interpreting silence as proof of either safety or misconduct.

Suppose a provider presents an information-security certificate. Establish the certified legal entity, the covered activities, the validity period and the relationship to the service under review. If the certificate concerns an office or a management system, do not extend its meaning to model accuracy. If the product is supplied through another company, clarify the connection rather than assuming that a shared brand establishes coverage.

The same discipline applies to a penetration-test summary, an audit report or a benchmark result. Each answers a particular question with a particular method. A restricted report may contain evidence that a public marketing page cannot show; the appropriate response is to arrange an authorised review, not require confidential material to be published. Public transparency and access to sufficient purchasing evidence are related but different objectives.

Separate three possible conclusions: the document supports the claim; it supports a narrower claim; or it does not establish the claim you need. This language produces a useful conversation without creating a competing certification scheme. Where specialist interpretation is required, assign it to a qualified reviewer and preserve the exact question. Buying a review service should never guarantee the supplier a favourable result.

A product demonstration usually follows a prepared path. Your evaluation should include the actual range of work: ordinary documents, incomplete records, conflicting sources, unsuitable requests and the point at which a person must intervene. Define acceptance before showing the supplier the test set. Otherwise the project can redefine success around whatever the tool happens to do well.

For a hypothetical service assistant, an accepted answer could require correct use of the current policy, support for each consequential statement and an appropriate escalation when the record is incomplete. Measure the whole case rather than counting a fluent paragraph as completion. Include the time a reviewer spends checking the answer. Record which cases were excluded, because an apparently strong rate can depend on leaving difficult work outside the denominator.

Use representative data under an appropriate agreement. Sensitive production records should not be copied into a public demonstration simply to make it realistic. A synthetic test can explore a failure mode, but it cannot establish production performance. When moving to a controlled pilot, explain what new evidence the pilot must supply and which earlier assumptions remain untested.

One supplier may charge per user, another per conversation and another per model usage. Those prices are not directly comparable until the workload is defined. Include minimum commitments, overages, required platform fees, implementation, human review and the cost of unresolved cases. Keep currency, tax basis and billing period visible. A lower headline price can produce a higher total cost when the scope differs.

The economics tools help explore assumptions; they do not promise a return. Use them to calculate the cost of an accepted outcome and inspect sensitivity to volume, acceptance rate and review effort. If a vendor's claimed savings assume that every suggested answer is used without review, show how the result changes when your actual controls are included.

Avoid presenting released staff time as cash savings by default. Identify whether the business plans to serve more demand, reduce overtime, avoid hiring or reduce an actual expense. The financial effect depends on that choice. A transparent scenario with a qualified assumption is more useful than a precise-looking ROI number whose economic mechanism is missing.

Sometimes two primary sources disagree. A pricing page may describe one allowance while a contractual schedule describes another; a product page may advertise a region that the selected plan does not offer. Show the particular difference, link the sources and say which fact must be confirmed. Do not average the statements or silently prefer the more attractive one.

Likewise, hide irrelevant empty fields instead of producing a wall of “unknown” labels. Preserve the unanswered questions that can change the decision. Ask about the relevant data region, training use, certifications, price, retention and deletion, and meaningful human review where the use requires it. A concise question gives the buyer a useful next action and lets the provider answer at the correct scope.

End the supplier conversation with three lists: facts established for the selected service, conditions that still require confirmation and assumptions to test in a pilot. Give each unresolved item a recipient and a decision consequence. For example, an unanswered deletion question may prevent the use of customer records while leaving a public-information test possible.

Ask the team to explain why the strongest remaining alternative was not chosen. This can expose an inconsistent comparison, such as treating one quote as complete while excluding integration from another. Preserve the reasons in plain language. When circumstances change, the organisation can revisit the actual trade-off rather than reconstruct it from a sales presentation. A concise ledger is valuable because another person can use it, not because it contains every document encountered during research.

The selected option should come with the reason it was chosen, the evidence relied on, the remaining conditions and the owner of the next review. Include an exit question: what happens to data, integrations and outstanding work when the contract ends? A purchase is the beginning of an operating relationship, not the end of evidence management.

Explore relevant alternatives in the Good AI Prism and carry the same questions through Profile and Compare. If you need a focused comparison of up to three offerings, inspect the AI Decision Brief. It is a professional analysis requested for a defined question, not an admission fee, product endorsement or purchase order. Its value should be a decision you can explain, including a well-supported reason to wait or choose a non-AI alternative.

A reviewed source-backed claim is not automatically an independently verified result. When verification is claimed, the evaluator, criteria, scope, method and outcome should be identifiable.

ETHORITY separates the public market profile from a customer-specific decision. The profile explains the offering, its sources and unresolved questions. An agreed advisory engagement adds the customer's intended use, constraints and decision criteria.

The aim is not a confident answer at any cost. It is a decision whose support and limitations another person can understand. Buying AI should leave you knowing what is supported, what remains uncertain and what to establish next.

Sources and scope [1] NIST, AI Risk Management Framework, official overview; checked 26 September 2026: https://www.nist.gov/itl/ai-risk-management-framework [2] ISO, ISO/IEC 27001:2022, official overview; checked 26 September 2026: https://www.iso.org/standard/27001 ETHORITY perspective, not legal advice, a vendor evaluation or certification. The customer-service use case is a scenario, not an observed customer result. External sources support the attributed points, not an endorsement of ETHORITY.