AI Product UX for Systems That Can Be Wrong

Start with a job, not a chat box

A blank chat box transfers product design to the user. For repeatable work, provide task-specific entry points, examples, accepted inputs, and clear outcomes. Collect constraints through ordinary controls where precision matters: date pickers, account selectors, permission scopes, and file choices. Natural language remains valuable for intent and nuance, but it should not replace every reliable interface element.

Set expectations about capability, data use, time, and whether actions can occur. Distinguish drafting from execution. A button labeled “Create campaign” should not ambiguously mean generate a proposal or publish ads. Show the sources and accounts the system can access before work begins. Users form trust from accurate boundaries, not theatrical claims that the AI can do anything.

Preserve user agency with editable plans and constraints. For complex tasks, summarize the interpreted objective and ask only about missing decisions that materially change the result. Do not interrogate users for details the system can safely discover. Defaults should be visible and reversible. The fastest experience is one that avoids both unnecessary questions and costly incorrect assumptions.

Make progress and evidence legible

Streaming text creates motion but not necessarily understanding. For long tasks, show meaningful stages such as searching approved sources, analyzing files, waiting for review, and verifying changes. Avoid exposing hidden chain-of-thought. Users need operational state, elapsed progress, and opportunities to cancel or amend. A truthful “waiting for access” is more useful than an endlessly animated thinking indicator.

Ground factual outputs with citations users can inspect in context. Deep links to source passages, timestamps, or records are more valuable than decorative footnote numbers. Distinguish source-backed facts, model suggestions, and assumptions visually and in data structure. When evidence conflicts, show the conflict rather than smoothing it into certainty. This makes correction faster and teaches users when extra review is warranted.

Confidence labels should correspond to calibrated product decisions, not arbitrary percentages. Use concrete language such as verified from account data, supported by two documents, or needs review because the scan is unclear. Pair uncertainty with an action: inspect source, provide missing information, or escalate. A warning without a recovery path becomes habituated noise and eventually loses protective value.

Confirm effects at the right boundary

Confirmations should appear before consequential actions and show exact scope: recipients, amounts, files, visibility, and irreversible consequences. Group low-risk reversible edits into one review while isolating high-impact operations. Do not repeatedly ask approval for safe reads; confirmation fatigue trains users to click through dangerous prompts. Risk-tier actions using domain rules, not the model’s judgment alone.

Preview generated artifacts in the form they will take after execution. An email preview should show recipients and attachments; a code change should show a diff and tests; a calendar action should show timezone and participants. Highlight values derived from untrusted or uncertain sources. Bind approval to that exact version so subsequent model edits require renewed review.

After execution, provide a receipt with result, identifier, timestamp, and undo or recovery where available. If status is ambiguous, say reconciliation is underway rather than claiming success. Make partial completion explicit. Users should never need to infer whether an agent merely proposed, attempted, or actually performed an action from conversational phrasing.

Design correction as a primary path

AI outputs should be easy to edit without restarting. Preserve structured fields, citations, and provenance as users make changes. Offer targeted regeneration for a section or field rather than replacing approved work. If the user corrects a fact, update the active state and show where that correction will apply. Do not silently write every correction into long-term memory.

Failure messages should identify what happened, what remains safe, and the next available action. Tool denial, missing evidence, timeout, and policy refusal need different recovery. Retain partial drafts when appropriate and avoid making users repeat inputs after transient failure. For dangerous or privacy-sensitive failures, clear affected state and explain why. Recovery is part of perceived intelligence.

Measure correction rate, time to accepted result, undo, abandonment, escalation, citation use, and repeated attempts by task. Pair behavioral data with research because a low edit rate may indicate perfection or resignation. Feed reviewed friction into model and interface improvements. The best AI UX does not hide fallibility; it contains it inside a workflow where users can understand, verify, and recover.

Research foundation and scope

Microsoft's human-AI interaction guidelines and Google's People + AI Guidebook emphasize setting expectations, supporting efficient correction, and designing around human needs [HAX] [PAIR]. NIST adds risk management and human-oversight concerns [NIST-GAI]. These sources establish the mechanism, but they do not remove the need to measure the local implementation. A paper's result belongs to its checkpoint, dataset, hardware, traffic, and experimental protocol; a standard describes a contract or risk practice, not a guarantee that a particular product follows it. The useful engineering move is to convert each cited idea into a hypothesis that can be falsified with representative inputs and observable intermediate state.

A conversational interface can hide uncertainty, source gaps, progress, cost, and side effects. The appropriate interaction depends on the user's job, consequence, expertise, time pressure, and ability to verify. Write these conditions into the design review before selecting a library or provider. Specify the authoritative source of truth, the authenticated principal, the versions that influence behavior, the maximum consequence of an error, and what the system will do when required evidence is missing. This boundary statement makes later optimization honest: teams can compare speed, cost, or convenience only after candidates satisfy the same correctness, authorization, privacy, and recovery contract.

Read [HAX], [PAIR], [NIST-GAI] together rather than treating one source as a recipe. Guidelines for Human-AI Interaction supplies one part of the foundation; People + AI Guidebook supplies a second perspective; Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile helps connect the mechanism to evaluation, governance, or operations. The citations are deliberately primary papers, standards, or official documentation. Product-specific claims still require local evidence, and any dated behavior must be rechecked against the cited version before an article or architecture decision is refreshed.

A production implementation blueprint

State capability and limits in context, show progress and evidence, distinguish draft from committed state, preview consequential actions, provide undo or recovery, preserve user edits, and offer structured controls when choices are known. Begin with one thin vertical path that preserves all decision evidence. Give each run a stable identifier; capture normalized input, authenticated scope, relevant dependency versions, selected policy, timing, result status, and any external receipt. Keep large or sensitive payloads in a separately governed store and pass bounded references through the orchestration layer. This produces a debuggable system without making raw customer content the default telemetry substrate.

Make state transitions explicit and make every external boundary return a typed result. Distinguish validation failure, permission denial, missing evidence, rate limit, timeout, dependency outage, user cancellation, and internal defect. The distinction matters because each permits a different response: clarify, retry, fall back, defer, compensate, or stop. A generic exception handler that asks the model to try again turns uncertainty into repeated cost and can duplicate irreversible effects. Retries require an idempotency strategy and a stored receipt, not optimism.

Roll out the implementation as a versioned policy. Record which cohort receives it, what baseline it replaces, which metrics can stop the rollout, and how to reverse stateful artifacts such as indexes, caches, adapters, or workflow histories. Review privacy and retention alongside performance. If raw content is needed temporarily for diagnosis, state the purpose, access, sampling, encryption, and deletion window. Production readiness means the team can explain, contain, and reverse behavior under load—not merely demonstrate the happy path once.

Failure lab and evaluation plan

Usability-test first use, uncertainty, partial failure, correction, long-running work, approval, cancellation, and recovery. Compare whether users can predict what the system did and detect an incorrect result before acting. Keep the arrival pattern, data distribution, authorization, and dependency versions close to production. Use a paired baseline where possible and retain per-case results so averages cannot hide a severe slice. Offline evaluation should localize the failing stage; shadow evaluation should expose integration and traffic effects; a canary should confirm user-facing outcomes with a small blast radius. Each stage needs an exit criterion rather than a vague request to 'look good.'

Include confident wrong output, missing source, delayed tool, revised answer, lost connectivity, unsafe recommendation, repeated confirmation, inaccessible streaming, small-screen overflow, and a correction the model later overwrites. Add adversarial boundary cases and partial failures, not only malformed inputs. Terminate a worker after an external system may have committed, revoke access after a cache was populated, change one dependency version, and delay one branch until its data is stale. Confirm that cancellation releases resources and that a fallback preserves semantics. The test passes only when the external outcome, audit evidence, and user-visible state agree; a fluent final message is not proof of completion.

Evaluation data needs lineage equal to production data. Record why each case exists, who reviewed the expected behavior, which source or policy supports it, and when it must be refreshed. Separate generated cases from production incidents and prevent test cases from leaking into training. When a metric changes, inspect changed examples before accepting an aggregate. A statistically small regression can still block release if it crosses an authorization, privacy, safety, or irreversible-action boundary.

Operating metrics and decision record

Use task success, time and steps to correction, calibrated trust, verification behavior, approval comprehension, undo and escalation, abandonment, accessibility completion, and severe unobserved error. Define every metric with a numerator, denominator, unit, time window, exclusions, and responsible owner. Segment by workload and consequence so a dominant easy class cannot hide a rare costly failure. Pair service signals with product outcomes: latency without completion encourages fast useless answers, while answer quality without queue and cost data can produce an uneconomic service. Review percentiles and distributions rather than relying on a single average.

Optimize for the user's ability to reach, verify, and recover the intended outcome—not for how human the conversation appears. Put that choice in a short architecture decision record containing context, alternatives, evidence, assumptions, selected policy, rollout, reversal trigger, and unresolved risk. Link the evaluation snapshot and source versions. This document prevents a benchmark from becoming folklore and tells the next engineer which evidence must be repeated when the model, provider, traffic, data, regulation, or product promise changes.

Monitor leading and lagging signals after release. Leading signals reveal pressure—queue age, cache allocation, disagreement, validation failure, policy denial, or retry growth—before users report harm. Lagging signals show whether the job was actually completed and trusted. Alert on a service objective tied to user consequence, and make the run identifier available to support without exposing private content. Every serious miss should become a reproducible regression case and, when relevant, a new threat or data-quality control.

Primary sources and further reading

1. Guidelines for Human-AI InteractionMicrosoft Research · official documentation
2. People + AI GuidebookGoogle · official documentation
3. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence ProfileNational Institute of Standards and Technology · standard