Production AI Architecture in Laravel and PHP

Put providers behind a domain interface

Controllers should not construct provider payloads or parse model-specific responses. Define application contracts such as GenerateStructuredAnswer, EmbedTexts, and StreamCompletion, with typed request and result objects. Provider adapters translate those contracts into APIs and normalize usage, finish reasons, errors, and tool calls. This keeps business services testable and makes provider routing possible without scattering conditionals across the codebase.

Configuration identifies model, timeout, token limits, and retry policy by task, not globally. Bind adapters through Laravel’s container and keep credentials in environment-backed secret configuration. Never expose keys to Blade, JavaScript, prompts, or logs. Validate all structured outputs with DTOs and domain rules after parsing. A model-produced object is external input even when it came through a trusted SDK.

Use fakes that replay recorded normalized results for unit tests, then maintain a smaller integration suite against real providers. Contract tests ensure every adapter implements streaming, cancellation, error mapping, and usage consistently. Pin model and prompt versions in traces. An interface is valuable only if semantic differences remain visible; do not normalize away a provider’s unsupported tool or context limitation.

interface StructuredGenerator
{
    public function generate(GenerationRequest $request): GenerationResult;
}

Move long AI work onto durable queues

Web requests should validate intent, create a run record, dispatch work, and return an identifier. Laravel queues handle model calls, ingestion, embeddings, and verification outside the request lifecycle. Persist each meaningful stage with status, attempts, configuration version, and result references. Configure worker timeouts above client deadlines but below job retry windows, and distinguish provider timeout from worker termination.

Jobs may run more than once, so assign an idempotency key to each logical model or tool operation. A generation retry is usually safe before effects; a tool that publishes or charges requires external idempotency and reconciliation. Use unique jobs or database constraints carefully because uniqueness expiration can outlive or undercut actual work. Record external operation IDs before acknowledging completion.

Use Horizon or equivalent queue telemetry to separate interactive, ingestion, and offline workloads. Set queue-specific concurrency, backoff, and priorities so a large embedding import does not stall user tasks. Propagate cancellation through run state and check it between expensive stages. Dead-letter failed jobs with enough structured context for recovery while excluding sensitive prompt payloads.

Stream without coupling work to the socket

Server-sent events can deliver tokens and progress to the browser, but the durable run should not depend on one PHP process holding a provider stream forever. A worker can publish bounded events to Redis or a database-backed channel while an authenticated endpoint streams them to clients. Sequence events and store a compact replay window so reconnecting browsers resume without duplicating text.

Separate token events from trusted status events. Never execute a tool because partial streamed text resembles JSON. Buffer and validate complete structured proposals at the worker. Escape rendered content, apply output policy, and treat markdown as untrusted. The client receives event types such as delta, stage, evidence, completed, and failed rather than inferring state from prose.

Apply backpressure and disconnect handling. If no client is listening, the run may continue when it creates a durable artifact, or cancel when its only value is live interaction. Make that policy explicit per task. Cap event storage and final output size. Streaming improves perceived latency, but it cannot substitute for queue deadlines, retries, and a recoverable final result.

Integrate retrieval, tools, and telemetry

Ingestion jobs parse sources, create versioned chunks, embed them in batches, and atomically publish an index generation. Store canonical content and permissions in authoritative tables; vector storage is an acceleration layer. Query services apply tenant and authorization filters before returning evidence. Cache keys include source generation, user scope, model, and query representation to avoid stale or cross-tenant results.

Expose model tools as narrow application services with form-request-like validation, policies, and typed results. Read tools and write tools use separate capabilities. Consequential actions return previews or approval requirements before execution. Laravel authorization policies run with the authenticated principal, never a user ID supplied by the model. Events and listeners can record receipts and trigger verification without hiding critical control flow.

Attach a run and trace ID to logs, queue jobs, model usage, retrieval candidates, tool calls, and user outcomes. Report tokens, provider cost, latency, retries, validation failures, and successful task completion. Use encrypted or redacted payload sampling for diagnosis. This architecture makes AI a well-governed subsystem of the Laravel application rather than a mysterious HTTP call embedded in a controller.

Research foundation and scope

Laravel's queues provide delayed and retried job execution, while its authorization policies centralize decisions about actions on resources [LARAVEL-QUEUES] [LARAVEL-AUTH]. Those mature application boundaries should contain AI providers, retrieval, tools, and streaming. These sources establish the mechanism, but they do not remove the need to measure the local implementation. A paper's result belongs to its checkpoint, dataset, hardware, traffic, and experimental protocol; a standard describes a contract or risk practice, not a guarantee that a particular product follows it. The useful engineering move is to convert each cited idea into a hypothesis that can be falsified with representative inputs and observable intermediate state.

A controller-bound model call inherits web-request deadlines and encourages provider details, authorization, and side effects to mix. Queue retries can duplicate effects unless jobs and tools are idempotent. Write these conditions into the design review before selecting a library or provider. Specify the authoritative source of truth, the authenticated principal, the versions that influence behavior, the maximum consequence of an error, and what the system will do when required evidence is missing. This boundary statement makes later optimization honest: teams can compare speed, cost, or convenience only after candidates satisfy the same correctness, authorization, privacy, and recovery contract.

Read [LARAVEL-QUEUES], [LARAVEL-AUTH], [OTEL-GENAI] together rather than treating one source as a recipe. Laravel queue documentation supplies one part of the foundation; Laravel authorization documentation supplies a second perspective; OpenTelemetry Generative AI semantic conventions helps connect the mechanism to evaluation, governance, or operations. The citations are deliberately primary papers, standards, or official documentation. Product-specific claims still require local evidence, and any dated behavior must be rechecked against the cited version before an article or architecture decision is refreshed.

A production implementation blueprint

Define provider-neutral domain interfaces, typed DTOs, policies, durable jobs, idempotency keys, effect receipts, transactional dispatch, stored stream events, and narrow tool services. Propagate tenant, principal, run, and trace identity. Begin with one thin vertical path that preserves all decision evidence. Give each run a stable identifier; capture normalized input, authenticated scope, relevant dependency versions, selected policy, timing, result status, and any external receipt. Keep large or sensitive payloads in a separately governed store and pass bounded references through the orchestration layer. This produces a debuggable system without making raw customer content the default telemetry substrate.

Make state transitions explicit and make every external boundary return a typed result. Distinguish validation failure, permission denial, missing evidence, rate limit, timeout, dependency outage, user cancellation, and internal defect. The distinction matters because each permits a different response: clarify, retry, fall back, defer, compensate, or stop. A generic exception handler that asks the model to try again turns uncertainty into repeated cost and can duplicate irreversible effects. Retries require an idempotency strategy and a stored receipt, not optimism.

Roll out the implementation as a versioned policy. Record which cohort receives it, what baseline it replaces, which metrics can stop the rollout, and how to reverse stateful artifacts such as indexes, caches, adapters, or workflow histories. Review privacy and retention alongside performance. If raw content is needed temporarily for diagnosis, state the purpose, access, sampling, encryption, and deletion window. Production readiness means the team can explain, contain, and reverse behavior under load—not merely demonstrate the happy path once.

Failure lab and evaluation plan

Feature-test provider adapters with recorded responses, run policy tests as multiple principals, retry jobs around every external boundary, and verify stream reconnection from persisted sequence numbers rather than rerunning generation. Keep the arrival pattern, data distribution, authorization, and dependency versions close to production. Use a paired baseline where possible and retain per-case results so averages cannot hide a severe slice. Offline evaluation should localize the failing stage; shadow evaluation should expose integration and traffic effects; a canary should confirm user-facing outcomes with a small blast radius. Each stage needs an exit criterion rather than a vague request to 'look good.'

Include worker timeout after provider success, database rollback after dispatch, duplicate webhook, expired model credential, validation error, disconnected SSE client, unauthorized model tool ID, stale retrieval index, and failed-job replay. Add adversarial boundary cases and partial failures, not only malformed inputs. Terminate a worker after an external system may have committed, revoke access after a cache was populated, change one dependency version, and delay one branch until its data is stale. Confirm that cancellation releases resources and that a fallback preserves semantics. The test passes only when the external outcome, audit evidence, and user-visible state agree; a fluent final message is not proof of completion.

Evaluation data needs lineage equal to production data. Record why each case exists, who reviewed the expected behavior, which source or policy supports it, and when it must be refreshed. Separate generated cases from production incidents and prevent test cases from leaking into training. When a metric changes, inspect changed examples before accepting an aggregate. A statistically small regression can still block release if it crosses an authorization, privacy, safety, or irreversible-action boundary.

Operating metrics and decision record

Track queue wait and age, attempts, provider latency and tokens, stream reconnect, policy denials, duplicate effects, completion by feature, failed-job recovery, cost per outcome, and trace completeness. Define every metric with a numerator, denominator, unit, time window, exclusions, and responsible owner. Segment by workload and consequence so a dominant easy class cannot hide a rare costly failure. Pair service signals with product outcomes: latency without completion encourages fast useless answers, while answer quality without queue and cost data can produce an uneconomic service. Review percentiles and distributions rather than relying on a single average.

Make AI an application subsystem governed by Laravel services, policies, queues, events, and tests; the provider SDK should remain an adapter at the edge. Put that choice in a short architecture decision record containing context, alternatives, evidence, assumptions, selected policy, rollout, reversal trigger, and unresolved risk. Link the evaluation snapshot and source versions. This document prevents a benchmark from becoming folklore and tells the next engineer which evidence must be repeated when the model, provider, traffic, data, regulation, or product promise changes.

Monitor leading and lagging signals after release. Leading signals reveal pressure—queue age, cache allocation, disagreement, validation failure, policy denial, or retry growth—before users report harm. Lagging signals show whether the job was actually completed and trusted. Alert on a service objective tied to user consequence, and make the run identifier available to support without exposing private content. Every serious miss should become a reproducible regression case and, when relevant, a new threat or data-quality control.

Primary sources and further reading

1. Laravel queue documentationLaravel · official documentation
2. Laravel authorization documentationLaravel · official documentation
3. OpenTelemetry Generative AI semantic conventionsOpenTelemetry · standard