The AI residency decision; where the foundation model inference physically runs; is one of the most consequential infrastructure choices in 2026 and one of the choices most often deferred to whatever the engineering team’s default cloud account happens to be. The default is wrong often enough that the decision deserves a deliberate frame: hyperscaler default region (the cheap option), hyperscaler-with-residency-pinning (the GDPR-and-region-aware option), sovereign cloud (the EU-data-sovereignty option), and on-prem or air-gapped (the regulated option). Each shape has a real cost, a real benefit, and a specific set of cases where it is the only viable answer. The HIPAA covered entity, the EU AI Act Annex III deployer, and the cleared contractor each have constraints that compress the decision differently. This piece names the four shapes, the four constraint dimensions (latency, residency, cost, talent), and the three regulated cases that compress the decision into a single answer. The goal is to make the residency decision visible at design time rather than discovered at audit time.
The AI build-vs-buy-vs-hire decision matrix for 2026 sets out, as its third principle, that foundation models are buy permanently; this piece is the residency dimension of that buy decision and where it stops being a pure buy.
Why residency is the under-named decision
In a generic SaaS context, residency is a checkbox: pick US-East, EU-West, or APAC, sign the DPA, move on. The vendor handles the rest, and the engineering team rarely thinks about it after deployment.
In an AI context, residency is a multi-dimensional decision that interacts with the foundation model availability matrix. Not most foundation model is available in most region; not most model is available in most cloud; not most cloud is approved for most regulated workload. The engineering team that makes a residency choice in week two of an engagement is making a choice that constrains which models the AI feature can use for the next three years. That constraint is invisible at the moment of the choice and load-bearing forever after.
The under-naming is structural. Cloud architects understand region selection. AI engineers understand model selection. The intersection; which models work in which regions, with which compliance posture, at which latency; is owned by neither role and is the seam where most residency mistakes happen.
The four constraint dimensions
Residency decisions weigh four dimensions.
Latency. Inference latency depends on where the user is, where the model runs, and the round-trip time. A US-West user calling a US-East model has 60 to 80 ms of network overhead before any inference time. A European user calling a US model has 100 to 150 ms. Streaming partially mitigates the perceived latency; non-streaming workloads (eval batches, function calls, tool use) feel the full cost.
Residency. Where the data physically resides during inference and what happens to it afterward. GDPR distinguishes data transfer to non-adequate countries; HIPAA requires PHI to flow only through covered entity infrastructure; EU AI Act Annex III applications inherit residency obligations from the underlying data. The vendor’s data handling defaults are rarely the buyer’s residency requirements.
Cost. The same model can cost 1.5x to 3x more in a sovereign region than in the cheapest hyperscaler region. On-prem inference has a different cost shape entirely; capex and ops rather than usage. The cost dimension is the dimension most likely to drive the default choice, but the savings can be illusory once compliance gaps are priced.
Talent. Operating different residency shapes requires different operational skill. Hyperscaler default is the easiest to staff. Sovereign cloud requires familiarity with the specific provider’s tooling. On-prem requires GPU operations capability that most teams do not have and cannot easily hire for. The talent dimension is the dimension most often underestimated.
The residency decision is a tradeoff across these four dimensions. Each shape below is a different combination of the tradeoffs.
Shape 1: hyperscaler default region
Shape 1 is what happens when no decision is made. The engineering team’s existing cloud account is the AI account. The foundation models run in whatever region the cloud account defaults to. The data flows where the vendor sends it. The compliance posture is whatever the vendor’s standard DPA happens to provide.
The pros: lowest cost, easiest to staff, fastest to deploy. The default works for the vast majority of non-regulated workloads where the data is not personal data and not subject to industry-specific regulation.
The cons: no defensible answer to residency questions. If a customer asks where their data is processed, the answer depends on what region the inference call routed to that day. If a regulator asks for documentation, there is none. If the buyer expands into a regulated industry segment, the residency choice becomes a re-platform.
Shape 1 is the right shape for early-stage products that are not in regulated industries and not handling regulated data. It is the wrong shape for any product whose residency posture might matter within a 24-month horizon.
Shape 2: hyperscaler with residency pinning
Shape 2 takes the hyperscaler convenience and adds explicit residency configuration. The cloud account is configured to pin AI inference to specific regions. The DPA is the regional DPA, not the global one. The data-flow documentation specifies which regions the data passes through and where it is processed.
The pros: most of the cost and talent advantages of shape 1, with a defensible residency posture. The buyer can answer regulator and customer questions about residency. Most major hyperscalers offer this configuration in 2026 with good model coverage in EU regions; coverage in APAC regions is still uneven.
The cons: model availability is constrained by region. The newest foundation models often launch in US regions first; EU and APAC regions can lag by 3 to 9 months. The engineering team must accept being on a slightly older model in non-US regions or accept the latency of cross-region calls. The pinning configuration is operationally fragile; engineers can accidentally bypass it with a misconfigured client.
Shape 2 is the right shape for most enterprises with EU users and a real residency posture requirement. It is the shape that the AI-aware MSA template described in the AI procurement maturity model is designed to support; model swap notification matters more here because regional model availability changes affect what the buyer can run.
Shape 3: sovereign cloud
Shape 3 uses a sovereign cloud provider; an EU-based cloud provider with strict data residency, often EU-owned and operated, with a compliance posture designed for EU regulatory frameworks. Examples include the major sovereign cloud offerings from European providers and the dedicated sovereign tenants offered by some hyperscalers.
The pros: defensible sovereignty posture for EU regulated workloads. The cloud provider is in EU jurisdiction; the data does not transit non-EU infrastructure; the operators are subject to EU law and not foreign government access regimes (the CLOUD Act question is moot). For EU public sector and regulated industries, this is often the only acceptable shape.
The cons: model availability is significantly more constrained. Sovereign clouds typically offer a smaller subset of foundation models than hyperscalers; some of the leading commercial models are not available at many. Cost is higher (typically 1.5x to 2x). Operational tooling differs from hyperscaler tooling; the talent pool is smaller. Build velocity is slower because the platform features the engineering team is used to may not exist.
Shape 3 is the right shape for EU public sector workloads, EU regulated industries with strict sovereignty requirements, and EU enterprises whose buyers (often other EU enterprises or governments) require sovereign hosting. It is overkill for most private-sector EU workloads where shape 2 with regional pinning is sufficient.
Shape 4: on-prem or air-gapped
Shape 4 runs the AI inference inside the buyer’s own infrastructure. The foundation model is either an open-weights model running on the buyer’s GPUs or a vendor-provided model running on dedicated hardware in the buyer’s data center. The data does not leave the buyer’s network.
The pros: maximum data control. The buyer’s infrastructure runs the inference; no third-party processor; no external data flow. For workloads where the data is so sensitive that any external processing is unacceptable (classified workloads, certain healthcare workloads, certain financial workloads), shape 4 is the only viable shape.
The cons: dramatically different cost shape. The buyer absorbs GPU capex, GPU operations, model deployment infrastructure, and ongoing model updates. The talent requirement is acute; GPU operations and on-prem MLOps are scarce skills, and experienced operators command top-of-market compensation. Build velocity is slower; iteration cycles measured in days rather than minutes. Model coverage is constrained to open-weights models or vendor-licensed models that can run on-prem.
Shape 4 is the right shape for a small set of regulated cases (classified, certain HIPAA, certain financial) and the wrong shape for almost everything else. The cost and talent overhead are large enough that shape 4 should be chosen because it is required, not because it sounded prudent.
The HIPAA case
HIPAA covered entities must ensure that PHI is processed under a Business Associate Agreement (BAA) with each AI vendor in the data flow. Major hyperscalers offer BAAs that cover their AI services in specific regions. Foundation model vendors offer BAAs that cover specific deployment configurations.
The HIPAA case typically resolves to shape 2 (hyperscaler with residency pinning to a HIPAA-eligible region) or shape 4 (on-prem) for the most sensitive subset of PHI workloads. The decision hinges on whether the BAA coverage is acceptable for the specific PHI types in scope.
The trap is that not most foundation model variant is BAA-covered. Even within a HIPAA-eligible region, some experimental or preview models may be excluded from the BAA. The engineering team that uses a non-BAA-covered model on PHI has produced an HHS-reportable incident, regardless of the model’s technical performance. The procurement function should explicitly enumerate which models are BAA-covered and which are not, and the engineering function should enforce the constraint at the routing layer.
The EU AI Act Annex III case
EU AI Act Annex III defines high-risk AI systems; systems used in employment decisions, education, essential services, law enforcement, migration, and justice. High-risk systems have substantial conformity assessment, transparency, human oversight, and post-market monitoring obligations.
The EU AI Act case rarely forces shape 3 or shape 4. EU AI Act compliance is largely about documentation and process rather than residency. A US-hosted AI service can be Annex III compliant if the provider produces the required Annex IV technical documentation, the conformity assessment, and the post-market monitoring artifacts.
The EU AI Act case does interact with residency for the post-market monitoring obligation. The deployer must monitor the AI system’s performance in production and report incidents to the relevant authority. If the AI system runs in a non-EU region, the data needed for monitoring may be subject to transfer restrictions that complicate the monitoring obligation. Shape 2 (EU-region pinning) is the most common Annex III-compatible shape; shape 3 is required only when the underlying data has explicit sovereignty requirements that pre-exist the AI system.
The cleared contracting case
The cleared contracting case; work for the US federal government at classified or controlled levels, or analogous government work in other jurisdictions; has hard residency requirements that compress the decision into shape 4 or a shape 3 variant operated by a cleared cloud provider.
For US federal classified work, the AI inference must run in a cleared environment (IL5, IL6 cloud regions, on-prem cleared facility). Foundation model availability in cleared environments is limited; the buyer typically must use models that have been specifically authorized for the cleared environment, which is a small subset of commercial models.
For controlled work (CUI, ITAR), the residency requirement is FedRAMP High or equivalent, which is available in cleared regions of the major hyperscalers but not in standard regions. The model availability matrix is better than in classified environments but still constrained.
The cleared contracting case is the most compressed of many the regulated cases. The engineering team’s freedom to choose models is heavily reduced; the procurement function’s freedom to choose vendors is reduced to vendors with cleared certifications. The talent requirement adds clearance overhead. Build velocity is dramatically slower than in commercial environments. Buyers in this case should plan for 2x to 3x build timelines compared to equivalent commercial work.
Frequently asked questions
Can we change residency shape mid-engagement?
Technically yes, operationally expensive. Migrating from shape 1 to shape 2 takes 4 to 8 weeks for a single AI feature. Shape 2 to shape 3 is a 3 to 6 month migration. Shape 1 to shape 4 is effectively a re-platform, typically 6 to 12 months. The migration cost makes the initial residency choice load-bearing.
Which shape is right for a B2B SaaS with EU customers?
Shape 2 (hyperscaler with EU-region pinning) is the default. The EU customer base requires defensible residency; the SaaS economics favor hyperscaler scale; the talent pool supports hyperscaler tooling. Shape 3 becomes appropriate when the EU customer base includes public sector or strict-sovereignty enterprises whose procurement requires sovereign hosting.
What about US state privacy laws?
State privacy laws (CCPA, CPRA, Colorado, Virginia, etc.) generally do not require US-only residency, but they do require defensible data handling and consumer rights mechanisms. Shape 2 with US-region pinning is sufficient for most state law compliance. The detail interacts with the AI vendor of record decision; the vendor-of-record carries much of the consumer rights obligation. See the AI vendor-of-record decision on who carries SOC 2 risk for that interaction.
How does this interact with the build-vs-buy decision?
Residency is largely a buy-side decision (which vendor’s infrastructure runs the inference) but has build-side implications when shape 4 is required. Shape 4 forces a build of the inference infrastructure that other shapes outsource to the cloud or vendor. The build/buy frame applies recursively; within shape 4, model selection is buy (open-weights or licensed) and operational tooling is build/buy/compose.
Can we run different residency shapes for different workloads?
Yes, and most enterprises with regulated workloads do. The regulated workloads run in shape 2 or shape 4; the non-regulated workloads run in shape 1. The cost of operating multiple shapes is real (each shape has its own talent and tooling overhead), but the cost is usually less than the cost of forcing many workloads into the most-restrictive shape.
What’s the worst residency mistake?
Shape 1 in a workload that turns out to be regulated 18 months later. The migration cost is substantial; the customer relationship damage of “we cannot serve you because our infrastructure does not meet your compliance posture” is expensive; the engineering velocity hit during the migration is a quarter or more of feature work converted to platform work.
Does on-prem AI work in 2026?
Yes, with caveats. Open-weights models in 2026 are competitive with commercial models on many capabilities (general reasoning, code, summarization, classification) and behind on others (frontier reasoning, multilingual coverage, agent capabilities). On-prem operations have improved with better tooling, but the talent gap remains acute. Shape 4 is viable when required; it remains an inferior choice when not required.
How does residency affect the agency engagement model?
Substantially. An agency operating in shape 1 cannot easily transition to a buyer who requires shape 4. The engagement should specify the production residency shape from day one and structure the build accordingly. The detail on agency engagement structure that supports residency requirements is in the AI hybrid playbook on which 30 percent to keep in-house.
What’s the role of edge inference?
Edge inference (running the model on the user’s device or near the edge of the network) is a fifth shape that is becoming viable for specific workloads; speech, vision, simple text generation. Edge inference solves the residency question by keeping the data on the device. It is constrained by model size and capability; the largest frontier models are not edge-deployable as of 2026. Treat edge as an emerging shape that resolves residency for the workloads that fit within the model size constraints.
Is residency a permanent decision or a quarterly re-litigation?
Both. The residency shape itself is a permanent commitment for any given workload; the migration cost is too high to revisit casually. The distribution of workloads across shapes is a quarterly re-litigation; new workloads should evaluate residency at design time, and the portfolio across shapes should be reviewed at each procurement review. The detail on the quarterly cadence is in why AI build-vs-buy decisions made in 2024 should be re-litigated this quarter.
Key takeaways
The AI residency decision has four shapes: hyperscaler default region (shape 1), hyperscaler with residency pinning (shape 2), sovereign cloud (shape 3), and on-prem or air-gapped (shape 4). Shape 1 is the cheap default and the right answer for early-stage non-regulated workloads. Shape 2 is the right answer for most enterprise workloads. Shape 3 is required for EU public sector and strict sovereignty cases. Shape 4 is required for the most regulated cases (HIPAA-sensitive PHI, classified work).
The four constraint dimensions; latency, residency, cost, talent; are tradeoffs that each shape balances differently. The dimension most often underestimated is talent; the dimension most often overweighted is cost. The right answer is rarely the cheapest answer.
Three regulated cases compress the decision into a single answer. HIPAA covered entities resolve to shape 2 (BAA-covered region) or shape 4 (on-prem). EU AI Act Annex III rarely forces shape 3 or 4 directly but interacts with regional monitoring obligations. Cleared contracting forces shape 4 or a cleared-cloud variant of shape 3 with severely constrained model availability.
Residency is a load-bearing decision that the engineering team often makes implicitly by deploying to the default cloud account. Making it explicit at design time costs less than discovering it at audit time. The discipline is naming the residency shape in the architecture document, listing which models are available in that shape, listing the compliance evidence required for that shape, and reviewing the choice on the quarterly cadence that the rest of the AI sourcing decisions follow.
Arthur Wandzel
