Cloud AI Platforms

Executive Summary

AWS Bedrock, Azure OpenAI Service, and Google Vertex AI represent the three dominant managed AI inference platforms for enterprise deployment. Each provides access to frontier and open-weight language models through a secure, scalable, HIPAA-eligible API — but they differ meaningfully in model selection, enterprise integration depth, data residency controls, pricing structure, and operational model. For enterprise AI architects, the choice of cloud AI platform is rarely purely technical; it is typically constrained by the organization's existing cloud provider relationship, compliance requirements, and enterprise integration needs. This chapter provides the framework for evaluating these platforms objectively and documents the specific technical differences that matter for production enterprise AI deployments.

Learning Objectives

  • Compare AWS Bedrock, Azure OpenAI Service, and Google Vertex AI across the dimensions relevant to enterprise selection
  • Identify which platform characteristics are genuinely differentiating vs. which are marketing-level distinctions
  • Design a cloud AI platform architecture that preserves optionality across providers through an AI gateway abstraction
  • Evaluate HIPAA eligibility and data residency controls for each platform
  • Apply the platform selection framework to a healthcare AI context

Business Problem

Selecting a cloud AI platform is a long-lived decision with significant switching costs. A healthcare organization that builds its clinical AI stack on Azure OpenAI — with Azure Private Link, Azure Active Directory integration, and Azure Monitor observability — faces significant re-engineering work to migrate to AWS Bedrock. Yet the model landscape evolves rapidly: today's best model for clinical documentation may not be on the same platform as next year's best model.

The business problem is: how do you select a cloud AI platform that serves current requirements without creating lock-in that constrains future optionality?

Why This Technology Exists

Before cloud AI platforms existed, enterprises accessing LLM APIs had two options: use the AI lab's native API (api.anthropic.com, api.openai.com) with its associated data handling terms, or self-host open-weight models with their associated operational burden. Neither option fit the enterprise requirement for native cloud integration, private networking, compliance certification, and consolidated billing.

Cloud AI platforms emerged to fill this gap: wrapping LLM API access in enterprise cloud packaging — VPC deployment, private endpoints, IAM integration, compliance certifications, SLA guarantees, and usage monitoring — while providing model access without requiring self-hosting.

Conceptual Explanation

All three platforms provide the same core service: managed LLM inference with enterprise packaging. The differentiation is in:

Model selection: Which models are available, when, and at what tier. Platform-exclusive models (GPT-4 on Azure OpenAI, Claude on Bedrock and Vertex) vs. multi-model catalogs (Bedrock offers Anthropic, Llama, Cohere, Titan, Stability).

Cloud integration depth: How tightly the AI service integrates with other services on the same cloud platform. Azure OpenAI integrates natively with Azure Cognitive Search, Azure Data Factory, and Azure Monitor. AWS Bedrock integrates with S3, Lambda, Bedrock Agents, and CloudWatch. GCP Vertex AI integrates with BigQuery, Dataflow, and Cloud Storage.

Data handling and residency: Where inference requests are processed, whether input data is used for model training, and what privacy controls apply.

Enterprise features: Private networking (VPC endpoints, Private Link), IAM integration, usage monitoring, content filtering, and SLA guarantees.

Core Architecture

Platform Architecture Comparison

Feature Matrix

Dimension AWS Bedrock Azure OpenAI Google Vertex AI
Key models Claude (Anthropic), Llama, Cohere, Amazon Titan, Stability GPT-4o, o1, o3 (OpenAI exclusive) Gemini, Claude, PaLM, Llama
Model exclusivity Claude requires Bedrock or Anthropic API GPT-4/o1/o3 require Azure or OpenAI Gemini requires GCP or Google AI
Private networking VPC Endpoint (PrivateLink) Azure Private Link VPC Service Controls
Identity integration IAM, AWS SSO Azure AD / Entra ID Google Cloud IAM
HIPAA BAA Yes (included in AWS BAA) Yes (Microsoft Azure BAA) Yes (Google Cloud BAA)
Data residency By AWS region By Azure region By GCP region
Training opt-out Yes (Bedrock doesn't train on inputs by default) Yes (Azure OpenAI) Yes (Vertex AI)
Content filtering Bedrock Guardrails Azure Content Safety Vertex AI Safety Filters
Observability CloudWatch, CloudTrail Azure Monitor, Azure Log Analytics Cloud Monitoring, Cloud Logging
Batch inference Bedrock Batch API Azure Batch Deployments Vertex AI Batch Prediction
Fine-tuning Bedrock Fine-tuning (select models) Azure OpenAI Fine-tuning Vertex AI Fine-tuning
Agent/orchestration Bedrock Agents Azure AI Foundry / Prompt Flow Vertex AI Agent Builder
SLA 99.9% (varies by service) 99.9% (varies by tier) 99.9% (varies by service)

Enterprise Considerations

Existing cloud commitment: Organizations with large existing cloud commitments (AWS Enterprise Discount Program, Azure Enterprise Agreement, GCP Committed Use Discounts) should evaluate whether AI platform usage can be applied against those commitments. AWS Bedrock usage applies to AWS consolidated billing; Azure OpenAI applies to Azure Enterprise Agreements.

Regional availability: Not all models are available in all cloud regions. HIPAA-eligible regions may be a subset of all regions. Validate that the required models are available in the required regions before committing to a platform.

Quota limits: Cloud AI platforms impose default quota limits (tokens per minute, requests per minute) that may be insufficient for enterprise production traffic. Request quota increases before production launch — they can take weeks to process.

Data handling verification: Verify current data handling terms directly with each provider before finalizing a compliance assessment. BAA terms, training data opt-out, and data retention policies change and the status described here may not reflect current terms.

Healthcare Example

⊕ Healthcare Example

Educational Example — Not intended for clinical use.

A Reference Healthcare Organization with an existing Azure Enterprise Agreement evaluates cloud AI platform selection for its HMS AI Platform. The evaluation criteria:

Criterion Azure OpenAI AWS Bedrock Decision Basis
Existing cloud relationship Primary cloud Secondary cloud Azure strongly preferred
HIPAA BAA ✅ Microsoft HIPAA BAA ✅ AWS HIPAA BAA Tie
Private networking ✅ Azure Private Link (already configured) Requires new VPC endpoint Azure easier
Model access (Claude) ❌ Not available ✅ Available Bedrock required for Claude
Epic FHIR integration ✅ Azure Health Data Services (existing) Requires setup Azure preferred
GPT-4o access ✅ Exclusive ❌ Not available Depends on model strategy

Decision: Primary platform = Azure OpenAI (for GPT-4o, existing EA, Azure ecosystem integration). Secondary platform = AWS Bedrock (for Claude access when required by specific use cases). All inference routed through LiteLLM gateway to enable model switching without application code changes.

Common Mistakes

1. Assuming model availability before verifying. Model availability on each platform changes frequently. Verify that the specific model version required is available in the required region before designing the architecture.

2. Building platform-specific integration without an abstraction layer. Code that calls azure<em>openai</em>client.chat.completions.create() directly cannot switch to Bedrock without rewriting. All inference calls should go through a platform-agnostic interface.

3. Not requesting quota increases before production launch. Default quotas on all platforms are insufficient for enterprise production. Quota increase requests must be submitted weeks before the target launch date.

4. Treating HIPAA BAA as a static guarantee. BAA terms and scope change. Verify current BAA terms annually and upon any significant platform update.

5. Ignoring content filter calibration. Default content filters on all platforms are calibrated for consumer use cases. Clinical content will trigger false positives on medical descriptions. Calibrate and test content filters before clinical deployment.

Best Practices

  • Implement an AI gateway abstraction (LiteLLM or equivalent) before committing to any single platform — preserves optionality
  • Align platform selection with existing cloud provider relationship when technically equivalent
  • Enable private networking for any inference involving PHI
  • Verify training data opt-out is configured and documented before processing PHI
  • Request quota increases 4–6 weeks before production launch
  • Test content filters against a representative clinical content sample before deployment
  • Review platform data handling terms annually — they change

Trade-offs

Model selection vs. ecosystem integration: The best model for a specific use case may not be available on the organization's preferred cloud platform. AWS Bedrock offers the broadest model selection (multi-vendor); Azure OpenAI offers exclusive access to the GPT-4/o1 family; Vertex offers native BigQuery integration.

Managed features vs. lock-in: Platform-specific features (Bedrock Agents, Azure AI Foundry, Vertex Agent Builder) reduce development effort but create deeper lock-in. Generic infrastructure (LiteLLM gateway + vLLM) preserves optionality at higher operational cost.

Interview Questions

Q: An enterprise client wants to use both Claude and GPT-4 in different parts of their AI platform. How would you architect this without creating application-level dependencies on each platform?

Category: Architecture Difficulty: Senior Role: AI Architect

Answer Framework:

The answer is an AI gateway pattern — a layer that translates a common API format (typically OpenAI-compatible) into platform-specific API calls. All application code calls the gateway using the common format; the gateway routes to the appropriate platform based on the model name requested.

LiteLLM is the standard open-source implementation. The gateway configuration maps logical model names ("claude-premium", "gpt4-enterprise") to platform-specific endpoints. Application code references only the logical name. Switching from Claude via Anthropic API to Claude via Bedrock, or from GPT-4 Azure to GPT-4 OpenAI, requires only gateway configuration changes.

The additional benefits of this pattern: centralized cost attribution, unified audit logging, failover routing (if Claude API is rate-limited, fall back to an alternative), and prompt injection for common headers (department attribution, audit trail).

Key Points to Hit:

  • AI gateway as the abstraction layer
  • LiteLLM as the practical implementation
  • Logical model names vs. platform-specific model IDs
  • Benefits beyond model switching: cost, logging, failover

Key Takeaways

  • AWS Bedrock offers the broadest multi-model catalog; Azure OpenAI offers exclusive GPT-4/o1 access; Google Vertex AI offers the deepest BigQuery and analytics integration
  • All three platforms provide HIPAA BAAs and private networking for PHI-in-cloud deployments
  • The AI gateway pattern (LiteLLM) is the architectural mechanism that preserves cross-platform optionality
  • Align cloud AI platform selection with existing cloud provider relationship when technically equivalent
  • Verify model availability in required regions before architecture commitment
  • Request quota increases 4–6 weeks before production launch — default quotas are insufficient for enterprise production
  • Training data opt-out must be verified and documented before processing PHI

Further Reading