Understanding AWS Bedrock: Basics, Pricing, and Cost Optimization

What is AWS Bedrock?

AWS Bedrock is a fully managed service designed to simplify the development of generative AI applications. It provides access to a selection of high-performing foundation models (FMs) from providers like Amazon, Anthropic, Cohere, Meta, and Stability AI, all accessible through a single API. 

Bedrock offers a comprehensive set of tools and capabilities essential for building secure, private, and responsible AI applications. Developers can experiment with different FMs, customize them using their data through techniques such as fine-tuning and Retrieval Augmented Generation (RAG), and create agents that interact with enterprise systems and data sources. 

Additionally, its serverless nature eliminates the need for infrastructure management, ensuring seamless integration with existing AWS services. This reduces operational overhead and allows businesses to focus on developing AI applications. 

AWS Bedrock’s 5 Pricing Models

1. On Demand and Batch

The on-demand pricing model charges users based on actual usage, with no long-term commitments. For text-generation models, pricing is determined by the number of input and output tokens. Embedding models are priced by the number of input tokens processed, while image-generation models are charged per image created.

On-demand usage supports cross-region inference for select models, allowing workloads to span multiple AWS Regions to manage traffic spikes and improve resilience. Pricing is based on the Region where the request originates.

In batch mode, users can submit multiple prompts in a single file and receive all responses in one output file, stored in Amazon S3. Batch inference supports select models and offers a 50% cost reduction compared to on-demand pricing.

2. Latency Optimized

The latency-optimized inference mode is currently in public preview and is intended for workloads requiring low-latency responses. This mode provides faster response times for supported foundation models, improving the end-user experience in real-time applications such as chatbots, assistants, or interactive systems.

Latency optimization is supported for a limited set of high-performance models, including:

• Claude 3.5 Haiku (Anthropic)
• Amazon Nova Pro
• Meta Llama 3.1, including both 405B and 70B parameter versions

According to benchmarks provided by model developers, Claude 3.5 Haiku delivers faster performance on AWS compared to any other platform. Likewise, Meta’s Llama 3.1 models run faster on AWS than on other major cloud providers when using latency-optimized inference.

3. Provisioned Throughput

The provisioned throughput pricing model is designed for high-volume, predictable inference workloads. Instead of paying per request, you purchase dedicated model units that provide guaranteed throughput—measured by the number of tokens processed per minute (both input and output). This ensures stable performance even during peak demand.

This mode is the only option available for running custom models within Bedrock. Pricing is charged hourly, and you can commit to either 1-month or 6-month terms, offering some flexibility in budgeting and planning. Provisioned Throughput is best suited for production systems with steady inference needs and performance guarantees.

4. Custom Model Import

The custom model import feature lets users bring in their own trained model weights to Amazon Bedrock and run them in the same managed environment as other foundation models. This allows you to reuse prior customization work and integrate it directly into your AWS-based AI workflows.

There is no cost to import a model. Once imported, your model is hosted in Bedrock and available on-demand. You are billed only for model inference, and the charges depend on the number of active model copies. A model copy is a ready-to-serve instance of the imported model. Billing is calculated in 5-minute increments, based on how long each copy is active.

Costs vary depending on:

• Model architecture
• Context length supported
• AWS Region
• Compute unit version (hardware generation)
• Size of the model copy

5. Marketplace Models

The Amazon Bedrock Marketplace enables users to discover and consume over 100 foundation models from various commercial and open-source providers. These models are deployed to endpoints that you configure based on your performance and scaling needs.

You can select:

• Instance type
• Number of instances
• Auto-scaling policies

For proprietary models, pricing includes two components:

1. A software fee set by the model provider (billed per hour, per second, or per request)
2. An infrastructure fee based on your selected instance type

For publicly available models, only the infrastructure cost applies. All pricing is transparently displayed before you subscribe and is also available from the model’s listing in the AWS Marketplace.

Using Model Customization and Native Tools to Reduce AWS Bedrock Costs

Model Customization

AWS Bedrock supports two main approaches for model customization: fine-tuning and model distillation.

Fine-Tuning and Continued Pretraining allow users to adapt foundation models with their own labeled or unlabeled data. Pricing is based on the total number of tokens processed—calculated by multiplying tokens in the dataset by the number of epochs. Storage for custom models is billed monthly. Inference using customized models requires purchasing provisioned throughput, with at least one model unit available without a long-term commitment. Additional units require either a 1-month or 6-month term.

Model Distillation offers a way to reduce costs and improve performance by transferring capabilities from a larger “teacher” model to a smaller “student” model. The process involves synthetic data generation—charged at on-demand rates of the teacher model—and subsequent training of the student model at customization pricing. Distilled models are treated like customized models and also require provisioned throughput for inference.

Prompt Caching

Prompt caching reduces costs and latency by reusing frequently used context in prompts. Users can cache prompt prefixes for up to five minutes using existing APIs. During this period, requests with matching prefixes receive up to 90% cost savings on cached tokens and up to 85% latency reduction. Cache performance and pricing benefits vary by model and prompt characteristics, and all caches are isolated to individual AWS accounts.

Prompt Management and Optimization

AWS Bedrock includes built-in tools for managing and improving prompts.

• Prompt Management provides a user interface and APIs for testing, versioning, and securely running prompts. It supports easy comparison of different versions and seamless integration into serverless environments.
• Prompt Optimization helps improve model response quality by automatically rewriting prompts for clarity and efficiency. It integrates with prompt management tools to support side-by-side comparisons and lifecycle management. These features are accessible via the Bedrock Playground or API.

Other Tools

Here are additional tools that can have an impact on your AWS Bedrock costs:

• Guardrails enable responsible AI practices by letting users define custom rules around safety, privacy, and truthfulness. They apply to all types of models—foundation, fine-tuned, and even external models. Guardrails can be integrated with Bedrock Agents and Knowledge Bases, and can also be used independently via the ApplyGuardrail API.
• Knowledge Bases and Data Automation support retrieval-augmented generation (RAG) by turning unstructured content into structured embeddings. Knowledge Bases work with multiple data sources, convert data for storage in vector databases, and enable retrieval using natural language. 
• Data Automation further enhances this by transforming multimodal content into structured outputs, either using default templates or custom-defined schemas, and supports downstream loading into data stores.
• Agents in Bedrock enable the creation of applications that respond dynamically by accessing enterprise systems, executing actions, and retaining memory across sessions. Agents can also interpret and run code in response to user prompts.
• Flows provide a no-code/low-code interface for building generative AI workflows. Flows let users visually connect models, prompts, agents, knowledge bases, and guardrails with business logic and AWS services. Workflows can be versioned, tested, and deployed in a fully managed environment.
• Evaluations offer tools for testing model quality. Users can run programmatic or human-in-the-loop evaluations, and pricing is based on model usage and completed human tasks. For LLM-as-a-judge evaluations, only model inference is billed. Human tasks cost $0.21 each. RAG evaluation supports detailed analysis of retrieval quality, with charges for generator and evaluator models, and any retrieval-related costs from knowledge base usage.

Amazon Bedrock Pricing Examples

On-Demand Text and Image Generation

A developer using the Amazon Titan Text Lite model to summarize 2,000 input tokens into 1,000 output tokens incurs a cost of $0.001 per request. The calculation is based on $0.0003 per 1,000 input tokens and $0.0004 per 1,000 output tokens.

In another case, generating 1,000 standard quality images (1024×1024) with Amazon Titan Image Generator costs $10, priced at $0.01 per image.

Using Anthropic Claude, summarizing 11,000 tokens into 4,000 output tokens costs $0.184 per request. The pricing is $0.008 per 1,000 input tokens and $0.024 per 1,000 output tokens.

Provisioned Throughput

A developer commits to two model units of Amazon Titan Text Express for 31 days. With each unit costing $18.40/hour, the total monthly cost is $27,379.20.

For image generation, a single model unit of Titan Image Generator at $16.20/hour totals $12,052.80 per month.

With Cohere Command, one model unit at $39.60/hour for 31 days results in a monthly cost of $29,462.40.

Model Customization

Fine-tuning Amazon Titan Image Generator with 1,000 image-text pairs across 500 steps and a batch size of 64 costs $160. Adding $1.95 for storage and $21 for one hour of inference results in a total of $182.95.

For Cohere Command, customizing with 1,000 tokens of data costs $4 for training, $1.95 for storage, and $49.50 for one hour of inference. The total is $55.45, excluding provisioned hosting costs.

Guardrails and Filtering

In a customer support chatbot using content filters and denied topics, 1,000 queries per hour (each with 1 input and 2 output text units) result in 3,000 units processed hourly. At $0.15 per 1,000 units per filter type, the total hourly cost is $0.90.

Summarizing 10,000 chat transcripts with sensitive info redaction, where each summary equals 4 text units, costs $4 total, using a $0.10 per 1,000 units rate.

Knowledge Bases and SQL Generation

In a structured data retrieval use case, a chatbot generates SQL queries for 10,000 user inputs per hour via the GenerateQuery API, priced at $0.002 per call. This results in a monthly cost of $1,440.

Embeddings and Lightweight Use

Generating embeddings for 10,000 tokens using Cohere’s Embed models costs just $0.001, based on $0.0001 per 1,000 tokens.

Custom Model Import

A developer imports a customized Llama 3.1 model (8B parameters, 128K sequence length), which requires two custom model units. If the model is active for one 5-minute billing window, the cost is $0.785. Monthly storage for this setup is $3.90. There is no charge for the import itself—only for inference and storage.

The Challenges of Forecasting and Managing AWS Bedrock Costs

Effectively forecasting and managing AWS Bedrock costs involves addressing several key challenges. These stem from the complexity of machine learning (ML) workloads and the dynamic nature of generative AI applications. Key challenges include:

1. Lack of Visibility

One of the primary challenges is the lack of granular visibility into spending. Without detailed insights into where, how, when, and by whom resources are being consumed, it becomes difficult to create accurate forecasts and manage cloud costs effectively. This lack of transparency can lead to inefficient cost management strategies, as organizations struggle to pinpoint specific areas for optimization​. 

2. Dynamic Usage Patterns

Generative AI applications often experience fluctuating usage patterns, influenced by factors such as user demand and the specific tasks being performed. Additionally, usage patterns can be affected by uncontrollable factors like organizational policy changes, cost constraints, or modernization. These unpredictable changes make it challenging to forecast costs accurately. Without efficient forecasting, organizations may struggle to leverage discounting opportunities with predictable usage patterns, thereby missing out on potential cloud cost savings​​.

3. Complexity of Cost Structures

AWS Bedrock's cost structure includes multiple components: data storage, compute resources, data processing, model training, and inference. Managing these cost drivers requires careful attention. Simplifying the approach to managing these components can be challenging, particularly for businesses new to ML operations. 

4. AWS Bedrock Cost Management

Effectively managing AWS Bedrock costs involves continuous monitoring, analysis, and optimization of resource usage. This process can be tough, as businesses need to stay vigilant in tracking their usage, identifying inefficiencies, and making necessary adjustments to optimize for cost efficiency every hour. Failure to do so can result in unexpected cost overruns and reduced overall efficiency of AI/ML operations​.

Addressing these challenges requires a strategic approach to cost management. This includes using tools for detailed forecasting and actively optimizing resource usage.

Cost Optimization Strategies for AWS Bedrock

Monitoring Your Workloads

Monitoring is critical for understanding usage patterns and identifying cost-saving opportunities in AWS Bedrock. Begin by enabling detailed billing and usage reports in the AWS Billing Console. Use AWS Cost Explorer to analyze spending over time, broken down by service, usage type, and tag. Pair this with Amazon CloudWatch to capture real-time metrics such as token consumption, model invocations, latency, and errors.

Consider setting CloudWatch alarms on token thresholds, unexpected model spikes, or usage anomalies. This allows teams to respond proactively when usage deviates from expectations. Integrate usage monitoring with custom dashboards in Amazon QuickSight or third-party analytics tools to visualize cost trends across business units or application environments. 

Utilizing Reserved Capacity

Provisioned throughput in Bedrock enables customers to lock in dedicated inference capacity with predictable performance and pricing. For stable, high-volume workloads, this model can offer substantial savings compared to on-demand usage. AWS offers one-month and six-month commitment options, with longer commitments yielding greater discounts.

To take advantage of reserved capacity, analyze past usage patterns using AWS Cost Explorer and workload telemetry to determine minimum and peak throughput needs. Purchase model units that align with these metrics and consider over-provisioning slightly to accommodate expected growth. For use cases like enterprise search, knowledge assistants, or automated summarization services with steady demand, reserved throughput ensures cost efficiency and removes the risk of throttling or unpredictable latency during peak hours.

Additionally, leverage Bedrock’s auto-scaling mechanisms within provisioned environments to maintain performance while containing costs, scaling units up only when traffic demands it.

AWS Tags and Cost Categories for Cost Visibility

Cost visibility in AWS Bedrock is significantly enhanced by tagging and organizing resources. Apply AWS tags consistently across your Bedrock workloads, including models, agents, knowledge bases, and infrastructure resources. Common tags include Project, Environment, Team, Application, and Owner. These tags feed into AWS Cost Explorer and allow for detailed chargeback or showback reporting.

Use AWS Cost Categories to group related tagged resources into logical billing groups, such as “Customer Support AI” or “R&D Experimentation.” This grouping simplifies reporting and allows stakeholders to monitor budgets specific to their operational scope. AWS Budgets can then be configured to enforce spend thresholds by tag or category, with alerts triggered when usage approaches predefined limits.

Integrating cost categorization into CI/CD workflows also improves traceability, ensuring every Bedrock resource has a purpose and owner, which reduces waste and improves accountability across teams.

Leverage Batch Processing for Non-Real-Time Tasks

Batch inference provides a low-cost solution for large-volume, latency-tolerant tasks. It allows developers to package multiple prompts into a single job and submit it for processing, with outputs returned via Amazon S3. This mode is priced at roughly half the cost of real-time on-demand inference and is ideal for scenarios like document summarization, content generation, data labeling, or daily report creation.

To optimize batch processing, consolidate prompt files to minimize overhead, and schedule jobs during off-peak hours to reduce load on other AWS services. Consider segmenting batch jobs by content type or model to streamline post-processing and monitoring.

Use Amazon S3 lifecycle policies to manage output storage costs and clean up stale results. Monitor batch job success rates and output accuracy using Amazon CloudWatch or custom validation scripts, and use results to iterate on prompt design or model selection before committing to real-time deployments.

Plan for Custom Model Costs

Custom models in Bedrock offer fine-grained control over AI behavior, but they come with significant cost components: training, storage, and dedicated inference capacity. Before launching full-scale customization, conduct a proof of concept using a reduced dataset to validate expected performance improvements. Use Bedrock’s training token calculator and logging tools to estimate the total token volume across epochs, which directly affects training costs.

Storage fees apply monthly for each version of a custom model, so clean up deprecated versions to avoid unnecessary charges. Provisioned throughput is mandatory for serving custom models, so plan how many model units are required to meet traffic and latency targets. Include failover or multi-AZ redundancy in your cost model if uptime is critical.

For ongoing optimization, explore model distillation to reduce inference costs by compressing larger models into faster, more efficient versions. Periodically re-evaluate the return on investment of custom models versus continuing with Bedrock’s hosted foundation models or fine-tuned public alternatives.

How Finout Can Help Manage and Optimize AWS Bedrock Costs