Agentic AI: From Per-Token Pricing to Scalable Inference on AWS

A Brazilian Agentic AI company focused on developing resilient, secure, and culturally aware AI agents capable of creating more humanized interactions between people and systems. Offered as AI-as-a-Service (AIaaS), its solutions enable organizations to deploy "digital people" on the front lines of customer service, operating at scale, integrated with existing software stacks, and collaborating with human teams.

The company serves large organizations, including financial sector institutions and environments with millions of users. In this context, customer experience systems must operate consistently under high volume while meeting strict security, governance, and legal accountability requirements. As a result, cost predictability and operational control are as critical as model quality.

The platform was using Amazon Bedrock, but the token-based inference economics began creating a ceiling for growth at scale. High token consumption generated elevated variable costs, and this variability made planning and sustainable operational expansion difficult. Additionally, at high volumes, constraints associated with managed services — such as throughput limits and performance control — become more visible, especially when predictable latency and high responsiveness are fundamental to the user experience.

In summary, the need was to control inference costs without compromising quality. A model was also needed that would allow scaling the user base without costs growing linearly with token usage, plus greater control over where inference is executed and where data resides — a particularly sensitive point for enterprise clients and regulated environments.

Elevata conducted a structured benchmarking and migration program to move operations from a managed, token-based inference model to a capacity-based inference model, using open-weight models hosted within the client's own AWS environment.

The work began with an inference benchmarking phase across different hardware options and instance families. Elevata tested open-weight models on chips and accelerators including AWS Inferentia2 and GPUs from the L40S, H100, and B200 Blackwell families. Hardware selection was not treated as merely an infrastructure decision: tests were directly tied to operational inference KPIs such as tokens per second, latency, output characteristics, and functional validation of candidate model quality.

This phase generated an objective decision basis, identifying which combinations offered the best balance of cost, throughput, and response time without compromising the client's functional requirements. With this baseline defined, Elevata implemented the migration from Bedrock usage to self-hosted open-weight models on AWS instances with dedicated inference GPUs and chips.

This shift fundamentally changed the financial model of the operation. Instead of paying a variable rate per token, the client now pays for infrastructure capacity. This capacity can be planned, measured, and scaled predictably, providing greater clarity on unit economics and reducing vendor-imposed limitations that tend to appear in high-volume scenarios. Additionally, the client can now support more users on the same infrastructure base and scale capacity for processing peaks without needing to restructure the application layer.

Security posture was treated as a core requirement from the start. The new inference stack was deployed within the client's VPC, keeping inference execution and data handling entirely within the company's AWS environment. For enterprise and financial sector clients, this means greater control over data boundaries, alignment with compliance requirements, and more direct governance over infrastructure configuration.