Open-Weight Models at Your Fingertips With Akamai Inference | Edge Case

Building AI-powered applications usually means choosing between two extremes: provisioning and wrestling with your own GPU infrastructure, or locking yourself into proprietary APIs that drain your budget at scale.

That’s why Akamai Inference is such a welcome addition. It bridges the gap by giving developers a fully managed service to run leading open-weight models, without having to manage a single GPU. Even better, it features an OpenAI-compatible API, meaning you can swap out endpoints in your existing codebase with almost zero friction.

In this post, we’ll take a quick look at what Akamai Inference is and how you can start interacting with the service.

Request Access

Akamai Inference is currently in beta. However, you can request access to the service and start exploring it now! Just follow the link down below:

Request accessAkamai Inference (Beta)Do you want to get in? Request access and start building today.

Why Akamai Inference?

Among others, here are my personal top four reasons why one should use Akamai Inference instead of provisioning and maintaining their own cloud infrastructure to do AI inferencing:

Zero Infrastructure Overhead: Akamai handles all the serving, autoscaling, and model lifecycle management.
OpenAI API-Compatible: If you already have code hitting api.openai.com, moving it over is as simple as swapping your base_url and api_key.
Predictable Economics (OpEx vs. CapEx): Skip the brutal upfront capital expenditure (CapEx) of purchasing (or renting) and maintaining expensive GPU clusters. With flexible pay-per-token serverless pricing, you convert heavy cloud-infra investments into operational expenditure (OpEx) that scales with your usage.
Enterprise Trust & Data Sovereignty: Stop worrying about shipping sensitive or business-critical data to consumer-focused third-party endpoints. Akamai Inference combines strict zero data retention for prompts and responses with the enterprise-grade security and compliance standards that your legal and security teams already trust.

The Chat Playground

Before writing a single line of code, you can use the built-in chat playground to test drive models and experiment with prompts directly inside the Akamai Cloud Manager.

Akamai Inference Chat Playground

It offers a low-friction space to evaluate different open-weight options from the catalog, tweak parameters, and validate outputs in real time. Once you find the right behavior and parameters, you can easily transition from sandbox prototyping to integrating the OpenAI-compatible endpoint into your application stack.

Scoped Tokens and Built-In Observability

Once you’re ready to move from the playground to real code, you’ll need an API token. Akamai Inference lets you create multiple tokens and configure permissions per token, including which models a given token is allowed to call. That makes it easy to hand a production service a token locked to a single approved model, while keeping a broader token for local experimentation, all without sharing one key across everything.

Observability is baked in too. Each token comes with usage insights, so you can see token economics, such as consumption and spend per token, at a glance, instead of bolting on your own metering. It’s a quick way to spot which service or environment is driving your usage before it shows up on a bill.

Akamai Inference token management and usage insights

Interacting with the Service: Simple Code Samples

Because the service uses a standard OpenAI-compatible API format, you can use your favorite HTTP client or SDKs right out of the box.

Using Python

If you’re already using the official openai SDK for Python, just point the client to your Akamai Inference endpoint:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.akamai-inference.com/v1",  
    api_key=os.environ.get("AKAMAI_INFERENCE_API_KEY"),
)

response = client.chat.completions.create(
    model="qwen3-5-9b",
    messages=[
        {"role": "system", "content": "You are a helpful developer assistant."},
        {"role": "user", "content": "Explain serverless inference in one sentence."},
    ],
    temperature=0.7,
)

print(response.choices[0].message.content)

Using TypeScript / JavaScript

The same pattern works seamlessly in Node.js or edge runtimes using the standard SDK:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.akamai-inference.com/v1",
  apiKey: process.env.AKAMAI_INFERENCE_API_KEY,
});

async function runInference() {
  const completion = await client.chat.completions.create({
    model: "qwen3-5-9b",
    messages: [
      { role: "user", content: "What makes open-weight models great for DevRel?" }
    ],
  });

  console.log(completion.choices[0].message.content);
}

runInference();

Using cURL

If you’re testing endpoints directly from the terminal or writing a lightweight script without heavy dependencies, a standard curl command does the trick:

curl https://api.akamai-inference.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AKAMAI_INFERENCE_API_KEY" \
  -d '{
    "model": "qwen3-5-9b",
    "messages": [
      {"role": "user", "content": "Hello from the edge!"}
    ]
  }'

Try Our Self-Paced & Interactive Workshop

We’ve got something for you, if you wanna dive deeper into Akamai Inference. Check out our self-paced, interactive workshop on Akamai Inference over on GitHub

Open sourceReady to RunGrab the full sample code and run it yourself!akamai-developers/akamai-workshop-serverless-inference

Wrapping Up

If you are scaling up AI features and want the flexibility of open-weight models without the infrastructure headache, head over to the Akamai Inference product page to request beta access and start experimenting.