Skip to content

Cost and FinOpsComparison

DeepSeek V4 on SageMaker JumpStart: self-host or call the API?

DeepSeek's API bills per token, with peak and off-peak rates. A SageMaker endpoint bills per instance-hour whether or not anyone calls it. This comparison shows where the two lines cross, and what changes for data location, scaling and your gateway once the endpoint is yours.

By INFRO Editorial

Published

9 min read

Glowing token streams being compressed into a precise efficient AI infrastructure pipeline

In short

Self-hosting a DeepSeek V4-family model on SageMaker only wins on cost when the endpoint stays busy enough to beat DeepSeek's per-token list rates, which for Flash-class models is a high bar. Data location and a dedicated capacity ceiling are the stronger reasons to host it yourself.

Key takeaways

  • DeepSeek bills Flash and Pro per token with off-peak rates at half of peak, so the API side of the comparison moves with your hour-of-day mix.3
  • A SageMaker endpoint is a fixed hourly cost; the effective per-token cost is the hourly rate divided by the tokens you actually push through it.
  • In the illustrative model below, an always-on endpoint beats the Flash API only above roughly 40% sustained utilization; below that the API is lower.
  • Choose JumpStart for data location, VPC isolation or reserved capacity, not for the rate card.
  • A self-hosted endpoint needs a gateway that handles overflow, cold capacity, cost estimates that reflect a fixed bill, and a clear answer on where prompts travel.

What the two options actually are

Calling DeepSeek's API means paying per token at DeepSeek's list rates, with a concurrency ceiling per account. Hosting the open weights from SageMaker JumpStart means paying AWS for instance-hours, whether a request arrives or not, and owning everything that follows: capacity, patching, cold starts, observability.

One caveat first. The JumpStart overview page I read names providers such as AI21 Labs, Cohere, Databricks, Hugging Face, Meta, Mistral AI, Stability AI and Alexa, and does not list DeepSeek.1 I could not confirm the V4 Pro, V4 Flash and V4.1 Flash listings, their instance types, regions or hourly rates. The SageMaker AI pricing page says you pay on-demand with no minimum fees, or commit through Savings Plans, and has a JumpStart section, but the instance rates did not come through as text.2 Everything about instance cost below therefore uses an illustrative rate you must replace with the number on your model card and the AWS pricing calculator.

Two things on DeepSeek's side change what "V4" means:

  • DeepSeek retired V4-Flash and V4-Flash-Vision-Exp; deepseek-flash now serves V4.1-Flash, and legacy names are routed to it at the Flash price.3
  • The September release note said all deepseek-v4-pro requests would route to V4.1-Flash at Flash rates until a V4.1-Pro launches.4 The current pricing page still lists deepseek-v4-pro as DeepSeek-V4-Pro-0813 with its own rates.3 Those two statements do not agree on their face, so confirm what a deepseek-v4-pro call actually returns before you budget for Pro.

A JumpStart "V4 Flash" listing is a pinned set of weights that DeepSeek's own API no longer serves under that name. Treat it as a different model from deepseek-flash when you compare quality and cost.

The API side: per-token rates with a clock

DeepSeek bills input and output tokens, with a separate lower rate for cache-hit input. Off-peak rates are half of peak.3

Flash peakFlash off-peakPro peakPro off-peak
Input, cache hit0.0060.0030.0440.022
Input, cache miss0.300.151.320.66
Output1.200.603.961.98
DeepSeek list rates per 1M tokens, USD, as published on the pricing page on 2026-11-13.

Peak is 01:00–04:00 and 06:00–10:00 UTC, Monday to Friday, excluding Chinese public holidays; every other hour is off-peak.3 That is 35 of 168 weekly hours, about 21%, before holidays. If your traffic is concentrated in European or US business hours, check how much of it falls in the 06:00–10:00 UTC window rather than assuming the 21% average.

Fees are deducted from a balance on your DeepSeek account, granted balance first.3 For finance, that means the API spend is a draw on a balance held with DeepSeek, not an instance bill that arrives in your AWS cost data.

The instance side: a fixed cost you divide

A SageMaker endpoint's cost per token is the hourly rate divided by tokens served per hour. Utilization is the share of the instance's token capacity you really use, averaged over the month, including nights, weekends and the capacity you hold in reserve for spikes.

The model is large. DeepSeek describes V4.1-Flash as a 552B-parameter mixture-of-experts model with 8B active parameters for input and 16B for output.4 Active parameters set compute per token; total parameters set how much memory the weights occupy. At one byte per parameter, 552B parameters is roughly 552 GB of weights before KV cache, so the endpoint needs a multi-GPU instance even though per-token compute is modest. The precision of the released checkpoint is not something I confirmed, so treat that as arithmetic, not a sizing guide.

DeepSeek also notes the model's KV cache needs a quarter of the HBM and an eighth of the SSD storage of the previous generation.4 That helps a self-hosted deployment, but only if your serving stack implements the same cache design. Ask whoever packages the JumpStart container.

The same arithmetic for Pro-class rates at the same request shape gives $5,940 per million requests at peak and $2,970 off-peak, about $3,590 blended. A self-hosted Pro-class model would need its own, probably larger, instance, so recompute with the real numbers rather than reusing the Flash figures.

breakeven.py
HOURS = 730

def endpoint_cost_per_m_requests(hourly_usd, capacity_m_requests_month, utilization):
    return hourly_usd * HOURS / (capacity_m_requests_month * utilization)

def api_cost_per_m_requests(in_tok, out_tok, in_peak, in_off, out_peak, out_off, peak_share=0.21):
    peak = in_tok * in_peak + out_tok * out_peak
    off = in_tok * in_off + out_tok * out_off
    return peak_share * peak + (1 - peak_share) * off  # USD per 1M requests

api = api_cost_per_m_requests(3000, 500, 0.30, 0.15, 1.20, 0.60)
breakeven = 100 * HOURS / (200 * api)   # illustrative inputs
print(round(api), round(breakeven, 2))  # ~908, ~0.40

Replace the hourly rate, the capacity you measured under your own prompts, and your real peak share. Capacity is the number most likely to be wrong, so measure it with production-shaped traffic.

Data location and control

This is the stronger reason to choose JumpStart. AWS states that JumpStart data is encrypted and does not leave your virtual private cloud, that none of your data is used to train the underlying models, and that admins control which models are visible to users in an organization.1 With your own endpoint you choose the region and the network path, and your cloud contracts and audit scope apply.

For the DeepSeek API, the pages I read do not state where requests are processed or retained. Do not assume; ask DeepSeek and get the answer in writing before sending regulated data. DeepSeek does give you a per-user lever: a user_id parameter that isolates content-safety handling, KV cache and scheduling per end user of yours, and it advises against putting personal information in the id.5

If data location is the requirement, check every hop. A self-hosted endpoint inside your VPC does not keep data in place if a fallback sends the same prompt to a public API or a hosted gateway in another region. See the related guide on multi-region traffic and data location.

Scaling and capacity

The API has account-level concurrency limits: 2,500 for deepseek-flash and 500 for deepseek-v4-pro. A request counts from send until the response completes, and requests above the limit get HTTP 429. DeepSeek will review a capacity-expansion request at no extra cost.5 That is a soft ceiling you ask to raise, not hardware you provision.

A self-hosted endpoint has the opposite shape. Capacity is whatever you provisioned. Scale-out takes minutes, not milliseconds, because new instances must load hundreds of gigabytes of weights. Autoscaling therefore lags a sudden spike, and the requests that arrive during the lag queue or time out. Plan for:

  • a warm floor of instances you pay for around the clock;
  • an overflow target for bursts beyond that floor;
  • a queue-depth or concurrency signal for scaling, not CPU.

DeepSeek says that for a large-scale deployment with 2,000 GPUs plus a storage cluster it wants to talk directly.4 That is far beyond a typical JumpStart endpoint, but it shows the intended range: the open weights are a real option at scale, and DeepSeek expects to be asked about it.

Two items are cheap to settle early. DeepSeek's API sends empty lines or : keep-alive SSE comments while a request waits and closes the connection if inference has not started after 10 minutes.5 Any proxy in front of it must tolerate those and set timeouts accordingly. The open-source V4.1-Flash-1017 release is on Hugging Face,6 so check the license and which snapshot the JumpStart listing carries before you pin anything.

What the gateway must handle when the endpoint is yours

Once the endpoint is yours, the gateway stops being a convenience and becomes the place where cost, failover and policy meet. The points that matter:

  1. Protocol. A SageMaker endpoint is invoked through AWS's runtime API, not an OpenAI-style URL. INFRO's provider page lists Bedrock and "any OpenAI-compatible endpoint" among what you can connect, and does not name SageMaker.7 Put an OpenAI-compatible front on the endpoint, or confirm support, before planning around it. The self-hosted vLLM guide covers the shape.
  2. Overflow and failover. Make the endpoint the priority target and a second target the overflow. A route that fails before the response starts can retry elsewhere; one that fails mid-stream cannot be silently replayed. Decide which prompts may leave your boundary and write that as a rule, not a hope.
  3. Cost that reflects a fixed bill. A per-token estimate is meaningless for an instance billed by the hour unless you enter the effective rate you computed from utilization, and revise it each month. Otherwise dashboards show a variable cost that does not match the AWS invoice.
  4. Idle cost is invisible per request. The 60% of the month the endpoint sat unused appears on no request. Track instance spend in cloud cost data, and join the two views.
  5. Model identity. Name the snapshot in the route. DeepSeek reroutes legacy names; a pinned endpoint does not, which is both a stability gain and a maintenance duty.
  6. Prompts in transit. A hosted gateway sees prompts even when the model is in your VPC. If that breaks your boundary, the gateway deployment matters as much as the model location.

INFRO can carry several of these today: health-aware routes with priority targets, failover, routing rules by prompt size or metadata, negotiated-rate entry so estimates use your effective rate, per-request usage and cost, budgets, and a FOCUS export to join AI spend with cloud cost. Its regional processing and self-hosted deployment options are not available yet, so do not count on them for a boundary requirement.

When each option wins

SituationLean toward
Spiky or low-volume traffic, under about 40% endpoint utilizationDeepSeek API
Flexible batch work that can run in off-peak hoursDeepSeek API
Regulated data that must stay in your VPC and regionJumpStart endpoint
Steady, high-volume load that keeps instances busy around the clockJumpStart endpoint, if the measured capacity holds
Need for a pinned snapshot and no upstream reroutingJumpStart endpoint
Small platform team with no on-call for GPU servingDeepSeek API
Decision guide.

Most teams end up with both: the endpoint carries the steady, boundary-sensitive base load, and the API takes overflow and non-sensitive batch traffic. That hybrid only holds if routing rules separate the two classes of prompt explicitly.

Checklist before you decide

  • Confirm the V4 Pro, V4 Flash and V4.1 Flash listings in your console: snapshot, license, instance type, regions.
  • Get the hourly rate from AWS for your region and compare Savings Plans with on-demand.
  • Measure tokens per second per instance with your own prompts and context lengths.
  • Compute average and peak utilization from a month of real traffic, split by peak and off-peak UTC hours.
  • Confirm what deepseek-v4-pro returns today.
  • Get DeepSeek's data-processing location in writing if you plan to use the API.
  • Define which prompts may overflow outside your boundary.
  • Decide who is on call for the endpoint.

Questions readers ask

Does the DeepSeek API price change by time of day?
Yes. DeepSeek lists peak and off-peak rates, with off-peak at half of peak. Peak is 01:00–04:00 and 06:00–10:00 UTC, Monday to Friday, excluding Chinese public holidays; all other hours are off-peak.3
Can I use a cache on a self-hosted endpoint like DeepSeek's cache-hit rate?
Only if your serving stack provides prefix caching and your traffic repeats prefixes. DeepSeek's API lists a separate, much lower cache-hit input rate; a self-hosted endpoint has no such rate, only the throughput the cache frees up.3

From the team that builds it

Where this shows up in INFRO

INFRO is a control plane in front of the provider accounts you already hold. The capabilities below are the parts of it this article touches, each with the documentation that defines its behaviour.

  • Negotiated provider rates

    Available with a plan

    Enter the rates your contracts actually carry, per model and optionally per provider, so estimated cost uses them before the vendor's list rate. Included on every plan.

    Read the documentation for Negotiated provider rates
  • Health-aware routing

    Available with a plan

    Named routes use priority or weighted targets over your own connections, with health checks, retries and failover.

    Read the documentation for Health-aware routing
  • Automatic failover

    Available with a plan

    A route that times out or 5xxs before the response starts is retried elsewhere without your client seeing it.

    Read the documentation for Automatic failover
  • Routing rules

    Available with a plan

    If-rules on a route send matching requests to their own targets, by metadata, feature, end user, key, project, prompt size, streaming, images, tools or time of day. Failure rules decide per failure kind whether to retry, move on or stop.

    Read the documentation for Routing rules
  • Usage and cost per request

    Available with a plan

    Every request records its model, latency, token or unit counts, and estimated provider cost, queryable by project and key.

    Read the documentation for Usage and cost per request
  • Usage, request and FOCUS exports

    Available with a plan

    GET /v1/usage and GET /v1/requests on any gateway key, the request log and usage series as CSV, plus a FOCUS v1.4 Cost and Usage dataset as JSON or CSV, so AI spend lands in your FinOps platform beside the rest of your cloud cost.

    Read the documentation for Usage, request and FOCUS exports
  • Budgets and spend ceilings

    Available with a plan

    A hard cap per organization, project, member, and key, with alerts at your own thresholds.

    Read the documentation for Budgets and spend ceilings
  • Provider connections

    Available with a plan

    Connect provider accounts you already hold. Each credential is encrypted when saved, scoped to your organization and never shown again; applications use INFRO gateway keys instead.

    Read the documentation for Provider connections

INFRO accounts are self-serve. Provider usage is billed to you by your providers; INFRO charges a software subscription and takes 0% of model spend. See plans.

Sources

Numbered in order of first citation. Each was opened and checked against the claim on the date shown.

  1. 1.Amazon SageMaker JumpStart (opens in a new tab) — Amazon Web Services. Accessed Nov 13, 2026.
  2. 2.Amazon SageMaker AI pricing (opens in a new tab) — Amazon Web Services. Accessed Nov 13, 2026.
  3. 3.Models & Pricing (opens in a new tab) — DeepSeek API Docs. Accessed Nov 13, 2026.
  4. 4.DeepSeek-V4.1-Flash: Smarter, Faster, More Efficient (opens in a new tab) — DeepSeek API Docs. Accessed Nov 13, 2026.
  5. 5.Rate Limit & Isolation (opens in a new tab) — DeepSeek API Docs. Accessed Nov 13, 2026.
  6. 6.DeepSeek-V4.1-Flash-1017 Release (opens in a new tab) — DeepSeek API Docs. Accessed Nov 13, 2026.
  7. 7.Connect a provider (opens in a new tab) — INFRO Docs. Accessed Nov 13, 2026.

About this article

Read on 2026-11-13: the SageMaker JumpStart and SageMaker AI pricing pages, DeepSeek's Models & Pricing, Rate Limit & Isolation and two release notes (V4.1-Flash, V4.1-Flash-1017), and INFRO's provider page. Left out as unconfirmed: the JumpStart listings of V4 Pro, V4 Flash and V4.1 Flash themselves, their instance types and hourly rates (the pricing page text carried placeholders, not rates), regions, and where DeepSeek's API processes requests.

Last checked against its sources on . The next review is due by . No separate named reviewer is recorded for this article.

How articles are written, sourced and corrected is set out in the editorial standards.