That works as long as an origin IP lives in one fixed place. Public cloud origins usually don't. They sit behind anycast or regional unicast front ends, so one origin IP can look equally close to a dozen Cloudflare data centers at once — and the latency probes have nothing to lock onto. Smart Tiered Cache handles this the safe way: when there's no clear winner, it falls back to several upper tiers. Nothing breaks. You just lose the thing that made a single closest tier worth it, which is cache efficiency.

Smart Tiered Cache for Public Cloud Regions fixes this by letting you provide a cloud region hint. With that hint, Cloudflare can map public cloud origins to the right region and select better primary and fallback upper tiers, even when the origin IP itself looks anycast or ambiguous.

Since it was launched, Smart Tiered Cache has become the most popular tiered cache topology among Cloudflare customers. It’s available to all plans, for free.

Much of our work aims to continually improve it. Over time, we’ve extended Smart Tiered Cache to handle more origin architectures, including:

• November 2024: Smart Tiered Cache for R2: We taught Smart Tiered Cache to automatically select the closest upper tier to where the R2 bucket actually lives, reducing latency with zero configuration.

• January 2025: Smart Tiered Cache for Load Balancing: We extended Smart Tiered Cache to select a single optimal upper tier for an entire Load Balancing pool, so all origins in the pool share the same cache, improving hit ratios.

Each of these improvements has shared a common goal: understand the customer’s origin infrastructure and automatically do the best thing for that infrastructure.

While we’ve been improving this system for a while, customers still had a common frustration: Smart Tiered Cache did not work when an origin is behind an anycast or regional unicast network, because this architecture prevented us from knowing where the origin is located. And this wasn’t an edge case, either. Origins hosted on public cloud providers behind anycast IPs are a growing slice of the Internet.

Today, we’re closing that gap for origins hosted on AWS, GCP, Azure, and Oracle Cloud.

Smart Tiered Cache works by measuring the latency from each Cloudflare data center to the origin’s IP address. The data center with the lowest latency becomes the upper tier: the single point through which all cache misses funnel on their way to your origin. By concentrating cache misses at one data center, you get higher cache hit ratios, fewer connections to your origin, and lower latency on origin pulls. This works well when the origin has a fixed, unicast IP address that can be reliably probed.

Many cloud providers use anycast or regional unicast networking for their load balancers, front-end services, and regional ingress points. When we probe these IPs, the origin appears to be “close” to many data centers simultaneously. That is because the IP address represents the cloud provider’s front end, not a single physical origin location. Different Cloudflare data centers may reach different nearby cloud edges for the exact same IP, and the provider then carries the request across its own network to the actual backend. So Smart Tiered Cache cannot confidently pick one best upper tier.

In practice, this could result in hairpin traffic across continents, adding a whole extra round trip. Say your origin sits in Singapore, behind an anycast IP from a cloud provider. Because of how anycast works, our Chicago data center might show the lowest probe latency to that IP. Smart Tiered Cache would then select Chicago as the upper tier. The result: a request from an end user in Asia hits a nearby Cloudflare data center, gets routed cross-continent to the upper tier in Chicago, and Chicago fetches from the origin back in Singapore, crossing the ocean twice. That hairpinning adds hundreds of milliseconds of latency, and it's one of the most consistently reported issues from customers with cloud-hosted origins.

^{An example of hairpinning is when traffic is routed to an upper tier in Chicago only to fetch data from an origin in Singapore, resulting in an unnecessary cross-continental round trip.}

To address this unnecessary back-and-forth, Smart Tiered Cache learned to detect anycast origins with a constraint from physics: the speed of light. We measure probe latencies from multiple checkpoint data centers around the world to the origin. If the combined latencies from two checkpoint data centers are faster than what light in fiber could physically travel between the two, the origin must be answering from multiple locations, not one. That means it's anycast.

^{We detect anycast origins by comparing probe latencies from multiple Cloudflare data centers. If two paths are faster than physically possible for a single origin location, the origin is likely answering from multiple places.}

When Smart Tiered Cache detects an anycast origin, it plays it safe: it won't pin that IP to a single upper tier. Instead, it falls back to a tiered cache topology with multiple upper tiers. Tiered caching still works, but spreading traffic across multiple tiers instead of one means more requests reach the origin. For some setups that’s a fine trade. But if you want one upper tier close to an origin that lives on a public cloud behind anycast IPs, there hasn't been a good option — until now. 

From the Cloudflare dashboard, go to Caching > Tiered Cache > Origin Configuration. Find your origin IP, click "Set Region Hint," and tell us the cloud region (for example, aws:us-east-1 or gcp:europe-west1). Smart Tiered Cache takes over from there. Note that, on the dashboard, region hints can only be set for origins whose IPs we've detected as anycast.

^{On the Tiered Cache page, go to the Origin Configuration table and click the edit icon next to an origin IP to set its region hint.}

You can set hints one IP at a time, or bulk-edit cloud regions for all your origin IPs at once. Beyond the dashboard, the same configuration is available via the API and through Terraform, so you can integrate it into your existing infrastructure-as-code workflows.

We're launching with AWS, GCP, Azure, and Oracle Cloud, with more providers coming.

Every few hours, we fetch the latest IP range files from each supported cloud provider. These files map every cloud region to its current set of IP prefixes, so when a provider adds, removes, or reassigns a subnet, we pick it up.

^{Smart Tiered Cache for Public Cloud system diagram}

We match those subnets against our upper tier database, which is built from continuous latency probing refreshed every 15 minutes. For each cloud region, each matching subnet contributes a weighted vote based on its current upper-tier assignment. The upper tier with the strongest signal becomes the region’s primary upper tier. Primary and fallback always come from different points of presence (PoPs), so losing one PoP can't take out both.

Some regions don't have enough probe data, for example, perhaps the new region’s cloud provider is still rolling out, or the region has no origin onboarded to Cloudflare yet — so there’s nothing to vote on. We fall back to geography: the closest of our Tier 1 PoPs. As origins come online and probe data builds up, the region quietly switches from that geographic guess to the option backed by real data.

All this means that the work of selecting the optimal region for your cache — the constant probing, the algorithmic choice of each region's best upper tier, the geographic fallbacks, the failover across PoPs — runs on our side. Your job is selecting the region hint.

If your anycast origin sits on a public cloud, you can turn this on now. In the dashboard, go to Caching > Tiered Cache > Origin Configuration. Find your origin IP, click Set Region Hint, and pick your region.

Next up, we’re expanding to more providers, and continuing to teach Smart Tiered Cache to recognize more origin setups and pick the right path on its own. To learn more about how Tiered Cache can benefit your service, check out our Tiered Cache documentation.

When Workers Cache is enabled, every cacheable request to your Worker hits Cloudflare's cache first. If there's a fresh cached response, Cloudflare returns it directly — your Worker doesn't run, and you don't pay CPU time for it. On a miss, your Worker runs, and if your response is cacheable, Cloudflare stores it for the next request. The next request from anywhere on Earth can be served straight from cache.

The whole thing is one config block:

{
  "name": "my-worker",
  "main": "src/index.ts",
  "compatibility_date": "2026-05-01",
  "cache": {
    "enabled": true
  }
}

After that, you control caching the way HTTP has always wanted you to — by setting headers on your responses:

return new Response(body, {
  headers: {
    "Cache-Control": "public, max-age=300, stale-while-revalidate=3600",
    "Cache-Tag": "products,product:123",
  },
});

And when content changes, your Worker purges its own cache:

await ctx.cache.purge({ tags: ["product:123"] });

That's the whole API. There is no zone to configure, no rules engine to set up, no separate cache to provision, and no second product to log into. The Worker's code is the configuration surface, and the cache follows the Worker wherever it runs — on a custom domain, on workers.dev, behind a service binding, in a preview, in a Workers for Platforms tenant. One Worker, one cache, configured once.

That's the surface area. There’s a lot underneath: tiered caching across our entire network, full support for stale-while-revalidate so stale responses never block a user, content negotiation via Vary, multi-tenant-safe cache keys via ctx.props, programmatic purges by tag or path prefix, and — the part we think is the biggest unlock — a cache that sits in front of every Worker entrypoint, not just the public one, with per-entrypoint control over which ones cache and which don't. That last piece means you can compose caching directly into the structure of your app: a chain of entrypoints with cache stages slotted in wherever you want them, configured by the code on either side. We'll walk through all of it below.

Workers Cache is available today to every Worker on any plan, enabled in Wrangler.

This is the caching API we've always wanted Workers to have. Here's why it took us this long, what becomes possible because of it, and what's coming next.

When we introduced Workers in 2017, the pitch was that you could run code on Cloudflare's network to transform requests on their way to your origin. The Worker sat in front of the cache and the origin:

This was the right model for the use cases we were targeting. If you wanted to add a header to every request, rewrite a URL, do an A/B split, or filter traffic before it reached your origin, putting the Worker in front of the cache and the origin gave you full control over what got cached and what didn't. Customers built incredible things with it.

But the world changed. Workers stopped being a thing you bolted onto an origin and started being the origin. Frameworks like Astro, TanStack Start, Next.js, Remix, and SvelteKit all ship a Cloudflare adapter that builds your app as a Worker. There's no origin behind them. The Worker is the server.

When the Worker is the origin, the original architecture has nothing to cache. Every request runs your code, even when the response would be byte-for-byte identical to the one you returned a second ago. The Workers runtime is fast enough that this works — it routinely handles tens of millions of requests per second without breaking a sweat — but "fast enough to render every request" still costs you latency on every page load and CPU time on every invocation. And on a server-rendered app, every page load is, by definition, a render.

Workers Cache flips the architecture. Cloudflare's cache now sits in front of the Worker:

On a cache hit, your Worker doesn't run at all. Cloudflare returns the cached response and your CPU billing stays at zero. On a miss, your Worker runs once, populates the cache, and the next request — from anywhere — gets served from cache without invoking your code.

This is what was missing for server-side rendering on Workers. You used to have to choose between two unsatisfying options:

• Prerender everything at build time ("static site generation"). Fast page loads, but every change requires a full rebuild and redeploy. For a docs site with a few thousand pages, that's 5–10 minutes. For a large e-commerce site, it's worse — and the build runs every single time you touch anything.

• Render every page on every request. Up-to-date content, but every page load pays the rendering cost and every visitor pays the latency.

Workers Cache gives you a third option: server-render on demand, cache the rendered response, refresh it on a time-to-live (TTL) you choose. The first request to a new page still renders. Every subsequent request, until the cache expires, is served as if the page were static. When the cache expires, the next request triggers a re-render — and with stale-while-revalidate, even that one doesn't wait.

You get the speed of a static site without the build time, and the freshness of server rendering without the cost. No framework-specific machinery like Incremental Static Regeneration. Just HTTP caching, working the way it was designed to work, in front of code that was designed to be the origin.

The stale-while-revalidate directive tells Cloudflare that when a cached response expires, it's allowed to serve the stale copy immediately while it refreshes the response in the background. Cloudflare shipped full support for stale-while-revalidate earlier this year, and it's the directive that turns "we cache your Worker" into "your Worker's site feels static."

Without it, the first request after a cache entry expires has to wait for the Worker to render the page from scratch. The user sees that latency. With it, the first request after expiration gets the stale page immediately (with a Cf-Cache-Status: UPDATING header), and the Worker runs in the background to refill the cache. Every user, including the one who triggered the refresh, gets a cache-speed response.

In practice, this looks like:

 export default {
  async fetch(request) {
    const html = await renderPage(request);
    return new Response(html, {
      headers: {
        "Content-Type": "text/html; charset=utf-8",
        // Treat as fresh for 5 minutes; serve stale for up to an hour
        // while a background refresh runs.
        "Cache-Control": "public, max-age=300, stale-while-revalidate=3600",
      },
    });
  },
};

The mental model that makes this click:

• Fresh window (max-age): Cloudflare serves the cached response. Your Worker doesn't run.

• Stale window (stale-while-revalidate): Cloudflare serves the cached response. Your Worker runs in the background to refresh it. No user waits.

• Outside both windows: Cloudflare runs your Worker to generate a fresh response, and the user waits for that one render.

You pick the windows. For a product catalog that updates every few minutes, max-age=300, stale-while-revalidate=3600 means visitors basically never wait, and your Worker still runs often enough to keep content fresh. For a blog archive that almost never changes, max-age=86400, stale-while-revalidate=2592000 means your Worker runs once a day per page.

The first request to a brand-new page is the only one that pays the full render cost. After that, the page behaves like static output for visitors, while your Worker still owns how the page gets generated.

Real apps rarely return the same bytes to every client. The same product page might be HTML for a browser and JSON for an API client. The same image might be WebP for clients that support it and JPEG for the ones that don't. The same homepage might come back in English, French, or Japanese depending on the user.

Doing this without a cache is easy — your Worker just reads the request header and returns the right thing. Doing it with a cache is where it usually gets ugly. Most caches give you two bad options: cache nothing on URLs that have multiple representations, or cache one representation and serve it to everyone.

Workers Cache supports the standard HTTP Vary header, which is the right way to solve this. When your Worker returns a response with Vary: Accept-Encoding (or Accept, or Accept-Language, or any other request header), Cloudflare stores a separate cached variant per distinct combination of those headers — and only returns a variant whose stored values match the incoming request.

export default {
  async fetch(request) {
    const accept = request.headers.get("Accept") ?? "";
    const wantsWebp = accept.includes("image/webp");

    const body = wantsWebp ? await fetchWebpImage() : await fetchJpegImage();

    return new Response(body, {
      headers: {
        "Content-Type": wantsWebp ? "image/webp" : "image/jpeg",
        "Cache-Control": "public, max-age=3600",
        // Cache a separate variant per distinct Accept header value.
        Vary: "Accept",
      },
    });
  },
};

One URL, two cached variants. A browser that sends Accept: image/webp,*/* gets the WebP. A browser that sends Accept: image/jpeg gets the JPEG. Both come from cache. Your Worker writes both variants on the first request to each, and then runs zero times for either after that.

This is the well-trodden HTTP standard for content negotiation, and Workers Cache implements it the way RFC 9110 and RFC 9111 describe. There's no allowlist of what headers you can Vary on. You list whatever you need, and Cloudflare keys variants on the verbatim values. The docs go through the edge cases — how to keep variant fan-out under control by normalizing headers in a gateway Worker, why purges invalidate all variants of a URL together, and the one case (Vary: *) that disables caching entirely.

Before we get to what becomes possible with all this, there's a conceptual shift worth naming.

Cloudflare has had a cache forever. It's configured at the zone level: Cache Rules, Page Rules, the cached-file-extensions list, Cache Reserve, Tiered Cache topology, custom cache keys. All of it is set per zone, and historically a Worker had to either fit into that zone's configuration or work around it.

Workers Cache is different. It's your Worker's cache — it belongs to the Worker, not to a zone. This has a bunch of consequences that turn out to matter:

• There is no zone configuration to manage. Cache Rules, cache level settings, the file-extensions list, Page Rules — none of them apply to Workers Cache. The Worker's Cache-Control headers are the configuration.

• The cache follows the Worker, not the hostname. A Worker that's bound to api.example.com, api.example.net, and invoked over a service binding shares one cache across all three. A request to /users/42 hits the same cached entry regardless of which way in it came.

• The cache works on workers.dev. It works in preview URLs (each preview gets its own cache, so testing a change doesn't poison production). It works in Workers for Platforms (each user Worker has its own cache, isolated from the dispatcher and from other tenants). All of these used to be second-class citizens for caching. They aren't anymore.

• Purges are scoped to the Worker’s entrypoint. When you call ctx.cache.purge({ purgeEverything: true }), you're only purging your Worker entrypoint's cache. No risk of nuking your zone's other content. No risk of one Worker's deploy invalidating another's data.

What you configure about caching, you configure in code: which paths get longer TTLs (branch on the path and set a different max-age), which requests bypass the cache (return Cache-Control: private), how the cache key is shaped (control what gets into ctx.props, normalize the URL in a gateway Worker before dispatching). The Worker you already wrote is the configuration surface.

The full docs go deep on this in Workers Cache: your Worker's cache.

Workers Cache is regionally tiered by default. There are two layers:

• A lower tier in the Cloudflare data center closest to the user. Every data center that receives traffic for your Worker has its own lower-tier cache.

• An upper tier that aggregates fills across the whole network. There are fewer of these, and every lower tier consults the upper tier on a miss.

A request hits the lower tier first. On a hit, the response is served and that's the end of it. On a miss, the lower tier asks the upper tier. On a hit there, the response is returned and also stored in the lower tier on the way back. Only if both tiers miss does your Worker actually run — and the response from that run gets stored in both tiers.

The reason this matters is that the first request anywhere in the world populates the upper tier. Every subsequent request, from any data center, can be served from the upper tier without your Worker running — even if the lower tier at that data center has never seen the request before. Cache hit ratios are dramatically higher than they would be with a single flat cache layer, which is exactly what you want when your Worker is the origin.

This is the same topology that powers Tiered Cache for zones today, except you don't configure it. There is no dialog for "turn on tiered cache for my Worker." Every Worker that has caching enabled gets tiering for free.

If your Worker uses Smart Placement, the cache composes cleanly with it: tiers are consulted first, and only if both miss does Smart Placement route execution close to your origin. We have more to say about how those layers interact, including a few rough edges we're planning to smooth out, in the docs.

There's a recurring tension in web performance that nobody has fully resolved: you want your code to run close to the user (because the round-trip between user and server is on the critical path), and you want your code to run close to the data (because every database query is also a round-trip). Pick one, and the other gets slow.

We've spent years chasing both. Our network puts us within ~50ms of about 95% of the world's Internet users. Smart Placement and Placement Hints let you keep your code close to your data without ever having to think about cloud regions. But until now, the two pieces didn't fully compose. You could do "near the user" or "near the data," and if you wanted both halves of your app to be in the right place at the same time, you had to be a Cloudflare expert. We knew we could do better.

Workers Cache is the piece that closes the gap. Because the cache belongs to the Worker (not the zone), and because service bindings and ctx.exports calls between Workers go through the cache, you can build an app as a chain of Workers — each one running where it should run — with the cache as the seam between them.

The architecture looks like this:

• Worker A runs near the user. It handles the cheap, latency-sensitive parts of every request: authentication, rate limiting, routing, header normalization, rendering the outer "shell" of an HTML page that doesn't depend on data.

• Worker B runs near the data, courtesy of Smart Placement or an explicit Placement Hint. It does the heavy work: server-rendering pages that fetch data, reading product catalogs, generating search results, aggregating APIs, expensive transforms.

• Workers Cache sits in front of Worker B. When Worker A calls Worker B over a service binding, Cloudflare checks Worker B's cache first. On a hit, Worker A receives the response and Worker B doesn't run at all — no data-center hop, no database query, no rendering work.

The cache hit path becomes: user → Worker A near the user → cache hit for Worker B → response. The data hop is paid only on a miss. Your hot pages run at the speed of code-in-front-of-the-user, and your cold pages still benefit from running near the data when they do execute.

You don't have to architect anything special to get this. Write your app as two Workers, point one at the other with a service binding, turn caching on in Worker B’s wrangler.jsonc file, and you're done.

If you're caching a Worker that returns user-specific data — say, an API that serves different content per logged-in user — you need a way to make sure one user can never see another user's cached response. The standard solution is "don't cache authenticated requests," and Cloudflare's automatic bypass for Authorization headers does exactly that. But "don't cache anything" gives up the entire performance win.

Workers Cache solves this by making the caller's ctx.props part of the cache key. When one Worker calls another over a service binding and passes ctx.props with a user ID, tenant ID, or any other identifier, callers with different props get separate cache entries. One user's response can never leak into another user's cache.

import { WorkerEntrypoint } from "cloudflare:workers";

interface Props { userId: string; }

export default class Backend extends WorkerEntrypoint<Env, Props> {
  async fetch(request: Request): Promise<Response> {
    // ctx.props.userId is part of the cache key. User A and User B
    // requesting the same URL get separate cached entries.
    const { userId } = this.ctx.props;
    const data = await loadUserData(userId);

    return new Response(JSON.stringify(data), {
      headers: {
        "Content-Type": "application/json",
        "Cache-Control": "public, max-age=300",
      },
    });
  }
}

The typical pattern is to authenticate the request in a gateway Worker, strip the Authorization header, set the authenticated user's ID into ctx.props, and then call the cached backend Worker. The gateway runs on every request (it has to, to authenticate), but the expensive backend only runs when there's no cache entry for that user yet. Auth'd APIs go from "uncacheable" to "cached per user with full safety," and the cache key does the isolation for you. The docs walk through this in detail in Multi-tenant safety with ctx.props and the example in Per-user authenticated responses.

Other CDNs make you choose between correctness and hit ratio: key the cache by each user’s token, or send every request back to origin for authorization. Workers Cache lets you share cached API responses at the edge while preserving per-request authorization boundaries. We don’t know of another CDN that offers this as a built-in model for authenticated, multi-tenant APIs. We’re pretty proud of it.

Here is the part of Workers Cache that we think is the biggest unlock, and it's the part that's hardest to see if you're thinking about it as "a CDN cache that happens to work in front of Workers."

Workers Cache sits in front of every Worker entrypoint — the default export, every named WorkerEntrypoint, and every call between entrypoints in the same Worker via ctx.exports. That last clause is the one that changes what you can build.

When one entrypoint calls another via ctx.exports, the cache evaluates that call the same way it would evaluate a request from a browser. A hit returns the cached response and the callee never runs. A miss runs the callee and stores its response under its own cache key — keyed by the callee's entrypoint, path, query string, and ctx.props. The caller still runs on every request, but anything it hands off to the callee is memoized independently.

You decide, per entrypoint, which ones cache. In your Wrangler config, the exports map lets you turn caching on or off for each entrypoint by name ("default" is the default export). Opt an entrypoint in to cache the responses it produces; opt one out to keep it running on every request. A gateway or router entrypoint — anything that authenticates, normalizes, or dispatches — should be opted out, so it always runs, and its own output is never served from cache.

That gives you a primitive you can compose. You can author a Worker as a chain of small entrypoints — auth, normalization, routing, the expensive read, the data layer — and let Workers Cache slot in wherever you want it. Each cached entrypoint is a unit of memoization with its own key, its own TTL, and its own tag namespace for purging. Anything you would want to configure about caching — when it runs, what it keys on, when it invalidates — is expressed as ordinary Worker code: which entrypoint you call, what request you forward, what ctx.props you pass, what Cache-Control you set.

To make this concrete, here's a single Worker that does three things you couldn't easily do together on any other platform: it authenticates every request, caches the expensive backend behind a multi-tenant-safe cache key, and invalidates that cache when data changes.

Caching is configured per entrypoint. The gateway must run on every request — both to authenticate and because a cached gateway response would skip that auth check — so we disable caching on the default entrypoint and enable it only on the inner one:

{
  "name": "my-worker",
  "main": "src/index.ts",
  "compatibility_date": "2026-05-01",
  "cache": { "enabled": true },
  "exports": {
    // The gateway runs on every request — don't cache it.
    "default": { "type": "worker", "cache": { "enabled": false } },
    // Cache the expensive inner entrypoint.
    "CachedBackend": { "type": "worker", "cache": { "enabled": true } }
  }
}

import { WorkerEntrypoint } from "cloudflare:workers";

interface Env { API_TOKEN: string; }
interface Props { userId: string; }

// Inner entrypoint: the expensive work. Workers Cache sits in front
// of this — on a hit, this code never runs.
export class CachedBackend extends WorkerEntrypoint<Env, Props> {
  async fetch(request: Request): Promise<Response> {
    // ctx.props.userId is part of the cache key, so this is cached
    // separately for every user.
    const { userId } = this.ctx.props;
    const data = await loadExpensiveData(userId);

    return new Response(JSON.stringify(data), {
      headers: {
        "Content-Type": "application/json",
        "Cache-Control": "public, max-age=300, stale-while-revalidate=3600",
        "Cache-Tag": `user:${userId}`,
      },
    });
  }

  // Invalidate a user's cached response. purge() is scoped to the
  // entrypoint that calls it, so it must run inside CachedBackend —
  // the entrypoint that owns the cached response.
  async invalidate(userId: string): Promise<void> {
    await this.ctx.cache.purge({ tags: [`user:${userId}`] });
  }
}

// Outer entrypoint: runs on every request to authenticate and route.
// Caching is disabled for it in Wrangler config (above), so it always
// runs and the auth check is never skipped by a cache hit.
export default {
  async fetch(request, env, ctx): Promise<Response> {
    const userId = await authenticate(request, env);
    if (!userId) return new Response("Unauthorized", { status: 401 });

    // Invalidate this user's cache on writes, from the entrypoint that
    // owns it.
    if (request.method === "POST") {
      await handleWrite(request, userId);
      await ctx.exports.CachedBackend.invalidate(userId);
      return new Response("OK");
    }

    // For reads: strip Authorization (otherwise Cloudflare's automatic
    // bypass fires and nothing caches), then dispatch to the cached
    // backend with the authenticated user's identity in ctx.props.
    const forwarded = new Request(request);
    forwarded.headers.delete("Authorization");

    return ctx.exports.CachedBackend.fetch(forwarded, {
      props: { userId },
    });
  },
} satisfies ExportedHandler<Env>;

The whole thing is one Worker. One source file. One deploy. But there are two execution stages — caching is turned off for the gateway and on for the backend in one small exports block — and a cache sits between them, keyed per user, invalidated by the write path, and serving stale during background refreshes. The cache stage isn't something you bolted on. It's a layer of the program, written in code.

The patterns this composes into are open-ended. The same shape works for:

• Caching a Durable Object. Wrap the Durable Object behind an entrypoint, set Cache-Control on the response, and reads stop touching the Durable Object on a hit. Writes go to the DO directly and purge the cache by tag. The DO stays unaware that caching is happening.

• Normalizing Accept-Encoding before Vary. The outer entrypoint restores the original encoding from request.cf.clientAcceptEncoding (Cloudflare's front line normalizes it for cache efficiency) and forwards to a cached entrypoint that varies on the real value. Hit ratios stay high; clients get the right encoding.

• Stripping tracking parameters before caching. The outer entrypoint canonicalizes the URL — or sets a custom cache key with cf.cacheKey on the ctx.exports call — so the cached inner entrypoint sees only the canonical form, and ?utm_source=anything collapses to a single cache entry.

Stack them. A single Worker can have an outer entrypoint that authenticates and routes, a normalization entrypoint that strips tracking parameters and restores encoding headers, a cached entrypoint that fronts a Durable Object, and a separate cached entrypoint for an unauthenticated public API — each connected by a cache stage you didn't configure, just decided where to put. The Examples page in the docs walks through several of these end-to-end.

We don't know of another platform where you can do this. CDN caches sit in front of an origin. Function platforms run functions. We don't know of another platform that gives you a cache that sits inside a single deployable unit, between the parts of your application, with each cache stage configured by the code on either side of it. That's what Workers Cache is. And because it composes with everything else the platform already gives you — Smart Placement, Durable Objects, service bindings, ctx.props, ctx.exports — the patterns you can build are open-ended. We've barely scratched the surface in this post.

If you're building with Astro, the Cloudflare adapter wires up Workers Cache for you. Just add the cacheCloudflare provider to your configuration:

// astro.config.mjs
import { defineConfig } from "astro/config";
import cloudflare from "@astrojs/cloudflare";
import { cacheCloudflare } from "@astrojs/cloudflare/cache";

export default defineConfig({
  adapter: cloudflare(),
  output: "server",
  experimental: {
    cache: { provider: cacheCloudflare() },
    routeRules: {
      "/products/*": { maxAge: 300, swr: 3600, tags: ["products"] },
      "/blog/*":     { maxAge: 60,  swr: 86400, tags: ["blog"] },
    },
  },
});

The adapter enables the cache, sets the right headers on the responses Astro generates, attaches Cache-Tag values for invalidation, and gives you a cache.invalidate() helper for purging tags when content changes. Astro pages that opt into server rendering automatically get the "render once, cache, refresh in the background" flow described above — no per-route configuration required, no framework-specific runtime layer to learn.

We're working with the maintainers of other frameworks to ship the same integration. If you build a framework adapter for Cloudflare, the Workers Cache APIs are exactly what you'd want them to be — header-driven configuration, programmatic purges, no platform-specific concepts to model.

Caching is only useful if you can see what it's doing. The Workers Observability dashboard now surfaces cache hit information per invocation:

You can see, per Worker:

• Cache hit ratio over time. The number you want trending up after you enable caching.

• Hits, misses, updates, bypasses broken down. If your hit ratio is low, this is where you find out why — too many BYPASS responses (because something is setting a cookie?), too many MISS responses (because the cache key is partitioning more than you thought?), too many UPDATING responses (because max-age is shorter than your traffic interval?).

Because all of this lives on the same dashboard as your Worker's other observability — logs, exceptions, CPU time, request counts — you don't have to context-switch between looking at your zone and your Worker to understand what's happening.

Cache hits don't run your Worker, and they don't bill CPU time. They do count as a request at the standard Workers request rate, the same as any other invocation. Cache misses and bypasses bill normally — request + CPU time, exactly as they would without caching.

There's no separate Workers Cache SKU and no per-GB cache storage fee. Tiered caching, purges, stale-while-revalidate, and the analytics described above are all included.  If a request would have run your Worker and Workers Cache serves it as a hit instead, you still pay the standard request rate, but you pay no CPU time for that request. Because of this, that cache hit costs less than rendering the same response in your Worker.

One thing to watch: when caching is enabled, requests that are normally free — static asset requests and worker-to-worker invocations through service bindings or ctx.exports — are billed at the standard request rate, because each one now consults the cache in front of your Worker.

Things we know we want to do next:

• Smarter co-location with Smart Placement. Today, Cloudflare chooses the upper-tier cache and Smart Placement target separately. On a full miss, the request may travel between Cloudflare locations twice: once to check the upper tier, and again to run your Worker near its data. We're working to coordinate those choices, so a miss only makes that long-distance trip once.

• Larger response size limits. At launch, all responses follow the Free plan’s cacheable size limit (512 MB), regardless of your account. That’s temporary — the standard per-plan cache limits will apply once we finish a few rollout steps.

• More framework integrations. Astro has built-in integration with Workers Cache. We’re working with maintainers to add similar integrations to other frameworks, including TanStack Start and Next.js via Vinext.

• An API to mark cached responses stale. ctx.cache.purge() removes matching responses from cache. We’re looking at a ctx.cache.invalidate() API that makes matching responses behave as expired, so the next request can still get a fast stale response with stale-while-revalidate while your Worker refreshes the cache in the background.

Workers Cache is available today to every Worker on any plan.

To get started, add "cache": { "enabled": true } to your wrangler.jsonc, redeploy, and start setting Cache-Control headers. The Workers Cache documentation walks through the full feature surface — including the quickstart, cache keys, purging, composition patterns and examples, and debugging.

Workers used to run in front of the cache. Now they can also run behind it. Use whichever side you need — or, with service bindings, both at once.

We can't wait to see what you build.

Today we’re excited to announce an update to our Tiered Cache offering: Regional Tiered Cache.

Tiered Cache allows customers to organize Cloudflare data centers into tiers so that only some “upper-tier” data centers can request content from an origin server, and then send content to “lower-tiers” closer to visitors. Tiered Cache helps content load faster for visitors, makes it cheaper to serve, and reduces origin resource consumption.

Regional Tiered Cache provides an additional layer of caching for Enterprise customers who have a global traffic footprint and want to serve content faster by avoiding network latency when there is a cache miss in a lower-tier, resulting in an upper-tier fetch in a data center located far away. In our trials, customers who have enabled Regional Tiered Cache have seen a 50-100ms improvement in tail cache hit response times from Cloudflare’s CDN.

First, a quick refresher on caching: a request for content is initiated from a visitor on their phone or computer. This request is generally routed to the closest Cloudflare data center. When the request arrives, we look to see if we have the content cached to respond to that request with. If it’s not in cache (it’s a miss), Cloudflare data centers must contact the origin server to get a new copy of the content.

Getting content from an origin server suffers from two issues: latency and increased origin egress and load.

Origin servers, where content is hosted, can be far away from visitors. This is especially true the more global of an audience a particular piece of content has relative to where the origin is located. This means that content hosted in New York can be served in dramatically different amounts of time for visitors in London, Tokyo, and Cape Town. The farther away from New York a visitor is, the longer they must wait before the content is returned. Serving content from cache helps provide a uniform experience to all of these visitors because the content is served from a data center that’s close.

Even when using a CDN, many different visitors can be interacting with different data centers around the world and each data center, without the content visitors are requesting, will need to reach out to the origin for a copy. This can cost customers money because of egress fees origins charge for sending traffic to Cloudflare, and it places needless load on the origin by opening multiple connections for the same content, just headed to different data centers.

When Tiered Cache is not enabled, all data centers in Cloudflare’s network can reach out to the origin in the event of a cache miss.

Performance improvements and origin load reductions are the promise of tiered cache.

Tiered Caching means that instead of every data center reaching out to the origin when there is a cache miss, the lower-tier data center that is closest to the visitor will reach out to a larger upper-tier data center to see if it has the requested content cached before the upper-tier asks the origin for the content. Organizing Cloudflare’s data centers into tiers means that fewer requests will make it back to the origin for the same content, preserving origin resources, reducing load, and saving the customer money in egress fees.

Cloudflare customers are given access to different Tiered Cache topologies based on their plan level. There are currently two predefined Tiered Cache topologies to select from – Smart and Generic Global. If either of those don’t work for a particular customer’s traffic profile, Enterprise customers can also work with us to define a custom topology.

In 2021, we announced that we’d allow all plans to access Smart Tiered Cache. Smart Tiered Cache dynamically finds the single closest data center to a customer’s origin server and chooses that as the upper-tier that all lower-tier data centers reach out to in the event of a cache miss. All other data centers go through that single upper-tier for content and that data center is the only one that can reach out to the origin. This helps to drastically boost cache hit ratios and reduces the connections to the origin. However, this topology can come at the cost of increased latency for visitors that are farther away from that single upper-tier.

When Smart Tiered Cache is enabled, a single upper tier data center can communicate with the origin, helping to conserve origin resources**.**

Enterprise customers may select additional tiered cache topologies like the Generic Global topology which allows all of Cloudflare’s large data centers on our network (about 40 data centers) to serve as upper-tiers. While this topology may help reduce the long tail latencies for far-away visitors, it does so at the cost of increased connections and load on a customer's origin.

When Generic Global Tiered Cache is enabled, lower-tier data centers are mapped to all upper-tier data centers in Cloudflare’s network which can all reach out to the origin in the event of a cache miss. 

To describe the latency problem with Smart Tiered Cache in more detail let’s use an example. Suppose the upper-tier data center is selected to be in New York using Smart Tiered Cache. The traffic profile for the website with the New York upper-tier is relatively global. Visitors are coming from London, Tokyo, and Cape Town. For every cache miss in a lower-tier it will need to reach out to the New York upper-tier for content. This means these requests from Tokyo will need to traverse the Pacific Ocean and most of the continental United States to check the New York upper-tier cache. Then turn around and go all the way back to Tokyo. This is a giant performance hit for visitors outside the US for the sake of improving origin resource load.

With Regional Tiered Cache we introduce a middle-tier in each region around the world. When a lower-tier fetches on a cache miss it tries the regional-tier first if the upper-tier is in a different region. If the regional-tier does not have the asset then it asks the upper-tier for it. On the response the regional-tier writes to its cache so other lower-tiers in the same region benefit.

By putting an additional tier in the same region as the lower-tier, there’s an increased chance that the content will be available in the region before heading to a far-away upper-tier. This can drastically improve the performance of assets while still reducing the number of connections that will eventually need to be made to the customer’s origin.

When Regional Tiered Cache is enabled, all lower-tier data centers will reach out to a regional tier close to them in the event of a cache miss. If the regional tier doesn’t have the content, the regional tier will then ask an upper-tier out of region for the content. This can help improve latency for Smart and Custom Tiered Cache topologies.

Regional Tiered Cache helps customers with Smart Tiered Cache or a Custom Tiered Cache topology with upper-tiers in one or two regions. Regional Tiered Cache is not beneficial for customers with many upper-tiers in many regions like Generic Global Tiered Cache .

Enterprise customers can enable Regional Tiered Cache via the Cloudflare Dashboard or the API:

• To enable Regional Tiered Cache, simply sign in to your account and select your website

• Navigate to the Cache Tab of the dashboard, and select the Tiered Cache Section

• If you have Smart or Custom Tiered Cache Topology Selected, you should have the ability to choose Regional Tiered Cache

Please see the documentation for detailed information about how to configure Regional Tiered Cache from the API.

GET

curl --request GET \
 --url https://api.cloudflare.com/client/v4/zones/zone_identifier/cache/regional_tiered_cache \
 --header 'Content-Type: application/json' \
 --header 'X-Auth-Email: '

PATCH

curl --request PATCH \
 --url https://api.cloudflare.com/client/v4/zones/zone_identifier/cache/regional_tiered_cache \
 --header 'Content-Type: application/json' \
 --header 'X-Auth-Email: ' \
 --data '{
 "value": "on"
}'

Regional Tiered Cache is the first of many planned improvements to Cloudflare’s Tiered Cache offering which are currently in development. We look forward to hearing what you think about Regional Tiered Cache, and if you’re interested in helping us improve our CDN, we’re hiring.

Cloudflare’s mission is to help build a better Internet for everyone. Building a better Internet means helping build more reliable and efficient services that everyone can use. To help realize this vision, we’re announcing the free distribution of two products, one old and one new:

• Tiered Caching is now available to all customers for free. Tiered Caching reduces origin data transfer and improves performance, making web properties cheaper and faster to operate. Tiered Cache was previously a paid addition to Free, Pro, and Business plans as part of Argo.

• Orpheus is now available to all customers for free. Orpheus routes around problems on the Internet to ensure that customer origin servers are reachable from everywhere, reducing the number of errors your visitors see.

Tiered Cache uses the size of our network to reduce requests to customer origins by dramatically increasing cache hit ratios. With data centers around the world, Cloudflare caches content very close to end users, but if a piece of content is not in cache, the Cloudflare edge data centers must contact the origin server to receive the cacheable content. This can be slow and places load on an origin server compared to serving directly from cache.

Tiered Cache works by dividing Cloudflare’s data centers into a hierarchy of lower-tiers and upper-tiers. If content is not cached in lower-tier data centers (generally the ones closest to a visitor), the lower-tier must ask an upper-tier to see if it has the content. If the upper-tier does not have it, only the upper-tier can ask the origin for content. This practice improves bandwidth efficiency by limiting the number of data centers that can ask the origin for content, reduces origin load, and makes websites more cost-effective to operate.

Dividing data centers like this results in improved performance for visitors because distances and links traversed between Cloudflare data centers are generally shorter and faster than the links between data centers and origins. It also reduces load on origins, making web properties more economical to operate. Customers enabling Tiered Cache can achieve a 60% or greater reduction in their cache miss rate as compared to Cloudflare’s traditional CDN service.

Additionally, Tiered Cache concentrates connections to origin servers so they come from a small number of data centers rather than the full set of network locations. This results in fewer open connections using server resources.

• Log into your Cloudflare account.

• Navigate to the Caching in the dashboard.

• Under Caching, select Tiered Cache.

• Enable Tiered Cache.

From there, customers will automatically be enrolled in Smart Tiered Cache Topology without needing to make any additional changes. Enterprise Customers can select from different prefab topologies or have a custom topology created for their unique needs.

Smart Tiered Cache dynamically selects the single best upper tier for each of your website’s origins with no configuration required. We will dynamically find the single best upper tier for an origin by using Cloudflare’s performance and routing data. Cloudflare collects latency data for each request to an origin. Using this latency data, we can determine how well any upper-tier data center is connected with an origin and can empirically select the best data center with the lowest latency to be the upper-tier for an origin.

Today, Smart Tiered Cache is being offered to ALL Cloudflare customers for free, in contrast to other CDNs who may charge exorbitant fees for similar or worse functionality. Current Argo customers will get additional benefits described here. We think that this is a foundational improvement to the performance and economics of running a website.

But what happens if an upper-tier can’t reach an origin?

Cloudflare is a reverse proxy that receives traffic from end users and proxies requests back to customer servers or origins. To be successful, Cloudflare needs to be reachable by end users while simultaneously being able to reach origins. With end users around the world, Cloudflare needs to be able to reach origins from multiple points around the world at the same time. This is easier said than done! The Internet is not homogenous, and diverse Cloudflare network locations do not necessarily take the same paths to a given customer origin at any given time. A customer origin may be reachable from some networks but not from others.

Cloudflare developed Argo to be the Waze of the Internet, allowing our network to react to changes in Internet traffic conditions and route around congestion and breakages in real-time, ensuring end users always have a good experience. Argo Smart Routing provides amazing performance and reliability improvements to our customers.

Enter Orpheus. Orpheus provides reachability benefits for customers by finding unreachable paths on the Internet in real time, and guiding traffic away from those paths, ensuring that Cloudflare will always be able to reach an origin no matter what is happening on the Internet.  

Today, we’re excited to announce that Orpheus is available to and being used by all our customers.

You may have seen this error before at one time or another.

This error indicates that a user was unable to reach content because Cloudflare couldn’t reach the origin. Because of the unpredictability of the Internet described above, users may see this error even when an origin is up and able to receive traffic.

So why do you see this error? The 522 error occurs when network instability causes traffic sent by Cloudflare to fail either before it reaches the origin, or on the way back from the origin to Cloudflare. This is the equivalent of either Cloudflare or your origin sending a request and never getting a response. Both sides think that they’re fine, but the network path between them is not reachable at all. This causes customer pain.

Orpheus solves that pain, ensuring that no matter where users are or where the origin is, an Internet application will always be reachable from Cloudflare.

Orpheus builds and provisions routes from Cloudflare to origins by analyzing data from users on every path from Cloudflare and ordering them on a per-data center level with the goal of eliminating connection errors and minimizing packet loss. If Orpheus detects errors on the current path from Cloudflare back to a customer origin, Orpheus will steer subsequent traffic from the impacted network path to the healthiest path available.

This is similar to how Argo works but with some key differences: Argo is always steering traffic down the fastest path, whereas Orpheus is reactionary and steers traffic down healthy (and not necessarily the fastest) paths when needed.

Let’s look at an example.

Barry has an origin hosted on WordPress in Chicago for his son’s band. This zone primarily sees traffic from three locations: the location closest to his son in Seattle, the location closest to him in Boston, and the location closest to his parents in Tampa, who check in on their grandson’s site daily for updates.

One day, a link between Tampa and the Chicago origin gets cut by a wandering backhoe. This means that Tampa loses some connectivity back to the Chicago origin. As a result, Barry’s parents start to see failures when connecting back to origin when connecting to the site. This reflects in origin reachability decreasing. Orpheus helps here by finding alternate paths for Barry’s parents, whether it’s through Boston, Seattle, or any location in between that isn’t impacted by the fiber cut seen in Tampa.

So even though there is packet loss between one of Cloudflare’s data centers and Barry’s origin, because there is a path through a different Cloudflare data center that doesn’t have loss, the traffic will still succeed because the traffic will go down the non-lossy path.

In our rollout of Orpheus for customers, we observed that Orpheus improved Origin reachability by 23%, from 99.87% to 99.90%. Here is a chart showing the improvement Orpheus provides (lower is better):

We measure this reachability improvement by measuring 522 rates for every data center-origin pair and then comparing traffic that traversed Orpheus routes with traffic that went directly back to origin. Orpheus was especially helpful at improving reachability for slightly lossy paths that could present small amounts of failure over a long period of time, whereas direct to origin would see those failures.

Note that we’ll never get this number to 0% because, with or without Orpheus, some origins really are unreachable because they are down!

Orpheus pairs well with some of our products that are already designed to provide highly available services on an uncertain Internet. Let’s go over the interactions between Orpheus and three of our products: Load Balancing, Cloudflare Network Interconnect, and Tiered Cache.

Orpheus and Load Balancing go together to provide high reachability for every origin endpoint. Load balancing allows for automatic selection of endpoints based on health probes, ensuring that if an origin isn’t working, customers will still be available and operational. Orpheus finds reachable paths from Cloudflare to every origin. These two products in tandem provide a highly available and reachable experience for customers.

Orpheus and Cloudflare Network Interconnect (CNI) combine to always provide a highly reachable path, no matter where in the world you are. Consider Acme, a company who is connected to the Internet by only one provider that has a lot of outages. Orpheus will do its best to steer traffic around the lossy paths, but if there’s only one path back to the customer, Orpheus won’t be able to find a less-lossy path. Cloudflare Network Interconnect solves this problem by providing a path that is separate from the transit provider that any Cloudflare data center can access. CNI provides a viable path back to Acme’s origin that will allow Orpheus to engage from any data center in the world if loss occurs.

Orpheus and Tiered Cache can combine to build an adaptive shield around an origin that caches as much as possible while improving traffic back to origin. Tiered Cache topologies allow for customers to deflect much of their static traffic away from their origin to reduce load, and Orpheus helps ensure that any traffic that has to go back to the origin traverses over highly available links.

The Internet is a growing, ever-changing ecosystem. With the release of Orpheus and Tiered Cache for everyone, we’ve given you the ability to navigate whatever the Internet has in store to provide the best possible experience to your customers.

Caching is a magic trick. Instead of a customer’s origin responding to every request, Cloudflare’s 200+ data centers around the world respond with content that is cached geographically close to visitors. This dramatically improves the load performance for web pages while decreasing the bandwidth costs by having Cloudflare respond to a request with cached content.

However, if content is not in cache, Cloudflare data centers must contact the origin server to receive the content. This isn’t as fast as delivering content from cache. It also places load on an origin server, and is more costly compared to serving directly from cache. These issues can be amplified depending on the geographic distribution of a website’s visitors, the number of data centers contacting the origin, and the available origin resources for responding to requests.

To decrease the number of times our network of data centers communicate with an origin, we organize data centers into tiers so that only upper-tier data centers can request content from an origin and then they spread content to lower tiers. This means content that loads faster for visitors, is cheaper to serve, and reduces origin resource consumption.

Today, I’m thrilled to announce a fundamental improvement to Argo Tiered Cache we’re calling Smart Tiered Cache Topology. When enabled, Argo Tiered Cache will now dynamically select the single best upper tier for each of your website’s origins while providing tiered cache analytics showing how your custom topology is performing.

Tiered Cache is part of Argo, a constellation of products that analyzes and optimizes routing decisions across the global Internet in real-time by processing information from every Cloudflare request to determine which routes to an origin are fast, which are slow, and what the optimum path from visitor to content is at any given moment. Previously, Argo Tiered Cache would use a static collection of upper-tier data centers to communicate with the origin. With the improvements we’re announcing today, Tiered Cache can now dynamically find the single best upper tier for an origin using Argo performance and routing data. When Argo is enabled and a request for particular content is made, we collect latency data for each request to pick the optimal path. Using this latency data, we can determine how well any upper-tier data center is connected with an origin and can empirically select the best data center with the lowest latency to be the upper tier for an origin.

Taking one step back, tiered caching is a practice where Cloudflare’s network of global data centers are subdivided into a hierarchy of upper tiers and lower tiers. In order to control bandwidth and number of connections between an origin and Cloudflare, only upper tiers are permitted to request content from an origin and must propagate that information to the lower tiers. In this way, Cloudflare data centers first talk to each other to find content before asking the origin. This practice improves bandwidth efficiency by limiting the number of data centers that can ask the origin for content, reduces origin load, and makes websites more cost-effective to operate. Argo Tiered Cache customers only pay for data transfer between the client and edge, and we take care of the rest. Tiered caching also allows for improved performance for visitors, because distances and links traversed between Cloudflare data centers are generally shorter and faster than the links between data centers and origins.

Previously, when Argo Tiered Cache was enabled for a website, several of Cloudflare’s largest and most-connected data centers were determined to be upper tiers and could pull content from an origin on a cache MISS. While utilizing a topology consisting of numerous upper-tier data centers may be globally performant, we found that cost-sensitive customers generally wanted to find the single best upper tier for their origin to ensure efficient data transfer of their content to Cloudflare’s network. We built Smart Tiered Cache Topology for this reason.

When you enable Argo Tiered Cache, Cloudflare now by default concentrates connections to origin servers so they come from a single data center. This is done without needing to work with our Customer Success or Solutions Engineering organization to custom configure the best single upper tier. Argo customers can generate this topology by:

• Logging into your Cloudflare account.

• Navigating to the Traffic tab in the dashboard.

• Ensuring you have Argo enabled.

• From there, Non-Enterprise Argo customers will automatically be enrolled in Smart Tiered Cache Topology without needing to make any additional changes.

Enterprise customers can select the type of topology they’d like to generate.

Self-serve Argo customers are automatically enrolled in Smart Tiered Cache Topology

Enterprise customers can determine the tiered cache topology that works best for them.

Once enabled, in addition to performance and cost improvements, Smart Tiered Cache Topology also delivers summary analytics for how the upper tiers are performing so that you can monitor the cost and performance benefits your website is receiving. These analytics are available in the Cache Tab of the dashboard in the Tiered Cache section. The “Primary Data Center” and “Secondary Data Center” fields show you which data centers were determined to be the best upper tier and failover for your origin. “Cached Hits” and the “Hit Ratio” shows you the proportion of requests that were served by the upper tier and how many needed to be forwarded on to the origin for a response. “Bytes Saved” indicates the total transfer from the upper-tier data center to the lower tiers, showing the total bandwidth saved by having Cloudflare’s lower tier data centers ask the upper tiers for the content instead of the origin.

Smart Tiered Cache Topology works with Cloudflare’s existing products to deliver you a seamless, easy, and performant experience that saves you money and provides you useful information about how your upper tiers are working with your origins. Smart Tiered Cache Topology stands on the shoulders of some of the most resilient and useful products at Cloudflare to provide even more benefits to webmasters.

If you’re interested in seeing how Argo and Smart Tiered Cache Topology can benefit your web property, please login to your Cloudflare account and find more information in the Traffic tab of the dashboard here.

A few years ago, we released Argo to help make the Internet faster and more efficient. Argo observes network conditions and finds the optimal route across the Internet for origin server requests, avoiding congestion along the way.

Tiered Cache is an Argo feature that reduces the number of data centers responsible for requesting assets from the origin. With Tiered Cache active, a request in South Africa won’t go directly to an origin in North America, but, instead, look in a large, nearby data center to see if the data requested is cached there first. The number and location of the data centers used by Tiered Cache is controlled by a piece of configuration called the topology. By default, we use a generic topology for every customer that strikes a balance between cache hit ratios and latency that is suitable for most users.

Today we’re introducing Smart Topology, which maximizes cache hit ratios by building on Argo’s internal infrastructure to identify the single best data center for making requests to the origin.

The standard method for caching assets is to let each data center be a reverse proxy for the origin server. In this scheme, a miss in any data center causes a request to the origin for an asset. A request to the origin for one asset could be made as many times as there are data centers.

A cache miss in any data center will result in a request being sent to the origin server even if the asset is cached in some other data center. This is because the data centers are completely oblivious of each other.

Theoretically, a request for the asset would have to be sent to every data center in order to reduce the cache misses to the minimum possible. However, sending every request to every data center is not practical.

The minimum possible cache hit latency is achieved if the asset is moved into the nearest cache before the request for it is made, but this kind of prediction is generally not possible. Instead, a good heuristic is to move the asset into the nearest cache after the first cache miss.

However, the asset has to be copied from somewhere and it isn’t possible to know where in the network it might be without querying each data center.

To avoid querying each data center, a copy of the asset has to be stored in a known location after the first cache miss so it is available to other data centers. This is precisely what Tiered Cache does.

Tiered Cache improves cache hit ratios by allowing some data centers to serve as caches for others, before the latter has to make a request to the origin. With Tiered Cache, certain data centers are reverse proxies to the origin for other data centers.

If the proxied data centers make requests for the same asset, the asset will already be cached in the proxying data center and can be retrieved from there rather than from the origin. Fewer requests to the origin are made overall.

In Tiered Cache, the topology describes which data center should serve as a proxy for others.

For customers, devising an optimal topology is a challenge requiring optimization and continuous maintenance. The best topology is a configuration based on information that is privately held by the customer and other information held only by Cloudflare.

For instance, knowing the desired balance of latency versus cache hit ratio is information only the customer has, but how to best make use of the Internet is something we would know. Enterprise customers usually have dedicated infrastructure teams that work with our solutions engineers to manually optimize and maintain their tiered cache topology.

Not every customer would want to personalize their topology. For this reason a generic topology exists.

The generic topology is designed to achieve good latency and cache efficiency for any origin, regardless of location. A balance is struck between two constraints —  cache efficiency and  latency.

The generic topology has multiple proxying data centers that are distributed around the world in order to ensure that requests that result in a cache miss do not take a very long detour before going to the origin. There is a balance between the number of proxying data centers and the cache hit ratio, because the proxying data centers are oblivious to each other.

If a proxying data center is taken offline, the proxied data centers either use a fallback (if the fallback is online) or revert to behaving like Tiered Cache is disabled.

To achieve the best balance for general usage, the generic topology instructs the smaller data centers to be proxied by the larger data centers in the same geographic region.

Smart Topology assumes the origin is in one place and then automatically configures itself to be optimal once the customer just flips a switch in the dashboard. In order to actually do this, Cloudflare needs to be able to determine which data center has the lowest latency to the origin without making the customer tell Cloudflare where the origin is.

There are a few ways to determine which data center has the lowest latency with respect to the origin.

IP geolocationPhysical distance can be used as an approximation for latency, but Smart Topology was not built this way for a couple of reasons. First, even the best commercial IP geo database doesn’t have the required coverage and accuracy. Second, even with perfect accuracy, physical distance is a questionable approximation of Internet latency.

ProbingLatency to an IP address can be determined exactly by probing that address. The probe can just be the time required to perform the TCP handshake. Each data center probes the origin so that the latencies can be directly measured and the minimum can be found. Except for edge cases involving Anycast and TCP termination, we can assume that the latency to an IP address is the same as the latency to the origin server behind that IP address.

The goal of the topology selection algorithm is to minimize cache misses and latency. The topology chooses a single proxying data center in order to maximize the cache hit ratio. The proxying data center is chosen to be close to the origin so that the latencies of cache misses in the proxied data centers are not much worse than they would be with tiered cache turned off.

The choice should eventually become stable. Stability is important because each time the choice changes, cache misses in proxied data centers are likely to cause cache misses in the new proxying data center. Capacity is important because when a data center goes offline, it can cause a large number of cache misses. Minimizing latency to the origin is important to ensure that the network is used efficiently.

The data center selection algorithm is rather like a leaderboard of the fastest data center for each origin. As data is collected, a faster data center can knock others off a given origin’s leaderboard. This competition is based on the 24 hour median latency and is held each hour. Only a subset of data centers deemed large enough are permitted to compete.

Eventually, the choice for proxying data centers becomes stable. Over time, data centers produce competing records for each origin and less competitive records in the leaderboard are replaced as necessary. Thus, latencies for any origin on the leaderboard can only monotonically decrease. There are always physical limits in the real world, so eventually the ideal data center will set a record that is too good to beat.

Also, the leaderboard actually includes both the lowest latency data center and the second lowest latency data center. The second lowest latency data center serves as a fallback if the preferred data center is taken offline for maintenance.

Anycast NetworksWe are measuring the latency to the origin IP address and assuming that it represents the latency to the origin server, but this can break down in certain cases. A few cloud providers other than Cloudflare also use Anycast technology to provide their services. In Anycast, multiple machines can share an IP address regardless of where they are connected to the Internet, and the Internet will typically route packets destined for that address to the closest machine. If an Anycast network is used to proxy an origin server, then the apparent latency to the IP address for the origin server is actually the latency to the edge of the Anycast network rather than the latency to the origin server. The real latency to the origin server cannot be determined by probing.

The algorithm would fail to select the single best proxying data center if the latencies are not representative of the actual latency between data center and origin. Selecting the wrong data center would adversely affect latencies for requests to the origin, and could be expensive.

For instance, imagine a cloud provider provides an IP address that actually routes to multiple data centers all over the world. Packets are routed through private infrastructure to the correct destination once they enter the network. The lowest latency data center to this Anycast IP address could potentially even be on a different continent than the actual origin server. Therefore, the apparent latency cannot actually be trusted as a true measure of actual latency to the origin.

The data center selection algorithm assumes that the origin is in a single geographic location and can be probed to determine latency from each data center. These networks break one or both of these assumptions, so a procedure had to be developed in order to detect them. First, it is assumed that the IP appears in a single geographic location and is not proxied by such a network. The latency to the origin is bounded by the speed of light through fiber. Although the distance between any data center and the origin server is not known, the distances between data centers is known by Cloudflare.

Imagine putting the origin server as a pitstop in that journey. Then, the theoretical minimum possible observable pair of latencies between the origin server and any two data centers can be computed. We have the latency probe data from both of these data centers and the origin, so we can check to see whether the observed latency is lower than what is possible.

The original assumption was that the origin IP address identifies an origin server that is in one location and the latency to that IP address is the latency to the origin server. If the observed latencies are faster than light then clearly the assumption is false. Smart Topology falls back to the generic topology when the original assumption does not hold. To be extra sure, we check this constraint on a bunch of data centers around the world and fall back if there is even a single physically impossible observation.

When Smart Topology is enabled many Cloudflare systems work together to ensure the correct data center is eventually used to request assets from the origin.

When the customer enables Tiered Cache Smart Topology, one of a few things can happen from the perspective of the origin. If a proxying data center has already been assigned to the CIDR block that encompasses the origin IP, the preferred or fallback data center is used to request assets from the origin. Otherwise, the generic topology is used to determine which proxying data centers to use to pull assets from the origin. The latency to the proxying data center should only decrease as the choice for proxying data center is updated over time.

Developing this technology offered a lot of opportunities to exercise great engineering and build an impactful product. It was not done in a vacuum; we used infrastructure that Cloudflare had already built, and we moved along that exponential gradient of using existing progress to make more progress. Building this framework opens a lot of doors to future progress too; for instance, in the future, we can explore ways to select the ideal proxying data center even for origins behind Anycast networks that hide the true latency to the origin.