Today we are thrilled to announce our support of a new set of HTTP response headers that provide surgical control over our CDN’s caching decisions. CDN-Cache-Control allows customers to directly control how our CDN behaves without affecting the behavior of downstream or upstream caches.
You might be thinking that this sounds a lot like the Cache-Control header we all know and love. And it’s very similar! CDN-Cache-Control has exactly the same directives as the Cache-Control header. The problem CDN-Cache-Control sets out to solve is that with Cache-Control, some directives are targeted at specific classes of caches (like s-maxage for shared caches), while other directives are not targeted at controlling any specific classes of intermediary caches (think stale-while-revalidate). As these non-specific directives are returned to downstream caches, they’re often not applied uniformly. This problem is amplified as the number of intermediary caches grows between an origin and the client.
For example, a website may deploy a caching layer on the origin server itself, there might be a cache on the origin’s network, the site might use one or more CDNs to cache content distributed throughout the Internet, and the visitor’s browser might cache content as well. As the response returns from the origin, each of these layers must interpret the set Cache-Control directives. These layers, however, are not guaranteed to interpret Cache-Control in the same way, which can cause unexpected behavior and confusion.
We set out to solve these problems and have been working with industry peers who also run large CDNs to create an industry standard solution through the Internet Engineering Task Force. CDN-Cache-Control is aimed at providing directives to manage how specific CDNs behave when caching objects.
Introducing CDN-Cache-Control
CDN-Cache-Control is a response header field set on the origin to control the behavior of CDN caches separately from other intermediaries that might handle a response. This feature can be used by setting the CDN-Cache-Control and/or Cloudflare-CDN-Cache-Control response header. Both of these new headers support the same directives currently supported by Cache-Control and also have the same semantics and directive precedence. In other words, if you were to copy and paste a Cache-Control value and insert it into either of these new headers, the same caching behavior should be observed.
Header precedence; or, which header should I use when?
When introducing a set of new cache response headers, a question at the forefront of the cache-conscious mind is how will these directives work when combined with each other or other Cache-Control directives? There are several options depending on how these headers are used. An origin can:
Return the CDN-Cache-Control response header which Cloudflare will evaluate to make caching decisions. Cache-Control, if also returned by the origin, will be proxied as is (more on this later) and will not affect caching decisions made by Cloudflare. In addition, CDN-Cache-Control will also be proxied downstream in case there are other CDNs between Cloudflare and the browser.
Return the Cloudflare-CDN-Cache-Control response header. This results in the same behavior as the origin returning CDN-Cache-Control except we will NOT proxy Cloudflare-CDN-Cache-Control downstream because it’s a header only used to control Cloudflare. This is beneficial if you want only Cloudflare to have a different caching behavior while having all downstream servers rely on Cache-Control, or you simply don’t want Cloudflare to proxy the CDN-Cache-Control header downstream.
Return both Cloudflare-CDN-Cache-Control and CDN-Cache-Control response headers. In this case, Cloudflare will only look at Cloudflare-CDN-Cache-Control when making caching decisions because it is the most specific version of CDN-Cache-Control and will proxy CDN-Cache-Control downstream. Only forwarding CDN-Cache-Control in this situation is beneficial if you want Cloudflare to have a different caching behavior than other CDNs downstream.
For example, a response leaves the origin server and can hit the following caches on the way to the browser and can be controlled by the following response headers (assuming the other CDNs support CDN-Cache-Control):
Caches
Control Headers
Origin Server Cache
Cache-Control
Shared Cache on the Origin Network
Cache-Control
CDN #1
CDN-Cache-Control
Cloudflare CDN
Cloudflare-CDN-Cache-Control/CDN-Cache-Control
CDN #N
CDN-Cache-Control
Browser Cache
Cache-Control
With these headers and directives set, intermediaries know whether it’s safe for something to be cached, how long it should be cached, and what to do once it’s no longer permitted to remain in cache.
Interaction with other Cloudflare features
Edge Cache TTL Page Rule
Edge Cache TTL is a page rule that is meant to override the amount of time an asset is cached on the edge (Cloudflare data centers) and therefore overrides directives in Cloudflare-CDN-Cache-Control/CDN-Cache-Control managing how long an asset is cached on the edge. This page rule can be set in the rules section of the dashboard.
Browser Cache TTL Page Rule
Browser Cache TTL is a page rule that is meant to override the amount of time an asset is cached by browsers/servers downstream of Cloudflare. Therefore, Browser Cache TTL will only modify the Cache-Control response header. Cloudflare-CDN-Cache-Control/CDN-Cache-Control response headers will not be modified by this page rule.
Interaction With Other Origin Response Headers
The Expires response header returned by the origin, which generally directs a browser on how long before an object should be considered stale, will not affect the caching decision at Cloudflare when Cloudflare-CDN-Cache-Control/CDN-Cache-Control is being used.
Interaction with Cloudflare Default Cache Values
In the situation where Cloudflare does not receive Cloudflare-CDN-Cache-Control, CDN-Cache-Control, or Cache-Control values, the general default values will be used for cacheable assets.
When should I use CDN-Cache-Control?
Caching is one of the most powerful tools available to ensure all possible requests are served from data centers near visitors to improve a website’s performance and limit origin egress. Cache-Control directives are the rules that dictate how caches should behave. These rules dictate how long something should stay in cache and direct the cache on what to do once that content has expired. However, in a setup where there are numerous caching layers between the origin and client, getting the desired control over each hop a response makes back to the client is complicated. This difficulty is exacerbated by unpredictable behavior by intermediary caches proxying or stripping cache control headers sent downstream.
Let’s walk through a few examples for how to use CDN-Cache-Control:
Acme Corp
Acme Corp is a user of Cloudflare’s CDN. They want to manage their cached asset’s TTLs separately for origin caches, CDN caches, and browser caches. Previously, Page Rules were required to specify their desired behavior. Now with CDN-Cache-Control, this common scenario can be accomplished solely through the use of origin-set response headers.
Before
Headers:
Cache-Control: max-age=14400, s-maxage=84000
Set an Edge Cache TTL Page Rule on Cloudflare for 24400 seconds fixed to the asset’s path
Cache Behavior:
Caches
Cache TTL (seconds)
Origin Server Cache
14400
Cloudflare Edge
24400
Other CDNs
84000
Browser Cache
14400
Now (no need for Page Rule configuration, and can set different TTLs on different CDNs)
Headers:
Cache-Control: max-age=14400, s-maxage=84000
Cloudflare-CDN-Cache-Control: max-age=24400
CDN-Cache-Control: 18000
Cache Behavior:
Caches
Cache TTL (seconds)
Origin Server Cache
14400
Network Shared Cache
84000
Cloudflare Edge
24400
Other CDNs
18000
Browser Cache
14400
ABC Industries
ABC Industries uses multiple CDNs stacked together serially and wants cache-specific control over when to serve stale content in the case of an error or during revalidation. This can more easily be expressed by using the new CDN-Cache-Control headers in combination with Cache-Control headers.
Previously, if a user wanted to specify when certain intermediaries should serve stale content, this could not be done. It was up to the cache to decide whether or not the directive applied to it and whether it should pass the header downstream. The new headers provide CDN-specific control for when to use stale assets to fulfill requests.
Before
Headers:
Cache-Control: stale-if-error=400
Behavior in response to 5XX Error:
Caches
Stale served (seconds) in response to error
Origin Cache Layer
400 (if it assumes the directive applies)
Cloudflare Edge
400 (we assume the directive applies if we get it from upstream)
Unknown CDN/Network caches/Browser Cache
0 (if they assume the directives doesn’t apply or they don’t get them from upstream); or 400 (if they do assume it applies)
Now (explicit indication of when directives apply to CDNs)
Headers:
Cache-Control: stale-if-error=400
Cloudflare-CDN-Cache-Control: stale-if-error=60
CDN-Cache-Control: stale-if-error=200
Behavior in response to 5XX Error:
Caches
Stale served (seconds) in response to error
Origin Cache Layer/Network Cache/Browser Cache
400 (if it assumes the directive applies)
Cloudflare Edge
60
Other CDN
200
Try it out!
Overall, CDN-Cache-Control allows finer grained control of how Cloudflare manages cache lifetimes and revalidation behavior on a per-asset basis.
If you’re looking for more control over how your CDNs’ cache objects, I encourage you to try these new headers out. And if you’re another CDN reading this, I recommend looking to add support for CDN-Cache-Control!