At Cloudflare, we’re committed to making it as easy as possible for developers to make their ideas come to life. Our announcements this week aim to give developers all the tools they need to build their next application on the edge. These include things like static site hosting, certificate management, and image services, just to name a few.
Today, we’re thrilled to announce that we’re exploring a new type of service at the edge: containers.
This announcement will be exciting to some and surprising to many. On this very blog, we’ve talked about why we believe isolates — rather than containers on the edge — will be the future model for applications on the web.
Isolates are best for Distributed Systems
Let us be clear: isolates are the best way to do edge compute, period. The Workers platform is designed to allow developers to treat our global network as one big computer. This has been a long-held dream of generations of engineers, inspiring slogans like "The Network is the Computer" — a trademark which, incidentally, we now own. Isolates and Durable Objects are finally making that vision possible.
In short, isolates excel at distributed systems. They are perfect for treating the network as one big computer.
Isolates are great for distributed systems because, by being extremely lightweight, they enable us to reduce the unit of compute to a very fine granularity. That in turn allows work to be more effectively distributed across a large network. It is completely reasonable and efficient (takes just a few milliseconds, less than a TLS handshake) to spin up an isolate to handle one single HTTP request on the edge, which means we can choose the ideal location for each request to be processed. In contrast, because containers and virtual machines are heavier weight, it's necessary to centralize traffic on a small number of instances to achieve economies of scale.
But there's still a place for containers
Some applications are not really meant to be distributed. Consider, for example, a modern, single-player 3D video game. Such a game can be processing dozens of gigabytes of data every second, which by some measures sounds like "Big Data." Can we make games like that better by running them as a distributed system across a cluster of machines? It turns out… probably not. The problem is that all that data is being compiled down into a single output stream (video frames) which must be delivered in a specific sequence with minimal latency. With today's technology, it just doesn't make sense to distribute this work across a network. As such, isolates don't offer much benefit for this use case.
Meanwhile, at least today, isolates present a challenge when supporting legacy systems. The ecosystem of tooling and technology stacks for isolates is still young and developing. Writing a new application on isolates is great, but taking a complex existing codebase and porting it to run in isolates takes considerable effort. In the case of something like a 3D game, it may not even be possible, as the APIs to access GPUs may not be available. We expect this to improve, but it won't happen overnight.
Isolates
Containers
Distributed/global systems
Legacy/single-user applications
Web application servers
3D rendering
Big data (e.g. MapReduce)
CI builds
We needed them too
We even have a small confession to make: we already built the capability to run containers at the edge for ourselves, specifically for our Browser Isolation product. This product lets you run your web browser on Cloudflare's servers and stream the graphics back to your client machine, increasing security and performance. We didn't build our own browser for this — our technology is based on Chromium.
Chromium is a big existing codebase that cannot realistically run inside isolates today. In fact, the "isolate engine" that Workers is built on — V8 — is itself a piece of Chromium. It's not designed to nest within itself — maybe someday, but not today.
Moreover, a web browser is another example of an application that doesn't make sense to be "distributed." A browser is extremely complex, but serves only one user. It doesn't need to be infinitely scalable or run all around the world at once.
So, instead of trying to build Browser Isolation on Workers, we deployed a container engine to our edge to run Chromium.
Another way to run isolates at the edge
“The edge”, of course, doesn’t have to mean running in all 200+ data centers all the time. We’ve also been able to use containers on the edge ourselves by running them in off-peak locations and for non-latency sensitive tasks. The scheduler for scheduled Workers, for example, runs on our internal container service. Since scheduled events don’t have an end user waiting on a timely response, we’re able to run events in data centers where it’s nighttime and the traffic levels are low.
Another great use case is running CI builds on the edge, though not for the reason you think. Web traffic in any particular location goes through daily cycles. During off-peak hours, a lot of compute is not used. These off-peak locations would be perfect for running batch work like builds in order to maximize compute efficiency.
What about migrating my containers to the edge to make them faster?
While there are some use cases better suited for containers, moving your container workload from its centralized location to the edge may not be the silver bullet you were hoping for.
A container-based web application running in Node.js or Django, for example, is unlikely to reap the same benefits from running on the edge. Due to the high overhead required by containers, your application will experience hundreds of milliseconds and often upwards of seconds of cold starts even when running on the edge. In that context, the saved network latency becomes negligible.
Even if the average request to a warmed-up container was faster, would you be willing to pay a premium for distributing it to 200+ data centers, rather than your current one or two?
Another thing to keep in mind is that being at the edge may introduce considerable cognitive overhead for legacy server stacks in containers. Managing the state of your application running in 200+ locations around the world is very different from managing it in one, two, or even three data centers. We've specifically designed Workers and Durable Objects to abstract away these concerns, but with classical server stacks running in containers, it may not be so easy.
With Cloudflare Workers and now Durable Objects — which were built with the edge in mind — we believe we have the right abstractions to allow developers to build for the edge first.
Container support is for a more limited class of applications that can’t be easily migrated today.
Still can’t contain your excitement?
If you have a use case for running containers at our edge, we’d love to know about it! Sign up for our early access (currently restricted to our enterprise plans) and let us know.