As applications grow in complexity, so does the definition of "healthy." We've heard from customers, big and small, that they need to validate multiple services to consider an endpoint ready to receive traffic. For example, they may need to confirm that an underlying API gateway is healthy and that a specific ‘/login’ service is responsive before routing users there. Until now, this required building custom, synthetic services to aggregate these checks, adding operational overhead and another potential point of failure.

Today, we are introducing Monitor Groups for Cloudflare Load Balancing. This feature provides a new way to create sophisticated, multi-service health assessments directly on our platform. With Monitor Groups, you can bundle multiple health monitors into a single logical entity, define which components are critical, and use an aggregated health score to make more intelligent and resilient failover decisions.

This new capability, available via the API for our Enterprise customers, removes the need for custom health aggregation services and provides a far more accurate picture of your application’s true availability. In the near future this feature will be available in the Dashboard for all Load Balancing users, not just Enterprise!

Monitor Groups function as a superset of monitors. Once you have created your monitors they can be bundled into a single unit – the Monitor Group! When you attach a Monitor Group to an endpoint pool, the health of each endpoint in that pool is determined by aggregating the results of all enabled monitors within the group. These settings, defined within the ‘members’ array of a monitor group, give you granular control over how the collective health is determined.

// Structure for a single monitor within a group
{
  "description": "Test Monitor Group",
  "members": [
    {
      "monitor_id": "string",
      "enabled": true,
      "monitoring_only": false,
      "must_be_healthy": true
    },
    {
      "monitor_id": "string",
      "enabled": true,
      "monitoring_only": false,
      "must_be_healthy": true
    }
  ]
}

Here’s what each property does:

• Critical Monitors (must_be_healthy): You can designate a monitor as critical. If a monitor with this setting fails its health check against an endpoint, that endpoint is immediately marked as unhealthy. This provides a definitive override for essential services, regardless of the status of other monitors in the group.

• Observational Probes (monitoring_only): Mark a monitor as "monitoring only" to receive alerts and data without it affecting a pool's health status or traffic steering. This is perfect for testing new checks or observing non-critical dependencies without impacting production traffic.

• Quorum-Based Health: In the absence of a failure from a critical monitor, an endpoint's health is determined by a quorum of all other active monitors. An endpoint is considered globally unhealthy only if more than 50% of its assigned monitors report it as unhealthy. This system prevents an endpoint from being prematurely marked as unhealthy due to a transient failure from a single, non-critical monitor.

You can add up to five monitors to a group.

^{A diagram showing three health monitors (HTTP, TCP, and Database) combined into a single Monitor Group. The group is attached to a Cloudflare Load Balancing pool, which assesses the health of three origin servers.}

The power of Monitor Groups is amplified by the scale of Cloudflare’s global network. Health checks aren't performed from a handful of static locations; they can be configured to execute from data centers in over 300 cities across the globe. While you can configure monitoring from every data center simultaneously ('All Datacenters' mode), we recommend a more targeted approach for most applications. Choosing a few diverse regions, like Western North America and Eastern Europe, or using the 'All Regions' setting provides a robust, global perspective on your application's health while reducing the volume of health monitoring traffic sent to your origins. This creates a distributed consensus on application health, preventing a localized network issue from triggering a false positive and causing an unnecessary global failover. Your application’s health is determined not by a single perspective, but by a global one. 

This same principle elevates Dynamic Steering when used in conjunction with Monitor Groups. The latency for a Monitor Group isn't just a single RTT measurement. It's a holistic performance score, averaged from, potentially, hundreds of points of presence, across all the critical services you’ve defined. This means your load balancer steers traffic based on a true, globally-aware understanding of your application’s performance.

For load balancers using Dynamic Steering and a Monitor Group, the latency used to make steering decisions is now calculated as the average Round Trip Time (RTT) of all active, non-monitoring-only members in the group. This provides a more stable and representative performance metric. Rather than relying on the latency of a single service, Dynamic Steering can now make decisions based on the collective performance of all critical components, ensuring traffic is sent to the endpoint that is truly the most performant overall.

Let's walk through an example to see how Cloudflare aggregates health signals from a Monitor Group to determine the overall health of a single endpoint. In this scenario, our application has three key components we need to check: a public-facing /health endpoint, another service running on a specific TCP port, and a database dependency. Privacy and security are paramount, so, to monitor the database without exposing it to the public Internet, you would securely connect it to Cloudflare using a Cloudflare Tunnel, allowing our health checks to reach it securely.

• Health Monitors in the Group:

  • HTTP check for /health (must_be_healthy: true)

  • TCP check for Port 3000 connectivity (must_be_healthy: false)

  • DB check for database health (must_be_healthy: false)

• Health Check Regions:

  • Western North America (3 data centers)

  • Eastern North America (3 data centers)

• Quorum Threshold: An endpoint is considered healthy if more than 50% of checking data centers report it as UP.

First, Cloudflare determines the health from the perspective of each individual data center. If the critical monitor fails, that data center’s result is definitively DOWN. Otherwise, the result is based on the majority status of the remaining monitors.

Here are the results from our six data centers:

[image description: A table showing health check results from six data centers across two regions. One of the six data centers report a "DOWN" status because the critical HTTP monitor failed. The other five report "UP" because the critical monitor passed and a majority of the remaining monitors were healthy.]

Finally, the results from all six checking data centers are combined to determine the final, global health status for the endpoint.

• Global Result: 5 out of the 6 total data centers (83%) report the endpoint as UP.

• Conclusion: Because 83% is greater than the 50% quorum threshold, the endpoint is considered globally healthy and will continue to receive traffic.

This multi-layered quorum system provides incredible resilience, ensuring that failover decisions are based on a comprehensive and geographically distributed consensus.

Monitor Groups are now available via the API for all customers with an Enterprise Cloudflare Load Balancing subscription and will be made available to self-serve customers in the near future. To get started with building more sophisticated health checks for your applications today, check out our developer documentation. 

To create a monitor group, you can use a POST request to the new /load_balancers/monitor_groups endpoint.

POST accounts/{account_id}/load_balancers/monitor_groups
{
  "description": "Monitor group for checkout service",
  "members": [
    {
      "monitor_id": "string",
      "must_be_healthy": true,
	"enabled": true
    },
    {
      "monitor_id": "string",
      "monitoring_only": false,
	"enabled": true
    }
  ]
}

Once created, you can attach the group to a pool by referencing its ID in the monitor_group field of the pool object. 

We are continuing to build a seamless platform experience that simplifies traffic management for both internal and external applications. Looking ahead, Monitor Groups will be making its way into the Dashboard for all users soon! We are also working on more flexible role-based access controls and even more advanced load-based load balancing capabilities to give you the granular control you need to manage your most complex applications.

In 2023, Cloudflare introduced a new load balancing solution supporting Private Network Load Balancing. This year, we took it a step further by introducing support for layer 4 load balancing to private networks via Spectrum. Now, organizations can seamlessly balance public HTTP(S), TCP, and UDP traffic to their privately hosted applications. Today, we’re thrilled to unveil our latest enhancement: support for end-to-end private traffic flows as well as WARP authenticated device traffic, eliminating the need for dedicated hardware load balancers! These groundbreaking features are powered by the enhanced integration of Cloudflare load balancing with our Cloudflare One platform, and are available to our enterprise customers. With this upgrade, our customers can now utilize Cloudflare load balancers for both public and private traffic directed at private networks.

Before discussing the new features, let's review Cloudflare's existing load balancing support and the challenges customers face.

Cloudflare currently supports four main load balancing traffic flows:

1. Internet-facing load balancers connecting to publicly accessible endpoints at layer 7, supporting HTTP(S).

2. Internet-facing load balancers connecting to publicly accessible endpoints at layer 4 (Spectrum), supporting TCP and UDP services

3. Internet-facing load balancers connecting to private endpoints at layer 7 HTTP(S) via Cloudflare Tunnels.

4. Internet-facing load balancers connecting to private endpoints at layer 4 (Spectrum), supporting TCP and UDP services via Cloudflare Tunnels.

One of the biggest advantages of Cloudflare’s load balancing solutions is the elimination of hardware costs and maintenance. Unlike hardware-based load balancers, which are costly to purchase, license, operate, and upgrade, Cloudflare’s solution requires no hardware. There's no need to buy additional modules or new licenses, and you won't face end-of-life issues with equipment that necessitate costly replacements.

With Cloudflare, you can focus on innovation and growth. Load balancers are deployed in every Cloudflare data center across the globe, in over 300 cities, providing virtually unlimited scale and capacity. You never need to worry about bandwidth constraints, deployment locations, extra hardware modules, downtime, upgrades, or supply chain constraints. Cloudflare’s global Anycast network ensures that every customer connects to a nearby data center and load balancer, where policies, rules, and steering are applied efficiently. And now, the resilience, scale, and simplicity of Cloudflare load balancers can be integrated into your private networks! We have worked hard to ensure that Cloudflare load balancers are highly available and disaster ready, from the core to the edge – even when datacenters lose power.

Before today's announcement, all of Cloudflare's load balancers operating at layer 4 have been connected to the public Internet. Customers have been able to secure the traffic flowing to their load balancers with WAF rules and Zero Trust policies, but some customers would prefer to keep certain resources private and under no circumstances exposed to the Internet. It’s been possible to isolate origin servers and endpoints this way, which can exist on private networks that are only accessible via Cloudflare Tunnels. And as of today, we can offer a similar level of isolation to customers’ layer 4 load balancers.

In our previous blog post, we discussed connecting these internal or private resources to the Cloudflare global network and how Cloudflare would soon introduce load balancers that are accessible via private IP addresses. Unlike other Cloudflare load balancers, these do not have an associated hostname. Rather, they are accessible via an RFC 1918 private IP address. In the land of load balancers, this is often referred to as a virtual IP (VIP). As of today, load balancers that are accessible at private IPs can now be used within a virtual network to isolate traffic to a certain set of Cloudflare tunnels, enabling customers to load balance traffic within their private network without exposing applications to the public Internet.

The question you might be asking is, “If I have a private IP load balancer and privately hosted applications, how do I or my users actually reach these now-private services?”

Cloudflare Magic WAN can now be used as an on-ramp in tandem with Cloudflare load balancers that are accessible via an assigned private IP address. Magic WAN provides a secure and high-performance connection to internal resources, ensuring that traffic remains private and optimized across our global network. With Magic WAN, customers can connect their corporate networks directly to Cloudflare's global network with GRE or IPSec tunnels, maintaining privacy and security while enjoying seamless connectivity. The Magic WAN Connector easily establishes connectivity to Cloudflare without the need to configure network gear, and it can be deployed at any physical or cloud location! With the enhancements to Cloudflare’s load balancing solution, customers can confidently keep their corporate applications resilient while maintaining the end-to-end privacy and security of their resources.

This enhancement opens up numerous use cases for internal load balancing, such as managing traffic between different data centers, efficiently routing traffic for internally hosted applications, optimizing resource allocation for critical applications, and ensuring high availability for internal services. Organizations can now replace traditional hardware-based load balancers, reducing complexity and lowering costs associated with maintaining physical infrastructure. By leveraging Cloudflare load balancing and Magic WAN, companies can achieve greater flexibility and scalability, adapting quickly to changing network demands without the need for additional hardware investments.

But what about latency? Load balancing is all about keeping your applications resilient and performant and Cloudflare was built with speed at its core. There is a Cloudflare datacenter within 50ms of 95% of the Internet-connected population globally! Now, we support all Cloudflare One on-ramps to not only provide seamless and secure connectivity, but also to dramatically reduce latency compared to legacy solutions. Load balancing also works seamlessly with Argo Smart Routing to intelligently route around network congestion to improve your application performance by up to 30%! Check out the blogs here and here to read more about how Cloudflare One can reduce application latency.

But what about when users are distributed and not connected to the local corporate network? Cloudflare WARP can now be used as an on-ramp to reach Cloudflare load balancers that are configured with private IP addresses. The Cloudflare WARP client allows you to protect corporate devices by securely and privately sending traffic from those devices to Cloudflare’s global network, where Cloudflare Gateway can apply advanced web filtering. The WARP client also makes it possible to apply advanced Zero Trust policies that check a device’s health before it connects to corporate applications.

In this load balancing use case, WARP pairs up perfectly with Cloudflare Tunnels so that customers can place their private origins within virtual networks to help either isolate traffic or handle overlapping private IP addresses. Once these virtual networks are defined, administrators can configure WARP profiles to allow their users to connect to the proper virtual networks. Once connected, WARP takes the configuration of the virtual networks and installs routes on the end users’ devices. These routes will tell the end user’s device how to reach the Cloudflare load balancer that was created with a private, non-publicly routable IP address. The administrator could then create a DNS record locally that would point to that private IP address. Once DNS resolves locally, the device would route all subsequent traffic over the WARP connection. This is all seamless to the user and occurs with minimal latency.

In contrast to public L4 or L7 load balancers, private L4 load balancers are not going to have publicly addressable hostnames or IP addresses, but we still need to be able to handle their traffic. To make this possible, we had to integrate existing load balancing services with private networking services created by our Cloudflare One team. To do this, upon creation of a private load balancer, we now assign a private IP address within the customer's virtual network. When traffic destined for a private load balancer enters Cloudflare, our private networking services make a request to load balancing to determine which endpoint to connect to. The information in the response from load balancing is used to connect directly to a privately hosted endpoint via a variety of secure traffic off-ramps. This differs significantly from our public load balancers where traffic is off-ramped to the public internet. In fact, we can now direct traffic from any on-ramp to any off-ramp! This allows for significant flexibility in architecture. For example, not only can we direct WARP traffic to an endpoint connected via GRE or IPSec, but we can also off-ramp this traffic to Cloudflare Tunnel, a CNI connection, or out to the public internet! Now, instead of purchasing a bespoke load balancing solution for each traffic type, like an application or network load balancer, you can configure a single load balancing solution to handle virtually any permutation of traffic that your business needs to run!

We are excited to be releasing these new load balancing features that solve critical connectivity issues for our customers and effectively eliminate the need for a hardware load balancer. Cloudflare load balancers now support end-to-end private traffic flows with Cloudflare One. To get started with configuring this feature, take a look at our load balancing documentation.

We are just getting started with our local traffic management load balancing support. There is so much more to come including user experience changes, enhanced layer 4 session affinity, new steering methods, refined control of egress ports, and more.