Image by Andrew D. Ferguson

The story of IPv6 support for customers of CloudFlare is about as long as the story of CloudFlare itself. June 6th, 2011 (four years ago) was the original World IPv6 Day, and CloudFlare participated. Each year since, the global Internet community has pushed forward with additional IPv6 deployment. Now, four years later, CloudFlare is celebrating June 6th knowing that our customers are being provided with a solid IPv6 offering that requires zero configuration to enable. CloudFlare is the only global CDN that provides IPv4/IPv6 delivery of content by default and at scale.

IPv6 has been featured in our blog various times over the last four years. We have provided support for legacy logging systems to handle IPv6 addresses, provided DDoS protection on IPv6 alongside classic IPv4 address space, and provided the open source community with software to handle ECMP load balancing correctly.

Today CloudFlare is celebrating four years of World IPv6 Day with a few new graphs:

First, let's measure IPv6 addresses (broken down by /64 blocks).

While CloudFlare operates the CDN portion of web traffic delivery to end-users, we don’t control the end-user's deployment of IPv6. We do, however, see IP addresses when they are used, and we clearly see an uptick in deployed IPv6 at the end-user sites. Measuring unique IP addresses is a good indicator of IPv6 deployment.

Next we can look at how end users, using IPv6, access CloudFlare customers' websites.

With nearly 25% of our IPv6 traffic being delivered to mobile devices as of today, CloudFlare is happy to help demonstrate that mobile operators have embraced IPv6 with gusto.

The third graph looks at traffic based on destination country and is a measure of end-users (eyeballs as they are called in the industry) that are enabled for IPv6. The graph shows the top countries that CloudFlare delivers IPv6 web traffic to.

On the Y Axis, we have the percentage of traffic delivered via IPv6. On the X Axis we have each of the last eight months. Let's just have a shoutout to Latin America. In Latin America CloudFlare operates five data centers: Argentina, Brazil, Chile, Columbia, and Peru. CloudFlare would like to highlight our Peru datacenter because it has the highest percentage of traffic being delivered over IPv6 in Latin America. Others agree.

The real question is: What's next for users stuck with only IPv4? Because the RIRs have depleted their IPv4 address space pools, ISPs are simply out of options regarding the need to embrace IPv6. Basically, there are no more IPv4 addresses available.

Supporting IPv6 is even more important today than it was four years ago and CloudFlare is happy that it provides this automatic IPv6 solution for all its customers. Come try it out and don’t let other web properties languish in the non-IPv6 world by telling a friend about our automatic IPv6 support.

The good news is that our IPv6 gateway is being widely used in order to enable the IPv6 Web. In fact, of the sites that support IPv6, more than 20% of them do so via CloudFlare. The bad news is that, while CloudFlare, Google, Facebook and other big people have shown IPv6 can be adopted without a performance penalty, still only 7% of the world's largest websites are available over IPv6. And, disappointingly, in spite of CloudFlare offering it for free, only about half of our customers have turned on our IPv6 gateway.

The silver lining is that, if we can just get all our current customers to enable our IPv6 gateway, we'd nearly reach the milestone of 10% of the world's largest sites being available over IPv6. With that goal in mind, we set out to find and solve the last stumbling blocks for our customers enabling IPv6.

CloudFlare makes supporting IPv6 on a website ridiculously easy. If your backend supports IPv6 then visitors arriving on an IPv6 connection will be transported via the protocol end-to-end. If your backend only supports IPv4, CloudFlare will accept a visitor over IPv6 and then seamlessly make a request to your server over IPv4. We've been defaulting the gateway on for new customers for several months and any existing customer can enable it with a single click. So, if we've made IPv6 so easy, what's stopping those sites that haven't yet enabled it?

The answer is usually legacy software that assumes an IPv4 world. Often this is software that handles sessions or stops fraud or abuse. For example, the popular, if sometimes controversial, website 4chan is a CloudFlare customer. 4chan has a notoriously mischievous audience. Sometimes, users will try to spam the service. To keep spammers at bay, 4chan uses algorithms which look for suspicious behavior. Unfortunately, these algorithms use a visitor's IP address as one of their parameters and they assume the IP will be in the IPv4 format.

The long term solution, of course, is for platforms like 4chan to rewrite their software to accommodate IPv6. That takes time. We wanted to provide a stopgap solution that would allow CloudFlare customers to enable IPv6 quickly while they worked to upgrade their software to fully support it.

To accommodate software that assumes a IPv4 world, today we're enabling a new CloudFlare option: Pseudo IPv4. The option will, whenever a connection is established over IPv6, add a HTTP header to requests with a "pseudo" IPv4 address. We are using Class E IP space (240.0.0.0 - 255.255.255.255) for these addresses. Class E address space is reserved as experimental and no actual traffic should originate from it. If, at some point, some or all of Class E space is designated for a specific use then we'll adjust the schema for the Pseudo IPv4 header.

Class E space gives us 268,435,456 possible unique IPv4 addresses. That's much smaller than the 340 undecillion possible IPv6 addresses, but it's larger than the number of IPs seen by all but the very largest websites. The Pseudo IPv4 service uses a hashing algorithm applied to top 64 bits of the connecting IPv6 address in order to map the visitor to one of the Class E IPs. Because the hashing algorithm will always produce the same output for the same input, the same IPv6 address will always result in the same Pseudo IPv4 address.

There are two options for the Pseudo IPv4 service: 1) you can have us automatically add a header (Cf-Pseudo-IPv4), which can then be parsed by software as needed; or 2) you can have us overwrite the existing Cf-Connecting-IP and X-Forwarded-For headers with a Pseudo IPv4 address. The advantage of the overwrite option is that, in most cases, it won't require any software changes. If you choose the overwrite option, we'll append a new header (Cf-Connecting-IPv6) in order to ensure you can still find the actual connecting IP address for debugging.

The Pseudo IPv4 service, like our IPv6 gateway, is available to all our customers, even those on the free plan. Our hope is that it will eliminate one of the last reasons for IPv6 holdouts. If you're already a CloudFlare customer, login to your account and make sure IPv6 is enabled on the CloudFlare Settings page. You can find the toggle for Pseudo IPv4 there as well. If you're not yet a CloudFlare customer, it only takes five minutes to sign up, so for most of the world there's still time to get your site on our network and join the modern web before World IPv6 Day 2014 comes to an end.

And, by the way, as of today, 4chan is now available over IPv6.

For the technical in the audience, here are some of the nitty gritty details on how we implemented Pseudo IPv4.

We chose to use the MD5 hashing algorithm. While MD5 has risks when it's used in cryptography, it is faster than many alternatives and has a acceptable uniformity over the hash space. Note that there is some risk of a collision (i.e., where two IPv6 addresses map to the same IPv4 pseudo address), however most abuse/fraud systems take this into account already in order to deal with NATs and other instances where multiple people may share a single IP.

At CloudFlare, we use a modified version of NGINX and wrote most of our request processing code in Lua, relying on the ngx_openresty app server. Here's the Lua code for the proof-of-concept prototype of the hashing algorithm:

function pseudo_ipv4(ipv6_top64) -- Grab bottom 32 bits from MD5 hash -- ngx.md5 does not suppress leading zeros so regexp will always match.local hash = ngx.md5(ipv6_top64) local mod = ngx.re.match(hash, "([a-f0-9]{2})([a-f0-9]{2})([a-f0-9]{2})([a-f0-9]{2})$", "joi")

-- Normalize first byte to fit in class E space. Done using subtraction as -- Lua doesn't have built-in bitwise operators local b1 = tonumber(mod[1], 16) if b1 >= 0xF0 then b1 = b1 - 0xF0 end

return string.format("%d.%d.%d.%d", b1 + 0xF0, tonumber(mod[2], 16), tonumber(mod[3], 16), tonumber(mod[4], 16)) end

Since every request needs to pass through the hashing algorithm, we wanted to make it as fast as possible. We set to work optimizing the Lua prototype for speed. Here's the result:

function pseudo_ipv4(ipv6_top64) -- Grab bottom 32 bits from MD5 hash local md5 = ngx.md5_bin(ipv6_top64) local b1, b2, b3, b4 = md5:byte(13, 16)

-- Normalize first byte to fit in class E space b1 = bit.bor(0xF0, bit.band(b1, 0x0F))

return string.format('%d.%d.%d.%d', b1, b2, b3, b4) end

PS - If writing Lua code like the above looks fun, we're always hiring.

Starting three months ago CloudFlare moved to make IPv6 enabled the default setting for all new accounts and for existing accounts that had simply sat on the default settings mode. This resulted in an uptick in the IPv6 traffic stats from our network and was noticed globally. Our support issues have been near zero; which makes sense as the previous three years has been a fantastic proving ground for IPv6 networking.

IPv6 is here today and it works.

But it's still not widely used. It's time for that to change.

Looking at the top one million web sites tracked by Alexa we see that around 7% are accessible using IPv6. Of that 7% a full 20% are accessible using IPv6 because they are CloudFlare customers with IPv6 enabled.

We hope and expect that both those percentages will grow over time.

However some complex sites, that use the origin server IP address within their codebase, have had issues enabling IPv6.

Many sites log the IP address of the visitor when a web page is accessed. This has caused some interesting headaches when migrating to IPv6. Why? Because when software developers hardcode a 4-byte IPv4-address into a database they will have a near-impossible time dealing with at 16-byteIPv6 address. Take a look at this IETF style graphic showing the overall size of the two addressing schemes.

IPv4 (32 bits) +----+----+ |####|####| +----+----+

IPv6 (128 bits) +----+----+----+----+----+----+----+----+ |####|####|####|####|####|####|####|####| +----+----+----+----+----+----+----+----+

When a developer uses a string (or character) area to store the IP address there’s a string length issue.

In IPv4, the maximum length could be represented by the IP address 255.255.255.255 (which is 15 bytes long). For IPv6 the address could be FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF (which is 39 bytes long). Some programming languages and databases can be string length agnostic; however most aren’t.

As a stopgap measure CloudFlare has introduced a pseudo IPv4 address that is mapped from the real IPv6 address and can be used in place of the IPv6. It's not a perfect solution and software and databases are going to have to be upgraded to support IPv6, but in the meantime this pseudo IPv4 address can be used where an IPv4 format address is expected.

This workaround is now available for every CloudFlare customer and we promise to persuade as many of our users who have chosen to disable IPv6 to enable IPv6 fully. Those few remaining sites will want to be part of the new Internet.

Happy third anniversary World IPv6 Day!

CloudFlare has supported IPv6 on our network for the last year and a half. We have become one of the largest providers of the IPv6 web because we offer a free IPv6 gateway that allows any website to be available over IPv6 even if a site's origin network doesn't yet support the protocol. For the last year, we've enabled IPv6 for customers on CloudFlare by default. Today, IPv6 is enabled for more than 1 million of our customers' websites.

Since the beginning of 2013, IPv6 connections as a percentage of CloudFlare's total traffic fluctuate daily with the minimum 0.849% on January 5 to a maximum of 1.645% on June 3, 2013. If look at the overall trend, IPv6 connections to our network have grown 26.5% since the start of the year.

Digging into where IPv6 connections are coming from it appears the majority of the growth has been from mobile network providers. Increasingly, traffic from mobile devices to the web has passed over IPv6. We saw a significant drop in IPv6 connection from mid-March through early-April when it appears a large mobile operator appears to have disabled and then reenabled IPv6 connectivity from their network.

While the overall increase in IPv6 usage is encouraging, the trend unfortunately indicates we are going to be living with IPv4 for some time to come. At current growth rates, assuming adoption of IPv6 is linear, it will take almost 67 years for IPv6 connections to surpass IPv4 connections and the last IPv4 connection won't be retired until May 10, 2148.

Things are a bit more optimistic if IPv6 adoption turns out to be exponential rather than linear. In that case, IPv6 connections will surpass IPv4 in about 5 years and 9 months. Not long thereafter, we'll extinguish IPv4 entirely on January 10, 2020. Our guess is the reality will be somewhere between the linear and exponential case. Regardless of what IPv6's adoption curve looks like, as a CloudFlare user you're covered. We anticipate we will be operating a dual-stack network with both IPv4 and IPv6 support for all our customers until IPv4 is fully retired, whether that takes 7 years or 140.

While the majority of IPv6 connections today are coming from legitimate users on mobile networks, over the last two months we've seen a marked increase in the number of IPv6-based web attacks. Largely these have been DDoS attacks. The attacks have typically been both Layer 4 (e.g., SYN floods) as well as Layer 7 (e.g., application layer attacks).

To date, the IPv6-based DDoS attacks have been relatively modest. The largest we've seen to date generated approximately 3 gigabits per second of traffic and accompanied a much larger traditional IPv4-based DDoS.

While a novelty, these attacks don't cause significant harm to CloudFlare's systems. We designed CloudFlare anticipating the transitionto IPv6, so our defenses assume an IPv6-enabled world. We speculate, however, that attackers may be targeting IPv6 as a way of bypassing older protections that base their protection largely on IPv4 blocklists.

IPv6 makes a strict blocklist on a per-IP basis much more challenging since the number of addresses available to an attacker can be significantly larger. This is a challenge that large blocklist operators like Spamhaus are currently thinking through. While IPv6 can present a challenge to some attack filtering strategies, it also presents opportunities. For example, since IPv6 reduces the need for NATs and provides users addresses that are routable all the way to the end device, we believe over time IPv6 will provide the ability to build significantly more accurate allowlists.

We will continue to monitor overall IPv6 growth rates as well as interesting trends in IPv6-based attacks. In the meantime, there's no better way to celebrate World IPv6 Day than signing up for CloudFlare and ensuring your site is automatically available for the increasing percentage of users that are accessing it over IPv6. It's free and will only take you 5 minutes to join the modern web.