CC BY-SA 2.0 image by Paul Miller

You may wonder why she wanted to inspect the TTL packet field (formally known as "IP Time To Live (TTL)" in IPv4, or "Hop Count" in IPv6)? The reason is simple - she wanted to ensure that the connections are routed outside of our datacenter. The "Hop Distance" - the difference between the TTL value set by the originating machine and the TTL value in the packet received at its destination - shows how many routers the packet crossed. If a packet crossed two or more routers, we know it indeed came from outside of our datacenter.

It's uncommon to look at TTL values (except for their intended purpose of mitigating routing loops by checking when the TTL reaches zero). The normal way to deal with the problem we had would be to blocklist IP ranges of our servers. But it’s not that simple in our setup. Our IP numbering configuration is rather baroque, with plenty of Anycast, Unicast and Reserved IP ranges. Some belong to us, some don't. We wanted to avoid having to maintain a hard-coded blocklist of IP ranges.

The gist of the idea is: we want to note the TTL value from a returned SYN+ACK packet. Having this number we can estimate the Hop Distance - number of routers on the path. If the Hop Distance is:

• zero: we know the connection went to localhost or a local network.

• one: connection went through our router, and was terminated just behind it.

• two: connection went through two routers. Most possibly our router, and one just near to it.

For our use case, we want to see if the Hop Distance was two or more - this would ensure the connection was routed outside the datacenter.

It's easy to read TTL values from a userspace application, right? No. It turns out it's almost impossible. Here are the theoretical options we considered early on:

A) Run a libpcap/tcpdump-like raw socket, and catch the SYN+ACK's manually. We ruled out this design quickly - it requires elevated privileges. Also, raw sockets are pretty fragile: they can suffer packet loss if the userspace application can’t keep up.

B) Use the IP_RECVTTL socket option. IP_RECVTTL requests a "cmsg" data to be attached to control/ancillary data in a recvmsg() syscall. This is a good choice for UDP connections, but this socket option is not supported by TCP SOCK_STREAM sockets.

Extracting the TTL is not so easy.

CC BY-SA 2.0 image by Lee Jordan

Wait, there is a third way!

You see, for quite some time it has been possible to attach a BPF filtering program to a socket. See socket(7)

SO_ATTACH_FILTER (since Linux 2.2), SO_ATTACH_BPF (since Linux 3.19)
    Attach a classic BPF (SO_ATTACH_FILTER) or an extended BPF
    (SO_ATTACH_BPF) program to the socket for use as a filter of
    incoming packets.  A packet will be dropped if the filter pro‐
    gram returns zero.  If the filter program returns a nonzero
    value which is less than the packet's data length, the packet
    will be truncated to the length returned.  If the value
    returned by the filter is greater than or equal to the
    packet's data length, the packet is allowed to proceed unmodi‐
    fied.

You probably take advantage of SO_ATTACH_FILTER already: This is how tcpdump/wireshark does filtering when you're dumping packets off the wire.

How does it work? Depending on the result of a BPF program, packets can be filtered, truncated or passed to the socket without modification. Normally SO_ATTACH_FILTER is used for RAW sockets, but surprisingly, BPF filters can also be attached to normal SOCK_STREAM and SOCK_DGRAM sockets!

We don't want to truncate packets though - we want to extract the TTL. Unfortunately with Classical BPF (cBPF) it's impossible to extract any data from a running BPF filter program.

This changed with modern BPF machinery, which includes:

• modernised eBPF bytecode

• eBPF maps

• bpf() syscall

• SO_ATTACH_BPF socket option

eBPF bytecode can be thought of as an extension to Classical BPF, but it's the extra features that really let it shine.

The gem is the "map" abstraction. An eBPF map is a thingy that allows an eBPF program to store data and share it with a userspace code. Think of an eBPF map as a data structure (a hash table most usually) shared between a userspace program and an eBPF program running in kernel space.

To solve our TTL problem, we can use eBPF filter program. It will look at the TTL values of passing packets, and save them in an eBPF map. Later, we can inspect the eBPF map and analyze the recorded values from userspace.

To use eBPF we need a number of things set up. First, we need to create an "eBPF map". There are many specialized map types, but for our purposes let's use the "hash" BPF_MAP_TYPE_HASH type.

We need to figure out the "bpf(BPF_MAP_CREATE, map type, key size, value size, limit, flags)" parameters. For our small TTL program, let's set 4 bytes for key size, and 8 byte value size. The max element limit is set to 5. It doesn't matter, we expect all the packets in one connection to have just one coherent TTL value anyway.

This is how it would look in a Golang code:

bpfMapFd, err := ebpf.NewMap(ebpf.Hash, 4, 8, 5, 0)

A word of warning is needed here. BPF maps use the "locked memory" resource. With multiple BPF programs and maps, it's easy to exhaust the default tiny 64 KiB limit. Consider bumping this with ulimit -l, for example:

ulimit -l 10240

The bpf() syscall returns a file descriptor pointing to the kernel BPF map we just created. With it handy we can operate on a map. The possible operations are:

• bpf(BPF_MAP_LOOKUP_ELEM, <key>)

• bpf(BPF_MAP_UPDATE_ELEM, <key>, <value>, <flags>)

• bpf(BPF_MAP_DELETE_ELEM, <key>)

• bpf(BPF_MAP_GET_NEXT_KEY, <key>)

More on this later.

With the map created, we need to create a BPF program. As opposed to classical BPF - where the bytecode was a parameter to SO_ATTACH_FILTER - the bytecode is now loaded by the bpf() syscall. Specifically: bpf(BPF_PROG_LOAD).

In our Golang program the eBPF program setup looks like:

ebpfInss := ebpf.Instructions{
	ebpf.BPFIDstOffSrc(ebpf.LdXW, ebpf.Reg0, ebpf.Reg1, 16),
	ebpf.BPFIDstOffImm(ebpf.JEqImm, ebpf.Reg0, 3, int32(htons(ETH_P_IPV6))),
	ebpf.BPFIDstSrc(ebpf.MovSrc, ebpf.Reg6, ebpf.Reg1),
	ebpf.BPFIImm(ebpf.LdAbsB, int32(-0x100000+8)),
...
	ebpf.BPFIDstImm(ebpf.MovImm, ebpf.Reg0, -1),
	ebpf.BPFIOp(ebpf.Exit),
}

bpfProgram, err := ebpf.NewProgram(ebpf.SocketFilter, &ebpfInss, "GPL", 0)

Writing eBPF by hand is rather controversial. Most people use clang (from version 3.7 onwards) to compile a code written in a C dialect into an eBPF bytecode. The resulting bytecode is saved in an ELF file, which can be loaded by most eBPF libraries. This ELF file also includes description of maps, so you don’t need to set them manually.

I personally don't see the point in adding an ELF/clang dependency for simple SO_ATTACH_BPF snippets. Don't be afraid of the raw bytecode!

Before we go further we should highlight couple of things about the eBPF environment. The official kernel documentation isn't too friendly:

• Documentation/networking/filter.txt

The first important bit to know, is the calling convention:

• R0 - return value from in-kernel function, and exit value for eBPF program

• R1-R5 - arguments from eBPF program to in-kernel function

• R6-R9 - callee saved registers that in-kernel function will preserve

• R10 - read-only frame pointer to access stack

When the BPF is started, R1 contains a pointer to ctx. This data structure is defined as struct __sk_buff. For example, to access the protocol field you'd need to run:

r0 = *(u32 *)(r1 + 16)

Or in other words:

ebpf.BPFIDstOffSrc(ebpf.LdXW, ebpf.Reg0, ebpf.Reg1, 16),

Which is exactly what we do in first line of our program, since we need to choose between IPv4 or IPv6 code branches.

Next, there are special instructions for packet payload loading. Most BPF programs (but not all!) run in the context of packet filtering, so it makes sense to accelerate data lookups by having magic opcodes for accessing packet data.

Instead of dereferencing context, like ctx->data[x] to load a byte, BPF supports the BPF_LD instruction that can do it in one operation. There are caveats though, the documentation says:

eBPF has two non-generic instructions: (BPF_ABS | <size> | BPF_LD) and
(BPF_IND | <size> | BPF_LD) which are used to access packet data.

They had to be carried over from classic BPF to have strong performance of
socket filters running in eBPF interpreter. These instructions can only
be used when interpreter context is a pointer to 'struct sk_buff' and
have seven implicit operands. Register R6 is an implicit input that must
contain pointer to sk_buff. Register R0 is an implicit output which contains
the data fetched from the packet. Registers R1-R5 are scratch registers
and must not be used to store the data across BPF_ABS | BPF_LD or
BPF_IND | BPF_LD instructions.

In other words: before calling BPF_LD we must move ctx to R6, like this:

ebpf.BPFIDstSrc(ebpf.MovSrc, ebpf.Reg6, ebpf.Reg1),

Then we can call the load:

ebpf.BPFIImm(ebpf.LdAbsB, int32(-0x100000+7)),

At this stage the result is in r0, but we must remember the r1-r5 should be considered dirty. For an instruction the BPF_LD looks very much like a function call.

Next note the load offset - we loaded the -0x100000+7 byte of the packet. This magic offset is another BPF context curiosity. It turns out that the BPF script loaded under SO_ATTACH_BPF on a SOCK_STREAM (or SOCK_DGRAM) socket, will only see Layer 4 and higher OSI layers by default. To extract the TTL we need access to the layer 3 header (i.e. the IP header). To access L3 in the L4 context, we must offset the data lookups by magical -0x100000.

This magic constant is defined in the kernel.

For completeness, the +7 is, of course, the offset of the TTL field in an IPv4 packet. Our small BPF program also supports IPv6 where the TTL/Hop Count is at offset +8.

Finally, the return value of the BPF program is meaningful. In the context of packet filtering it will be interpreted as a truncated packet length.Had we returned 0 - the packet would be dropped and wouldn't be seen by the userspace socket application. It's quite interesting that we can do packet-based data manipulation with eBPF on a stream-based socket. Anyway, our script returns -1, which when cast to unsigned will be interpreted as a very large number:

ebpf.BPFIDstImm(ebpf.MovImm, ebpf.Reg0, -1),
ebpf.BPFIOp(ebpf.Exit),

Our running BPF program will set a key on our map for any matched packet. The key is the recorded TTL value, the value is the packet count. The value counter is somewhat vulnerable to a tiny race condition, but it's ignorable for our purposes. Later on, to extract the data from userspace program we use this Golang loop:

var (
	value   MapU64
	k1, k2  MapU32
)

for {
	ok, err := bpfMap.Get(k1, &value, 8)
	if ok {
		// k1 is TTL, value is counter
		...
	}

	ok, err = bpfMap.GetNextKey(k1, &k2, 4)
	if err != nil || ok == false {
		break
	}
	k1 = k2
}

Now with all the pieces ready we can make it a proper runnable program. There is little point in discussing it here, so allow me to refer to the source code. The BPF pieces are here:

• ebpf.go

We haven't discussed how to catch inbound SYN+ACK in the BPF program. This is a matter of setting up BPF before calling connect(). Sadly, it's impossible to customize net.Dial in Golang. Instead we wrote a surprisingly painful and awful custom Dial implementation. The ugly custom dialer code is here:

• magic_conn.go

To run all this you need kernel 4.4+ Kernel with the bpf() syscall compiled in. BPF features of specific kernels are documented in this superb page from BCC:

• docs/kernel-versions.md

Run the code to observe the TTL Hop Counts:

$ ./ttl-ebpf tcp4://google.com:80 tcp6://google.com:80 \
             tcp4://cloudflare.com:80 tcp6://cloudflare.com:80
[+] TTL distance to tcp4://google.com:80 172.217.4.174 is 6
[+] TTL distance to tcp6://google.com:80 [2607:f8b0:4005:809::200e] is 4
[+] TTL distance to tcp4://cloudflare.com:80 198.41.215.162 is 3
[+] TTL distance to tcp6://cloudflare.com:80 [2400:cb00:2048:1::c629:d6a2] is 3

In this blog post we dived into the new eBPF machinery, including the bpf() syscall, maps and SO_ATTACH_BPF. This work allowed me to realize the potential of running SO_ATTACH_BPF on fully established TCP sockets. Undoubtedly, eBPF still requires plenty of love and documentation, but it seems to be a perfect bridge to expose low level toggles to userspace applications.

I highly recommend keeping the dependencies small. For small BPF programs, like the one shown, there is little need for complex clang compilation and ELF loading. Don't be afraid of the eBPF bytecode!

We only touched on SO_ATTACH_BPF, where we analyzed network packets with BPF running on network sockets. There is more! First, you can attach BPFs to a dozen "things", XDP being the most obvious example. Full list. Then, it's possible to actually affect kernel packet processing, here is a full list of helper functions, some of which can modify kernel data structures.

In February LWN jokingly wrote:

Developers should be careful, though; this could
prove to be a slippery slope leading toward something 
that starts to look like a microkernel architecture.

There is a grain of truth here. Maybe the ability to run eBPF on variety of subsystems feels like microkernel coding, but definitely the SO_ATTACH_BPF smells like STREAMS programming model from 1984.

Thanks to Gilberto Bertin and David Wragg for helping out with the eBPF bytecode.

Doing eBPF work sound interesting? Join our world famous team in London, Austin, San Francisco, Champaign and our elite office in Warsaw, Poland.

CC BY 2.0 image by Jeremy Keith

The source address was being spoofed, as usual, and apparently chosen randomly, but something else was going on. He offered a bottle of Scotch to the first person to come up with a satisfactory solution.

Here's what some of the packets looked like:

$ tcpdump -ni eth0 -c 10 "ip[8]=40 and udp and port 53"
1.181.207.7.46337 > x.x.x.x.53: 65098+
1.178.97.141.45569 > x.x.x.x.53: 65101+
1.248.136.142.63489 > x.x.x.x.53: 65031+
1.207.241.195.52993 > x.x.x.x.53: 65072+

$ tcpdump -ni eth0 -c 10 "ip[8]=41 and udp and port 53"
2.10.30.2.2562 > x.x.x.x.53: 65013+
2.4.9.36.1026 > x.x.x.x.53: 65019+
2.98.1.99.25090 > x.x.x.x.53: 64925+
2.109.69.229.27906 > x.x.x.x.53: 64914+

$ tcpdump -ni eth0 -c 10 "ip[8]=42 and udp and port 53"
4.72.42.184.18436 > x.x.x.x.53: 64439+
4.240.78.0.61444 > x.x.x.x.53: 64271+
5.73.44.84.18693 > x.x.x.x.53: 64182+
4.69.99.10.17668 > x.x.x.x.53: 64442+

I've removed the destination IP address, but left in the DNS ID number (the number with the plus after it), the spoofed source IP, and port. There are three different TTLs represented (40, 41, and 42) by filtering on ip[8].

Stop reading here if you want to go figure this out for yourself.

I couldn't resist Marek's challenge, so I did what I always do with anything involving packets: I went straight for the hex. Since Marek hadn't given me a pcap, I manually converted the first few to hex.

The reason hex is useful is that bytes, words, dwords, and qwords are the real stuff of computing and looking at decimal obscures data.

Taking the first three (one for each TTL), I saw this pattern:

1.181.207.7.46337
2.10.30.2.2562
4.72.42.184.18436

Converted to hex they are

01.b5.cf.07.b501
02.0a.1e.02.0a02
04.48.2a.b8.4804

It's immediately obvious that the 'random' source port is the first two bytes of the random IP source address reversed: 01.b5.cf.07 has source port b501.

A little bit of tinkering revealed a relationship between the TTL and the first byte of the IP address.

TTL = first byte >> 1 + 40

This relationship was confirmed by a later packet that had a TTL of 150 and a source IP of 220.255.141.181. The shift right also explained why the same TTL was used when the first byte differed by one (see the third group above for an example: 4.x.x.x and 5.x.x.x have the same TTL).

I then spotted that the DNS ID field was also related to the IP address.

1.181.207.7.46337 > x.x.x.x.53: 65098+

Converted to hex:

01.b5.cf.07.b501 > x.x.x.x.35: fe4a+

It's probably not obvious at first glance (unless you're a hex-level hacker), but fe4a is the one's complement of 01b5. So the DNS ID field was simply the one's complement of the first two bytes of the source IP.

Finally, Marek gave me a pcap, and I had one more relationship to find. The ID value in the IPv4 header was also related to the random source IP address—in fact, it was just the first two bytes of the IP address.

One mystery remains (and there's a CloudFlare T-shirt for person with the most convincing explanation): How are the random source IPs being generated?

The answer might be boring (i.e. it might just be reading bytes from /dev/random), but given the author's love of relationships between fields, perhaps there's something else going on. We've seen IP addresses get reused which leads us to think that there's something to discover here.

Here's a sequence of actual source IPs seen:

218.254.187.151
8.187.160.236
123.73.134.186
68.133.199.20
205.26.91.155
169.235.56.120
96.160.119.221
44.226.72.236
205.26.91.155
140.206.27.92
70.62.151.0
161.98.197.249

We have no real guarantee that the attacker generated them in this order, but perhaps there's an interesting way these are being generated.

Answers in comments if you manage to figure it out!

One thing we haven't spent enough time talking about so far though also happens to be at the core of the way our service, as well as any service on the Internet works: DNS. CloudFlare offers DNS services for free- you don't even need to use any of our other free performance or security features to take advantage of our DNS. One of the side effects of building a network such as ours is that our DNS has properties unlike that of nearly all other DNS providers: even within the cutting edge world of Content Delivery Networks. These unique properties allow you to do things that would be much harder or impossible with other providers.

First though, a recap for the uninitiated. DNS (Domain Name System) is the way in which the human-readable addresses such as www.cloudflare.com are turned into the IP addresses that computers use to communicate with each other, such as

198.41.213.157 or 2400:cb00:2048:1::c629:d59d.

You can think of it as the world's greatest phone book, turning human-readable names such Joe Bloggs into the string of numbers you need to call them. DNS entries (or 'records', in the correct parlance) are announced by DNS 'nameservers', which hold the records for a particular domain name or list of domains, such as cloudflare.com. (As a side note, a common misconception is that the domain name includes the www at the beginning: the address www.cloudflare.com is actually a subdomain of the cloudflare.com domain.) Anyone can run a DNS nameserver, and become part of the DNS network.

When you enter the web site name www.cloudflare.com into your web browser and hit enter, the first thing it does is ask a DNS 'resolver' server (usually provided by your ISP) to find the machine-readable IP address that corresponds with the human-readable domain name. In order to do that, much like a postman searching for the correct house to post a letter, the DNS resolver starts at the end of the address, and works its way to the beginning.

First, it asks the 'root' nameservers for the location of the .com records. Once that's found, it asks the .com nameservers for the location of the nameservers for the cloudflare.com domain. Once we have that, it can finally ask the cloudflare.com nameservers for the IP address of www.cloudflare.com. The web browser can then carry on with the web request, now that it has the right IP address to send it to.

Note that in reality it's not necessary to find the root server and then the .com server every time because the DNS server will cache that information.

All great in theory, but there are a few ways in which this process is hampered. Firstly, let's say you run a web site with a DNS nameserver in London, and someone from San Francisco wants to visit your site. To find the correct IP address, they need to make a request all the way from San Francisco to London. A distance of 8,611km is going to add some latency to the request, and slow down the load time of your web site. Secondly, the DNS records have a property known as Time To Live (or TTL). This specifies the length of time that a DNS name should be cached with an ISP's nameservers before being refreshed. This caching means that a commonly accessed name like www.google.com isn't constantly being sent to Google's servers. The lower the TTL (it's usually specified in seconds) the more often DNS resolvers will ask the nameserver for the DNS records of a particular domain name.

Often to save on traffic and processing power, web hosts and other DNS providers will set this to hours or even days. That means that if you want to make some changes to your DNS, it can often take hours or days for the change to be seen by visitors to your web site, as the resolvers don't know to check for changes to your domain. (Web hosts often refer to this as 'propagation time', though the idea that DNS records need to 'propagate-out' is largely a myth: in most cases, it's the TTL set by the web host or DNS provider that causes delays in DNS changes.) Also like all servers, DNS nameservers will fail from time to time, and when they do your web site will drop off the internet.

The way CloudFlare routes traffic allows us to get around these issues, and provide interesting advantages to boot. As with the rest of CloudFlare's networking, our DNS works over Anycast. That means that instead of having to make requests all the way back to the web host, who could be on a different continent, the request is instead made to the nearest of CloudFlare's 24 data centres worldwide. For example, I'm based in the UK, but cloudflare.com is hosted in California. Even so, when I perform a traceroute to CloudFlare's DNS nameserver dns2.cloudflare.com, it's only 7 short hops to reach a server in our London data centre.

$ traceroute dns2.cloudflare.com traceroute to dns2.cloudflare.com (173.245.58.99), 64 hops max, 52 byte packets 1 10.0.1.1 (10.0.1.1) 1.572 ms 1.875 ms 2.047 ms 2 lo0-central10.pcl-ag07.plus.net (195.166.128.188) 16.613 ms 16.897 ms 15.741 ms 3 link-a-central10.pcl-gw01.plus.net (212.159.2.184) 15.757 ms 15.601 ms 15.641 ms 4 xe-9-0-0.pcl-cr01.plus.net (212.159.0.216) 15.787 ms 16.378 ms 15.766 ms 5 ae1.ptw-cr01.plus.net (195.166.129.0) 15.888 ms 15.742 ms 15.549 ms 6 195.66.225.179 (195.66.225.179) 46.333 ms * * 7 dns2.cloudflare.com (173.245.58.99) 16.925 ms 15.713 ms 15.698 ms

This can significantly reduce the time that your web site takes to load for visitors, anywhere in the world. Not only that, but because there's no one single physical DNS server, the chances that DNS would fail for the domain are greatly reduced.

To solve the issue of having to wait ages for new DNS records to set correctly, CloudFlare has TTLs of 5 minutes on all DNS records by default. That means that if you make a change to, or add a new DNS record, you can expect visitors to your site to see the change in under 5 minutes (2 and a half minutes on average).

As well as never having to wait for DNS propagation again, this allows you to do things with DNS records that you wouldn't be able to do with other providers. Let's say you've been working on a replacement for your current website in a staging area on a different IP address, and it's now ready to go live. All you need to do is change the IP address set through your account with us and seconds later the new site will be live and publicly accessible.

Alternatively, let's say you have a server at home which contains personal files such as documents, music and other things you might want to access from work or a coffee shop. Some DNS providers including CloudFlare offer an API so you can write a script to constantly update an address such as home.example.com to point to your server at home, similar to a service such as DynDNS but with your own domain name. With a TTL of hours or days, an update to your home's IP address would make such a script pretty unreliable, as you would need to wait for your domain's records to change. With our DNS though, any downtime for that address wouldn't ever last more than a handful of minutes, allowing you to access your files remotely at all times.

Low TTLs can have more serious uses as well: let's say you run an enterprise-level service, and near-100% uptime is a major concern. If one of your servers stopped working, a quick DNS change to the IP address of your fail-over server is all that's need to get it back up again. As before, this process is scriptable, and can be set to happen automatically if the script detects issues connecting to the primary server.

Another advantage of low TTLs is it means we can move the IP addresses used for customers around at short notice. This is really important for CloudFlare as an attack mitigation service. When an attack reaches our network, our primary goal is to isolate it so that it doesn't affect any other customers, and 'null-route' it so that it doesn't go anywhere. Lowering the TTLs for DNS records even further allows us to do this on-the-fly, allowing us to react to attacks within a few seconds of them hitting our network edge.

Looking to the future, we are investigating other things we can do with the network we've built, and other DNS-related services we can add on top of the ones we already offer. One interesting feature would be the ability for traffic to be sent to different IP addresses depending on where the visitor is located, allowing customers with servers in multiple data centres to direct requests based on where it comes from. For the time-being though, our DNS offers some of the best reliability and speed in the industry, and can help to speed up your website, for free, regardless of whether you use any of the other services CloudFlare offers.

Sam Howson is a member of the Support team for CloudFlare, based in London. When he isn't helping customers to take advantage of CloudFlare's awesome features, he enjoys irritating his housemates with the sounds of his violin and mandolin, going to gigs, concerts & plays and exploring London on his bike.

CloudFlare is hiring! Do you love helping people, and want to work in one of the fastest growing and most exciting companies in tech? If so, check out our careers page. We are looking for team members for both our London and San Francisco offices, in a variety of roles including Technical Support.

CloudFlare makes caching easy. Our service automatically determines what files to cache based on file extensions. Performance benefits kick in automatically.

For customers that want advanced caching, beyond the defaults, we have Cache Everything available as Page Rules. Designate a URL and CloudFlare will cache everything, including HTML, out at the edges of our global network.

With Cache Everything, we respect all headers. If there is any header in place from the server or a CMS solution like WordPress, we will respect it. However, we got many requests from customers who wanted an easy way to override any existing headers. Today, we are releasing a new feature called 'Edge cache expire TTL' that does just that.

Edge cache expire TTL is the setting that controls how long CloudFlare's edge servers will cache a resource before requesting a fresh copy from your server. When you create a Cache Everything Page Rule, you now may choose whether to respect all existing headers or to override any headers that are in place from your server. By overwriting the headers, CloudFlare will cache more content at the CloudFlare edge network, decreasing load to your server.

Common situations where you may choose to overwrite existing headers:

• You expect a large surge in traffic

• You are under DDOS attack

• You are not sure what the headers on WordPress or your server are set to

• You are using WordPress and want to easily overwrite the default settings

It is important to emphasize when you do not want to use Cache Everything. If you have any personalized information on the page like login information or credit card information, you do not want to use the Cache Everything option.**What is Browser Cache Expire TTL?**Browser cache expire TTL is the time that CloudFlare instructs a visitor's browser to cache a resource. Until this time expires, the browser will load the resource from its local cache thus speeding up the request significantly. CloudFlare will respect the headers that you give us from your web server, and then we will communicate with the browser based on the time selected in this drop down menu.

When you'd like to have CloudFlare cache your content but want your visitors to always get a fresh copy of the page, you can use the new 'Edge cache expire TTL' setting to express this differentiation. Set a value for 'Edge cache expire TTL' to how often you want the CloudFlare CDN to refresh from your server, and 'Browser cache expire TTL' to how often you want your visitors' browsers to refresh the page content. This is useful when you have a rapidly changing page but still want the benefit of the CloudFlare cache to reduce your server load.

CloudFlare offers a range of edge cache expire TTLs based on plan type:

• Free 2 hours

• Pro 1 hour

• Business 30 minutes

• Enterprise as low as 30 seconds

A Pro customer may set the refetch time to 1 hour. After 60 minutes, we return to your server for a fresh copy of the resource. Business customers may lower the refetch interval to 30 minutes. Enterprise customers may set this interval as low as 30 seconds.

Login in to your CloudFlare account and choose "Page Rules" from under the gear icon. Enter the URL that you want to Cache Everything (under Custom Caching):

The edge cache server TTL option will appear:

The default setting is set to "Respect all existing headers." To override this setting, choose a time from the drop down menu:

You can find more information in our knowledge base articles here and here.

Give it a try and let us know what you think.

At CloudFlare, we think a lot about the Domain Name Service -- better known as DNS, the Internet's address book. CloudFlare uses DNS to bring performance and security to our customers. But, we don't spend a lot of time talking about our DNS service, behind the scenes.

One recent customer comment reminded us of an unsung advantage:

_"The thing is, we had this new design running on a QA server for the past week, doing final testing and updating. Yesterday we changed the IP to point the domain to the new QA server, away from the "old" prod server.

BUT unlike every other time when we change a site's IP - the change was instantaneous‚ since the only IP the world has for our sites is CloudFlare, so when we change servers, IPs, etc.

So with CloudFlare, instant server move, web host move, IP change -- never deal with DNS propagation again. That is awesome."_

Because the public IP addresses don't change, using CloudFlare removes the wait for TTL (Time To Live) record expiration and DNS propagation when you make a change to one of your site's DNS records. This waiting period is often an excruciating part of development and deployment changes, because it's completely out of your control. Yes, you can lower your TTLs ahead of time, but it's still a seemingly endless wait for the world's recursive DNS servers to come get the new record.

If DNS isn't something you spend a lot of time thinking about, that's fine. Just know that once you're on CloudFlare, your changes are live faster, and you can get back to everything else on your plate. Making running a website easier...that's part of CloudFlare's goal, even in the small things.

P.S. - We recently made major improvements to our DNS management interface. You can also manage DNS via API.