This announcement takes us to Cloudflare’s SASE platform, Cloudflare One, used by enterprise security and IT teams to manage the security of their employees, applications, and third-party tools, all in one place.

Starting today, Cloudflare One users can now use the CASB (Cloud Access Security Broker) product to integrate with and scan Amazon Web Services (AWS) S3 and Google Cloud Storage, for posture- and Data Loss Prevention (DLP)-related security issues. Create a free account to check it out.

Scanning both point-in-time and continuously, users can identify misconfigurations in Identity and Access Management (IAM), bucket, and object settings, and detect sensitive information, like Social Security numbers, credit card numbers, or any other pattern using regex, in cloud storage objects.

Over the last few years, our customers — predominantly security and IT teams — have told us about their appreciation for CASB’s simplicity and effectiveness as a SaaS security product. Its number of supported integrations, its ease of setup, and speed in identifying critical issues across popular SaaS platforms, like files shared publicly in Microsoft 365 and exposed sensitive data in Google Workspace, has made it a go-to for many.

However, as we’ve engaged with customers, one thing became clear: the risks of unmonitored or exposed data at-rest go far beyond just SaaS environments. Sensitive information – whether intellectual property, customer data, or personal identifiers – can wreak havoc on an organization’s reputation and its obligations to its customers if it falls into the wrong hands. For many of our customers, the security of data stored in cloud providers like AWS and GCP is even more critical than the security of data in their SaaS tools.

That’s why we’ve extended Cloudflare CASB to include Cloud DLP (Data Loss Prevention) functionality, enabling users to scan objects in Amazon S3 buckets and Google Cloud Storage for sensitive data matches​.

With Cloudflare DLP, you can choose from pre-built detection profiles that look for common data types (such as Social Security Numbers or credit card numbers) or create your own custom profiles using regular expressions​. As soon as an object matching a DLP profile is detected, you can dive into the details, understanding the file’s context, seeing who owns it, and more. These capabilities provide the insight needed to quickly protect data and prevent exposure in real time.

And as with all CASB integrations, this new functionality also comes with posture management features, meaning whether you’re using AWS or GCP, we’ll help you identify misconfigurations and other cloud security issues that could leave your data vulnerable​, like buckets that are publicly-accessible or have critical logging settings disabled, access keys needing rotation, or users without multi-factor authentication (MFA). It’s all included.

Cloudflare CASB and DLP are simple to use by default, making it easy to get started right away. But it’s also highly configurable, giving you the flexibility to fine-tune the scanning profiles to suit your specific needs.

For example, you can adjust which storage buckets or file types to scan, and even sample only a percentage of objects for analysis​. The scanning also runs within your own cloud environment, so your data never leaves your infrastructure​. This approach keeps your cloud storage secure and your costs managed while allowing you to tailor the solution to your organization’s unique compliance and security requirements.

Looking ahead, our roadmap also includes expanding support to additional cloud storage environments, such as Azure Blob Storage and Cloudflare R2, further extending our comprehensive, multi-cloud security strategy. Stay tuned for more on that!

From the start, we knew that to deliver DLP capabilities across cloud environments, it would require an efficient and scalable design to enable real-time detection of sensitive data exposure.

An early design decision was made to leverage a serverless architecture approach to ensure sensitive data discovery is both efficient and scalable. Here’s how it works:

• Compute Account: The entire process runs within a cloud account owned by your organization, known as a Compute Account. This design ensures your data remains within your boundaries, avoiding costly cloud egress fees. The Compute Account can be launched in under 15 minutes using a provided Terraform template.

• Controller function: Every minute, a lightweight, serverless controller function in your cloud environment communicates with Cloudflare’s APIs, fetching the latest DLP configurations and security profiles from your Cloudflare One account.

• Crawler process: The controller triggers an object discovery task, which is processed by a second serverless function known as the Crawler. The Crawler queries cloud storage accounts, like AWS S3 or Google Cloud Storage, via API to identify new objects. Redis is used within the Compute Account to track which objects have yet to be evaluated.

• Scanning for sensitive data: Newly discovered objects are sent through a queue to a third serverless function called the Scanner. This function downloads the objects and streams their contents to the DLP engine in the Compute Account, which scans for matches against predefined or custom DLP Profiles.

• Finding generation and alerts: If a DLP match is found, metadata about the object, such as context and ownership details, is published to a queue. This data is ingested by a Cloudflare-hosted service and presented in the Cloudflare Dashboard as findings, giving security teams the visibility needed to take swift action.

The DLP pipeline ensures that sensitive data never leaves your cloud environment — a privacy-first approach. All communication between the Compute Account and Cloudflare's APIs are initiated by the controller, also meaning there is no need to perform any extra configuration to allow ingress traffic.

To get started, reach out to your account team to learn more about this new data security functionality and our roadmap. If you want to try this out on your own, you can login to the Cloudflare One dashboard (create a free account here if you don’t have one) and navigate to the CASB page to set up your first integration.

Today we are excited to announce Magic Cloud Networking, supercharged by Cloudflare’s recent acquisition of Nefeli Networks’ innovative technology. These new capabilities to visualize and automate cloud networks will give our customers secure, easy, and seamless connection to public cloud environments.

Public clouds offer organizations a scalable and on-demand IT infrastructure without the overhead and expense of running their own datacenter. Cloud networking is foundational to applications that have been migrated to the cloud, but is difficult to manage without automation software, especially when operating at scale across multiple cloud accounts. Magic Cloud Networking uses familiar concepts to provide a single interface that controls and unifies multiple cloud providers’ native network capabilities to create reliable, cost-effective, and secure cloud networks.

Nefeli’s approach to multi-cloud networking solves the problem of building and operating end-to-end networks within and across public clouds, allowing organizations to securely leverage applications spanning any combination of internal and external resources. Adding Nefeli’s technology will make it easier than ever for our customers to connect and protect their users, private networks and applications.

Compared with a traditional on-premises data center network, cloud networking promises simplicity:

• Much of the complexity of physical networking is abstracted away from users because the physical and ethernet layers are not part of the network service exposed by the cloud provider.

• There are fewer control plane protocols; instead, the cloud providers deliver a simplified software-defined network (SDN) that is fully programmable via API.

• There is capacity — from zero up to very large — available instantly and on-demand, only charging for what you use.

However, that promise has not yet been fully realized. Our customers have described several reasons cloud networking is difficult:

• Poor end-to-end visibility: Cloud network visibility tools are difficult to use and silos exist even within single cloud providers that impede end-to-end monitoring and troubleshooting.

• Faster pace: Traditional IT management approaches clash with the promise of the cloud: instant deployment available on-demand. Familiar ClickOps and CLI-driven procedures must be replaced by automation to meet the needs of the business.

• Different technology: Established network architectures in on-premises environments do not seamlessly transition to a public cloud. The missing ethernet layer and advanced control plane protocols were critical in many network designs.

• New cost models: The dynamic pay-as-you-go usage-based cost models of the public clouds are not compatible with established approaches built around fixed cost circuits and 5-year depreciation. Network solutions are often architected with financial constraints, and accordingly, different architectural approaches are sensible in the cloud.

• New security risks: Securing public clouds with true zero trust and least-privilege demands mature operating processes and automation, and familiarity with cloud-specific policies and IAM controls.

• Multi-vendor: Oftentimes enterprise networks have used single-vendor sourcing to facilitate interoperability, operational efficiency, and targeted hiring and training. Operating a network that extends beyond a single cloud, into other clouds or on-premises environments, is a multi-vendor scenario.

Nefeli considered all these problems and the tensions between different customer perspectives to identify where the problem should be solved.

Consider a train system. To operate effectively it has three key layers:

• tracks and trains

• electronic signals

• a company to manage the system and sell tickets.

A train system with good tracks, trains, and signals could still be operating below its full potential because its agents are unable to keep up with passenger demand. The result is that passengers cannot plan itineraries or purchase tickets.

The train company eliminates bottlenecks in process flow by simplifying the schedules, simplifying the pricing, providing agents with better booking systems, and installing automated ticket machines. Now the same fast and reliable infrastructure of tracks, trains, and signals can be used to its full potential.

In networking, there are an analogous set of three layers, called the networking planes:

• Data Plane: the network paths that transport data (in the form of packets) from source to destination.

• Control Plane: protocols and logic that change how packets are steered across the data plane.

• Management Plane: the configuration and monitoring interfaces for the data plane and control plane.

In public cloud networks, these layers map to:

• Cloud Data Plane: The underlying cables and devices are exposed to users as the Virtual Private Cloud (VPC) or Virtual Network (VNet) service that includes subnets, routing tables, security groups/ACLs and additional services such as load-balancers and VPN gateways.

• Cloud Control Plane: In place of distributed protocols, the cloud control plane is a software defined network (SDN) that, for example, programs static route tables. (There is limited use of traditional control plane protocols, such as BGP to interface with external networks and ARP to interface with VMs.)

• Cloud Management Plane: An administrative interface with a UI and API which allows the admin to fully configure the data and control planes. It also provides a variety of monitoring and logging capabilities that can be enabled and integrated with 3rd party systems.

Like our train example, most of the problems that our customers experience with cloud networking are in the third layer: the management plane.

Nefeli simplifies, unifies, and automates cloud network management and operations.

One common approach to tackle management problems in cloud networks is introducing Virtual Network Functions (VNFs), which are virtual machines (VMs) that do packet forwarding, in place of native cloud data plane constructs. Some VNFs are routers, firewalls, or load-balancers ported from a traditional network vendor’s hardware appliances, while others are software-based proxies often built on open-source projects like NGINX or Envoy. Because VNFs mimic their physical counterparts, IT teams could continue using familiar management tooling, but VNFs have downsides:

• VMs do not have custom network silicon and so instead rely on raw compute power. The VM is sized for the peak anticipated load and then typically runs 24x7x365. This drives a high cost of compute regardless of the actual utilization.

• High-availability (HA) relies on fragile, costly, and complex network configuration.

• Service insertion — the configuration to put a VNF into the packet flow — often forces packet paths that incur additional bandwidth charges.

• VNFs are typically licensed similarly to their on-premises counterparts and are expensive.

• VNFs lock in the enterprise and potentially exclude them benefitting from improvements in the cloud’s native data plane offerings.

For these reasons, enterprises are turning away from VNF-based solutions and increasingly looking to rely on the native network capabilities of their cloud service providers. The built-in public cloud networking is elastic, performant, robust, and priced on usage, with high-availability options integrated and backed by the cloud provider’s service level agreement.

In our train example, the tracks and trains are good. Likewise, the cloud network data plane is highly capable. Changing the data plane to solve management plane problems is the wrong approach. To make this work at scale, organizations need a solution that works together with the native network capabilities of cloud service providers.

Nefeli leverages native cloud data plane constructs rather than third party VNFs.

The Nefeli team has joined Cloudflare to integrate cloud network management functionality with Cloudflare One. This capability is called Magic Cloud Networking and with it, enterprises can use the Cloudflare dashboard and API to manage their public cloud networks and connect with Cloudflare One.

Just as train providers are focused only on completing train journeys in their own network, cloud service providers deliver network connectivity and tools within a single cloud account. Many large enterprises have hundreds of cloud accounts across multiple cloud providers. In an end-to-end network this creates disconnected networking silos which introduce operational inefficiencies and risk.

Imagine you are trying to organize a train journey across Europe, and no single train company serves both your origin and destination. You know they all offer the same basic service: a seat on a train. However, your trip is difficult to arrange because it involves multiple trains operated by different companies with their own schedules and ticketing rates, all in different languages!

Magic Cloud Networking is like an online travel agent that aggregates multiple transportation options, books multiple tickets, facilitates changes after booking, and then delivers travel status updates.

Through the Cloudflare dashboard, you can discover all of your network resources across accounts and cloud providers and visualize your end-to-end network in a single interface. Once Magic Cloud Networking discovers your networks, you can build a scalable network through a fully automated and simple workflow.

Resource inventory shows all configuration in a single and responsive UI

Public clouds are used to deliver applications and services. Each cloud provider offers a composable stack of modular building blocks (resources) that start with the foundation of a billing account and then add on security controls. The next foundational layer, for server-based applications, is VPC networking. Additional resources are built on the VPC network foundation until you have compute, storage, and network infrastructure to host the enterprise application and data. Even relatively simple architectures can be composed of hundreds of resources.

The trouble is, these resources expose abstractions that are different from the building blocks you would use to build a service on prem, the abstractions differ between cloud providers, and they form a web of dependencies with complex rules about how configuration changes are made (rules which differ between resource types and cloud providers). For example, say I create 100 VMs, and connect them to an IP network. Can I make changes to the IP network while the VMs are using the network? The answer: it depends.

Magic Cloud Networking handles these differences and complexities for you. It configures native cloud constructs such as VPN gateways, routes, and security groups to securely connect your cloud VPC network to Cloudflare One without having to learn each cloud’s incantations for creating VPN connections and hubs.

Returning to our train system example, what if the railway maintenance staff find a dangerous fault on the railroad track? They manually set the signal to a stop light to prevent any oncoming trains using the faulty section of track. Then, what if, by unfortunate coincidence, the scheduling office is changing the signal schedule, and they set the signals remotely which clears the safety measure made by the maintenance crew? Now there is a problem that no one knows about and the root cause is that multiple authorities can change the signals via different interfaces without coordination.

The same problem exists in cloud networks: configuration changes are made by different teams using different automation and configuration interfaces across a spectrum of roles such as billing, support, security, networking, firewalls, database, and application development.

Once your network is deployed, Magic Cloud Networking monitors its configuration and health, enabling you to be confident that the security and connectivity you put in place yesterday is still in place today. It tracks the cloud resources it is responsible for, automatically reverting drift if they are changed out-of-band, while allowing you to manage other resources, like storage buckets and application servers, with other automation tools. And, as you change your network, Cloudflare takes care of route management, injecting and withdrawing routes globally across Cloudflare and all connected cloud provider networks.

Magic Cloud Networking is fully programmable via API, and can be integrated into existing automation toolchains.

The interface warns when cloud network infrastructure drifts from intent

We are thrilled to introduce Magic Cloud Networking as another pivotal step to fulfilling the promise of the Connectivity Cloud. This marks our initial stride in empowering customers to seamlessly integrate Cloudflare with their public clouds to get securely connected, stay securely connected, and gain flexibility and cost savings as they go.

Join us on this journey for early access: learn more and sign up here.

How great would it be to have a dashboard with a holistic view of threats, malicious server activity, vulnerabilities, sensitive data access levels and a daily scan of resources across all of your applications and services? Now you can.

Cloudflare is thrilled to announce its integration with Cloud Security Command Center (Cloud SCC) for Google Cloud Platform: A security and data risk platform helping enterprises gather data, identify threats, and act on them before they result in business damage or loss.

The advantage of the Cloud SCC solution is that it surfaces insights from both the Google Cloud Platform, as well as Cloudflare’s edge, in a unified dashboard.

Through Cloudflare’s API endpoints, data is pushed to Google’s Cloud SCC dashboard and domain name information mapped to the appropriate Google Cloud asset. Cloudflare’s branded card in the Cloud SCC dashboard is automatically populated with a summary of top theat origins, top types of threats, and latest Web Application Firewall (WAF) events.

To view a full list of Cloudflare events, click on the Cloudflare card in Cloud SCC and it will take you to a “Cloudflare Findings” page. From there, you can select a time period of visible events. The “Findings” table shows what type of event, when it happened, and which asset (website, application, or API) was involved.

Every Cloudflare finding offers detailed information, including: country source, IP source of original request, requested Hostname, requested URI, User Agent, protocol type, method (GET/POST), action taken, rule triggered, and more.

Cloudflare’s findings in the Cloud SCC dashboard highlight which requests were blocked or challenged, and why. To take action on this data, you’ll need to login to your Cloudflare dashboard (link available directly from within Cloud SCC) or configure changes through the Cloudflare API.

One of the great things about container technology is that it delivers the same experience and functionality across different platforms. This frees you as a developer from having to rewrite or update your application to deploy it on a new cloud provider—or lets you run it across multiple cloud providers. With a containerized application running on multiple clouds, you can avoid lock-in, run your application on the cloud for which it’s best suited, and lower your overall costs.

If you’re using Kubernetes, you probably manage traffic to clusters and services across multiple nodes using internal load-balancing services, which is the most common and practical approach. But if you’re running an application on multiple clouds, it can be hard to distribute traffic intelligently among them. In this blog post, we show you how to use Cloudflare Load Balancer in conjunction with Kubernetes so you can start to achieve the benefits of a multi-cloud configuration.

To continue reading follow the Google Cloud blog here or if you are ready to get started we created a guide on how to deploy an application using Kubernetes on GCP and AWS along with our Cloudflare Load Balancer.

Earlier this year, Mary Meeker published her annual Internet Trends report which revealed that 22% of respondents viewed Cloud Vendor Lock-In as a top 3 concern, up from just 7% in 2012. This is in contrast to previous top concerns, Data Security and Cost & Savings, both of which dropped amongst those surveyed.

At Cloudflare, our mission is to help build a better internet. To fulfill this mission, our customers need to have consistent access to the best technology and services, over time. This is especially the case with respect to storage and compute providers. This means not becoming locked-in to any single provider and taking advantage of multiple cloud computing vendors (such as Amazon Web Services or Google Cloud Platform) for the same end user services.

There are a number of potential challenges when selecting a single cloud provider. Though there may be scenarios where it makes sense to consolidate on a single vendor, our belief is that it is important that customers are aware of their choice and downsides of being potentially locked-in to that particular vendor. In short, know what trade offs you are making should you decide to continue to consolidate parts of your network, compute, and storage with a single cloud provider. While not comprehensive, here are a few trade-offs you may be making if you are locked-in to one cloud.

For some companies, there may be a cost savings involved in spreading traffic across multiple vendors. Some can take advantage of free or reduced cost tiers at lower volumes. Vendors may provide reduced costs for certain times of day that are lower utilized on their infrastructure. Applications can have varying compute requirements amongst layers of the application: some may require faster, immediate processing while others may benefit from delayed processing at a lower cost.

One of the most important reasons to consider deploying in multiple cloud providers is to minimize your reliance on a single vendor’s technology for your critical business processes. As you become more vertically integrated with any vendor, your negotiation posture for pricing or favorable contract terms becomes diminished. Having production ready code available on multiple providers allows you to have less technical debt should you need to change. If you go a step further and are already sending traffic to multiple providers, you have minimized the technical debt required to switch and can negotiate from a position of strength.

While the major cloud providers are generally reliable, there have been a few notable outages in recent years. The most significant in recent memory being Amazon’s US-EAST S3 outage in February. Some organizations may have a policy specifying multiple providers for high availability while others should consider it where necessary and feasible as a best practice. A multi-cloud strategy can lower operational risk from a single vendor’s mistakes causing a significant outage for a mission critical application.

One of the exciting things about having competition in the space is the level of innovation and feature velocity of each provider. Every year there are major announcements of new products or features that may have a significant impact on improving your organization's competitive advantage. Having test and production environments in multiple providers gives your engineers the ability to understand and experiment with a new capability in the context of your technology stack and data. You may even try these features for a portion of your traffic and get real world data on any benefits realized.

Cloudflare is an independent third party in your multi-cloud strategy. Our goal is to minimize the layers of lock-in between you and a provider and lower the effort of change. In particular, one area where we can help right away is to minimize the operational changes necessary at the network, similar to what Kubernetes can do at the storage and compute level. As a benefit of our network, you can also have a centralized point for security and operational control.

Cloudflare’s Load Balancing can easily be configured to act as your global application traffic aggregator and distribute your traffic amongst origins at as many clouds as you choose to utilize. Active layer 7 health checks continually probe your origins and can automatically move traffic in the case of network or application failure. All consolidated web traffic can be inspected and acted upon by Cloudflare’s best of breed Security services, providing a single control point and visibility across all application traffic, regardless of which cloud the origin may be on. You also have the benefit of Cloudflare’s Global Anycast Network, providing for better speed and higher availability regardless of which clouds your origins are hosted on.

Billforward is a San Francisco and London based startup that is focused and mission driven on changing the way people bill and charge their customers, providing a solution to the complexities of Quote-to-Cash. Their platform is built on a number of Rest APIs that other developers call to bill and generate revenue for their own companies.

Billforward is using Cloudflare for its core customer facing application to failover traffic between Google Compute Engine and Amazon Web Services. Acting as a reverse proxy, Cloudflare receives all requests for and decides which of Billforward’s two configured cloud origins to use based upon the availability of that origin in near real-time. This allows Billforward to completely manage the connections to and from two disparate cloud providers using Cloudflare’s UI or API. Billforward is in the process of migrating all of their customer facing domains to a similar setup.

Billforward has a single load balanced hostname with two available Pools. They’ve named the two Pools with “gce” and “aws” labels and each Pool has one Origin associated with it. All of the Pools are enabled and the entire LB/hostname is proxied through Cloudflare (as indicated by the orange cloud).

Cloudflare probes Billforward’s Origins once every minute from all of Cloudflare’s data centers around the world (a feature available to all Load Balancing Enterprise customers). If Billforward’s GCE Origin goes down, Cloudflare will quickly and automatically failover to the AWS Origin with no actions required from Billforward’s team.

Google Compute Engine was chosen as the primary provider for this application by virtue of cost. Martin Lee, Site Reliability Engineer at Billforward says, “Essentially, GCE is cheaper for our general purpose computing needs but we're more experienced with deployments in AWS. This strategy allows us to switch back and forth at will and avoid being tied in to either platform.” It is likely that Billforward will change the priority as pricing models evolve.

“It's a fairly fast moving world and features released by cloud providers can have a meaningful impact on performance and cost on a week by week basis - it helps to stay flexible,” says Martin. “We may also change priority based on features.”

For orchestration of the compute and storage layers, Billforward uses Docker containers managed through Rancher. They use distinct environments between cloud providers but are considering bridging an environment across cloud providers and using VPNs between them, which will enable them to move load between providers even more easily. “Our system is loosely coupled through a message queue,” adds Martin. “Having a container system across clouds means we can really take advantage of this - we can very easily move workloads across clouds without any danger of dropping tasks or ending up in an inconsistent state.”

Billforward manages these connections at Cloudflare’s edge. Through this interface (or via the Cloudflare APIs), they can also manually move traffic from GCE to AWS by just disabling the GCE pool or by rearranging the Pool priority and make AWS the primary. These changes are near instant on the Cloudflare network and require no downtime to Billforward’s customer facing application. This allows them to act on potential advantageous pricing changes between the two cloud providers or move traffic to hit pricing tiers.

In addition, Billforward is now not “locked-in” to either provider’s network; being able to move traffic and without any downtime means they can make traffic changes independent of Amazon or Google. They can also integrate additional cloud providers any time they deem fit: adding Microsoft Azure, for example, as a third Origin would be as simple as creating a new Pool and adding it to the Load Balancer.

Billforward is a good example of a forward thinking company that is taking advantage of technologies from multiple providers to best serve their business and customers, while not being reliant on a single vendor. For further detail on their setup using Cloudflare, please check their blog.

Cloudflare logs provide real time insight into traffic, malicious activity, attack incidents, and infrastructure health checks. The output is used to help customers adjust their settings, manage costs and resources, and plan for expansion.

Working with Google, we created an end-to-end solution that allows customers to retrieve Cloudflare access logs, store and process data in a simple way. GCP components such as Google Storage, Cloud Function, BigQuery and Data Studio come together to make this possible.

One of the biggest challenges of data analysis is to store and process large volume of data within a short time period while avoiding high costs. GCP Storage and BigQuery easily address these challenges.

Cloudflare customers can decide if they wish to obtain and process data from Cloudflare access logs on demand or on a regular basis. The full solution is described in this Knowledge Base article. Initial setup takes no more than 30 minutes to an hour. Moreover, customers can still replace any part of the process with their own tool or solution.

Below is a simple visualization of the data flow:

Cloudflare logs are obtained via a REST API. Usually this service can be run on your local workstation or Virtual Machine. The illustrated solution uses GCP Compute micro-instance.

For storing and managing log files we used GCP Storage bucket. All logs are stored in JSON format. Google Cloud Storage allows you to adjust the storage capacity when needed and set the retention policy.

Analyzing large data sets can be challenging. Google BigQuery makes it straightforward. When there is a new log file uploaded to the GCP Storage bucket, GCP Cloud Function triggers the process to import data from the new log file into BigQuery. BigQuery allows you to access your data almost immediately by running a simple query. As illustrated below you can, for example, pull top requested URIs with status code 404.

Based on feedback from our customers about which data they are interested in, we used GCP Data Studio to create visual reports. Data Studio Cloudflare logs analysis template can be found here. The following reports can be created in Data Studio using BigQuery as an input: top client IP address requests, requests by URL, error types, cached or uncached URLs, top triggered WAF rules, traffic types by device or location and many more.

Google Cloud is offering a $500 credit towards a new Google Cloud account to help you get started. In order to receive a credit, please follow these instructions.

Costs depend on several factors including the number of requests, storage, retention policy and number of queries in BigQuery, among others. For more pricing details, please use the GCP Pricing Calculator.

Please reach out to your Cloudflare Enterprise Solution Engineer or Customer Success Manager for more information.

OnAir Video Presentation

Looking to host your website, application, or API in the cloud, or migrate to a new cloud provider while keeping your data secure? In this webinar, Trey Guinn, Head of Solutions Engineering at Cloudflare, will discuss how companies should approach security, during and after migration. We'll highlight the migration story of LUSH, one of the largest global e-Commerce cosmetic retailers, and how they took the right steps to migrate from their previous cloud provider to Google Cloud Platform, in less than 3 weeks. Trey will be performing a live demo on setting up Cloudflare load balancing across cloud providers, as well as optimizing security through web application firewall (WAF), SSL / TLS Encryption, and Rate Limiting.

Asad BaheriSecurity & Networking Partner ManagerGoogle Cloud Platform

Trey GuinnHead of Solutions EngineeringCloudflare

Asad Baheri

Today we're going to talk about LUSH's migration to Google Cloud and how Cloudflare, one of our top security and performance partners, can help you with your own cloud migration. Throughout our presentation, we'll be talking about security best practices, how CDNs and the CDN Interconnect program works, and we're also going to also give you a demo of Cloudflare's load balancing to start your migration.

One of the main things that many people don't realize is the amount of effort that Google has put into security. You may be familiar with safe browsing, which protects over three billion devices every day. That's something Google has done not for profit, but just out of our own commitment to security.

If you actually look at what Google does, they have over 600 engineers dedicated to security and privacy with Google, which is more than a lot of other pure-play security companies. And the result of that is we're trying to make the Internet a more secure place for our users. We believe that raising the security awareness level makes it easier for our customers. So we'll talk about how that philosophy actually translates into Google cloud. If you look at some of the open source projects out there, even if you look at iOS, you'll see Google was one of the top reporters of bugs and security vulnerabilities. Again, we are trying to make the Internet more secure for everyone using our products.

When we talk about Google Cloud, it really starts from the bottom up, at the hardware level. From the second device boots up, the OS, the application, the network, the storage, etc. So if you look more in depth, you get a pretty good idea of all the different areas that we have security.

As you probably are aware, Google has many different data centers which are maintained by us, but we're also in a lot of third party data centers. When a hardware device boots up, we want to make sure that it is one of our devices. We know what it is, what it's supposed to do, and then we have actual Google written security codes to monitor that. And that works hand-in-hand with the Titan chip to actually make sure we have that secure level of trust from the second the device is powered-on, to when users can use it when network traffic is going across.

When we look at that, network plays a big part. As you can see here, there are a lot of data center locations; you'll see all of the lease and owned fiber that Google has. What that really translates to for our users is: No matter where you are in the world, you're going to be close to a Google data center or Google location that can handle your traffic.

Trey GuinnCloudflare also has one of the world's largest networks, although we are focused less on compute and infrastructure as a service, we're focused on providing a smart network which sits in-between the web visitor or API consumer and the origin infrastructure — which could be GCP. Cloudflare is something you may not have heard of, but you've definitely used us if you use the Internet. Cloudflare proxies around 10% of all HTTP requests today. With over six million domains on Cloudflare, we're a fairly large presence. Our job is to make sure that we stop threats, and try to improve performance as as traffic is flowing through our network.

Asad BaheriWhen you say 6 million domains, you mean you're actually managing the DNS records? Whether it's A records, CNAMEs, AAA zones, all of that?

Trey GuinnCorrect. One of the services at Cloudflare is to be an authoritative DNS provider; we're the world's largest and fastest authoritative DNS provider. Customers can also CNAME specific subdomains over to us, and we're handling not only DNS for a lot of these customers, but we're also handling proxying of HTTP and HTTPS traffic.

Through the Cloudflare CDN Interconnect program, you can see we sit in-between the visitor and the Google Cloud Platform. We try to remove all of the online threats, while at the same time speeding up the communication which goes from the visitor to GCP. The best thing is the direct interconnections between our services; it's essentially wire between the Cloudflare datacenter and the Google Cloud platform data centers at 53 locations (which is nearly half of the locations). This interconnection has all kinds of advantages because it's higher performance and you're not having to worry about congestion or fighting the public Internet. But on top of that, as GCP customer, you pay significantly discounted egress pricing when you're using CDN interconnect.

One of our many joint customers is our customer LUSH. They had a big migration over to GCP, and they had to do it really quickly. Interestingly enough, they were already a Cloudflare customer before they moved to GCP. Tell me a little bit about their migration to GCP.

Asad BaheriThis is something LUSH decided on a Friday, and they had to get started on Monday. They had 22 days to move everything. They had a very short period of time. Part of the reason for their migration was around general availability and scale during the holiday season traffic spikes. The other part of it was that they wanted to have their online presence match their corporate philosophy. A very large percentage of our network electrical costs come from renewable sources, and that was something that was important to them. From technology to philosophy, it was just very well aligned and made sense to make that move quickly. LUSH here had a great amount of savings, correct?

Trey GuinnLUSH as a customer sees about a 75% bandwidth savings, but also a 95% reduction in the number of requests to their origin server. That reduces the amount of infrastructure that they need on the origin, because we're either filtering out bad traffic or caching content. Part of that, of course, is that we stop about 60,000 threats a month for LUSH. And that's something to be expected with Cloudflare's infrastructure and network. We run one of the largest security networks in the world, and are versed in stopping these threats.

Asad BaheriThe hit rate seems dependent on the website or API using Cloudflare's services, correct? If you have a lot of cat pictures versus a super dynamic API, it's going to be a difference experience.

Trey GuinnExactly. If you're serving a bunch of GIFs, you can get your cache hit rate up into the high 90s, maybe over 99%. If you are a weather application, and someone's checking the weather, your cache hit rate might be a little bit lower because if you're delivering whether by zip code there's not that many people per zip code or checking for weather.

Asad BaheriCan I actually set my cache rates by different regions and say "Hey for this geography, I want this cached versus another geography I want to have something else cached?"

Trey GuinnIt's possible to go into deeper levels of customization, but generally the caching is going to be around per URL; you can customize it based on certain headers, cookies etc. And the flip-side of that is you can get fine-grained control around rate limiting, in addition our web application firewall (WAF) and IP reputation database allow you to add security and performance at the same time.

Asad BaheriOne of the most important things is actually making sure you have a SSL certificate, that's following best practices; not all SSL certificates are created equally.

Trey GuinnIndependent of whatever origin network or security network that you run, we want to share some security best practices and, as you mentioned, everything should be on SSL. If you didn't know that, this is your last warning, and everything should be over SSL. Chrome as of next month is going to start positively identifying websites which are not encrypted with an "insecure" flag. If you want to be able to retain the trust of your customers, it's required. And a key thing is that not all SSL is created equal: some SSL is more secure than others, some SSL is faster than others, etc. We shared a link on on these slides for you to go look at the SSL labs best practices; they're a third party, but we want to do things like support session resumption and HTTP/2, etc.

Asad BaheriGoogle also has a series of SSL best practices for anyone who's using SSL; it'll walk you through how to create the right type of key requests, where you want to load it, etc. And when using Cloudflare, where does the SSL session terminate?

Trey GuinnCloudflare will terminate the SSL session at the Cloudflare network edge, and we'll make sure that you have the best SSL. If you check your SSL grade, you'll get an A or an A+ with our SSL. We decrypt because we need to be able to protect against layer 7 attacks and look within the application layer. Then we re-encrypt we go back to the origin, so it sits and it's encrypted as it goes across the network.

Asad BaheriAnd then also, you can provide protection for DNS services, because that's another vector of attack where someone can just knock out your DNS or try to inject a bad DNS record.

Trey GuinnExactly. A lot of things people look at DDoS prevention and security but they forget about the fact that DNS infrastructure is one of the key things that you need to protect. It's the Internet's phonebook and if you can find someone's phone number, you can't call them and you're knocked offline. We all number the Dyn attack in October of last year... it took around a third of the internet offline.

Beyond DNS and SSL, other things to be prepared for are large layer 3 and layer 4 floods, this is sort of like UDP floods from DNS amplification — that's sort of a “caveman with a club”, not very sophisticated, but it fills up your pipes. Luckily, if you're on GCP, you have very very big pipes. Beyond that though, you also have to worry about layer 7 attacks. Less sophisticated is: What if someone goes in and searches or scrapes the product pages on your ecommerce site, but they just decide to search your product pages 10,000 times a second? Can your application infrastructure handle that? And even if it can scale up to that, do you want to pay to scale up to that? And then beyond that, while those attacks are occurring, you should also be aware of application vulnerabilities. This is where attacks are going to try to extract data using SQL injection, cross-site scripting, etc. So these are the layers that we want to make sure that you're taking care of.

Asad BaheriSo those are the basics of it, but today we're going to actually show people a demo on how they can set up some of these service and protections on Cloudflare.

Trey GuinnWe're going to jump into the Cloudflare demo, and see how it is that Cloudflare can be configured to sort of meet some of those requirements that we just talked about. This is going to be a live demo, so feel free to play along from home if you'd like. We have a web application; let's assume we grew from a US-only audience. We had lots of folks in North America, but my business has taken off and it's doing great, and it's now distributed out to Australia and the UK.

Trey GuinnPreviously, let's say I had a single origin on AWS. In order to support a geographically distributed origin, I needed the google Spanner database and I'm moving over to GCP. Now I can have an origin within GCP in Australia, U.S. West, and Europe, all at the same time.

Asad BaheriAnd for those not familiar with BigQuery or Google Spanner, this is really where some of Google's technologies shine, where you have these globally distributed databases.

Trey GuinnBut you know once I've added a globally distributed database, I need to be able to access it from everywhere.

So we're going to set up a geographic load balancing that'll migrate our traffic over to GCP. We're also going to look at those layer 7 protections; we'll set up a rate limiting rule and see that come into effect. We'll also make sure that Cloudflare's SSL is working. We'll turn on our WAF and block SQL injections. And then, a little special thing, like any migration project there's always that extra person who wasn't reading their email or didn't check in at the meetings, and we’re going see how to handle that.

So I've already added a domain multicloud.tech to Cloudflare. The signup process takes about five minutes, and most of that is part of the DNS change. And really what you're doing is making Cloudflare your authoritative DNS provider or you can CNAME specific subdomains. In our DNS infrastructure, we've four records but we can see the WWW record is going to the domain apex, the domain apex is going to this IP address, and that's our AWS origin to start with. In our DNS servers listed, we'll see the orange cloud and grey cloud. And what is the difference there?

If we do a dig on www.multicloud.tech, we will see that these are a bunch of Cloudflare IPs. And so we see those are the orange clouded record, the WWW. But what I also had created here, just for usability, is this origin. Warning here: You shouldn't really have things pointing to your origin that are gray clouded, because it allows people to hit your origin directly. Just for demonstration purposes, I've this record is grey clouded. If I do a DNS lookup on origin.multicloud.tech, then we'll see that it's returning back the origin IP address.

What this is doing is when you orange cloud a record in Cloudflare, it's routing all of your customers through the Cloudflare network. And if you gray cloud the record, then all traffic is just going to go straight to your origin server. So it's just a simple sort of "on / off" switch for the Cloudflare network.

So we're now sending traffic through Cloudflare. So if you went to www.multicloud.tech, wherever you are, you'd be going through the Cloudflare network to hit this website. So if I want to look at multicloud.tech, I'm going through the website and we're hitting our AWS infrastructure.

So let's go ahead and set up a load balancer. We've got folks in Australia and UK, and we want to make sure it's fast for everybody. I'm going to create a load balancer here called lb1.multicloud.tech. So when you create a load balancer, it just creates another DNS record that can be used to accept traffic. Now I'm going to create a few origin pools because the idea is that we could have 10 origins in Australia, 30 in North America, and 15 in Europe.

Asad BaheriAm I going to that one DNS record, and then that's getting sent out to different locations, or am I going to come to multiple DNS records that are just going to do round robin?.

Trey GuinnThat’s a great question; so what's happening is you're just seeing the Cloudflare IP on the outside. It's an Anycast network, so it's the same IP all over the world; you connect to Cloudflare and then this is all happening behind the scenes.

Asad BaheriSo, I don't need to worry about maintaining the DNS records, what I want to do for load balancing, etc... you've taken care of all that?

Trey GuinnExactly. Now we're going to set up our first origin pool. This origin pool only to have one origin in it, because this is a demo. But we'll start with the first one in US West; we're going to go with our GCP origin in US West. And we're going to do an active health check of just the root page.

Asad BaheriAnd I can customize that obviously, if I want something more specific?

Trey GuinnExactly. In that health check, that's where you can set the number of times it has to fail before it's unhealthy, and does it check specifically URL, does a check for certain status codes, etc.

So we've got our U.S. West set up, so let's also set up Australia. We're going to go ahead and set the same monitor, and we're going to save that.

And then we still need to do our our European origin. Same health check monitor and we'll hit save. We've created three origin pools and, as I was saying, those origin pools could hold more than one origin each.

And if you add multiple origins in a pool, it would round robin between those. Now we have three pools and we can do a clever migration between them. If you had an active / passive data center setup, you could use two pools and put them in the right order so you'd say origin pool #1 / origin pool #2 and then active / passive. In this instance, we're going to say if, say Europe fails, we want to fail back over the U.S., we'll make US our primary, Europe secondary globally, and then Australia third.

But we also want to do some geo-routing, so let's choose which regions we're going to do some geo-routing.

So we're going to get Europe, Oceania, The Middle East, Africa, Southern Africa, India, and Asia. So we're going to take Europe, and we're going to send that to the European origin. Eastern Europe to European origin. Oceana will go to Australia. Middle East we'll send to Europe. North Africa we'll send that to Europe. South Africa let's send that to Europe. India let's send to Australia. Northeast Asia we'll also send that to Australia.

Now we’re going to hit next; and we're ready to save and deploy.

Asad BaheriThat was less than 10 minutes; we actually set up an infrastructure, set up our pools, and set up our geo-routing.

Trey GuinnSo we're doing active health checks and probes to each of these data centers. Now we're going to say that WWW, which was going to the apex record and went to AWS, will be sent to the load balancer. The other thing I want to do is take the zone apex, multicloud.tech, and I want to send that to the load balancer, as well. This is something else that is special with Cloudflare; if you've ever had to CNAME your apex record, it’s a real bugger; we allow you to do that on our infrastructure, because we do a thing called CNAME flattening. So now all the traffic now is going to the load balancer. We're also going to set up a WAF and setup Rate Limiting rules and we can make all that work together.

Asad BaheriSo now we're just getting into protection; we've setup the infrastructure, the routing, and now we're going to protect it.

Trey GuinnExactly. I'm running out of time already on my demo, so how hard is it to set up a WAF?

Asad BaheriIt's pretty much an all day process setting up rules, I think?

Trey GuinnSo now our WAF is setup; it's pretty easy. We made this product easy to use. We'll also turn on the OWASP top 10 ruleset and put that into block mode.

Now that our WAF is fully engaged, let's set up a rate limiting rule, because you want to be able to stop someone from hammering away at your website. So we'll just call this rate limiting rule “Global” on http & https * it'll just say any URL that you're checking.

Asad BaheriIt's great to have different rulesets on different parts of my website; if I'm managing multiple customers or there's different regions, I can say "hey I want this part of my site or this subdomain to have one type or protection vs. another part."

Trey GuinnOne of the common use cases is to protect your login page, and we can look at the response code and say if you're getting 200's that's fine, but if you're getting 401s or 403s, then clearly you're logging in with the wrong password, so we're going to really restrict you. So if you make more than 10 requests in 30 seconds, we're going to block you for 30 seconds.

Asad BaheriAnd that's really going to stop those automated bot attacks, where people are trying credential stuffing and they're just trying to see how fast can I hammer on a login page, through these credential dumps that I've gotten from somewhere, and see which ones work.

Trey GuinnExactly. Before we reload this webpage, it came from AWS. Now, if I do a refresh, it says "Google Cloud Platform". And I'm coming from US West; if you happen to be watching this from Australia, you'll see that you're coming from the Australian data center, if you're watching this from Europe and hit this website you'd see Europe.

Now let's check our web application firewall (WAF); I snuck a little trick in here in one of my notes, because I wanted to remember a SQL injection command. So maybe with this command I was trying to dump the customer recordbase, and we'll see I've been blocked by Cloudflare. The WAF is in place and working; that command never even made it to GCP, it never hit your infrastructure.

And in our Cloudflare dashboard is the bad request that came through, and it was just blocked.

And the last thing we need to do test out rate limiting. I've setup rate limiting already, so I'm going to do 200 requests against our website. I've just curled multicloud.tech and I'm grepping for the HTTP status code that comes back. And it’s blocked.

Like all migrations, there's always something that comes up; Jerry came to us, and said: "Hey my website stopped working, and I can't find it anymore; what's going on?" And if we look, he's got this awesome old marketing website.

And you know what: We're not going to migrate it to GCP, because it's getting killed off in about six months from now. But how can you create a path and route it over to AWS?

I have the legacy AWS origin defined here as “legacyorigin”, and what I'll do in Cloudflare is create a thing called a "Page Rule".

And this way, we can take the website **www.multicloud.tech/legacy/**\*, so it takes anything under the legacy path and override the resolution of the origin and resolve it to legacyorigin.multicloud.tech. Now this has actually replicate out globally to a bunch of data centers after hitting it here.

Asad BaheriSo in this 10 minutes: We've set up pools, we've set up rules, we've actually kept some of our legacy stuff back where it was, we've shown protection against script kiddies and credential stuffing. And all in 15 minutes, which is pretty amazing. Especially for something which can easily take a month.