Great documentation empowers users to be successful with a new product as quickly as possible, showing them how to use the product and describing its benefits. Relevant, timely, and accurate content can save frustration, time, and money. Open source documentation adds a few more benefits, including building inclusive and supportive communities that help reduce the learning curve. We love being open source!

While the Cloudflare content team has scaled to deliver documentation alongside product launches, the open source documentation site itself was not scaling well. developers.cloudflare.com had outgrown the workflow for contributors, plus we were missing out on all the neat stuff created by developers in the community.

Just like a software product evaluation, we reviewed our business needs. We asked ourselves if remaining open source was appropriate? Were there other tools we wanted to use? What benefits did we want to see in a year or in five years? Our biggest limitations in addition to the contributor workflow challenges seemed to be around scalability and high maintenance costs for user experience improvements. 

After compiling our wishlist of new features to implement, we reaffirmed our commitment to open source. We valued the benefit of open source in both the content and the underlying framework of our documentation site. This commitment goes beyond technical considerations, because it's a fundamental aspect of our relationship with our community and our philosophy of transparency and collaboration. While the choice of an open source framework to build the site on might not be visible to many visitors, we recognized its significance for our community of developers and contributors. Our decision-making process was heavily influenced by two primary factors: first, whether the update would enhance the collaborative ecosystem, and second, how it would improve the overall documentation experience. This focus reflects that our open source principles, applied to both content and infrastructure, are essential for fostering innovation, ensuring quality through peer review, and building a more engaged and empowered user community.

Cloudflare’s developer documentation is open source on GitHub, with content supporting all of Cloudflare’s products. The underlying documentation engine has gone through a few iterations, with the first version of the site released in 2020. That first version provided dev-friendly features such as dark mode and proper code syntax. 

In 2021, we introduced a new custom documentation engine, bringing significant improvements to the Cloudflare content experience. The benefits of the Gatsby to Hugo migration included:

• Faster development flow: The development flow replicated production behavior, increasing iteration speed and confidence. Preview links via Cloudflare Pages were also introduced, so the content team and stakeholders could quickly review what content would look like in production.

• Custom components: Introduced features like resources-by-selector which let us reference content throughout the repository and gave us the flexibility to expand checks and automations.

• Structured changelog management: Implementation of structured YAML changelog entries which facilitated sharing with various platforms like RSS feeds, Developer Discord, and within the docs themselves.

• Improved performance: Significant page load time improvements with the migration to HTML-first and almost instantaneous local builds.

These features were non-negotiable as part of our evaluation of whether to migrate. We knew that any update to the site had to maintain the functionality we’d established as core parts of the new experience.

After careful evaluation, we chose to migrate from Hugo to the Astro (and by extension, JavaScript) ecosystem. Astro fulfilled many items on our wishlist including:

• Enhanced content organization: Improved tagging and better cross-referencing of  related pages.

• Extensibility: Support for user plugins like starlight-image-zoom for lightbox functionality.

• Development experience: Type-checking at build time with astro check, along with syntax highlighting, Intellisense, diagnostic messages, and plugins for ESLint, Stylelint, and Prettier. 

• JavaScript/TypeScript support: Aligned the docs site framework with the preferred languages of many contributors, facilitating easier contribution.

• CSS management: Introduction of Tailwind and scoped styles.

• Content collections: Offered various ways to manage and enhance tagging practices including Markdown front matter validated by Zod schemas, JSON schemas for Intellisense, and a JavaScript callback for filtering returned entries.

Starlight, Astro’s documentation theme, was a key factor in the decision. Its powerful component overrides and plugins system allowed us to leverage built-in components and base styling.

Content needed to be migrated quickly. With dozens of pull requests opened and merged each day, entering a code freeze for a week simply wasn’t feasible. This is where the nature of abstract syntax trees (ASTs) came into play, only parsing the structure of a Markdown document rather than details like whitespace or indentation that would make a regular expression approach tricky.

With Hugo in 2021, we configured code block functionality like titles or line highlights with front matter inside the code block.

---
title: index.js
highlight: 1
---
const foo = "bar";

Starlight uses Expressive Code for code blocks, and these options are now on the opening code fence.

js title="index.js" {1}
const foo = "bar";

With astray, this is a simple as visiting the `code` nodes and:

1. Parsing `node.value` with front-matter.

2. Assigning the attributes from `front-matter` to `node.meta`.

3. Replacing `node.value` with the rest of the code block.

import { fromMarkdown } from "mdast-util-from-markdown";
import { toMarkdown } from "mdast-util-to-markdown";
 
import * as astray from "astray";
import type * as MDAST from "mdast";
import fm from "front-matter";
 
const markdown = await Bun.file("example.md").text();
 
const AST = fromMarkdown(markdown);
 
astray.walk<MDAST.Root, void, any>(AST, {
    code(node: MDAST.Code) {
        const { attributes, body } = fm(node.value);
        const { title, highlight } = attributes;
 
        if (title) {
            node.meta = `title="${title}"`;
        }
 
        if (highlight) {
            node.meta += ` {${highlight}}`;
        }
 
        node.value = body;
 
        return;
    }
})

When we migrated from Gatsby to Hugo in 2021, the pull request included 4,850 files and the migration took close to three weeks from planning to implementation. This time around, the migration was nearly twice as large, with 8,060 files changed. Our planning and migration took six weeks in total:

• 10 days: Evaluate platforms, vendors, and features 

• 14 days: Migrate the components required by the documentation site

• 5 days: Staging and user acceptance testing (UAT) 

• 8 hours: Code freeze and migrate to Astro/Starlight

The migration resulted in removing a net -19,624 lines of code from our maintenance burden.

While the number of files had grown substantially since our last major migration, our strategy was very similar to the 2021 migration. We used Markdown AST and astray, a utility to walk ASTs, created specifically for the previous migration!

A website migration like our move to Astro/Starlight is a complex process that requires time to plan, review, and coordinate, and our preparation paid off! Including our Cloudflare Community MVPs as part of the planning and review period proved incredibly helpful. They provided great guidance and feedback as we planned for the migration. We only needed one day of code freeze, and there were no rollbacks or major incidents. Visitors to the site never experienced downtime, and overall the migration was a major success.

During testing, we ran into several use cases that warranted using experimental Astro APIs. These APIs were always well documented, thanks to fantastic open source content from the Astro community. We were able to implement them quickly without impacting our release timeline.

We also ran into an edge case with build time performance due to the number of pages on our site (4000+). The Astro team was quick to triage the problem and begin investigation for a permanent fix. Their fast, helpful fixes made us truly grateful for the support from the Astro Discord server. A big thank you to the Astro/Starlight community!

Migrating developers.cloudflare.com to Astro/Starlight is just one example of the ways we prioritize world-class documentation and user experiences at Cloudflare. Our deep investment in documentation makes this a great place to work for technical writers, UX strategists, and many other content creators. Since adopting a content like a product strategy in 2021, we have evolved to better serve the open source community by focusing on inclusivity and transparency, which ultimately leads to happier Cloudflare users. 

We invite everyone to connect with us and explore these exciting new updates. Feel free to reach out if you’d like to speak with someone on the content team or share feedback about our documentation. You can share your thoughts or submit a pull request directly on the cloudflare-docs repository in GitHub.

We’re excited to share that the next iteration of Cloudflare’s API reference documentation is now available. The new docs standardize our API content and improve the overall developer experience for interacting with Cloudflare’s API.

Everyone talks about how important APIs are, but not everyone acknowledges the critical role that API documentation plays in an API’s usability. Throwing docs together is easy. Getting them right is harder.

At Cloudflare, we try to meet our users where they are. For the majority of customers, that means providing clear, easy-to-use products in our dashboard. But developers don’t always want what our dashboard provides. Some developers prefer to use a CLI or Wrangler to have a higher level of control over what’s happening with their Cloudflare products. Others want more customization and deeper ties into their company’s internal applications. Some want all the above.

A developer’s job is to create, debug, and optimize code - whether that’s an application, interface, database, etc. - as efficiently as possible and ensure that code runs as efficiently as possible. APIs enable that efficiency through automation. Let’s say a developer wants to run a cache purge every time content is updated on their website. They could use Cloudflare’s dashboard to enable cache purge, but they might want it to happen automatically instead. Enter Cloudflare’s API.

In the same way that the Cloudflare dashboard is an interface for humans, an API is an interface for computers. For a computer to execute on instructions, there is no room for interpretation. The instructions, formatted as requests, have to follow a specific set of rules and include certain requirements. API documentation details what those rules and requirements are.

It’s a frustrating experience for developers when you’re in the details of a complex project and can’t troubleshoot an error because the docs aren’t comprehensive or don’t load. Unfortunately, that’s been the reality for developers using Cloudflare’s API. Figuring out how to use Cloudflare’s API was a “choose your own adventure” story for the Cloudflare users who made 126 million visits monthly to our API documentation. If the APIs you needed were fully documented, you encountered long page load times and a site that couldn’t render on mobile.

From a technical standpoint, we were using api.cloudflare.com for both the API documentation site and the Cloudflare API access point, which is awkward. We needed a more sustainable way for internal teams to create and document their APIs according to an accepted standard.

Like with all of our documentation projects at Cloudflare, we started by thinking about the user’s workflow - in this case, a developer’s workflow as they’re getting started with the Cloudflare API.

Providing docs on mobileWe know developers research products before diving into projects, usually exploring API documentation before writing any code. That means making docs available on mobile for working on the go. Check.

Improving the navigationAs a developer, at the point you’re looking at API docs, you generally have an idea of what you want to use the API for. Our goal with the new API docs is to make it as easy as possible for you to find the information you need. We recognize that endpoints are organized a bit haphazardly in the current API docs. To address that clunkiness, we’ve grouped endpoints by product or setting and alphabetized that list on the new site, making it easier to navigate and find what you’re looking for. And while this may seem like a small thing, you now can use a keyboard shortcut to search the page.

Clarifying authenticationOnce developers start writing code, you first have to handle authentication to start using the API. Authentication information is now readily available for every endpoint, and we link to the full developer docs about authentication from the API overview page.

Adding examplesGood API docs have clear descriptions, but the best API docs share templates and examples to minimize a developer’s friction to deploying code. Every endpoint now includes an example and clearly indicates which parameters are required, removing yet another decision developers have to make when working with our API.

Gathering feedback and iteratingAs we implemented all of these improvements, we kept some of our biggest users - our internal developers - informed along the way. We shared the test site early and often to get internal feedback, which caught bugs and helped us refine the site’s usability. The Discord and Community MVPs also volunteered to test the site, giving us valuable outside perspective on what we built. We incorporated all of that feedback to provide a vetted, deliberate UX at launch.

Since this week is all about what you can build on our developer platform, we wanted to share the details about how this all works under the hood (hint: it’s mostly Workers).

We used a combination of open-source tools and Cloudflare products to revamp the API doc site. Previously the content on api.cloudflare.com was sourced from the JSON hyper-schema files that described our APIs, but over the years we repeatedly heard that published schemas would help you better integrate with Cloudflare. Several Cloudflare engineering teams started adopting the OpenAPI specification, and with a little research, planning, and testing, we pivoted from those JSON hyper-schemas to the OpenAPI framework. Check out the blog post about our Transition to OpenAPI for more details.

Because the OpenAPI specification defines how to describe APIs, our schema files now have consistency - not just among the various products, but also with an industry-accepted standard. Hundreds of documentation and code generation tools exist to pull from that standard, which means we have options that weren’t available for our homegrown JSON hyper-schemas.

We chose Stoplight Elements, an open-source React framework, for our site design because it has a clean layout and is easily customizable. While there are a number of both dynamic and static tools for parsing OpenAPI schemas and rendering documentation sites, we chose React because of its ubiquity, performance, and because it plays well with Cloudflare Pages, our deployment tool of choice. Within Stoplight you can generate code samples for a variety of  programming languages, like JavaScript, Java, and Python, as well as for tools like cURL, HTTPie, and wget.

We build and deploy the React application with Cloudflare Pages, and using Pages functions, we optimize the OpenAPI schema file for Stoplight’s UI and cache it on Cloudflare’s network, reducing the latency needed to request the schema. Whenever teams add new API endpoints or definitions to the schema, we just update the schema file in GitHub. Because our API documentation loads the schema dynamically, this means we don’t have to wait for Cloudflare Pages to deploy a new version of the documentation site. The automation helps us ensure that everything we expose in the API is documented without manual interaction. Deploying with Pages gives us yet another opportunity to test our own products out on ourselves, helping us find areas for improvement that we turn around and pass along to you.

Moving our schemas to the OpenAPI specification allows us to use ready-to-go tools like Stoplight, but we’re just getting started. Soon we’ll be adding search functionality, letting you access all relevant information across developer, API, and support docs. The mobile experience will evolve, providing an even cleaner way to read API content when you’re on your phone. We’ll also add a “try it out” functionality to test out your requests right in the browser before writing your own code.

Over time, we want an even tighter integration between the API reference documentation, our how-to content in developer docs, and the Cloudflare dashboard. The standardized structure we get with the OpenAPI spec sets us up to reuse the schema files across our client libraries, Terraform, API gateway, and Trakal. The consistency across API descriptions makes it easier for us to do things like localize API content, customize out-of-the-box documentation generation tools, and continue innovating like we always do.

As you’re using the new API doc site, send us your feedback and let us know if there are any feature improvements you’d like to see in the future. We want the site to be as useful as possible for your day-to-day interactions with Cloudflare’s API. We hope the improvements to our API doc experience will help you seamlessly use our API and efficiently deploy and maintain your Cloudflare products.

Thanks for your patience and perspective as we iterate on the API docs!

As a SaaS provider, you’re juggling many challenges while building your application, whether it’s custom domain support, protection from attacks, or maintaining an origin server. In 2021, we were proud to announce Cloudflare for SaaS for Everyone, which allows anyone to use Cloudflare to cover those challenges, so they can focus on other aspects of their business. This product has a variety of potential implementations; now, we are excited to announce a new section in our Developer Docs specifically devoted to Cloudflare for SaaS documentation to allow you take full advantage of its product suite.

You may remember, from our October 2021 blog post, all the ways that Cloudflare provides solutions for SaaS providers:

• Set up an origin server

• Encrypt your customers’ traffic

• Keep your customers online

• Boost the performance of global customers

• Support custom domains

• Protect against attacks and bots

• Scale for growth

• Provide insights and analytics

However, we received feedback from customers indicating confusion around actually using the capabilities of Cloudflare for SaaS because there are so many features! With the existing documentation, it wasn’t 100% clear how to enhance security and performance, or how to support custom domains. Now, we want to show customers how to use Cloudflare for SaaS to its full potential by including more product integrations in the docs, as opposed to only focusing on the SSL/TLS piece.

Cloudflare for SaaS can be overwhelming with so many possible add-ons and configurations. That’s why the new docs are organized into six main categories, housing a number of new, detailed guides (for example, WAF for SaaS and Regional Services for SaaS):

Once you get your SaaS application up and running with the Get Started page, you can find which configurations are best suited to your needs based on your priorities as a provider. Even if you aren’t sure what your goals are, this setup outlines the possibilities much more clearly through a number of new documents and product guides such as:

• Regional Services for SaaS

• Querying HTTP events by hostname with GraphQL

• Migrating custom hostnames

Instead of pondering over vague subsection titles, you can peruse with purpose in mind. The advantages and possibilities of Cloudflare for SaaS are highlighted instead of hidden.

This setup facilitates configurations much more easily to meet your goals as a SaaS provider.

For example, consider performance. Previously, there was no documentation surrounding reduced latency for SaaS providers. Now, the Performance section explains the automatic benefits to your performance by onboarding with Cloudflare for SaaS. Additionally, it offers three options of how to reduce latency even further through brand-new docs:

• Early Hints for SaaS

• Cache for SaaS

• Argo Smart Routing for SaaS

Similarly, the new organization offers WAF for SaaS as a previously hidden security solution, extending providers the ability to enable automatic protection from vulnerabilities and the flexibility to create custom rules. This is conveniently accompanied by a step-by-step tutorial using Cloudflare Managed Rulesets.

While this transition represents an improvement in the Cloudflare for SaaS docs, we’re going to expand its accessibility even more. Some tutorials, such as our Managed Ruleset Tutorial, are already live within the tile. However, more step-by-step guides for Cloudflare for SaaS products and add-ons will further enable our customers to take full advantage of the available product suite. In particular, keep an eye out for expanding documentation around using Workers for Platforms.

Visit the new Cloudflare for SaaS tile to see the updates. If you are a SaaS provider interested in extending Cloudflare benefits to your customers through Cloudflare for SaaS, visit our Cloudflare for SaaS overview and our Plans page.

Docs-as-code is an approach to writing and publishing documentation with the same tools and processes developers use to create code. This philosophy has become more popular in recent years, especially in tech companies. Automatic link checking is part of this process, which ensures that writer's changes are sound and safe to deploy. By setting the stage with a docs-as-code approach, technical writers can focus on what they do best: ensure that our readers get useful and accurate information that is easy to find, and our documentation speaks a single language.

Besides following a docs-as-code approach, at Cloudflare we handle our documentation changes in public, in our cloudflare-docs GitHub repository. Having our documentation open to external contributions has helped us improve our documentation over time — our community is great at finding issues! While we need to review these contributions and ensure that they fit our style guide and content strategy, the contributions provided by the Cloudflare community have been instrumental in making our documentation better every day. While Cloudflare helps build a better Internet, our community helps build better documentation.

At Cloudflare, we follow a docs-as-code approach to create and publish product documentation in Developer Docs.

Such an approach involves different components. We use Git with a public GitHub repository to keep track of changes and to handle branching and merging. We rely on GitHub Actions and Cloudflare Pages for continuous integration and delivery (CI/CD). Our continuous integration pipeline checks if the documentation builds successfully and if there are any broken links. Users, both internal and external, can preview the changes in each pull request, which means that collaborators don't have to set up local environments to preview the changes they're proposing. However, if they wish to do so, users can set up a local environment using open-source tools and create local builds of the documentation.

Following this docs-as-code strategy has a few advantages. We use well-known workflows that technical folks use on a daily basis. This makes it easier to get feedback from engineers, while still being simple enough to get contributions from less tech-inclined people in the company and from the general Cloudflare community. After reviewing and approving a contribution to the documentation, we can deploy those changes to the live documentation site with the click of a button. Thanks to recent changes in our documentation engine, both deployments and local builds are fast, which helps increase our team's velocity. Additionally, it's safe to restructure parts of the documentation without being afraid of breaking stuff — our CI/CD pipeline ensures we don't have broken links.

It's possible to use a docs-as-code approach internally, using private repositories. However, we would not be extending some benefits of this approach to contributions provided by the larger Cloudflare community.

As a technical writer at Cloudflare, the great part about documentation-related work is that it’s done in our public GitHub repository. I can push changes to the cloudflare-docs GitHub repository with new or updated documentation that is then reviewed by my Cloudflare stakeholders, such as product managers and engineers.

Besides this work done together with other Cloudflare employees, I can also address issues and review pull requests from external stakeholders, since our documentation is open source. For example, pull request #4601 from an external contributor fixed an expression that was incorrect in the documentation:

Thanks to these contributions, we have been able to identify many issues, not just in the documentation but also in our products!

We handle these contributions as part of maintaining our own documentation product, with its own bugs and feature requests. Issues and pull requests get assigned, prioritized, handled, and reviewed just like internal tickets – even though they're not all in the same backlog.

Our work is similar both for internal and external contributions. We review the proposed changes, making sure that the new or updated content follows our style guide and our content strategy. When required, we ask for technical validation of the content. Finally, we merge the changes when they're approved.

For issues that are not related to the documentation, we have a feedback mechanism in every documentation page that allows readers to provide feedback about the product itself. This information is then reviewed internally and addressed by the correct team.

For more information on how you can contribute to the documentation by creating an issue or a pull request, refer to the Contributing to Cloudflare's Documentation page in our public GitHub repository.

For content that we cannot disclose just yet, we currently have several approaches, depending on the exact technical writer and on the involved stakeholders.

In some situations, we work on a shared document where we receive and address direct feedback. Close to the feature release date, we create the corresponding Markdown version that we push to the public GitHub repository at the right time.

In other cases, we work in a private Git repository, getting early feedback using the same processes we have in place for our public repository. This method of handling non-public content is easy to implement because we're already following a docs-as-code approach. In this case, pushing the content to GitHub is a straightforward operation when the time comes — it's just a matter of pushing the branch to a different remote and creating a public pull request.

Much of our work is already done in public, but we can still improve. While we do provide issue creation templates and pull request guidelines, we'll eventually make our style guide and content strategy public. This will allow users to know in advance what we will check for (and enforce) for every contribution to the public documentation, making our review process more transparent.

We recently updated developers.cloudflare.com, the Cloudflare Developers documentation website, to a new version of our custom documentation engine. This change consisted of a significant migration from Gatsby to Hugo and converged a collection of Workers Sites into a single Cloudflare Pages instance. Together, these updates brought developer experience, performance, and quality of life improvements for our engineers, technical writers, and product managers.

In this blog post, we’ll cover the history of Cloudflare’s developer docs, why we made this recent transition, and why we continue to dogfood Cloudflare’s products as we develop applications internally.

Cloudflare’s Developer Docs, which are open source on GitHub, comprise documentation for all of Cloudflare’s products. The documentation is written by technical writers, product managers, and engineers at Cloudflare. Like many open source projects, contributions to the docs happen via Pull Requests (PRs). At time of writing, we have 1,600 documentation pages and have accepted almost 4,000 PRs, both from Cloudflare employees and external contributors in our community.

The underlying documentation engine we’ve used to build these docs has changed multiple times over the years. Documentation sites are often built with static site generators and, at Cloudflare, we’ve used tools like Hugo and Gatsby to convert thousands of Markdown pages into HTML, CSS, and JavaScript.

When we released the first version of our Docs Engine in mid-2020, we were excited about the facelift to our Developer Documentation site and the inclusion of dev-friendly features like dark mode and proper code syntax highlighting.

Most importantly, we also used this engine to transition all of Cloudflare’s products with documentation onto a single engine. This allowed all Cloudflare product documentation to be developed, built, and deployed using the same core foundation. But over the next eighteen months and thousands of PRs, we realized that many of the architecture decisions we had made were not scaling.

While the user interface that we had made for navigating the documentation continued to receive great feedback from our users and product teams, decisions like using client-side rendering for docs had performance implications, especially on resource-constrained devices.

At the time, our decision to dogfood Workers Sites — which served as a precursor to Cloudflare Pages — meant that we could rapidly deploy our documentation across all of Cloudflare’s network in a matter of minutes. We implemented this by creating a separate Cloudflare Workers deployment for each product’s staging and production instances. Effectively, this meant that more than a hundred Workers were regularly updated, which caused significant headaches when trying to understand the causes and downstream effects of any failed deployments.

Finally, we struggled with our choice of underlying static site generator, Gatsby. We still think Gatsby is a great tool of choice for certain websites and applications, but we quickly found it to be the wrong match for our content-heavy documentation experience. Gatsby inherits many dependency chains to provide its featureset, but running the dependency-heavy toolchain locally on contributors’ machines proved to be an incredibly difficult and slow task for many of our documentation contributors.

When we did get to the point of deploying new docs changes, we began to be at the mercy of Gatsby’s long build times – in the worst case, almost an entire hour – just to compile Markdown and images into HTML. This negatively impacted our team’s ability to work quickly and efficiently as they improved our documentation. Ultimately, we were unable to find solutions to many of these issues, as they were core assumptions and characteristics of the underlying tools that we had chosen to build on — it was time for something new.

Built using Go, Hugo is incredibly fast at building large sites, has an active community, and is easily installable on a variety of operating systems. In our early discovery work, we found that Hugo would build our docs content in mere seconds. Since performance was a core motive for pursuing a rewrite, this was a significant factor in our decision.

When comparing frameworks, the most significant difference between Hugo and Gatsby – from a user’s standpoint – is the allowable contents of the Markdown files themselves. For example, Gatsby makes heavy use of MDX, allowing developers to author and import React components within their content pages. While this can be effective, MDX unfortunately is not CommonMark-compliant and, in turn, this means that its flavor of Markdown is required to be very flexible and permissive. This posed a problem when migrating to any other non-MDX-based solution, including Hugo, as these frameworks don’t grant the same flexibilities with Markdown syntax. Because of this, the largest technical challenge was converting the existing 1,600 markdown pages from MDX to a stricter, more standard Markdown variant that Hugo (or almost any framework) can interpret.

Not only did we have to convert 1,600 Markdown pages so that they’re rendered correctly by the new framework, we had to make these changes in a way that minimized the number of merge conflicts for when the migration itself was ready for deployment. There was a lot of work to be done as part of this migration – and work takes time! We could not stall or block the update cycles of the Developer Documentation repository, so we had to find a way to rename or modify every single file in the repository without gridlocking deployments for weeks.

The only way to solve this was through automation. We created a migration script that would apply all the necessary changes on the morning of the migration release day. Of course, this meant that we had to identify and apply the changes manually and then record that in JavaScript or Bash commands to make sweeping changes for the entire project.

For example, when migrating Markdown content, the migrator needs to take the file contents and parse them into an abstract syntax tree (AST) so that other functions can access, traverse, and modify a collection of standardized objects representing the content instead of resorting to a sequence string manipulations… which is scary and error-prone.

Since the project started with MDX, we needed a MDX-compatible parser which, in turn, produces its own AST with its own object standardizations. From there, one can “walk” – aka traverse – through the AST and add, remove, and/or edit objects and object properties. With the updated AST and a final traversal, a “stringifier” function can convert each object representation back to its string representation, producing updated file contents that differ from the original.

Below is an example snippet that utilizes mdast-util-from-markdown and mdast-util-to-markdown to create and stringify, respectively, the MDX AST and astray to traverse the AST with our custom modifications. For this example, we’re looking for heading and anchor nodes – both names are provided by the mdast-* utilities – so that we can read the primary header () text and ensure that all internal Developer Documentation links are consistent:

import * as fs from 'fs';
import * as astray from 'astray';
import { toMarkdown } from 'mdast-util-to-markdown';
import { fromMarkdown } from 'mdast-util-from-markdown';

/**
 * @param {string} file The "*.md" file path.
 */
export async function modify(file) {
  let content = await fs.promises.read(file, 'utf8');
  let AST = fromMarkdown(content);
  let title = '';

  astray.walk(AST, {
    /**
     * Read the page's <h1> to determine page's title.
     */
    heading(node) {
      // ignore if not <h1> header
      if (node.depth !== 1) return;

      astray.walk(node, {
        text(t: MDAST.Text) {
          // Grab the text value of the H1
          title += t.value;
        },
      });

      return astray.SKIP;
    },
    
    /**
     * Update all anchor links (<a>) for consistent internal linking.
     */
    link(node) {
      let value = node.url;
      
      // Ignore section header links (same page)
      if (value.startsWith('#')) return;

      if (/^(https?:)?\/\//.test(value)) {
        let tmp = new URL(value);
        // Rewrite our own "https://developers.cloudflare.com" links
        // so that they are absolute, path-based links instead.
        if (tmp.origin === 'https://developers.cloudflare.com') {
          value = tmp.pathname + tmp.search + tmp.hash;
        }
      }
      
      // ... other normalization logic ...
      
      // Update the link's `href` value
      node.url = value;
    }
  });
  
  // Now the AST has been modified in place.
  // AKA, the same `AST` variable is (or may be) different than before.
  
  // Convert the AST back to a final string.
  let updated = toMarkdown(AST);
  
  // Write the updated markdown file
  await fs.promises.writeFile(file, updated);
}

https://gist.github.com/lukeed/d63a4561ce9859765d8f0e518b941642#file-cfblog-devdocs-0-js

The above is an abbreviated snippet of the modifications we needed to make during our migration. You may find all the AST traversals and manipulations we created as part of our migration on GitHub.

We also took this opportunity to analyze the thousands and thousands of code snippets we have throughout the codebase. These serve an important role as they are crucial aides in reference documentation or are presented alongside tutorials as recipes or examples. So we added a code formatter script that utilizes Prettier to apply a consistent code style across all code snippets. As a bonus side effect, Prettier would throw errors if any snippets had invalid syntax for their given language. Any of these were fixed manually and the `format` script has been added as part of our own CI process to ensure that all JavaScript, TypeScript, Rust, JSON, and/or C++ code we publish is syntactically valid!

Finally, we created a Makefile that coordinated the series of Node scripts and git commands we needed to make. This orchestrated the entire migration, boiling down all our work into a single make run command.

In effect, the majority of the migration Pull Request was the result of automated commits – over one million changes were applied across nearly 5,000 files in less than two minutes. With the help of product owners, we reviewed the newly generated documentation site and applied any fine-tuning adjustments where necessary.

Previously, with the Gatsby-based Workers Sites architecture, each Cloudflare product needed to be built and deployed as its own individual documentation site. These sites would then be managed and proxied by an umbrella Worker, listening on developers.cloudflare.com, which ensured that all requests were handled by the appropriate product-specific Worker Site. This worked well for our production needs, but made it complicated for contributors to replicate a similar setup during local development. With the move to Hugo, we were able to merge everything into a single project – in other words, 48 moving pieces became 1 single piece! This made it extremely easy to build and develop the entire Developer Docs locally, which is a big confidence booster when working.

A unified Hugo project also means that there’s only one build command and one deployable unit… This allowed us to move the Developer Docs to Cloudflare Pages! With Pages attached and configured for the GitHub repository, we immediately began making use of preview deployments as part of our PR review process and our production branch commits automatically queued new production deployments to the live site.

After all the changes were in place, we ended up with a near-identical replica of the Developer Documentation site. However, upon closer inspection, a number of major improvements had been made:

1. Our application now has fewer moving pieces for development and deployment, which makes it significantly easier to understand and onboard other contributors and team members.

2. Our development flow is a lot snappier and fully replicated the production behavior. This hugely increased our iteration speed and confidence.

3. Our application was now built as an HTML-first static site. Even though it was always a content site, we are now shipping 90% less JavaScript bytes, which means that our visitors’ browsers are doing less work to view the same content.

The last point speaks to our web pages’ performance, which has real-world implications. These days, websites with faster page-load times are preferred over competitor sites with slower response times. This is true for human and bot users alike! In fact, this is so true that Google now takes page speed into consideration when ranking search results and offers tools like WebMasters and Lighthouse to help site owners track and improve their scores. Below, you can see the before-after comparison of our previous JS-first site to our HTML-first replacement:

Here you can see that our Performance grade has significantly improved! It’s this figure, which is a weighted score of the Metrics like First Contentful Paint, that is tied to Page Speed. While this does have SEO impact, the SEO score in a Lighthouse report has to do with Google Crawler’s ability to parse and understand the page’s metadata. This remains unchanged because the content (and its metadata) were not changed as part of the migration.

Developer documentation is incredibly important to the success of any product. At Cloudflare, we believe that technical docs are a product – one that we can continue to iterate on, improve, and make more useful for our customers.

One of the most effective ways to improve documentation is to make it easier for our writers to contribute to them. With our new Documentation Engine, we’re giving our product content team the ability to validate content faster with instantaneous local builds. Preview links via Cloudflare Pages allows stakeholders like product managers and engineering teams the ability to quickly review what the docs will actually look like in production.

As we invest more into our build and deployment pipeline, we expect to further develop our ability to validate both the content and technical implementation of docs as part of review – tools like automatic spell checking, link validation, and visual diffs are all things we’d like to explore in the future.

Importantly, our documentation continues to be 100% open source. If you read Cloudflare’s developer documentation, and have feedback, feel free to check out the project on GitHub and submit suggestions!

At Cloudflare, we talk a lot about how to help build a better Internet. On the Product Content Experience (PCX) team, we treat content like a product that represents and fulfills this mission. Our vision is to create world-class content that anticipates user needs and helps build accessible Cloudflare products. We believe we can impact the Cloudflare product experience and make it as wonderful as possible by intentionally designing, packaging, and testing the content.

I like taking on projects. A singular goal is met, and I clearly know I’m successful because the meaning of “done” is normally very clear. For example, I volunteer some of my time editing academic papers about technology. My role as an editor is temporary and there is a defined beginning and end to the work. I send my feedback and my task is largely complete.

“Content like a product” is when you shift your mindset from completing projects to maintaining a product, taking into consideration the user and their feedback. Product content at Cloudflare is an iterative, living, breathing thing. Inspired by the success of teams that adopt an agile mindset, along with some strategic functions you might find in a product management organization, treating content like a product means we treat content much like how a software project is created and maintained. This strategy allows for content development behaviors that closely align with the release of actual products, while also allowing technical writers and content designers to be laser-focused on doing what’s best for the user.

When the content team was new, we initially adopted many traditional agile methodologies. Why agile? Before I joined Cloudflare I was a product owner and was a huge advocate for sprint planning, retrospectives, and daily stand-ups. I liked Agile — I could easily keep up with a technical team, focus on priorities, and get things done quickly and efficiently. However, the rigidity of agile was just a bit too much for a content team. Over time, we modified and chose our favorite parts of the methodology while letting the rest go.

Shifting to a product development process created a lot of flexibility, but we didn’t want to abandon all process. Situationally, we take a process-focused mindset. For writing tasks that need to be predictable and consistent, like choosing inclusive terminology throughout our documentation, we have automated and manual processes to ensure we’re following our best practices.

Aligning content to the product development process means that when a new product is shipping, we have developer documentation ready to publish. Whenever the UI of the product changes, screenshots in the docs are updated accordingly. When new features are launched, we provide how-to guides and configuration content. Better alignment with the product team not only means the content team maintains accuracy of staying on top of all changes, it allows us to be user-focused. Above all, writers are aligned to the most important priority — shipping fast and often.

As you know, Cloudflare ships fast. You can see just a small sample of what I mean by fast here, here, and here. That speed was driven home within my first few weeks. I started just before Birthday Week 2020, and was super excited because I just wanted to jump in and create a lot of great content. But wow. What an intense start time. After Birthday Week, my main concerns were how to balance quality while meeting demand. I also wanted to create a quality environment for a team.

In retrospect, Birthday Week was a great time to start because it highlighted that keeping pace with products was going to be a big priority. Here’s how the content team met the demand.

First, the writers and I established that our focus was creating the most important content for the user, which allowed us to establish a product development mindset. We were now aligned with the product team.

Second, we moved content to an open source platform. This helped writers ship content fast because our authoring tools were consolidated to fewer platforms, and we were now in the same environment as our users.

We actually started publishing content as fast as products shipped within a few months! The content team began chipping away at the backlog once we understood the product team’s release cadence, and within less than six months we were ahead of the backlog and focusing on bigger initiatives including how to make content accessible, more consistent, and approachable to a wider group of users. It happened fast and was thrilling as a content creator.

The open source authoring tools on developers.cloudflare.com have evolved since 2020, continuing to help writers and contributors publish content faster by improving the review and build processes. We moved the docs platform to Cloudflare Pages earlier this year, allowing the writers to help build a more robust open source docs community while also providing valuable feedback to the Cloudflare Pages team.

Adopting a “content as a product” strategy requires buy-in from product managers and engineers, but it scales really well once established because everyone is focused on supporting the user versus the specifics of a content strategy itself. We go through the same planning, research, and analytics tasks you might find for a product to identify if we are creating the right content for folks who read the docs or use Cloudflare products. While everything we do with content is done so that we can create better content for our users, we also intentionally communicate that the content strategy is just a tool that enables a great user experience.

Over the next few weeks expect to see more about how Cloudflare writers have embraced the “content as a product” methodology as part of their own specific roles. In addition to learning more about how the developers.cloudflare.com site was moved to Cloudflare Pages, writers will share how they leaned into content creation for an open source community, their journey from technical writer to UX writing and content design, and share more specifics about our content strategy including the customer journey and success metrics.

I’ll admit. I’m fortunate to work for a company with overwhelming support regarding content. Great documentation is important to so many folks, and we’ve created the type of writing environment I always wanted to be a part of. It’s an exciting time to be helping build a better Internet through excellent product content.