Today, we're announcing Cloudflare Sequence Analytics for APIs. Using Sequence Analytics, Customers subscribed to API Gateway can view the most important sequences of API requests to their endpoints. This new feature helps customers to apply protection to the most important endpoints first.
What is a sequence? It is simply a time-ordered list of HTTP API requests made by a specific visitor as they browse a website, use a mobile app, or interact with a B2B partner via API. For example, a portion of a sequence made during a bank funds transfer could look like:
\n
\n\n
\n
Order
\n
Method
\n
Path
\n
Description
\n
\n\n\n
\n
1
\n
GET
\n
/api/v1/users/{user_id}/accounts
\n
user_id is the active user
\n
\n
\n
2
\n
GET
\n
/api/v1/accounts/{account_id}/balance
\n
account_id is one of the user’s accounts
\n
\n
\n
3
\n
GET
\n
/api/v1/accounts/{account_id}/balance
\n
account_id is a different account belonging to the user
\n
\n
\n
4
\n
POST
\n
/api/v1/transferFunds
\n
Containing a request body detailing an account to transfer funds from, an account to transfer funds to, and an amount of money to transfer
\n
\n\n
Why is it important to pay attention to sequences for API security? If the above API received requests for POST /api/v1/transferFunds without any of the prior requests, it would seem suspicious. Think about it: how would the API client know what the relevant account IDs are without listing them for the user? How would the API client know how much money is available to transfer? While this example may be obvious, the sheer number of API requests to any given production API can make it hard for human analysts to spot suspicious usage.
In security, one approach to defending against an untold number of threats that are impossible to screen by a team of humans is to create a positive security model. Instead of trying to block everything that could potentially be a threat, you allow all known good or benign traffic and block everything else by default.
Customers could already create positive security models with API Gateway in two main areas: volumetric abuse protection and schema validation. Sequences will form the third pillar of a positive security model for API traffic. API Gateway will be able to enforce the precedence of endpoints in any given API sequence. By establishing precedence within an API sequence, API Gateway will log or block any traffic that doesn’t match expectations, reducing abusive traffic.
When attackers attempt to exfiltrate data in an abusive way, they rarely follow the patterns of expected API traffic. Attacks often use special software to ‘fuzz’ the API, sending several requests with different request parameters hoping to find unexpected responses from the API indicating opportunities to exfiltrate data. Attackers can also manually send requests to APIs that attempt to trick the API in performing unauthorized actions, like granting an attacker elevated privileges or access to data through a Broken Object Level Authentication attack. Protecting APIs with rate limits is a common best practice; however, in both of the above examples attackers may deliberately execute request sequences slowly, in an attempt to thwart volumetric abuse detection.
Think of the sequence of requests above again, but this time imagine an attacker copying the legitimate funds transfer request and modifying the request payload in an attempt to trick the system:
\n
\n\n
\n
Order
\n
Method
\n
Path
\n
Description
\n
\n\n\n
\n
1
\n
GET
\n
/api/v1/users/{user_id}/accounts
\n
user_id is the active user
\n
\n
\n
2
\n
GET
\n
/api/v1/accounts/{account_id}/balance
\n
account_id is one of the user’s accounts
\n
\n
\n
3
\n
GET
\n
/api/v1/accounts/{account_id}/balance
\n
account_id is a different account belonging to the user
\n
\n
\n
4
\n
POST
\n
/api/v1/transferFunds
\n
Containing a request body detailing an account to transfer funds from, an account to transfer funds to, and an amount of money to transfer
\n
\n
\n
… attacker copies the request to a debugging tool like Postman …
\n
\n
\n
5
\n
POST
\n
/api/v1/transferFunds
\n
Attacker has modified the POST body to try and trick the API
\n
\n
\n
6
\n
POST
\n
/api/v1/transferFunds
\n
A further modified POST body to try and trick the API
\n
\n
\n
7
\n
POST
\n
/api/v1/transferFunds
\n
Another, further modified POST body to try and trick the API
\n
\n\n
If the customer knew beforehand that the funds transfer endpoint was critical to protect and only occurred once during a sequence, they could write a rule to ensure that it was never called twice in a row and a GET /balance always preceded a POST /transferFunds. But without prior knowledge of which endpoint sequences are critical to protect, how would the customer know which rules to define? A low rate limit is too risky, since an API user might legitimately have a few funds transfer requests to perform in a short amount of time. In the present reality there are few tools to prevent this type of abuse, and most customers are left with reactive efforts to clean up abuse with their application teams and fraud departments after it’s happened.
Ultimately, we believe that providing our customers with the ability to define positive security models on API request sequences requires a three-pronged approach:
Sequence Analytics: Determining which sequences of API requests occurred and when, as well as summarizing the data into readily understandable form.
Sequence Abuse Detection: Identifying which sequences of API requests are likely of benign or malicious origin.
Sequence Mitigation: Identifying relevant rules on sequences of API requests for deciding which traffic to allow or block.
Sequence Analytics presents some difficult technical challenges, because sessions may be long-lived and may consist of many requests. As a result, it is not sufficient to define sequences by session identifier alone. Instead, it was necessary for us to develop a solution capable of automatically identifying multiple sequences which occur within a given session. Additionally, since important sequences are not necessarily characterized by volume alone and the set of possible sequences is large, it was necessary to develop a solution capable of identifying important sequences, as opposed to simply surfacing frequent sequences.
To help illustrate these challenges for the example of api.cloudflare.com, we can group API requests by session and plot the number of distinct sequences versus sequence length:
\n \n \n \n \n
The plot is based on a one hour snapshot comprising approximately 88,000 sessions and 260 million API requests, with 301 distinct API endpoints. We process the data by applying a fixed-length sliding window to each session, then we count the total number of different fixed-length sequences (‘n-grams’) that we observe as a result of applying the sliding window. The plot displays results for a window size (‘n-gram length’) varying between 1 and 10 requests. The number of distinct sequences ranges from 301 (for a window size of 1 request) to approximately 780,000 (for a window size of 10 requests). Based on the plot, we observe a large number of possible sequences which grows with sequence length: As we increase the sliding window size, we see an increasingly large amount of different sequences in the sample. The smooth trend can be explained by the fact that we apply a sliding window (sessions may themselves contain many sequences) in combination with many long sessions relative to the sequence length.
Given the large number of possible sequences, trying to find abusive sequences is a ‘needles in a haystack’ situation.
Here is a screenshot from the API Gateway dashboard highlighting Sequence Analytics:
\n \n \n \n \n
Let’s break down the new functionality seen in the screenshot.
API Gateway intelligently determines sequences of requests made by your API consumers using the methods described earlier in this article. API Gateway scores sequences by a metric we call Correlation Score. Sequence Analytics displays the top 20 sequences by highest correlation score, and we refer to these as your most important sequences. High-importance sequences contain API requests which are likely to occur together in order.
You should inspect each of your sequences to understand their correlation scores. High correlation score sequences may consist of rarely used endpoints (potentially anomalous user behavior) as well as commonly used endpoints (likely benign user behavior). Since the endpoints found in these sequences commonly occur together, they represent true usage patterns of your API. You should apply all possible API Gateway protections to these endpoints (rate limiting suggestions, Schema Validation, JWT Validation, and mTLS) and check their specific endpoint order with your development team.
We know customers want to explicitly set allowable behavior on their APIs beyond the active protections offered by API Gateway today. Coming soon, we’re releasing sequence precedence rules and enabling the ability to block requests based on those rules. The new sequence precedence rules will allow customers to specify the exact order of allowable API requests, bringing yet another way of establishing a positive security model to protect your API against unknown threats.
All API Gateway customers now have access to Sequence Analytics. Navigate to a zone in the Cloudflare dashboard, then click the Security tab > API Gateway tab > Sequences tab. You’ll see the most important sequences that your API consumers request.
Enterprise customers that haven’t purchased API Gateway can get started by enabling the API Gateway trial inside the Cloudflare Dashboard or contacting their account manager.
Sequence-based detection is a powerful and unique capability that unlocks many new opportunities to identify and stop attacks. As we fine-tune the methods of identifying these sequences and shipping them to our global network, we will release custom sequence matching and real-time mitigation features at a future date. We will also ensure you have the actionable intelligence to take back to your team on who the API users were that attempted to request sequences that don’t match your policy.
"],"published_at":[0,"2023-03-15T13:00:00.000+00:00"],"updated_at":[0,"2024-10-09T23:23:15.620Z"],"feature_image":[0,"https://cf-assets.www.cloudflare.com/zkvhlag99gkb/3CxCrd84oy1gSzNzVE0ghN/cc490e9bf999783c92868b0affd7855b/api-sequence-analytics.png"],"tags":[1,[[0,{"id":[0,"3DmitkNK6euuD5BlhuvOLW"],"name":[0,"Security Week"],"slug":[0,"security-week"]}],[0,{"id":[0,"6GvYnn2MD0ye88h10BplGd"],"name":[0,"API Shield"],"slug":[0,"api-shield"]}],[0,{"id":[0,"7aSCe0ZArYqX0ThFK2gRm0"],"name":[0,"API Gateway"],"slug":[0,"api-gateway"]}],[0,{"id":[0,"4lmuF3Pb7F4hTBUe3svrD0"],"name":[0,"API Security"],"slug":[0,"api-security"]}],[0,{"id":[0,"6Foe3R8of95cWVnQwe5Toi"],"name":[0,"AI"],"slug":[0,"ai"]}]]],"relatedTags":[0],"authors":[1,[[0,{"name":[0,"John Cosgrove"],"slug":[0,"john-cosgrove"],"bio":[0,null],"profile_image":[0,"https://cf-assets.www.cloudflare.com/zkvhlag99gkb/aQvqUtMeYIUH7r1CngAse/21a1670cea398792be0f5b3213645c09/john-cosgrove.jpg"],"location":[0,null],"website":[0,null],"twitter":[0,"@cameracoz"],"facebook":[0,null]}],[0,{"name":[0,"Peter Foster"],"slug":[0,"peter-foster"],"bio":[0,null],"profile_image":[0,"https://cf-assets.www.cloudflare.com/zkvhlag99gkb/WvbnoXMjb0d3R9kQEvYoo/1dec8ad195d76f4b49a4905d5d853fb1/peter-foster.jpeg"],"location":[0,null],"website":[0,null],"twitter":[0,null],"facebook":[0,null]}]]],"meta_description":[0,"Today, we're announcing Cloudflare Sequence Analytics for APIs. Using Sequence Analytics, Customers subscribed to API Gateway can view the most important sequences of API requests to their endpoints. This new feature helps customers to apply protection to the most important endpoints first."],"primary_author":[0,{}],"localeList":[0,{"name":[0,"Detecting API abuse automatically using sequence analysis Config"],"enUS":[0,"English for Locale"],"zhCN":[0,"Translated for Locale"],"zhHansCN":[0,"No Page for Locale"],"zhTW":[0,"Translated for Locale"],"frFR":[0,"Translated for Locale"],"deDE":[0,"Translated for Locale"],"itIT":[0,"No Page for Locale"],"jaJP":[0,"Translated for Locale"],"koKR":[0,"Translated for Locale"],"ptBR":[0,"No Page for Locale"],"esLA":[0,"No Page for Locale"],"esES":[0,"Translated for Locale"],"enAU":[0,"No Page for Locale"],"enCA":[0,"No Page for Locale"],"enIN":[0,"No Page for Locale"],"enGB":[0,"No Page for Locale"],"idID":[0,"No Page for Locale"],"ruRU":[0,"No Page for Locale"],"svSE":[0,"No Page for Locale"],"viVN":[0,"No Page for Locale"],"plPL":[0,"No Page for Locale"],"arAR":[0,"No Page for Locale"],"nlNL":[0,"No Page for Locale"],"thTH":[0,"No Page for Locale"],"trTR":[0,"No Page for Locale"],"heIL":[0,"No Page for Locale"],"lvLV":[0,"No Page for Locale"],"etEE":[0,"No Page for Locale"],"ltLT":[0,"No Page for Locale"]}],"url":[0,"https://blog.cloudflare.com/api-sequence-analytics"],"metadata":[0,{"title":[0,"Detecting API abuse automatically using sequence analysis"],"description":[0,"Today, we're announcing Cloudflare Sequence Analytics for APIs. Using Sequence Analytics, Customers subscribed to API Gateway can view the most important sequences of API requests to their endpoints. This new feature helps customers to apply protection to the most important endpoints first."],"imgPreview":[0,"https://cf-assets.www.cloudflare.com/zkvhlag99gkb/4sxU0DXJoCSLWsRCHRlvnc/c53e617b0a3ac9cf0d45e1d6d2d1ffc6/api-sequence-analytics-cCOTZh.png"]}]}],"translations":[0,{"posts.by":[0,"By"],"footer.gdpr":[0,"GDPR"],"lang_blurb1":[0,"This post is also available in {lang1}."],"lang_blurb2":[0,"This post is also available in {lang1} and {lang2}."],"lang_blurb3":[0,"This post is also available in {lang1}, {lang2} and {lang3}."],"footer.press":[0,"Press"],"header.title":[0,"The Cloudflare Blog"],"search.clear":[0,"Clear"],"search.filter":[0,"Filter"],"search.source":[0,"Source"],"footer.careers":[0,"Careers"],"footer.company":[0,"Company"],"footer.support":[0,"Support"],"footer.the_net":[0,"theNet"],"search.filters":[0,"Filters"],"footer.our_team":[0,"Our team"],"footer.webinars":[0,"Webinars"],"page.more_posts":[0,"More posts"],"posts.time_read":[0,"{time} min read"],"search.language":[0,"Language"],"footer.community":[0,"Community"],"footer.resources":[0,"Resources"],"footer.solutions":[0,"Solutions"],"footer.trademark":[0,"Trademark"],"header.subscribe":[0,"Subscribe"],"footer.compliance":[0,"Compliance"],"footer.free_plans":[0,"Free plans"],"footer.impact_ESG":[0,"Impact/ESG"],"posts.follow_on_X":[0,"Follow on X"],"footer.help_center":[0,"Help center"],"footer.network_map":[0,"Network Map"],"header.please_wait":[0,"Please Wait"],"page.related_posts":[0,"Related posts"],"search.result_stat":[0,"Results {search_range} of {search_total} for {search_keyword}"],"footer.case_studies":[0,"Case Studies"],"footer.connect_2024":[0,"Connect 2024"],"footer.terms_of_use":[0,"Terms of Use"],"footer.white_papers":[0,"White Papers"],"footer.cloudflare_tv":[0,"Cloudflare TV"],"footer.community_hub":[0,"Community Hub"],"footer.compare_plans":[0,"Compare plans"],"footer.contact_sales":[0,"Contact Sales"],"header.contact_sales":[0,"Contact Sales"],"header.email_address":[0,"Email Address"],"page.error.not_found":[0,"Page not found"],"footer.developer_docs":[0,"Developer docs"],"footer.privacy_policy":[0,"Privacy Policy"],"footer.request_a_demo":[0,"Request a demo"],"page.continue_reading":[0,"Continue reading"],"footer.analysts_report":[0,"Analyst reports"],"footer.for_enterprises":[0,"For enterprises"],"footer.getting_started":[0,"Getting Started"],"footer.learning_center":[0,"Learning Center"],"footer.project_galileo":[0,"Project Galileo"],"pagination.newer_posts":[0,"Newer Posts"],"pagination.older_posts":[0,"Older Posts"],"posts.social_buttons.x":[0,"Discuss on X"],"search.icon_aria_label":[0,"Search"],"search.source_location":[0,"Source/Location"],"footer.about_cloudflare":[0,"About Cloudflare"],"footer.athenian_project":[0,"Athenian Project"],"footer.become_a_partner":[0,"Become a partner"],"footer.cloudflare_radar":[0,"Cloudflare Radar"],"footer.network_services":[0,"Network services"],"footer.trust_and_safety":[0,"Trust & Safety"],"header.get_started_free":[0,"Get Started Free"],"page.search.placeholder":[0,"Search Cloudflare"],"footer.cloudflare_status":[0,"Cloudflare Status"],"footer.cookie_preference":[0,"Cookie Preferences"],"header.valid_email_error":[0,"Must be valid email."],"search.result_stat_empty":[0,"Results {search_range} of {search_total}"],"footer.connectivity_cloud":[0,"Connectivity cloud"],"footer.developer_services":[0,"Developer services"],"footer.investor_relations":[0,"Investor relations"],"page.not_found.error_code":[0,"Error Code: 404"],"search.autocomplete_title":[0,"Insert a query. Press enter to send"],"footer.logos_and_press_kit":[0,"Logos & press kit"],"footer.application_services":[0,"Application services"],"footer.get_a_recommendation":[0,"Get a recommendation"],"posts.social_buttons.reddit":[0,"Discuss on Reddit"],"footer.sse_and_sase_services":[0,"SSE and SASE services"],"page.not_found.outdated_link":[0,"You may have used an outdated link, or you may have typed the address incorrectly."],"footer.report_security_issues":[0,"Report Security Issues"],"page.error.error_message_page":[0,"Sorry, we can't find the page you are looking for."],"header.subscribe_notifications":[0,"Subscribe to receive notifications of new posts:"],"footer.cloudflare_for_campaigns":[0,"Cloudflare for Campaigns"],"header.subscription_confimation":[0,"Subscription confirmed. Thank you for subscribing!"],"posts.social_buttons.hackernews":[0,"Discuss on Hacker News"],"footer.diversity_equity_inclusion":[0,"Diversity, equity & inclusion"],"footer.critical_infrastructure_defense_project":[0,"Critical Infrastructure Defense Project"]}]}" ssr="" client="load" opts="{"name":"PostCard","value":true}" await-children="">
Today, we're announcing Cloudflare Sequence Analytics for APIs. Using Sequence Analytics, Customers subscribed to API Gateway can view the most important sequences of API requests to their endpoints...
We are making the machine learning empowered WAF and Security analytics view available to our Business plan customers, to help detect and stop attacks before they are known...
One year ago we published our first Application Security Report. For Security Week 2023, we are providing updated insights and trends around mitigated traffic, bot and API traffic, and account takeover attacks....
Today, we are excited to announce that Cloudflare One now offers Data Loss Prevention (DLP) detections for Microsoft Purview Information Protection labels. Simply integrate with your Microsoft account, retrieve your labels, and build rules to guide the movement of your labeled da...
Cloudflare Access and Ping Identity offer a powerful solution for organizations looking to implement Zero Trust security controls to protect their applications and data....
Cloudflare CASB can now integrate and scan Atlassian products, Confluence and Jira, for critical security issues, like misconfigurations, data exposure, and third-party app risks. Start scanning in just a few clicks!...
We’re expanding the phishing protections available to Cloudflare One customers by automatically identifying—and blocking—so-called “confusable” domains....
Mutual TLS is used to secure a range of network services and applications: APIs, web applications, microservices, databases and IoT devices. With mTLS support for Workers you can use Workers to authenticate to any service secured by mTLS directly!...
We are thrilled to introduce an innovative new approach to secure hosted applications via Cloudflare Access without the need for any installed software or custom code on your application server....
Phishing attacks come in all sorts of ways to fool people. Email is definitely the most common, but there are others. Following up on our "Top 50 Most Impersonated Brands in Phishing Attacks" post, here are some tips to help you catch these scams before you fall for them....
Starting today, using Page Shield, Cloudflare’s client side security solution, you can ensure only vetted and secure JavaScript is being executed by your user’s browsers. Stop unwanted JavaScript and keep your end user data safe with Page Shield policies....
Welcome to Security Week 2023. This week we’ll demonstrate how Cloudflare is making it as easy as possible to shift from protecting applications, to protecting employees, and making sure they are protected everywhere....
Autonomous Systems (ASes) are a group of routable IP prefixes belonging to a single entity or organization, and is one of the fundamental building blocks of the Internet...