How to Scrape Kasada-Protected Websites With Python

By Rajesh Kannan, Co-Founder & CTO, Gramosoft · Last updated: November 3, 2026 · Lab results from our own local test on November 3, 2026 (Python 3.13, requests 2.34, httpx 0.28, Playwright 1.64)

Your Python scraper hits a ticketing or sneaker site and gets a 429 on the very first request, or a 200 with nothing in it. That's often Kasada. This guide shows how to recognise it from Python, why plain HTTP clients never get the page, how to stop counting empty pages as wins, and which routes hold up, including GcrawlAI.

Three routes to the same data

Protected pages, one request each: GcrawlAI

Kasada only serves a page to a client that runs its JavaScript in a convincing browser, from a reasonable IP, at a human pace. Assembling that yourself means browsers, proxies and retries to run and maintain. GcrawlAI does it inside one API request and hands back the rendered page, so your Python code only has to check the content.

The homepage says it plainly: JavaScript rendering, anti-bot stealth and proxies are included in every request, with no credit system. That matters on Kasada sites because the expensive part, a real browser through a residential IP, isn't a premium option priced at a multiple of a basic request.

Here's the whole client. It asks GcrawlAI for the page, then accepts it only if the listing really holds items, because on Kasada sites a 200 alone proves nothing:

# pip install gcrawl-sdk parsel
"""Fetch a protected page you're allowed to collect through GcrawlAI, then accept it only
if the content is really there. A 200 with an empty grid counts as a failure."""
import os
import time

from gcrawl_sdk import GcrawlClient, ScrapeOptions
from gcrawl_sdk.exceptions import GcrawlAPIError, GcrawlTimeoutError
from parsel import Selector

gc = GcrawlClient(api_key=os.getenv("GCRAWL_API_KEY", "gc-YOUR_API_KEY"))
OPTS = ScrapeOptions(html_enabled=True, html_clean=False, markdown_enabled=True, proxy_geo="US", timeout=60)


def has_content(html: str, item_css: str, min_items: int) -> bool:
    return len(Selector(html or "").css(item_css)) >= min_items


def fetch(url: str, item_css: str, min_items: int = 5, tries: int = 3):
    for attempt in range(1, tries + 1):
        try:
            page = gc.scrape(url=url, options=OPTS, wait=True)
            if has_content(page.html, item_css, min_items):
                return page
            print(f"attempt {attempt}: page came back without {min_items}+ items")
        except (GcrawlAPIError, GcrawlTimeoutError) as e:
            print(f"attempt {attempt}: {e}")
        time.sleep(2 ** attempt)                              # back off between tries
    return None


if __name__ == "__main__":
    page = fetch("https://example.com/category/shoes", item_css="li.product")
    print(page.markdown[:500] if page else "no usable page")

• Rendering and access: a real browser through residential IPs in the country you choose, with retries and stealth escalation inside the request.
• Output you can check: HTML, Markdown or JSON, plus extractors that return structured fields for common page types.
• Scrape, crawl, screenshots and Search API: scrape single pages, crawl whole sites, capture screenshots, or use the Search API.
• For developers and AI agents: a Python SDK, an MCP server for Claude, Cursor and other agents, the Playground to test a page before you write code, and the docs.

The MIT-licensed core also runs on your own servers:

git clone https://github.com/GramosoftAI/GcrawlAI.git && cd GcrawlAI
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt && playwright install
cp .env.example .env                      # add your database, Redis and proxy settings
python -m api.core.db_setup
redis-server &
celery -A web_crawler.crawler.celery_config worker -l info &
uvicorn api.api:app --port 8000

Pricing is a free plan of 50 requests a month with no card, paid plans from $0.67 to $1.45 per 1,000 requests, and a self-hosted core that costs nothing. Failed requests are never charged. The full plan table is in the cost section.

Where it falls short: Kasada is one of the harder protections, and no service passes every Kasada site every time, GcrawlAI included; we haven't been part of a public Kasada benchmark yet, so test your own targets on the free plan. It won't log in for you, and it doesn't make collecting a site's data acceptable when that site's terms say no.

"On Kasada sites the dangerous failure isn't the 429, because you notice that. It's the run that reports 100% success and stored empty pages. We judge every response on its content before we count it."

Rajesh Kannan, Co-Founder & CTO at Gramosoft

What Kasada is, and why Python sees nothing

Kasada is a bot-management service that sits in front of a website and decides whether each client is a real browser before the site answers. Founded in 2015, it's common on ticketing, sneaker, gaming and travel sites. Unlike a CAPTCHA, it shows no puzzle: a client that fails just gets an error or an empty page.

The core idea is making the client prove it can run code. On the first visit, Kasada returns a small page whose script runs heavy, obfuscated JavaScript in the browser, including proof-of-work, and sends the result back. Only then does the server issue a session token, sent on later requests in headers such as x-kpsdk-ct, and serve the real page.

That's why a requests script fails at step one: it downloads the challenge page and stops, because it can't run JavaScript. According to DataImpulse's write-up, Kasada's own site describes a deployment where 90% of requests were bad bots and 98% of those never executed its JavaScript. ScrapingBee notes Kasada was named a leader in the Forrester Wave for bot and agent trust management, Q2 2026.

How to tell a site uses Kasada

Before changing anything, confirm what's blocking you, because Cloudflare, Akamai and DataDome also return 403 and 429. Kasada leaves a recognisable set of fingerprints in the response: its own headers, a script on a distinctive path, and a near-empty HTML page. Two or more of them together are a reliable sign.

The header names come from ScrapingBee and ZenRows; the UUID-and-ips.js path and window.KPSDK from Decodo's analysis. This detector checks for all five and needs only httpx:

# pip install httpx
"""Tell whether a response came from a Kasada-style challenge instead of the real page."""
import re
import sys

import httpx

SCRIPT_PATH = re.compile(r"/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/"
                         r"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/(ips|p)\.js")


def kasada_signals(status: int, headers: httpx.Headers, html: str) -> list[str]:
    found = []
    kp = sorted(h for h in headers if h.lower().startswith("x-kpsdk-"))
    if kp:
        found.append("headers: " + ", ".join(kp))
    if status in (403, 429):
        found.append(f"status: {status}")
    if SCRIPT_PATH.search(html):
        found.append("script: /<uuid>/<uuid>/" + SCRIPT_PATH.search(html).group(1) + ".js")
    if "KPSDK" in html:
        found.append("global: window.KPSDK")
    if len(html) < 2_000 and "<title></title>" in html.replace(" ", ""):
        found.append(f"body: {len(html)} bytes, empty title")
    return found


if __name__ == "__main__":
    url = sys.argv[1] if len(sys.argv) > 1 else "http://127.0.0.1:8772/products"
    r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 "
                                              "(KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36"})
    signals = kasada_signals(r.status_code, r.headers, r.text)
    print(f"{r.status_code} {url}")
    for s in signals:
        print("  -", s)
    print("verdict:", "Kasada-style challenge" if len(signals) >= 2 else "no challenge signals")

Against our lab edge, which copies those public signals, it found all five:

429 http://127.0.0.1:8772/products
  - headers: x-kpsdk-ct, x-kpsdk-r, x-kpsdk-v
  - status: 429
  - script: /<uuid>/<uuid>/ips.js
  - global: window.KPSDK
  - body: 281 bytes, empty title
verdict: Kasada-style challenge

Log the verdict with every failed request. Knowing which vendor stopped you tells you whether to look at your client, your pace or the route you chose, instead of guessing.

The layers Kasada checks

Kasada doesn't publish its scoring, but public write-ups agree on the broad shape: several layers, each able to stop a request on its own. Knowing which layer stopped you explains the symptom. A block before any script loads is a different problem from a page that loads, then empties.

• TLS and HTTP/2: the handshake requests sends looks nothing like Chrome's, so a request can be judged before any page exists.
• IP reputation: datacenter and shared proxy ranges score worse, though DataImpulse's own test got the same 200 from datacenter and residential IPs on one page.
• The JavaScript challenge: no JavaScript engine, no token, so plain HTTP clients stop here.
• Browser environment: signs that a browser is automated or inconsistent. A stock headless browser often fails here even though it ran the script.
• Behaviour and session: pace and repetition over time, which is where silent blocks come from.

We don't cover patching browsers to hide automation or reusing Kasada tokens outside the browser that earned them. Those techniques exist to defeat a control the site chose to deploy, they break with each Kasada update, and they're the riskiest part of this work legally.

What Python can and can't do here

Python has three realistic options against a Kasada page: an HTTP client, a browser you drive with Playwright, or an API that does the browsing for you. We ran the first two against a local toy edge that copies Kasada's public behaviour, to show what each one actually receives before any fingerprinting comes into play.

The toy edge is not Kasada and models none of its fingerprinting. It only issues a 429 with x-kpsdk-* headers, serves an ips.js proof-of-work script, and quietly empties pages for sessions that browse too fast. The full lab is in the code download; this is the test:

# pip install requests httpx parsel playwright && playwright install chromium
"""Run three clients against the local lab edge and grade every response on content."""
import time

import httpx
import requests
from playwright.sync_api import sync_playwright

from content_check import check_page
from detect_kasada import kasada_signals

URL = "http://127.0.0.1:8772/products"
UA = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36"

print("1) HTTP clients, no JavaScript engine")
for name, get in (("requests", lambda: requests.get(URL, headers={"User-Agent": UA})),
                  ("httpx", lambda: httpx.get(URL, headers={"User-Agent": UA}))):
    r = get()
    print(f"  {name:<9} {r.status_code}  signals={len(kasada_signals(r.status_code, r.headers, r.text))}  -> {check_page(r.status_code, r.text)}")

print("2) A real browser that runs the page's JavaScript")
with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    first = page.goto(URL)
    page.wait_for_selector("li.product", timeout=15_000)        # the script solves, then reloads
    print(f"  chromium  first response {first.status}, after reload -> {check_page(200, page.content())}")

    print("3) Same session, 30 page views: status code vs content")
    for label, gap in (("no delay", 0.0), ("0.6 s apart", 0.6)):
        ctx = browser.new_context(); pg = ctx.new_page()
        pg.goto(URL); pg.wait_for_selector("li.product", timeout=15_000)
        codes, verdicts = [], []
        for _ in range(30):
            resp = pg.goto(URL)
            codes.append(resp.status); verdicts.append(check_page(resp.status, pg.content()))
            time.sleep(gap)
        ok200 = codes.count(200); real = sum(v.startswith("ok") for v in verdicts)
        print(f"  {label:<12} 200 OK: {ok200}/30   real content: {real}/30   silent blocks: {30 - real}")
        ctx.close()
    browser.close()

1) HTTP clients, no JavaScript engine
  requests  429  signals=5  -> challenge
  httpx     429  signals=5  -> challenge
2) A real browser that runs the page's JavaScript
  chromium  first response 429, after reload -> ok (24 items)
3) Same session, 30 page views: status code vs content
  no delay     200 OK: 30/30   real content: 19/30   silent blocks: 11
  0.6 s apart  200 OK: 30/30   real content: 30/30   silent blocks: 0

Both HTTP clients got the challenge page and nothing else, which is exactly what happens on a real Kasada site. The browser ran the script and got all 24 products. On a real Kasada site that last step is far less certain, because the browser-environment layer often flags stock headless automation, which our toy doesn't check.

• requests and httpx: fine for detection and for reading the response, never enough to get the page.
• Your own Playwright browser: runs the challenge, but you own the detection risk, the proxies and the maintenance.
• A managed browser API: one HTTP call from Python, with the browser, IPs and retries on the provider's side.

Catch silent blocks with content checks

The most expensive Kasada failure is the one you don't see. Instead of an error, a flagged session can get a 200 OK with an empty or misleading page. A scraper that only checks status codes keeps running, reports success, and fills your database with nothing. The fix is simple: judge every page on its content.

"""Judge a page by what it holds, not by its status code: catches silent 200 blocks."""
from parsel import Selector

MIN_ITEMS = 5              # what a real listing page on this site always has


def check_page(status: int, html: str) -> str:
    sel = Selector(html)
    items = sel.css("li.product")
    priced = [i for i in items if i.css(".price::text").get()]
    if status in (403, 429):
        return "challenge"
    if status != 200:
        return f"error {status}"
    if not sel.css("h1::text").get():
        return "not the page (no heading)"
    if len(priced) < MIN_ITEMS:
        return f"silent block ({len(priced)} items)"
    return f"ok ({len(priced)} items)"

In the lab, the fast session got 30 responses with status 200, and this check flagged 11 of them as empty: the edge allowed 20 page views in 10 seconds, counting the one that solved the challenge. Scrapeway's Kasada benchmark makes the same point about real sites, warning that silent blocking leaves you paying for empty 200 responses unless you check content.

• Assert on something only a real page has: a minimum number of items, a price field, a heading you expect.
• Treat an empty 200 as a failure: retry it later, don't store it.
• Track the content rate, not the status rate: a drop in items per page is your early warning.

"Our dashboards don't count 200s, they count items. When the item rate drops and the status rate doesn't, that's a silent block, and it's the first number we look at on any protected site."

Rajesh Kannan, Co-Founder & CTO at Gramosoft

Pace sessions and back off

Kasada's last layer watches behaviour over time, so how fast you go matters as much as what you send. In our lab, slowing the same browser session to one page view every 0.6 seconds took silent blocks from 11 to zero. Real thresholds are unknown and vary by site, so start slow and measure.

• Start well below human browsing speed and raise the rate only while your content rate stays flat.
• Back off on 429 and on empty pages: wait longer after each failure instead of retrying at once.
• Cap concurrency per site and read robots.txt, including any crawl delay.
• Collect only what you need: fewer pages means fewer chances to be flagged and less load on the site.

Rotating IPs faster doesn't fix a pace problem. It spreads the same behaviour across more addresses, which is the pattern behaviour layers are built to notice.

How scraping APIs score on Kasada

If you go the managed route, results vary widely between providers on Kasada. The independent Scrapeway benchmark tested six scraping APIs against a live Kasada-protected target, Realtor.com, in October 2026, and success rates ranged from 100% down to 8%. Price per 1,000 requests didn't track success.

Source: Scrapeway, October 2026 Kasada benchmark. GcrawlAI wasn't in this run, so we make no claim against these numbers. One target is a small sample, and results change month to month: run your own pages through any provider, GcrawlAI included, and measure the content rate.

What scraping protected pages costs

On Kasada sites every page needs JavaScript rendering, and usually premium IPs, so compare providers on their protected-page price, not their headline rate. For 100,000 successful protected pages a month, credit-based APIs cost $249 to $399 at list price and GcrawlAI costs $101. For plain pages, they're cheaper.

List-price arithmetic, not a benchmark. Credit costs and billing rules come from ScraperAPI's credit docs, ZenRows' pricing docs and ScrapingBee's FAQ, checked November 1, 2026. The honest read: for plain HTML, ZenRows, ScraperAPI and ScrapingBee cost less than GcrawlAI. And on Kasada, a status-200 bill can include silent blocks, so the price that counts is the one per page with content.

GcrawlAI plans (GcrawlAI pricing, checked October 28, 2026):

Every plan, Free included, has scrape, crawl, the Search API, JavaScript rendering, residential IP rotation and stealth escalation in each request, and each feature counts as one request. Failed requests are never charged. Annual billing charges 10 months for 12, unused requests on paid plans roll over once, and Indian customers are billed in INR with a GST invoice.

Which route fits your project

Start with the route that carries the least risk and the least maintenance, and move on only when it can't give you the data. For Kasada sites that order is usually an official source, then a managed browser API, then your own browsers. The table shows where GcrawlAI fits in each case.

When your own Python is enough

Plenty of Kasada-related work needs no service at all: detecting which sites use it, monitoring whether your access still returns content, or collecting a few pages slowly from a site that allows it. The detector and content check above cost nothing and belong in every pipeline that touches protected pages, whatever fetches them.

When volume grows or the content rate drops, move the fetching to GcrawlAI and keep your checks. Its free Cloud plan's 50 requests a month are enough to test your real targets, the Playground shows the result without code, and the MIT core self-hosts for free. For the build-or-buy trade-off, read web scraping with Python vs a scraping API.

Is it OK to scrape a Kasada-protected site?

A site that pays for Kasada is telling you it doesn't want automated traffic it hasn't approved. That doesn't make every collection unlawful, but it raises the bar. Read the terms and robots.txt, ask for an API or data licence, stay on public pages, and keep your pace gentle.

Don't defeat the protection itself: no reverse-engineering Kasada's script, forging or replaying its tokens, or solving challenges outside the browser that received them. Don't scrape behind a login, don't buy tickets or limited stock with bots, and don't collect personal data you have no basis to hold. For the legal background, read what web scraping is and where it's legal, and get advice for your own case.

"When a customer brings us a Kasada site, the first question is whether there's an official feed. Often there is, and it's cheaper than any scraper. The pages we do fetch are public ones they're entitled to, at a pace the site won't notice."

Rajesh Kannan, Co-Founder & CTO at Gramosoft

Related guides

• How to Bypass Cloudflare When Web Scraping (2026)
• How to Fix 403 Forbidden Errors in Python Requests (2026)
• 429 Too Many Requests: How to Fix It When Web Scraping
• How to Scrape Dynamic JavaScript Websites with Python (2026 Guide)
• ZenRows Alternatives: Best Anti-Bot Scraping APIs in 2026
• Python Web Scraping Frameworks Compared: Scrapy vs Crawlee vs Scrapling (2026)
• Web Scraping with Python vs a Scraping API: When to Stop Writing Your Own Scraper
• What Is Web Scraping? How It Works, Uses & Legality