Scrapling Audited: 92,848 Stars, 1.9 Issues per 1,000 Stars, and an Adaptive Matcher That Scores 47% on the Wrong Element

Scrapling is a Python scraping framework that has, as of this writing, 92,848 GitHub stars, 54 releases, and a README that opens with “The BEST Web Scraping framework”. Version 0.5 landed on 17 October 2026 and, in the project’s own words, “upgrades almost everything: fetching, parsing, spiders, browser automation, AI tools, and docs in ten languages”, accompanied by a warning that “this release introduces a lot of breaking changes that don’t fit here”.

Three claims in that release are worth measuring rather than repeating: that the parser is dramatically faster than its rivals, that adaptive element tracking survives site redesigns, and that the browser fetcher is stealthy. All three can be tested on a single 2-vCPU Linux box in under an hour, so I did.

The results are mixed in an interesting way. The library is real, is used at scale, and does several things well. Two of its headline numbers, however, do not survive reproduction — and the adaptive feature, the one thing in the README nobody else ships, behaves in a way that will silently corrupt a dataset before it ever throws an error.

What Scrapling actually is

A single package that replaces the usual stack of requests + BeautifulSoup/parsel + playwright, with four layers that are installed separately:

The bare pip install scrapling gives you only the parser: from scrapling.fetchers import Fetcher raises ModuleNotFoundError. That is documented and correct, and it matters for the benchmark section below, because the published tables are parser-only numbers being compared against whole scraping stacks.

Codebase shape, measured on a shallow clone of main:

• 61 Python files, 16,108 lines under scrapling/
• 99 test files, 25,529 lines under tests/ — 1.58 lines of test per line of source
• Python 99.7% of bytes, BSD-3-Clause, Development Status :: 4 - Beta
• 41 contributors, of whom D4Vinci (Karim Shoair) has 1,683 of 1,780 commits (94.6%); 37 of the 41 have 8 commits or fewer

A test-to-source ratio above 1 is not typical for a side project, and the project also ships AI_POLICY.md, which requires contributors to disclose AI assistance and reserves the right to label undisclosed AI output AI-Slop. Governance here is more mature than the issue tracker suggests.

The star count and the issue count do not move together

GitHub stars are a bookmark button, and this project is a good illustration of how far they can drift from every other engagement signal. All figures read from the GitHub API on 3 November 2026:

Scrapling has 1.4x scrapy’s stars and a twenty-seventh of its issue rate. Only four issues are open, and the forks/stars ratio is 10.5% (9,778 forks), which is high even for a library people vendor into their own code.

The obvious reading — that the stars are fake — is not what the rest of the data says. PyPI download statistics tell a different story:

So usage is roughly a fifth of Scrapy’s while stars are 1.4x Scrapy’s. That is a per-download star rate about 6.7x higher than Scrapy’s, and it is consistent with a project whose audience arrives through social channels rather than through engineering evaluation. The repo pushes a Discord invite and an X account in its own description; there is no GitHub Discussions tab.

The clearest single signal is Hacker News: the highest-scoring submission of Scrapling in the Algolia index is from 13 October 2024 with 4 points and 1 comment. A 92k-star developer tool that has never had a front-page discussion is not being discovered by the audience that files issues. None of that is misconduct. It does mean that “92k stars” tells you almost nothing about how much scrutiny the code has had — which is exactly why the next two sections exist.

Claim 1: the benchmark table, re-run

The README publishes this table under “Scrapling isn’t just powerful - it’s also blazing fast”:

The methodology link points to benchmarks.py in the repo, so the whole thing is reproducible. It is not:

$ python benchmarks.py
File ".../selectolax/parser.py", line 13, in <module>
    raise ImportError(DEPRECATION_MESSAGE)
ImportError: Modest backend is deprecated since selectolax 1.0. The engine has not
been updated since 2021, while lexbor has been available since 2021 and is actively
developed. Modest is outdated, not maintained, contains bugs and does not follow
modern HTML5 standards. Please use lexbor backend instead

The script imports from selectolax.parser import HTMLParser, the Modest backend, which has been removed from current selectolax. So the published “Selectolax: 197.02 ms, ~99x slower” row was produced against an HTML engine that selectolax’s own maintainers describe as outdated, unmaintained, buggy and pre-HTML5. That number cannot be regenerated today without editing the benchmark file.

I edited exactly one line — from selectolax.lexbor import LexborHTMLParser as HTMLParser — and reduced the run count from 100 to 10 to keep it inside a coffee break (the ratios are stable; the point stands either way):

Two things changed. Parsel is now marginally faster than Scrapling rather than 3.5% slower — which is expected, because Scrapling’s Selector is built on the same lxml tree and the same cssselect translator (“This project includes code adapted from: Parsel”). And the headline multipliers compress by a third, though not enough to explain themselves.

What explains them is the fixture. large_html is 5,000 nested <div class="item"> elements, so there are 5,000 matching nodes and the innermost one contains 5,000 levels of descendants. A text extraction that walks the whole subtree on each match is therefore O(n²) work on this fixture, and a text extraction that reads only the node’s own text is O(n). The benchmark compares those two different operations and reports the difference as parser speed:

• test_scrapling uses .css(".item::text") — direct text only
• test_parsel also uses ::text — direct text only
• test_lxml uses e.text — direct text only
• test_selectolax uses node.text() — whole subtree
• test_bs4_* uses e.text — whole subtree
• test_pyquery uses e.text() — whole subtree

To show how much of the published gap is semantic rather than implementation, I measured the same libraries on both fixtures with both extraction modes:

Read the first row against the repo’s table: the 99x becomes 5.3x by changing one keyword argument inside the same library on the same page, and on a flat 5,000-item list — the shape of an actual search-results page — selectolax is 3.7x faster than Scrapling, not 99x slower. BS4’s 785x collapses to 26x with the same substitution.

This is not a hit piece on the library: Scrapling’s parser is genuinely fast, within 2% of parsel and ahead of raw lxml on the nested fixture. It is a hit piece on the table, which uses a pathological synthetic document and inconsistent extraction semantics to produce multipliers that would make any reader misjudge the trade-off.

The second table has the same problem in the other direction. The docs report AutoScraper 12.58 ms, 5.47x slower; the same script on the same machine produced 3.88 ms for Scrapling and 220.08 ms for AutoScraper — 56.7x, ten times the published ratio. Both measurements call build() on the same page, so the difference is version drift in the dependency, not in the claim’s direction.

Claim 2: adaptive tracking, measured

The adaptive feature is the reason people pick Scrapling over parsel. Here is the mechanism from scrapling/parser.py, which is worth reading before trusting the marketing sentence “Smart Element Tracking: Relocate elements after website changes using intelligent similarity algorithms”:

1. On auto_save=True, it serialises the matched element into a dict: tag, cleaned attributes, element.text (the node’s own text, i.e. what precedes its first child), path (a tuple of ancestor tag names, with no indices), parent_name, parent_attribs, parent_text, and siblings (tag names of the siblings, also without positions).
2. On adaptive=True, when the selector returns nothing, relocate() iterates over every element in the page, scores each candidate against that dict, and returns the elements tied at the highest score, provided the top score is at least percentage — default 40.
3. The score is the mean of several difflib.SequenceMatcher ratios plus one binary check, multiplied by 100: tag equality, text similarity, attribute dict similarity (keys and values), separate class/id/href/src similarity, path similarity, parent similarity, sibling similarity. Dividing by the number of checks means a candidate that matches on cheap signals can pass 40 without matching on the ones that identify the element.

Because path and siblings contain no positions, and because element.text is empty for any card whose title lives in a child node, cards in an identical list are largely indistinguishable to this scoring function.

I built a 12-card product list and ran four scenarios. Full script in the reproduction section:

Three findings, in order of how much they will cost you:

The correct match scores 61.9% and the wrong match scores 47.08%. The margin between “found the right element” and “found the price tag instead of the title” is 14.8 percentage points, and the default threshold of 40 sits 7 points below the wrong answer. If your selector loses its target, the library does not return nothing — it returns confident garbage.

Nothing warns you at default verbosity. The only signal is log.debug("Highest probability was 61.9%"), which is invisible unless you configure DEBUG logging. The failure branch does warn properly, and it is worth reading what that warning looks like, because it tells you the library knows this is a guess:

WARNING: Adaptive relocation found no element above the 80% threshold
(top score: 47.08%). Lower `percentage` if this is the right element.

The count changes silently. A selector that returned 12 rows returns 1 after relocation. This one is documented, but only under “Known Issues” at the bottom of the adaptive page: “In the adaptive save process, only the unique properties of the first element in the selection results are saved. So if the selector you are using selects different elements on the page in other locations, adaptive will return the first element to you only when you relocate it later.” The feature page above it, and the README, describe tracking elements in the plural.

There is also a cost curve. Because relocate() scores every node, the price is linear in page size but with a large constant — each candidate costs a dozen SequenceMatcher calls:

At 0.09 ms per node, a 40,000-node listing page costs about 3.7 seconds per adaptive call, and adaptive is exactly the code path you fall back to when a site has changed and you are scraping every page. Use percentage deliberately, run adaptive only for single-element targets (a price, a stock flag), and assert cardinality and a content hash after every relocation.

Where the database lives

Selector(..., adaptive=True) writes to Path(scrapling/parser.py).parent / "elements_storage.db" — inside site-packages. In this venv that resolves to:

/opt/data/scripts2/recon/scrapling_venv/lib/python3.13/site-packages/scrapling/elements_storage.db

Make that directory read-only, which is what happens in a hardened container image, and the save fails with a second, misleading error on top of the first:

OperationalError: attempt to write a readonly database
Exception ignored in: <function SQLiteStorageSystem.__del__>
AttributeError: 'SQLiteStorageSystem' object has no attribute 'cursor'

The AttributeError comes from __del__ calling close() on a connection that was never opened. Anyone debugging that traceback will look in the wrong place. Pass storage_args={"storage_file": ...} and put the file on a writable volume.

The storage key is the base domain, resolved through tld, or the literal string "default" when no url= was passed: parsing HTML strings from several sites without url= puts them all in one namespace, where the same selector identifier overwrites the previous record. The docs say this outright — “The save process overwrites previous data, and the adaptive feature uses only the latest saved properties” — but it is one omitted keyword argument away from cross-contaminating two scrapers.

Claim 3: fetching and stealth

The fetch layer is less controversial and mostly does what it says. Six pages, same machine, sequential, warm browser process reused. It is the same trade-off I measured yesterday for a headless browser that removed Chromium entirely: the HTTP path is cheap, the rendering path is not, and only one of them sees the content.

* The Playwright figure counts the Python process only; the browser is a child process, so memory is not comparable across rows. Wall time is.

The useful lesson is in the third column: the HTTP fetcher returns HTTP 200 with the right <title> and zero of the content, because the page renders its quotes in JavaScript. A pipeline that checks response.status == 200 will record that page as successfully scraped and write an empty row. That is the same class of failure as the adaptive matcher: no exception, no warning, wrong data.

For the “undetectable” claim, I served a local page that reports the classic fingerprint surface and fetched it with both browser paths:

Patchright fixes the three cheapest tells and leaves two. A browser claiming to be Chrome 156 with five plugins and granted notification permission but no window.chrome.runtime is internally inconsistent, and 2 cores in UTC is a server. Whether that matters depends entirely on the target’s detector — which is why the README’s “can easily bypass all types of Cloudflare’s Turnstile/Interstitial” needs your own numbers, not mine. I tried nowsecure.nl, the usual Cloudflare test page, and got 200 with ~180 KB from all three methods from this datacenter IP, which means that test discriminated nothing and I have no evidence either way. Report your own success rate per target before you build a business on it.

One default worth knowing: robots_txt_obey is False. In v0.5, turning it on also changed behaviour — a robots.txt fetch failure now blocks the whole domain for the crawl, where previously it allowed access.

The v0.5 upgrade is not a version bump

Version 0.5 removes the plain Playwright backend entirely. DynamicFetcher, DynamicSession, AsyncDynamicSession and fetchers.chrome.PlayWrightFetcher are gone; everything browser-based is Patchright now. If your code imports playwright.sync_api types in a callback, it must import patchright.sync_api instead.

The breaking changes most likely to alter data without an error:

The project’s own upgrade advice is the right one, and unusually blunt: “Run a small crawl before your full job. Check the extracted data, followed URLs, sent request bodies, saved file encoding, and any cache or checkpoint behaviour you depend on.”

What I would actually do with it

• Use the parser. Selector is fast, the API is a superset of parsel’s, and being able to swap parsel for it in a Scrapy pipeline is the one feature no competitor has. Pin the version.
• Do not use adaptive as a correctness mechanism. Use it as an alert: run the selector normally, and when it returns zero, run the adaptive path and fail the job if the score, the count, or a content hash changed. Treat a 47%-confidence match as a bug report, not a row.
• Always pass url= and storage_args. Otherwise every site shares the default namespace and the database lives inside site-packages.
• Set percentage explicitly — 70 or higher for anything that feeds a database, and log the score at INFO yourself, because the library logs it at DEBUG.
• Keep the HTTP and browser paths separate. Fetcher for static HTML (1.9 s for six pages, 57 MB), StealthyFetcher for anything that renders, and assert on extracted content, never on status == 200. For a pure-HTTP crawler built around the same idea, MediaCrawler’s approach is worth reading side by side.
• Test the upgrade before you trust it. 0.4.15 → 0.5 is a rewrite of the browser layer with a half-dozen silent output changes; a diff of a 50-page sample crawl takes ten minutes and will catch all of them.

Reproduce this in an hour

python3.13 -m venv venv && ./venv/bin/pip install "scrapling[fetchers]"
export PLAYWRIGHT_BROWSERS_PATH=/path/you/can/write
./venv/bin/python -m patchright install chromium   # only for StealthyFetcher

The adaptive scenario, reduced to its essentials — the four rows of the table above differ only in the HTML passed to the third constructor:

from scrapling import Selector

def page(names, cls="product", tclass="ptitle"):
    return "<html><body>" + "".join(
        f'<div class="{cls}"><span class="{tclass}">{n}</span>'
        f'<span class="price">$9.99</span></div>' for n in names
    ) + "</body></html>"

names = [f"Widget-{i:02d}" for i in range(1, 13)]
url = "https://shop.example.com/list"

# day 1: save
saved = Selector(page(names), url=url, adaptive=True).css(".product .ptitle", auto_save=True)
print(len(saved))                                  # 12

# day 2: classes renamed -> adaptive
moved = Selector(page(names, "item-x", "ttl"), url=url, adaptive=True)
print(len(moved.css(".product .ptitle", adaptive=True)))   # 1, not 12

# day 3: titles deleted -> adaptive invents an answer
gone = ("<html><body>" + "".join(
    '<div class="item-x"><span class="x">n/a</span><span class="price">$9.99</span></div>'
    for _ in names) + "</body></html>")
hit = Selector(gone, url=url, adaptive=True).css(".product .ptitle", adaptive=True)
print(len(hit), hit[0].html_content)   # 12 <span class="price">$9.99</span>

Enable logging.getLogger().setLevel(logging.DEBUG) on scrapling.core.utils.log to see the score (Highest probability was 47.08%). The benchmark reproduction is benchmarks.py from the repo with the selectolax import switched to selectolax.lexbor.LexborHTMLParser and repeat=10.

FAQ

Is Scrapling’s 92,848 stars fake? I found no evidence of that, and two findings argue against it: 2.26 million PyPI downloads in October 2026 and 310 repositories listing it as a dependency. What the numbers do show is that stars and engineering scrutiny have decoupled — 327 watchers, 179 issues total, and a maximum of 4 points on Hacker News. Treat the star count as reach, not as a code review.

Is it actually faster than BeautifulSoup? Yes, by a wide margin, but not by 785x. On the repo’s own nested fixture the honest gap between direct text extraction in Scrapling and deep text extraction in BS4 is about 26x (2.43 ms vs 51.76 ms); the 785x figure comes from comparing different operations and from an outdated selectolax backend. Against parsel, Scrapling is a tie (2.50 vs 2.54 ms median in my re-run).

Does adaptive scraping work? It works on the case it was designed for — one element, with a threshold you set. It found the right card after a class rename at 61.9%. It also returned 12 wrong elements at 47.08% when the target was deleted, without a warning at default log level, and it collapses a 12-row match to 1. Check counts and content, not just emptiness.

Do I need a browser to use it? No. pip install "scrapling[fetchers]" without scrapling install gives you the HTTP fetcher, which handled six pages in 1.9 seconds total. The browser is only needed for StealthyFetcher, and each launch costs roughly a second of overhead.

Is it production-ready? The package metadata says Development Status :: 4 - Beta, the changelog calls v0.5 full of breaking changes, and the storage layer writes into site-packages by default. The parser is production-grade; the crawler and adaptive layer are beta in the literal sense. Pin versions, own your storage path, and diff your output after every upgrade.

Can it bypass Cloudflare? The README claims it “can easily bypass all types of Cloudflare’s Turnstile/Interstitial”. I could not verify that: the standard test page returned 200 to a vanilla Playwright browser from this IP as well, so the test proved nothing. The fingerprint table above shows three patched signals and two remaining ones (window.chrome.runtime, 2 cores in UTC). Measure on your own targets.

The shape of the finding

A library can be good and its marketing can still be wrong, and the distance between the two is where the actual work is. Scrapling’s parser is within 2% of parsel; its HTTP fetcher is four times faster than a browser for static pages; its test suite is larger than its source. None of that requires a 92,848-star chart or a 785x benchmark.

The two numbers that should change your implementation are the ones nobody published: 47.08% is enough similarity for the adaptive matcher to return the wrong element twelve times over, and 61.9% is what a correct match scores — 14.8 points of separation between a working scraper and silent data corruption. Set the threshold yourself, and assert on what came back.