scrapy /scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
★ 65,415⑂ 12,146👁 1,744⑃ 5,052● 3,325▤ 55
+26
Daily
+128
Weekly
65,415
stars
12,146
forks
1,744
watchers
5,052
prs
3,325
issues
55
releases
Top issues
#1306Speedup & fix URL parsing39 comments · 2015-06-15#6597Scrapy needs to have sane defaults that do no harm32 comments · 2025-01-01#4240Wrong type(response) for binary responses28 comments · 2019-12-17#1144Awaiting on Requests in callbacks directly25 comments · 2015-04-10#356Support list type arguments from command line.24 comments · 2013-07-24#1988"Content-Encoding" header gets stripped from response headers20 comments · 2016-05-12#1225dupefilter skips a request when a page is redirected to itself 20 comments · 2015-05-12#713Bring docstrings back17 comments · 2014-05-07
Discussions
#8368PR automatically flagged as spam, looking for maintainer help1 comments · 2026-10-17#7582Choosing Issue1 comments · 2026-06-08#7362Newbie question about scraping3 comments · 2026-03-25#7343How to Automatically Retrieve Web Resources That Require On-Site Search to Locate, Instead of Just Recursively Crawling URLs?2 comments · 2026-03-21#7326Scrapy doesn't seem to understand what 403 and 444 mean.3 comments · 2026-03-11#7325Providing Spider specific settings to custom DownloadHandlers1 comments · 2026-03-11#7322License and Attribution of the Scrapy Logo?2 comments · 2026-03-09#7320Localization of The Scrapy Documentation0 comments · 2026-03-09
Latest release
2.20.0 2.20.0 2026-11-06- Web-browser-like detection of response encodings, base URLs and meta refresh redirects, powered by frostwork - Requests can be awaited to get their response - Non-asyncio Twisted reactors are deprecated Full changelog