Build maintainable web crawlers with Scrapy.
What it does
Guides development of maintainable Scrapy crawlers with structured data extraction and processing pipelines.
When to use it
Use when building web scrapers or crawlers with the Scrapy Python framework.
How to use it
It changes how the agent develops Scrapy crawlers, emphasizing maintainable code, modular architecture, error handling, and ethical scraping.
What you provide
- An existing project
Latest review
What you need · 2
The crawler code uses Python; no runtime version is specified.
Scrapy is the framework the guidance targets and is listed as a key dependency.
Access · 0
No additional actions listed.