Lightlines
Catalogue
Sign in
By jamditis

web-scraping

jamditis

Extract public web data without bypassing access controls.

web scraping

What it does

Guides authorized web scraping with fallback extraction, access-failure handling, and safeguards for untrusted content.

When to use it

Use for social-media scraping, yt-dlp workflows, and handling CAPTCHA or HTTP 403 blocks.

How to use it

It changes how the agent performs authorized web scraping, validates destinations, selects fallbacks, and handles denied access.

Uses


Access · 7

This skill

public web pages

Read

Reads public web pages

robots.txt

Read

Reads sites' robots.txt rules

browser network requests

Read

Inspects browser network requests

YouTube

Read

Reads YouTube metadata and media

What you need · 8

The supplied scraping implementations are Python code.

The HTTP extraction strategies import and use requests.

The extraction strategies import BeautifulSoup from bs4.

The first extraction strategy uses trafilatura.


About this skill

Visibility
Public
Repository
jamditis/claude-skills-journalism
Created
Oct 8, 2026
Updated
Oct 8, 2026
Files
2