跳转到主要内容
此内容尚未提供您的语言版本,正在以英文显示。

Rotate Scraping Proxies

技能 已验证 活跃

Escalate blocked scraping campaigns with provider-neutral proxy rotation — decide between datacenter, residential, and mobile pools, integrate rotation with scrapling, configure session stickiness for stateful flows, monitor cost and health, and stay inside legal and ethical boundaries. Use as the next step after `headless-web-scraping` client-side stealth (StealthyFetcher, rate limiting, robots.txt) is insufficient and traffic is legitimate.

目的

To enable users to overcome scraping blocks by ethically and effectively rotating proxy IPs, managing costs, and ensuring compliance with legal and ethical guidelines when standard stealth techniques fail.

功能

  • Provider-neutral proxy pool selection (datacenter, residential, mobile)
  • Integration with scraping frameworks (e.g., scrapling)
  • Configuration for sticky sessions and per-request rotation
  • Monitoring of proxy pool health, cost, and traffic limits
  • Step-by-step guidance on legal and ethical considerations

使用场景

  • When standard client-side stealth (User-Agent, rate limiting) fails to bypass target website blocks.
  • For legitimate scraping of public data that requires overcoming geo-blocking or IP-based rate limits.
  • To manage complex stateful scraping flows (e.g., logins, multi-page processes) that require persistent proxy IPs.
  • As an escalation step for scraping campaigns where a public API is unavailable and the use case is defensible.

非目标

  • Bypassing site Terms of Service prohibitions against automated access.
  • Circumventing geo-licensing or paywalls.
  • Enabling fraudulent activities like credential stuffing or content piracy.
  • Replacing the use of official APIs when they are available and suitable.

安装

/plugin install agent-almanac@pjt222-agent-almanac

质量评分

已验证
99 /100
about 22 hours ago 分析

信任信号

最近提交2 days ago
星标14
许可证MIT
状态
查看源代码

类似扩展

Hybrid Cloud Networking

100

Configure secure, high-performance connectivity between on-premises infrastructure and cloud platforms using VPN and dedicated connections. Use when building hybrid cloud architectures, connecting data centers to cloud, or implementing secure cross-premises networking.

技能
wshobson

High Performance Browser Networking

100

Optimize web performance through network protocols, resource loading, and browser rendering internals. Use when the user mentions "page load speed", "Core Web Vitals", "HTTP/2", "resource hints", "network latency", "render blocking", "TCP optimization", "service worker", or "critical rendering path". Also trigger when diagnosing slow page loads, optimizing time to first byte, choosing between WebSocket and SSE, or reducing bundle sizes. Covers TCP/TLS optimization, caching strategies, WebSocket/SSE, and protocol selection. For UI visual performance, see refactoring-ui. For font loading, see web-typography.

技能
wondelai

Node Connect

100

Diagnose OpenClaw Android, iOS, or macOS node pairing, QR/setup code, route, auth, and connection failures.

技能
steipete

Embedding Strategies

100

Select and optimize embedding models for semantic search and RAG applications. Use when choosing embedding models, implementing chunking strategies, or optimizing embedding quality for specific domains.

技能
wshobson

Aws Cdk Development

100

AWS Cloud Development Kit (CDK) 专家,用于使用 TypeScript/Python 构建云基础设施。在创建 CDK 堆栈、定义 CDK 构造、实现基础设施即代码,或当用户提及 CDK、CloudFormation、IaC、cdk synth、cdk deploy,或希望以编程方式定义 AWS 基础设施时使用。涵盖 CDK 应用结构、构造模式、堆栈组合和部署工作流。

技能
zxkane

Fit Drift Diffusion Model

100

Fit cognitive drift-diffusion models (Ratcliff DDM) to reaction time and accuracy data with parameter estimation (drift rate, boundary separation, non-decision time), model comparison, and parameter recovery validation. Use when modeling binary decision-making with reaction time data, estimating cognitive parameters from experimental data, comparing sequential sampling model variants, or decomposing speed-accuracy tradeoff effects into latent cognitive components.

技能
pjt222