初级开发工程师，基于 http 写的爬虫扩展包。请不要随意下载里面有很多坑。

Empower World Travel Information Technology

This little library helps people download images from different subs much easier. It's actually like a crawler for the images posted on a subreddit. Actually, it's a great tool to have your favorite memes locally!

Iudex is a general purpose web crawler and feed processor in ruby/java. This gem is a Jetty HTTP Client based implementation of the iudex-http interfaces.

Fake User-Agents of about %80 of real devices to use in headers of web crawlers. It keeps your script away from being nested by many UA strings.

This gem crawls the latest CircleCI artifact file you specified. For Example, you can get the result JSON of simplecov.gem etc.

webget gem - a web (go get) crawler incl. web cache

Allows to crawl bandcamp sites, including release and track information

Discovery Mission is an easy-to-use website crawler. Use it for generating sitemaps.

FileCrawler searches and controls files in local directory

RegexpCrawler is a Ruby library for crawl data from website using regular expression.

Cangrejo lets you consume crabfarm crawlers using a simple DSL

Iudex is a general purpose web crawler and feed processor in ruby/java. This gem is an rjack-async-httpclient based implementation of the iudex-http interfaces.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-http-test gem contains a HTTP test server for testing HTTP client implementations.

株価情報を取得してあれこれするライブラリ

Simple little website crawler.

A simple web crawler for ruby

A crawler and parser to the Currículo Lattes.

Crawl websites

Add support for Cassandra in Polipus crawler