A crawler and parser to the Currículo Lattes.

A web scrawler to get a Marmiton's recipe

A gem that collects mangas from websites

Grab the movies information from the atmovies.com

A Crawler for NewRank

Add support for ElasticSearch in Polipus crawler

Crawl websites

Add support for Cassandra in Polipus crawler

A web crawler written in ruby

The wrapper of capybara for crawler

A ruby social media stat crawler

A crawler for such as mailing lists.

CIA World Factbook crawler

Gem para acessar os dados do TIA Mackenzista

A web crawler dsl.

A ruby based zhihu content crawler.

Another Web crawler running with Amazon SQS and ElastiCache(Redis)

Rack middleware that executes javascript before serving pages to crawlers.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-barc gem contains support for the BARC Basic ARChive format.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-simhash gem contains support for generation and searching over simhash fingerprints