A little website crawler.

Cangrejo lets you consume crabfarm crawlers using a simple DSL

DSL to build crawlers easily

A crawler for a single domain web application

Simple little website crawler.

A crawler and parser to the Currículo Lattes.

A Crawler for NewRank

crawler of niconico-douga

Add support for ElasticSearch in Polipus crawler

This is web comic crawler that Akihabara otaku.

Crawl websites

Add support for Cassandra in Polipus crawler

A crawler for such as mailing lists.

Gem para acessar os dados do TIA Mackenzista

Server browser and Crawler for many games (L4D2, TF2, CS:S, KZMOD, The Ship)

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-barc gem contains support for the BARC Basic ARChive format.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-char-detector gem provides charset detection support.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-da gem provides a PostgreSQL-based content meta-data store and work priority queue.

Iudex is a general purpose web crawler and feed processor in ruby/java. This gem is an rjack-httpclient-3 based implementation of the iudex-http interfaces.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-rome gems is an adaption of rjack-rome for feed parsing in Iudex.