A crawler and parser to the Currículo Lattes.

Grab the movies information from the atmovies.com

A simple web crawler for ruby

A Crawler for NewRank

crawler of niconico-douga

Simple little website crawler.

Add support for ElasticSearch in Polipus crawler

A web crawler written in ruby

The wrapper of capybara for crawler

Friendly, neighborhood web crawler for quality assurance.

株価情報を取得してあれこれするライブラリ

Another Web crawler running with Amazon SQS and ElastiCache(Redis)

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-barc gem contains support for the BARC Basic ARChive format.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-simhash gem contains support for generation and searching over simhash fingerprints

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-html gem contains filters for HTML parsing, filtering, exracting text and links.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-char-detector gem provides charset detection support.

ITunesCrawler provides an easy way to download the requested iTunes data through Apple's Search API.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-da gem provides a PostgreSQL-based content meta-data store and work priority queue.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-brutefuzzy-service provides a fuzzy simhash lookup index as a distributed service.

Iudex is a general purpose web crawler and feed processor in ruby/java. This gem is an rjack-httpclient-3 based implementation of the iudex-http interfaces.