Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-http-test gem contains a HTTP test server for testing HTTP client implementations.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-http gem contains and http client agnostic abstraction layer.

株価情報を取得してあれこれするライブラリ

Lightweight web crawler

Simple little website crawler.

A simple web crawler for ruby

A crawler and parser to the Currículo Lattes.

DSL to build crawlers easily

Cangrejo lets you consume crabfarm crawlers using a simple DSL

A Crawler for NewRank

crawler of niconico-douga