A simple movie converter, queue and directory crawler, all in one!

Botch is a DSL for quickly creating web crawlers. Inspired by Sinatra.

Iudex is a general purpose web crawler and feed processor in ruby/java. This gem is a Jetty HTTP Client based implementation of the iudex-http interfaces.

book's price crawler

This will be crawling data from websites. Need to give the xpaths clearly. Will be updating with new functionalities in future

Discovery Mission is an easy-to-use website crawler. Use it for generating sitemaps.

FileCrawler searches and controls files in local directory

Mobile App Review Crawler

A little website crawler.

DSL to build crawlers easily

A crawler for a single domain web application

A ruby stock price & option chain crawler

Show DMM and DMM.R18's crawled data. e.g. ranking

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-filter gem contains a fundamental filtering/chain of responsbility sub-system.

Lightweight web crawler

Headless HTTP crawler/scraper

Iudex is a general purpose web crawler and feed processor in ruby/java. This gem is an rjack-async-httpclient based implementation of the iudex-http interfaces.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-http-test gem contains a HTTP test server for testing HTTP client implementations.

Simple little website crawler.

Grab the movies information from the atmovies.com