Easy to use DSL that helps scraping data from websites. Thanks to it, writing web crawlers would be very fast and intuitive. Traversing through html nodes and fetching all of the HTML attributes, would be possible. Just like in jQuery - you will find methods like parent, children, first, find, si...

Mobile App Review Crawler

The application crawls a URL and extracts links, tags and sequences. These features are written to an output file

Jobs crawler

RegexpCrawler is a Ruby library for crawl data from website using regular expression.

This gem offers: classes to subclass and create a manga site crawler; a dowloader to use with these classes; some site-specific scripts.

plugin-container totally sucks..

A simple web crawler gem

Ruby Web crawler

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-http-test gem contains a HTTP test server for testing HTTP client implementations.

Crawler Guru provides all basic functionalities to extract data from web pages

Iudex is a general purpose web crawler and feed processor in ruby/java. This gem is an rjack-async-httpclient based implementation of the iudex-http interfaces.

Crawler Engine provides function of crawl all news from the customized website

A simple web crawler gem

A simple news crawler. You can specify the structure of your xml or rss feeds.

A simple web crawler gem

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-worker gem provides a worker deamon for feed/page processing.

A web crawler dsl.

Crawl websites

Lightweight web crawler