Ruby web crawler to access omelete informations

A set of classes for dealing with options. It includes a crawler for Yahoo!Finance.

Lightweight web crawler

A very simple crawler for RubyGems.org used to demo the power of ElasticSearch at RubyConf 2013

Samao is a web crawler written in ruby.

The SimpleCrawler module is a library for crawling web sites. The crawler provides comprehensive data from the page crawled which can be used for page analysis, indexing, accessibility checks etc. Restrictions can be specified to limit crawling of binary files.

Easy to use DSL that helps scraping data from websites. Thanks to it, writing web crawlers would be very fast and intuitive. Traversing through html nodes and fetching all of the HTML attributes, would be possible. Just like in jQuery - you will find methods like parent, children, first, find, siblings etc. Furthermore, you are able to download images, web pages, and store all content in the database. Please visit my Github account for more details.

The application crawls a URL and extracts links, tags and sequences. These features are written to an output file

This gem offers: classes to subclass and create a manga site crawler; a dowloader to use with these classes; some site-specific scripts.

plugin-container totally sucks..

A simple web crawler gem