Crawler Guru provides all basic functionalities to extract data from web pages

it starts a crawler for Middleman sites

Stupid crawler that looks for URLs on a given site. Result is saved as two CSV files one with found URLs and another with failed URLs.

Simple web crawler to crawl a domain and generate sitemap

Web crawler help you with parse and collect data from the web

crawler of niconico

A simply web crawler.

A periodic crawler that fetches the latest CVE additions, parses them, and filters them

MurmuringSpider is a concise Twitter crawler. When we write a data-mining / text-mining application based on twitter timeline, we have to collect and store tweets first. I am irritated with writing such crawler repeatedly, so I wrote this. What you have to do is only to add query and to run th...

A crappy crawler for crashlytics

A simple web crawler gem

Simple ruby web crawler

A watir based web crawler

A simple web crawler gem

a simple web crawler using DSL of capybara

A flexible, modular web crawler

Retrieves a list of URLs to seed the crawler by publishing them to a RabbitMQ exchange.

Crawler for downloading comics from komiks.gildia.pl

Simple Gem Using Watir For Phantom Crawler

Crawl instagram photos, posts and videos for download.