A Crawler for NewRank

crawler of niconico-douga

Add support for ElasticSearch in Polipus crawler

A web crawler written in ruby

Crawler that auto generates SitePrism Page Objects

A simple web crawler gem

A ruby social media stat crawler

Crawler Guru provides all basic functionalities to extract data from web pages

A simple crawler to list SESC events schedule on Terminal

CIA World Factbook crawler

Gem para acessar os dados do TIA Mackenzista

Crawl multiple torrent sites.

Server browser and Crawler for many games (L4D2, TF2, CS:S, KZMOD, The Ship)

A watir based web crawler

A web crawler dsl.

A simple directory crawler DSL.

A ruby based zhihu content crawler.

Another Web crawler running with Amazon SQS and ElastiCache(Redis)

Retrieves a list of URLs to seed the crawler by publishing them to a RabbitMQ exchange.

Checks a user agent for a web crawler