Cangrejo lets you consume crabfarm crawlers using a simple DSL

Simple site crawler using Capybara

Empower World Travel Information Technology

ActiveRecord adapter for kudzu crawler

株価情報を取得してあれこれするライブラリ

your friendly neighborhood web crawler

A simple web crawler for ruby

This crawler will use my personnal scraper named 'RecipeScraper' to dowload recipes data from Marmiton, 750g or cuisineaz

A flexible, modular web crawler

webget gem - a web (go get) crawler incl. web cache

A ruby social media stat crawler

it starts a crawler for Middleman sites

Checks a user agent for a web crawler

This rubygem does not have a description or summary.

web crawler that generates a sitemap to a neo4j database. It will also store broken_links and total number of pages on site

Discovery Mission is an easy-to-use website crawler. Use it for generating sitemaps.

This file crawler helps to decect if there are new files in a directory.

A simply web crawler.

A gem that collects mangas from websites

Block crawlers who spam your site with fake HTTP referers