Simple site crawler using Capybara

This gem crawls the latest CircleCI artifact file you specified. For Example, you can get the result JSON of simplecov.gem etc.

A simple, fast web crawler

Show DMM and DMM.R18's crawled data. e.g. ranking

Crawl instagram photos, posts and videos for download.

This gem allows to crawl news articles from RSS feeds.

Voight-Kampff detects bots, spiders, crawlers and replicants

ActiveRecord adapter for kudzu crawler

This plugin need set the initial url.

Crawls the mails in a IMAP Folder

This file crawler helps to decect if there are new files in a directory.

Lightweight interface to communicate with the Marvel Comics API

A simple crawler to list SESC events schedule on Terminal

Crawler that auto generates SitePrism Page Objects

webget gem - a web (go get) crawler incl. web cache

Gem para acessar os dados do TIA Mackenzista

A web scrawler to get a Marmiton's recipe

A demo of Web Crawler using arb-crawler

FileCrawler searches and controls files in local directory

A simply web crawler.