Crawler that auto generates SitePrism Page Objects

This crawler will use my personnal scraper named 'RecipeScraper' to dowload recipes data from Marmiton, 750g or cuisineaz

This gem helps Crawler Writers to interact with the PromoQui REST API

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-barc gem contains support for the BARC Basic ARChive format.

This plugin need set the initial url.

FileCrawler searches and controls files in local directory

image download from instagram

Lightweight interface to communicate with the Marvel Comics API

This gem allows to crawl news articles from RSS feeds.

A demo of Web Crawler using arb-crawler

A simple web crawler gem

This gem offers: classes to subclass and create a manga site crawler; a dowloader to use with these classes; some site-specific scripts.

Multithreaded web crawler with transparent DSL and requests caching.

A simple crawler to list SESC events schedule on Terminal

A ruby social media stat crawler

Stupid crawler that looks for URLs on a given site. Result is saved as two CSV files one with found URLs and another with failed URLs.

Gem para acessar os dados do TIA Mackenzista

Web page crawler.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-da gem provides a PostgreSQL-based content meta-data store and work priority queue.

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-html gem contains filters for HTML parsing, filtering, exracting text and links.