A simple solution to provide on-demand service access (e.g. port 80 on webserver), where a more robust and secure VPN solution is not available. Essentially, it is a more user-friendly form of "port knocking". The original proof-of-concept implementation was run for almost three years by ...

Cangrejo lets you consume crabfarm crawlers using a simple DSL

SemanticCrawler is a ruby library that encapsulates data gathering from different sources. Currently microdata from websites, country information from Freebase, Factbook and FAO (Food and Agriculture Organization of the United Nations), crisis information from GDACS.org and geo data from LinkedGe...

Cosmicrawler is crawler library for Ruby. It provides scalable asynchronous crawling by (http|file|etc) using EventMachine.

Simple site crawler using Capybara

Gem para acessar os dados do TIA Mackenzista

Using paperclip to generate images from sensible attributes like e-mails and telephone numbers, in order to reduce crawler's success

Add support for Cassandra in Polipus crawler

A web scrawler to get a Marmiton's recipe

Checks a user agent for a web crawler

This crawler will use my personnal scraper named 'RecipeScraper' to dowload recipes data from Marmiton, 750g or cuisineaz

Crawler4J filter plugin for Embulk

Ruby Cheerio is a jQuery style HTML parser, which take selectors as input. This is a Ruby version NodeJS package named 'Cheerio', which is extensively used by crawlers. Please visit the home page for usage details.

Botch is a DSL for quickly creating web crawlers. Inspired by Sinatra.

== Medusa: a ruby crawler framework {rdoc-image:https://badge.fury.io/rb/medusa-crawler.svg}[https://rubygems.org/gems/medusa-crawler] rdoc-image:https://github.com/brutuscat/medusa-crawler/workflows/Ruby/badge.svg?event=push Medusa is a framework for the ruby language to crawl and collect usefu...

Gem for crawling data from external sources

The Taiwan VSCinema crawler to get latest film list.

The wrapper of capybara for crawler

A simple web crawler for ruby

Add support for ElasticSearch in Polipus crawler