A simple crawler demo crawler

Using paperclip to generate images from sensible attributes like e-mails and telephone numbers, in order to reduce crawler's success

A crappy crawler for a crappy bank interface

Allows to crawl bandcamp sites, including release and track information

book's price crawler

This will be crawling data from websites. Need to give the xpaths clearly. Will be updating with new functionalities in future

BFS webcrawler that implements Observable

web crawler that generates a sitemap to a neo4j database. It will also store broken_links and total number of pages on site

This is a crawler framework.

Driller is a command line Ruby based web crawler based on Anemone. Driller can crawl website and reports error pages and slow pages and generates HTML reports.

A crappy crawler for a crappy bank interface