0.03
Post URLs to Wayback Machine (Internet Archive), using a crawler, from Sitemap(s) or a list of URLs.
2019
2020
2021
2022
2023
2024
0.01
JavaScript enabled web crawler kit
2019
2020
2021
2022
2023
2024
0.0
Server browser and Crawler for many games (L4D2, TF2, CS:S, KZMOD, The Ship)
2019
2020
2021
2022
2023
2024
0.01
This is a crawler framework.
2019
2020
2021
2022
2023
2024
0.01
SemanticCrawler is a ruby library that encapsulates data gathering from different sources. Currently microdata from websites, country information from Freebase, Factbook and FAO (Food and Agriculture Organization of the United Nations), crisis information from GDACS.org and geo data from LinkedGe...
2019
2020
2021
2022
2023
2024
0.0
Email crawler: crawls the top ten Google search results looking for email addresses and exports them to CSV.
2019
2020
2021
2022
2023
2024
0.0
Ruby Cheerio is a jQuery style HTML parser, which take selectors as input. This is a Ruby version NodeJS package named 'Cheerio', which is extensively used by crawlers. Please visit the home page for usage details.
2019
2020
2021
2022
2023
2024
0.0
Bulbasaur is a helper for crawler operations used in Pread.ly
2019
2020
2021
2022
2023
2024
0.01
Driller is a command line Ruby based web crawler based on Anemone. Driller can crawl website and reports error pages and slow pages and generates HTML reports.
2019
2020
2021
2022
2023
2024
0.0
A web crawler written in ruby
2019
2020
2021
2022
2023
2024
0.0
Checks a user agent for a web crawler
2019
2020
2021
2022
2023
2024
0.0
RegexpCrawler is a Ruby library for crawl data from website using regular expression.
2019
2020
2021
2022
2023
2024
0.01
A simple, fast web crawler
2019
2020
2021
2022
2023
2024
0.0
FileCrawler searches and controls files in local directory
2019
2020
2021
2022
2023
2024
0.0
Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-worker gem provides a worker deamon for feed/page processing.
2019
2020
2021
2022
2023
2024
0.0
Using paperclip to generate images from sensible attributes like e-mails and telephone numbers, in order to reduce crawler's success
2019
2020
2021
2022
2023
2024
0.0
Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-html gem contains filters for HTML parsing, filtering, exracting text and links.
2019
2020
2021
2022
2023
2024
0.0
Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-barc gem contains support for the BARC Basic ARChive format.
2019
2020
2021
2022
2023
2024
0.0
A simple web crawler for ruby
2019
2020
2021
2022
2023
2024
0.0
Iudex is a general purpose web crawler and feed processor in ruby/java. This gem is a Jetty HTTP Client based implementation of the iudex-http interfaces.
2019
2020
2021
2022
2023
2024