Categories
No matching categories were found
0.01
== Medusa: a ruby crawler framework {rdoc-image:https://badge.fury.io/rb/medusa-crawler.svg}[https://rubygems.org/gems/medusa-crawler] rdoc-image:https://github.com/brutuscat/medusa-crawler/workflows/Ruby/badge.svg?event=push
Medusa is a framework for the ruby language to crawl and collect usefu...
2019
2020
2021
2022
2023
2024
0.13
Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites which is much more performant than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.
2019
2020
2021
2022
2023
2024
0.54
Generic Web crawler with a DSL that parses structured data from web pages
2019
2020
2021
2022
2023
2024
0.07
CrawlerDetect is a library to detect bots/crawlers via the user agent
2019
2020
2021
2022
2023
2024
0.0
Crawler Engine provides function of crawl all news from the customized website
2019
2020
2021
2022
2023
2024
0.0
Crawler4J filter plugin for Embulk
2019
2020
2021
2022
2023
2024
0.0
A crappy crawler for a crappy bank interface
2019
2020
2021
2022
2023
2024
0.26
Voight-Kampff detects bots, spiders, crawlers and replicants
2019
2020
2021
2022
2023
2024
0.02
is_crawler does exactly what you might think it does: determine if the supplied string matches a known crawler or bot.
2019
2020
2021
2022
2023
2024
0.0
A simple crawler demo crawler
2019
2020
2021
2022
2023
2024
0.0
A very simple crawler for RubyGems.org used to demo the power of ElasticSearch at RubyConf 2013
2019
2020
2021
2022
2023
2024
0.0
it starts a crawler for Middleman sites
2019
2020
2021
2022
2023
2024
0.0
Simple web crawler to crawl a domain and generate sitemap
2019
2020
2021
2022
2023
2024
0.0
a simple web crawler using DSL of capybara
2019
2020
2021
2022
2023
2024
0.0
This file crawler helps to decect if there are new files in a directory.
2019
2020
2021
2022
2023
2024
0.0
crawler of niconico
2019
2020
2021
2022
2023
2024
0.0
Crawler for http://legendas.tv to see the most dowloaded subtitles
2019
2020
2021
2022
2023
2024
0.0
Web page crawler.
2019
2020
2021
2022
2023
2024
0.0
Simple ruby web crawler
2019
2020
2021
2022
2023
2024
0.0
Simple Gem Using Watir For Phantom Crawler
2019
2020
2021
2022
2023
2024