Categories
No matching categories were found
0.01
== Medusa: a ruby crawler framework {rdoc-image:https://badge.fury.io/rb/medusa-crawler.svg}[https://rubygems.org/gems/medusa-crawler] rdoc-image:https://github.com/brutuscat/medusa-crawler/workflows/Ruby/badge.svg?event=push
Medusa is a framework for the ruby language to crawl and collect usefu...
2019
2020
2021
2022
2023
2024
0.13
Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites which is much more performant than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.
2019
2020
2021
2022
2023
2024
0.54
Generic Web crawler with a DSL that parses structured data from web pages
2019
2020
2021
2022
2023
2024
0.07
CrawlerDetect is a library to detect bots/crawlers via the user agent
2019
2020
2021
2022
2023
2024
0.0
Crawler Engine provides function of crawl all news from the customized website
2019
2020
2021
2022
2023
2024
0.0
Crawler4J filter plugin for Embulk
2019
2020
2021
2022
2023
2024
0.0
A crappy crawler for a crappy bank interface
2019
2020
2021
2022
2023
2024
0.26
Voight-Kampff detects bots, spiders, crawlers and replicants
2019
2020
2021
2022
2023
2024
0.02
is_crawler does exactly what you might think it does: determine if the supplied string matches a known crawler or bot.
2019
2020
2021
2022
2023
2024
0.0
A simple crawler demo crawler
2019
2020
2021
2022
2023
2024
0.0
A simple web crawler gem
2019
2020
2021
2022
2023
2024
0.0
Web crawler with JSON-based DSL and EventMachine-powered page fetching
2019
2020
2021
2022
2023
2024
0.0
A demo of Web Crawler using arb-crawler
2019
2020
2021
2022
2023
2024
0.0
Web page crawler.
2019
2020
2021
2022
2023
2024
0.0
A periodic crawler that fetches the latest CVE additions, parses them, and filters them
2019
2020
2021
2022
2023
2024
0.0
Simple site crawler using Capybara
2019
2020
2021
2022
2023
2024
0.0
Simple Gem Using Watir For Phantom Crawler
2019
2020
2021
2022
2023
2024
0.0
web crawler that generates a sitemap to a neo4j database. It will also store broken_links and total number of pages on site
2019
2020
2021
2022
2023
2024
0.0
Crawler for downloading comics from komiks.gildia.pl
2019
2020
2021
2022
2023
2024
0.0
A simple web crawler gem
2019
2020
2021
2022
2023
2024