Categories
No matching categories were found
0.01
== Medusa: a ruby crawler framework {rdoc-image:https://badge.fury.io/rb/medusa-crawler.svg}[https://rubygems.org/gems/medusa-crawler] rdoc-image:https://github.com/brutuscat/medusa-crawler/workflows/Ruby/badge.svg?event=push
Medusa is a framework for the ruby language to crawl and collect usefu...
2019
2020
2021
2022
2023
2024
0.13
Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites which is much more performant than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.
2019
2020
2021
2022
2023
2024
0.55
Generic Web crawler with a DSL that parses structured data from web pages
2019
2020
2021
2022
2023
2024
0.07
CrawlerDetect is a library to detect bots/crawlers via the user agent
2019
2020
2021
2022
2023
2024
0.0
Crawler Engine provides function of crawl all news from the customized website
2019
2020
2021
2022
2023
2024
0.0
A crappy crawler for a crappy bank interface
2019
2020
2021
2022
2023
2024
0.0
Crawler4J filter plugin for Embulk
2019
2020
2021
2022
2023
2024
0.26
Voight-Kampff detects bots, spiders, crawlers and replicants
2019
2020
2021
2022
2023
2024
0.02
is_crawler does exactly what you might think it does: determine if the supplied string matches a known crawler or bot.
2019
2020
2021
2022
2023
2024
0.0
A simple crawler demo crawler
2019
2020
2021
2022
2023
2024
0.0
Web crawler help you with parse and collect data from the web
2019
2020
2021
2022
2023
2024
0.0
A periodic crawler that fetches the latest CVE additions, parses them, and filters them
2019
2020
2021
2022
2023
2024
0.0
A simple web crawler gem
2019
2020
2021
2022
2023
2024
0.0
A generic web crawler that doesn't crawl outside URLs.
2019
2020
2021
2022
2023
2024
0.0
A simple web crawler gem
2019
2020
2021
2022
2023
2024
0.0
Jobs crawler
2019
2020
2021
2022
2023
2024
0.0
Simple Gem Using Watir For Phantom Crawler
2019
2020
2021
2022
2023
2024
0.0
a crawler toolkit
2019
2020
2021
2022
2023
2024
0.0
This crawler will use my personnal scraper named 'RecipeScraper' to dowload recipes data from Marmiton, 750g or cuisineaz
2019
2020
2021
2022
2023
2024
0.0
A simple web crawler gem
2019
2020
2021
2022
2023
2024