0.0
The Baidu Crawler is to crawl data with your demmand
2019
2020
2021
2022
2023
2024
0.0
A simple web crawler gem
2019
2020
2021
2022
2023
2024
0.0
Retrieves a list of URLs to seed the crawler by publishing them to a RabbitMQ exchange.
2019
2020
2021
2022
2023
2024
0.0
a simple web crawler using DSL of capybara
2019
2020
2021
2022
2023
2024
0.0
A simple web crawler gem
2019
2020
2021
2022
2023
2024
0.0
it starts a crawler for Middleman sites
2019
2020
2021
2022
2023
2024
0.0
Web crawler to get courses from XML pages at mymav.uta
2019
2020
2021
2022
2023
2024
0.0
A flexible, modular web crawler
2019
2020
2021
2022
2023
2024
0.0
Generic Web crawler with a DSL that parses event-related data from web pages
2019
2020
2021
2022
2023
2024
0.0
Easy way to enable AdSense crawler to login and see private or custom pages in your rails application. Basically one custom login filter. Gem enables you to easily slightly increase revenues from Google AdSense/AdWords. It makes it easy to enable crawling on private pages and so get better target...
2019
2020
2021
2022
2023
2024
0.08
Crawl instagram photos, posts and videos for download.
2019
2020
2021
2022
2023
2024
0.08
Asynchronous web crawler, scraper and file harvester
2019
2020
2021
2022
2023
2024
0.07
An easy to use distributed web-crawler framework based on Redis
2019
2020
2021
2022
2023
2024
0.03
Arachnid is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.
2019
2020
2021
2022
2023
2024
0.03
Post URLs to Wayback Machine (Internet Archive), using a crawler, from Sitemap(s) or a list of URLs.
2019
2020
2021
2022
2023
2024
0.03
Rack Middleware adhering to the Google Ajax Crawling Scheme, using a headless browser to render JS heavy pages and serve a dom snapshot of the rendered state to a requesting search engine.
2019
2020
2021
2022
2023
2024
0.03
validate-website is a web crawler for checking the markup validity with XML Schema / DTD and not found urls.
2019
2020
2021
2022
2023
2024
0.02
Ruby web crawler using PhantomJS
2019
2020
2021
2022
2023
2024
0.02
Cosmicrawler is crawler library for Ruby. It provides scalable asynchronous crawling by (http|file|etc) using EventMachine.
2019
2020
2021
2022
2023
2024
0.01
A web scrawler to get a Marmiton's recipe
2019
2020
2021
2022
2023
2024