0.0
crawler of niconico
2020
2021
2022
2023
2024
2025
0.0
a crawler toolkit
2020
2021
2022
2023
2024
2025
0.0
Crawler Guru provides all basic functionalities to extract data from web pages
2020
2021
2022
2023
2024
2025
0.0
Web crawler to get courses from XML pages at mymav.uta
2020
2021
2022
2023
2024
2025
0.0
Simple Gem Using Watir For Phantom Crawler
2020
2021
2022
2023
2024
2025
0.0
Easy way to enable AdSense crawler to login and see private or custom pages in your rails application. Basically one custom login filter. Gem enables you to easily slightly increase revenues from Google AdSense/AdWords. It makes it easy to enable crawling on private pages and so get better target...
2020
2021
2022
2023
2024
2025
0.0
Simple ruby web crawler
2020
2021
2022
2023
2024
2025
0.0
A simple web crawler gem
2020
2021
2022
2023
2024
2025
0.0
A very simple crawler for RubyGems.org used to demo the power of ElasticSearch at RubyConf 2013
2020
2021
2022
2023
2024
2025
0.0
Crawler for http://legendas.tv to see the most dowloaded subtitles
2020
2021
2022
2023
2024
2025
0.06
Crawl instagram photos, posts and videos for download.
2020
2021
2022
2023
2024
2025
0.06
Asynchronous web crawler, scraper and file harvester
2020
2021
2022
2023
2024
2025
0.05
An easy to use distributed web-crawler framework based on Redis
2020
2021
2022
2023
2024
2025
0.02
Ruby web crawler using PhantomJS
2020
2021
2022
2023
2024
2025
0.02
Arachnid is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.
2020
2021
2022
2023
2024
2025
0.02
validate-website is a web crawler for checking the markup validity with XML Schema / DTD and not found urls.
2020
2021
2022
2023
2024
2025
0.02
Rack Middleware adhering to the Google Ajax Crawling Scheme, using a headless browser to render JS heavy pages and serve a dom snapshot of the rendered state to a requesting search engine.
2020
2021
2022
2023
2024
2025
0.02
Post URLs to Wayback Machine (Internet Archive), using a crawler, from Sitemap(s) or a list of URLs.
2020
2021
2022
2023
2024
2025
0.01
A simple, fast web crawler
2020
2021
2022
2023
2024
2025
0.01
render_static allows you to make your single-page apps (Backbone, Angular, etc) built on Rails SEO-friendly. It works by injecting a small rack middleware that will render pages as plain html, when the requester is one of the most common crawlers/bots out there (Google, Yahoo Baidu and Bing)
2020
2021
2022
2023
2024
2025