0.0
Generic Web crawler with a DSL that parses event-related data from web pages
2021
2022
2023
2024
2025
2026
0.0
Web crawler to get courses from XML pages at mymav.uta
2021
2022
2023
2024
2025
2026
0.0
Jobs crawler
2021
2022
2023
2024
2025
2026
0.0
A simple web crawler gem
2021
2022
2023
2024
2025
2026
0.0
Web crawler with JSON-based DSL and EventMachine-powered page fetching
2021
2022
2023
2024
2025
2026
0.0
Simple site crawler using Capybara
2021
2022
2023
2024
2025
2026
0.0
A demo of Web Crawler using arb-crawler
2021
2022
2023
2024
2025
2026
0.0
Web page crawler.
2021
2022
2023
2024
2025
2026
0.0
A periodic crawler that fetches the latest CVE additions, parses them, and filters them
2021
2022
2023
2024
2025
2026
0.0
Easy way to enable AdSense crawler to login and see private or custom pages in your rails application. Basically one custom login filter. Gem enables you to easily slightly increase revenues from Google AdSense/AdWords. It makes it easy to enable crawling on private pages and so get better target...
2021
2022
2023
2024
2025
2026
0.0
The Baidu Crawler is to crawl data with your demmand
2021
2022
2023
2024
2025
2026
0.0
A generic web crawler that doesn't crawl outside URLs.
2021
2022
2023
2024
2025
2026
0.05
Asynchronous web crawler, scraper and file harvester
2021
2022
2023
2024
2025
2026
0.05
Crawl instagram photos, posts and videos for download.
2021
2022
2023
2024
2025
2026
0.05
An easy to use distributed web-crawler framework based on Redis
2021
2022
2023
2024
2025
2026
0.02
Arachnid is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.
2021
2022
2023
2024
2025
2026
0.02
validate-website is a web crawler for checking the markup validity with XML Schema / DTD and not found urls.
2021
2022
2023
2024
2025
2026
0.02
Rack Middleware adhering to the Google Ajax Crawling Scheme, using a headless browser to render JS heavy pages and serve a dom snapshot of the rendered state to a requesting search engine.
2021
2022
2023
2024
2025
2026
0.02
Post URLs to Wayback Machine (Internet Archive), using a crawler, from Sitemap(s) or a list of URLs.
2021
2022
2023
2024
2025
2026
0.02
Ruby web crawler using PhantomJS
2021
2022
2023
2024
2025
2026