初级开发工程师，基于 http 写的爬虫扩展包。请不要随意下载里面有很多坑。

Retrieves a list of URLs to seed the crawler by publishing them to a RabbitMQ exchange.

Simple Web Crawler

Hushes worthless Rails exceptions & logs, such as those caused by bots and crawlers.

A simple crawler demo crawler

The Taiwan VSCinema crawler to get latest film list.

CrawlerDetect is a library to detect bots/crawlers via the user agent

validate-website is a web crawler for checking the markup validity with XML Schema / DTD and not found urls.

your friendly neighborhood web crawler

Empower World Travel Information Technology

Empower World Travel Information Technology

Empower World Travel Information Technology

Empower World Travel Information Technology

Empower World Travel Information Technology

a demo for study ruby

== Medusa: a ruby crawler framework {rdoc-image:https://badge.fury.io/rb/medusa-crawler.svg}[https://rubygems.org/gems/medusa-crawler] rdoc-image:https://github.com/brutuscat/medusa-crawler/workflows/Ruby/badge.svg?event=push Medusa is a framework for the ruby language to crawl and collect usefu...

A simple web crawler for ruby

Simple Gem Using Watir For Phantom Crawler

Generic Web crawler with a DSL that parses structured data from web pages

Post URLs to Wayback Machine (Internet Archive), using a crawler, from Sitemap(s) or a list of URLs.