Crawl all pages on a domain, checking for errors

Super DRY Sitemaps for Rails and Sinatra Apps

Write a longer description or delete this line.

BFS webcrawler that implements Observable

Crawler Engine provides function of crawl all news from the customized website

This rubygem does not have a description or summary.

a crawler toolkit

This gem is a web crawler sample code.So I don't reccmmend that you use.

Crawls a page, with no limits and without visiting external domains

Crawls and scrapes search engine results pages.

Crawlr is a modern Ruby web scraping framework built for respectful and efficient data extraction. Features async HTTP requests, robots.txt compliance, cookie management, proxy rotation, flexible CSS/XPath selectors, and comprehensive error handling. Designed for both simple scraping tasks and co...

A small Ruby gem for sitemap-driven SEO validation with structured issues, configurable rules and schema registries, optional browser rendering, and Rails rake task integration.

Crayfish - PDF templating for Rails

CrazyHarry is a high-level html fragment sanitiser/cleaner based on Loofah.

Provides React integration with Cuba

Retrieve temporary AWS credentials via an identity provider.

A Ruby gem that streams and parses large Excel(xlsx and xlsm) files fast and efficiently. Based on Creek gem, but with Multi-threading and Garbage Collection

Creepin so logically

web crawler that generates a sitemap to a neo4j database. It will also store broken_links and total number of pages on site

Ruby client for the Creditreform API.