An easy to use utility to scrape websites using a DSL similar to rake.

A command to scrape page title and convert it to Markdown link format.

Clients contains instruments to work with websites and local services.

Utilities to help make planningalerts scrapers, especially multi authority scrapers, easier to develop, run and debug.

Scrapetor is a Ruby HTML parsing + scraping toolkit. The parser is a native C arena DOM with structural indexes built at parse time and NEON SIMD scanners in the SAX hot loop. A streaming extraction engine compiles the schema DSL into a single forward pass — no DOM materialised, one Ruby boundary crossing per document. On builds where libcurl is available, Scrapetor::Fetcher adds an HTTP/2-capable fetch layer with per-thread connection cache, shared DNS + TLS session pool, in-process gzip / deflate / brotli / zstd decoding, iconv charset transcoding, retry + exponential backoff, ETag / Last-Modified disk cache with bulk revalidation, per-host throttle, cookie jar, basic + bearer auth, proxy, and three bulk concurrency models (parallel_fetch / multi_fetch / streaming multi_each). Scrapetor::Session ties the cookie / auth / throttle / retry policies together. Also ships robots.txt + sitemap.xml parsers, a bounded-memory streaming HTML parser, and structured-data extractors (JSON-LD, OpenGraph, Schema.org, Microdata, RDFa, Twitter Cards). The Net::HTTP-based Scrapetor.fetch is preserved as the no-libcurl fallback.

Framework for mapping ruby objects to web resources,which you can then manipulate.

A scraper which will output every and every link it finds on a page, to the console.

Interactive and declarative XPath driven HTML scraper

Through scrapping and standard get/post calls we enable simple integractions with websites that host companys information for BOVESPA and enable access to it's webservice. For the webservice you will need a registry, which can be done in this website http://blog.cedrotech.com/como-integrar-o-market-data-bmfbovespa-via-servicos-rest-xml-ou-json/ .

Extract structured data from HTML/XML files.

Rack middleware to relocate JavaScript tags to the end of an HTML response body. Only handles non-streaming `text/html` reponses.

Rip shell scripts out of instructional web pages

Crawling framework

Styleguide generator, prototype and test tool powered by Middleman

Secure Device Event Exchange (SDEE) is a simple HTTP-based protocol used by security appliances to exchange events and alerts. Results are returned in XML. This is a very bare-bones ruby implementation to get SDEE events from a Cisco IPS in JSON format.

Listing of events happening in San Diego today

Find information about all the animals listed on the San Diego Zoo website, including life span, conservation status, taxonomic information and a fun fact! The app/author is not affiliated with the SD zoo.

Seagull makes managing XCode projects easy as flying is for seagulls

A simple CLI Gem that provides information about various sea creatures.

Shared library to search through the GeoServer by Range, Tier and Section