Empower World Travel Information Technology

Empower World Travel Information Technology

Empower World Travel Information Technology

Fake User-Agents of about %80 of real devices to use in headers of web crawlers. It keeps your script away from being nested by many UA strings.

s7o crawler optimized for programmer happiness and sustainable productivity.

This gem crawls the latest CircleCI artifact file you specified. For Example, you can get the result JSON of simplecov.gem etc.

Hushes worthless Rails exceptions & logs, such as those caused by bots and crawlers.

This gem helps Crawler Writers to interact with the PromoQui REST API

Using paperclip to generate images from sensible attributes like e-mails and telephone numbers, in order to reduce crawler's success

Iudex is a general purpose web crawler and feed processor in ruby/java. This gem is a Jetty HTTP Client based implementation of the iudex-http interfaces.

book's price crawler

Discovery Mission is an easy-to-use website crawler. Use it for generating sitemaps.

A simple crawler that gets posts and pages from wordpress websites that have an exposed api

A crawler for a single domain web application

A ruby stock price & option chain crawler

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-filter gem contains a fundamental filtering/chain of responsbility sub-system.

Lightweight web crawler

Iudex is a general purpose web crawler and feed processor in ruby/java. The iudex-http gem contains and http client agnostic abstraction layer.

Headless HTTP crawler/scraper

A crawler and parser to the Currículo Lattes.