Web crawler to get courses from XML pages at mymav.uta

A flexible, modular web crawler

This crawler will use my personnal scraper named 'RecipeScraper' to dowload recipes data from Marmiton, 750g or cuisineaz

A demo of Web Crawler using arb-crawler

Stupid crawler that looks for URLs on a given site. Result is saved as two CSV files one with found URLs and another with failed URLs.

A crappy crawler for crashlytics

Web crawler help you with parse and collect data from the web

Jobs crawler

A periodic crawler that fetches the latest CVE additions, parses them, and filters them

Simple Gem Using Watir For Phantom Crawler

Easy way to enable AdSense crawler to login and see private or custom pages in your rails application. Basically one custom login filter. Gem enables you to easily slightly increase revenues from Google AdSense/AdWords. It makes it easy to enable crawling on private pages and so get better target...

MurmuringSpider is a concise Twitter crawler. When we write a data-mining / text-mining application based on twitter timeline, we have to collect and store tweets first. I am irritated with writing such crawler repeatedly, so I wrote this. What you have to do is only to add query and to run th...

Simple site crawler using Capybara

Crawler Guru provides all basic functionalities to extract data from web pages

A simple web crawler gem

A generic web crawler that doesn't crawl outside URLs.

A simple news crawler. You can specify the structure of your xml or rss feeds.

Retrieves a list of URLs to seed the crawler by publishing them to a RabbitMQ exchange.

it starts a crawler for Middleman sites

crawler of niconico