Project

trawler

0.0
No commit activity in last 3 years
No release in over 3 years
Scrapes the web
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Dependencies

Development

~> 1.3
>= 0
~> 2.14
~> 0.7
~> 2.6
~> 1.14

Runtime

~> 0.0.22
~> 1.6
 Project Readme

Code Climate

Trawler

Trawl the web for meta tags.

Installation

Add this line to your application's Gemfile:

gem 'trawler'

And then execute:

$ bundle

Or install it yourself as:

$ gem install trawler

Usage

Trawler is designed to be simple to use. Give the Trawler a URL and some options and it will bring back a catch of Images, Meta Descriptions and Video URLS in a ParsedDocument object.

  doc = Trawler.fetch("www.foobar.com") 
  doc.url         # => "http://www.foobar.com"
  doc.description # => "Descriptive meta tag"
  doc.title       # => "Foobar.com The place of Foo"
  doc.images      # => [
                  #      http://www.foobar.com/assets/bar.png,
                  #      http://www.foobar.com/assets/baz.jpg,
                  #      http://www.foobar.com/assets/bat.gif
                  #     ]
  doc.video       # => [
                  #      http://www.vimeo.com/354357349
                  #     ]

Testing

 rspec/

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request