Discovery Mission¶ ↑

A SPACE ODYSSEY

This is just a simple, basic (dummy?) and lightweight crawler. It is used to reference all URLs of your website in order to record them on a database, and use them for a sitemap manager.

Just install it:

# sudo gem install discovery-mission

And then you’re ready to take off:

require ‘discovery_mission’

# Use a block to do something with each path # response is a ‘net/http’ response object, therefore responds to :body, :code …

DiscoveryMission.for(“www.my-domain.com”) do |url, response| puts “Dave Bowman landed on #{url}” case response when Net::HTTPSuccess puts “Fuck you HAL 9000” when Net::HTTPRedirection puts “Shit! That’s Jupiter!” else puts “Bad trip indeed” end end

# Or just without a block because an Array with all paths is returned

my_sitemap_entries = DiscoveryMission.for(“www.my-domain.com”)