Project

ronin-web

0.02
No release in over 3 years
Low commit activity in last 3 years
There's a lot of open issues
Ronin Web is a Ruby library for Ronin that provides support for web scraping and spidering functionality. Ronin is a Ruby platform for exploit development and security research. Ronin allows for the rapid development and distribution of code, exploits or payloads over many common Source-Code-Management (SCM) systems.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
 Dependencies

Development

>= 2.3.3
>= 0.4.1
>= 1.2.8
= 1.2.3

Runtime

>= 0.9.3
>= 0.3.0
>= 0.9.4
>= 0.2.0
 Project Readme

ronin-web

CI Code Climate

Description

{Ronin::Web} is a Ruby library for Ronin that provides support for web scraping and spidering functionality.

Features

  • HTML/XML parsing/building (using Nokogiri).
  • Automated Web Browsing (using Mechanize)
  • Provides popular User Agent strings.
  • Integrates Spidr into {Ronin::Web::Spider}.
  • Provides {Ronin::Web::Server}, a Sinatra based Web Server.
  • Provides {Ronin::Web::Proxy}, a Sinatra based Web Proxy.

Synopsis

Start the Ronin console with Ronin Web preloaded:

$ ronin-web

Examples

Get a web-page:

Web.get('http://www.rubyinside.com/')

Get only the body of the web-page:

Web.get_body('http://www.rubyinside.com/')

Get a Mechanize agent:

agent = Web.agent

Parse HTML:

Web.html(open('some_file.html'))
# => <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
# <html>
#   <head>
#     <script type="text/javascript" src="redirect.js"></script>
#   </head>
# </html>

Build a HTML document:

doc = Web.build_html do
  html {
    head {
      script(:type => 'text/javascript', :src => 'redirect.js')
    }
  }
end

puts doc.to_html
# <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
# <html><head><script src="redirect.js" type="text/javascript"></script></head></html>

Parse XML:

Web.xml(some_text)
# => <?xml version="1.0"?>
# <users>
#   <user>
#     <name>admin</name>
#     <password>0mni</password>
#   </user>
# </users>

Build a XML document:

doc = Web.build_xml do
  playlist {
    mp3 {
      file { text('02 THE WAIT.mp3') }
      artist { text('Evil Nine') }
      track { text('The Wait feat David Autokratz') }
      duration { text('1000000000') }
    }
  }
end

puts doc.to_xml
# <?xml version="1.0"?>
# <playlist>
#   <mp3>
#     <file>02 THE WAIT.mp3</file>
#     <artist>Evil Nine</artist>
#     <track>The Wait feat David Autokratz</track>
#     <duration>1000000000</duration>
#   </mp3>
# </playlist>

Spider a web site:

Web::Spider.host('www.example.com') do |spider|
  spider.every_url do |url|
    # ...
  end

  spider.every_page do |page|
    # ...
  end
end

Serve files via a Web Server:

require 'ronin/web/server'

Web.server do
  file '/opensearch.xml', '/tmp/test.xml'
  directory '/downloads/', '/tmp/downloads/'
end

Web.server.get '/test' do
  'Test 1 2 1 2'
end

Requirements

Install

$ gem install ronin-web

Development

  1. Fork It!
  2. Clone It!
  3. cd ronin-web
  4. bundle install
  5. git checkout -b my_feature
  6. Code It!
  7. bundle exec rake spec
  8. git push origin my_feature

License

Ronin Web - A Ruby library for Ronin that provides support for web scraping and spidering functionality.

Copyright (c) 2006-2021 Hal Brodigan (postmodern.mod3 at gmail.com)

This file is part of ronin-web.

Ronin is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Ronin is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with Ronin. If not, see https://www.gnu.org/licenses/.