Project

s3x

0.0
The project is in a healthy, maintained state
Scrape public AWS S3 buckets with ease.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Dependencies
 Project Readme

S3x

Gem Version Test Coverage Status

Found something spicy? Want to check it out? Here is the tool for ya 😎

Zero dependencies, no AWS SDK bloat, pure Ruby.

Installation

Install the gem and add to Gemfile:

bundle add s3x

Or install it manually:

gem install s3x

Configuration

Initialize a bucket of interest:

bucket = S3x::Bucket.new("http://ftp.ruby-lang.org/")

You can set prefix to pre-filter items and/or override default page_size:

bucket = S3x::Bucket.new(
  "http://ftp.ruby-lang.org/",
  prefix: "pub/ruby/binaries", # default: nil
  page_size: 666               # default: 1000
)

Usage

Get first X (page_size) items:

bucket.items
# => [{
#   key:           "pub/media/irb_multiline.mp4",
#   etag:          "03fa58ca64375c23cb567be6b129ab11",
#   size:          239988,
#   storage_class: "INTELLIGENT_TIERING",
#   last_modified: 2019-04-20 09:10:30 UTC
# }, ...]

Get next page items:

bucket.next_items
# => [{...}, ...]

Check if the next page actually exists:

bucket.next_items?
# => true/false

Get all items in one take:

bucket.all_items
# => [{...}, ...]

Download selected item using its key:

bucket.download("pub/misc/ci_versions/cruby-jruby.json")
# => "[\"3.1\",\"3.2\",\"3.3\",\"head\",\"jruby\",\"jruby-head\"]\n"

Return bucket name:

bucket.name
# => "ftp.r-l.o"

Development

bin/setup         # install deps
bin/console       # interactive prompt to play around
rake spec         # run tests!
rake spec:no_vcr  # run tests with VCR cassettes disabled!
rake rubocop      # lint code!
rake rubocop:md   # lint docs!

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/ocvit/s3x.

License

The gem is available as open source under the terms of the MIT License.