S3FileHandler

S3FileHandler is a simple, powerful, and unified Ruby wrapper for Amazon S3 operations. Instead of dealing with complex AWS SDK error handling and verbose configurations, this gem provides a clean interface returning a standardized Result object for every operation.

Whether you need to manage buckets, handle virtual folders, stream files, or process CSVs directly in memory, S3FileHandler has you covered.

Requirements
Installation
Configuration
Usage
- 1. Bucket Operations
- 2. Folder Operations
- 3. File Operations
- 4. CSV Processors
- 5. Real-World Example: Image Upload
- 6. Real-World Example: CSV Automation Script
Contributing
License

Requirements

Ruby: >= 3.0.0
aws-sdk-s3: ~> 1.120
csv: ~> 3.2
rexml: ~> 3.2

Installation

Add this line to your application's Gemfile:

gem 's3_file_handler'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install s3_file_handler

Configuration

Before using the gem, you need to configure your AWS credentials. If you are using Ruby on Rails, you can place this in an initializer (e.g., config/initializers/s3_file_handler.rb).

S3FileHandler.configure do |config|
  config.access_key_id     = ENV['AWS_ACCESS_KEY_ID']
  config.secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
  config.region            = ENV['AWS_REGION'] # e.g., 'us-east-1'
end

Usage

All methods in S3FileHandler return a standardized Result object. You can always check if an operation was successful using .success?. If it fails, .errors will contain an array of error messages. If it succeeds, .data will contain a hash with the requested information.

1. Bucket Operations

# Create a Bucket
result = S3FileHandler.create_bucket('my-awesome-bucket')

# Check if a Bucket exists
exists = S3FileHandler.bucket_exists?('my-awesome-bucket') # => true or false

# List all Buckets
result = S3FileHandler.list_buckets
result.data[:buckets] # => ['bucket-1', 'bucket-2']

# Delete a Bucket
result = S3FileHandler.delete_bucket('my-awesome-bucket')
puts "Bucket deleted!" if result.success?

2. Folder Operations

Note: S3 is a flat storage system. "Folders" are simulated using prefixes and zero-byte marker objects.

# Create a Folder
result = S3FileHandler.create_folder('my-bucket', 'assets/images')
puts "Folder created!" if result.success?

# Check if a Folder exists
exists = S3FileHandler.folder_exists?('my-bucket', 'assets/images') # => true or false

# List Folders (Returns subfolders within a prefix)
result = S3FileHandler.list_folders('my-bucket', prefix_path: 'assets')
result.data[:folders] # => ['assets/images/', 'assets/videos/']

# Delete a Folder (Safe mode: fails if the folder is not empty)
result = S3FileHandler.delete_folder('my-bucket', 'assets/images')
puts result.errors unless result.success?

# Delete a Folder (Force mode: recursively deletes the folder and ALL its contents)
result = S3FileHandler.delete_folder('my-bucket', 'assets/images', force: true)
puts "Folder and contents deleted!" if result.success?

3. File Operations

You can upload files using a local file path or by passing raw bytes (memory content) directly. This makes it highly compatible with web frameworks like Rails for both HTML multipart uploads and API Base64 uploads.

# Upload from a Local File
result = S3FileHandler.upload_file('my-bucket', 'pets/pug.jpg', local_path: '/path/to/pug.jpg')

# Upload from Memory (Raw Content/Bytes)
result = S3FileHandler.upload_file('my-bucket', 'settings.json', content: '{"theme": "dark"}')
puts "File uploaded successfully!" if result.success?

# Read/Download a File
result = S3FileHandler.read_file('my-bucket', 'pets/pug.jpg')
if result.success?
  raw_bytes = result.data[:content]
  # You can save it to disk, send it via Rails `send_data`, or parse it!
end

# Check if a File exists
exists = S3FileHandler.file_exists?('my-bucket', 'settings.json') # => true or false

# Delete a File
result = S3FileHandler.delete_file('my-bucket', 'settings.json')
puts "File deleted!" if result.success?

4. CSV Processors

Modify CSV files directly without worrying about the AWS SDK. The gem handles downloading, parsing, and re-uploading the file behind the scenes.

# Create a new CSV with headers
result = S3FileHandler.create_csv('my-bucket', 'reports/users.csv', headers: ['id', 'name', 'email'])
puts "CSV created!" if result.success?

# Append a new row to an existing CSV
result = S3FileHandler.append_csv_row('my-bucket', 'reports/users.csv', row_data: ['1', 'John Doe', 'john_doe@example.com'])
puts "Row appended!" if result.success?

# Add a new column to an existing CSV (Fills existing rows with an empty string)
# Note: Fails if the column already exists.
result = S3FileHandler.add_csv_column('my-bucket', 'reports/users.csv', column_name: 'status')
puts result.success? ? "Column added!" : result.errors

5. Real-World Example: Image Upload

This complete example demonstrates the end-to-end flow for uploading an image, from creating the infrastructure on S3 to using it in an application.

Step 1: Create the Bucket and Folder

First, we ensure that our bucket and destination folder exist.

BUCKET_NAME = 'my-company-assets'
FOLDER_PATH = 'uploads/avatars'

# Create the bucket (if it doesn't already exist)
S3FileHandler.create_bucket(BUCKET_NAME) unless S3FileHandler.bucket_exists?(BUCKET_NAME)

# Create the virtual folder
S3FileHandler.create_folder(BUCKET_NAME, FOLDER_PATH)

Step 2: Upload the Image

You can upload from a local file or from data received in a request (for example, in a Rails API).

Option A: Upload from a local file

# Path to the image on your file system
local_image_path = '/path/to/user_avatar.png'
file_key = "#{FOLDER_PATH}/user_123_avatar.png"

result = S3FileHandler.upload_file(
  BUCKET_NAME,
  file_key,
  local_path: local_image_path
)

puts "Upload successful!" if result.success?

Option B: Upload from in-memory data (e.g., Rails API)

Suppose you received an image file in a Rails controller.

# params[:avatar] is an ActionDispatch::Http::UploadedFile object
uploaded_file = params[:avatar]
file_key = "#{FOLDER_PATH}/user_#{current_user.id}_avatar.png"
file_content = uploaded_file.read

result = S3FileHandler.upload_file(
  BUCKET_NAME,
  file_key,
  content: file_content,
)

if result.success?
  render json: { message: 'Avatar updated successfully!' }, status: :ok
else
  render json: { errors: result.errors }, status: :unprocessable_entity
end

Step 3: Use the Image

After the upload, you can read the file content to send it via an API or render it directly in an HTML view using Base64 encoding. This approach is highly recommended because it keeps your S3 bucket entirely private.

Usage in an API (serve the image bytes directly)

# In a controller, to serve the image privately without exposing any S3 URL
file_key = "#{FOLDER_PATH}/user_123_avatar.png"
result = S3FileHandler.read_file(BUCKET_NAME, file_key)

if result.success?
  # The gem returns the raw bytes in :content. 
  send_data result.data[:content], type: 'image/png', disposition: 'inline'
else
  head :not_found
end

Usage in a View (render directly via Base64)

Instead of making your bucket public to generate URLs, you can fetch the file securely and embed it straight into your HTML using Base64.

require 'base64'

# In your controller or helper:
file_key = "#{FOLDER_PATH}/user_123_avatar.png"
result = S3FileHandler.read_file(BUCKET_NAME, file_key)

if result.success?
  # Convert the raw binary content to a Base64 string
  @avatar_base64 = Base64.strict_encode64(result.data[:content])
end

And in your view (.html.erb):

<% if @avatar_base64 %>
  <img src="data:image/png;base64,<%= @avatar_base64 %>" alt="User Avatar">
<% else %>
  <p>Avatar not found.</p>
<% end %>

6. Real-World Example: CSV Automation Script

This example shows how to use the gem in a typical automation script: creating a daily sales report, updating it, and then reading the data to calculate totals.

Step 1: Initialize the Report

BUCKET = 'company-reports'
FILE_KEY = "sales/#{Time.now.strftime('%Y-%m-%d')}_vendas.csv"

# Create a new CSV with initial headers
S3FileHandler.create_csv(BUCKET, FILE_KEY, headers: ['id', 'product', 'price'])

Step 2: Update Data (Append Rows & Add Columns)

# Add a new sale
S3FileHandler.append_csv_row(BUCKET, FILE_KEY, row_data: ['101', 'Mechanical Keyboard', '150.00'])
S3FileHandler.append_csv_row(BUCKET, FILE_KEY, row_data: ['102', 'Gaming Mouse', '80.00'])

# Need to track delivery? Add a new column to all existing rows
S3FileHandler.add_csv_column(BUCKET, FILE_KEY, column_name: 'delivery_status')

Step 3: Read and Process in a Script

This is the most common use case for developers: fetching the data from S3 and using Ruby's native CSV library to perform calculations.

require 'csv'

# Fetch the file content
result = S3FileHandler.read_file(BUCKET, FILE_KEY)

if result.success?
  # The gem returns the raw string in :content. 
  # Simply pass it to CSV.parse with headers: true
  csv_data = CSV.parse(result.data[:content], headers: true)
  
  total_sales = 0
  
  csv_data.each do |row|
    puts "Processing Product: #{row['product']} - Price: $#{row['price']}"
    total_sales += row['price'].to_f
  end
  
  puts "Total Sales Today: $#{total_sales}"
else
  puts "Failed to read report: #{result.errors.join(', ')}"
end

Contributing

Bug reports and pull requests are welcome on GitHub. This project is intended to be a safe, welcoming space for collaboration.

License

The gem is available as open source under the terms of the MIT License.

s3_file_handler

Runtime