petl 
Pretty good ETL framework
Features
- Batching support
- Automatic validity check
- Logging of running times
Installation
Add this line to your application's Gemfile:
gem 'petl'
And then execute:
$ bundle
Or install it yourself as:
$ gem install petl
Usage
require 'petl'
module ETL::Example
extend Petl
extend self
def extract
# Grab all data from source(s) here.
# Perferrably return an array of hashes.
end
def transform rows
# Manipulate the data extracted by the previous extract method.
end
def load rows
# Load the transformed data here into the destination(s).
end
def source_count
# Count the number of records from your source(s).
end
def destination_count
# Same as #source_count but with your destination(s).
end
end
# Run it!
ETL::Example.performContributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Add some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request