FF1 Format Preserving Encryption Gem
A Ruby implementation of the FF1 Format Preserving Encryption algorithm from NIST SP 800-38G.
Overview
Format Preserving Encryption (FPE) allows you to encrypt data while maintaining its original format. This is particularly useful when you need to encrypt sensitive data like credit card numbers, social security numbers, or other structured data while keeping the same format for compatibility with existing systems.
The FF1 algorithm is one of two methods specified in NIST Special Publication 800-38G for Format-Preserving Encryption.
Features
- Full implementation of NIST FF1 algorithm
- Dual-mode operation: Reversible and Irreversible encryption
- Support for any radix from 2 to 65, 536
- Full UTF-8 text encryption support - encrypt arbitrary text while maintaining FF1 security properties
- Rails/ActiveRecord integration - seamless database encryption with GDPR compliance
-
Simplified soft delete - Uses single
deleted_at
timestamp column for tracking - Tweak support for additional security
- GDPR "right to be forgotten" compliance with soft delete
- Automatic encryption/decryption in Rails models
- Rails migration generators and configuration
- Query scopes for active/deleted records
- Restore functionality for soft-deleted records
- Proper input validation and error handling
- Comprehensive test suite
- Thread-safe implementation
Installation
Add this line to your application's Gemfile:
gem 'ff1'
And then execute:
$ bundle install
Or install it yourself as:
$ gem install ff1
Usage
Basic Usage
require 'ff1'
# Create a cipher with a 128-bit key for decimal numbers (radix 10)
key = "\x2B\x7E\x15\x16\x28\xAE\xD2\xA6\xAB\xF7\x15\x88\x09\xCF\x4F\x3C"
# Reversible mode (default) - can decrypt back to original
cipher = FF1::Cipher.new(key, 10, FF1::Modes::REVERSIBLE)
plaintext = "4111111111111111"
ciphertext = cipher.encrypt(plaintext)
decrypted = cipher.decrypt(ciphertext)
puts "#{plaintext} → #{ciphertext} → #{decrypted}"
# => "4111111111111111 → 8224999410799188 → 4111111111111111"
# Irreversible mode - cannot decrypt (for GDPR compliance)
secure_cipher = FF1::Cipher.new(key, 10, FF1::Modes::IRREVERSIBLE)
secure_ciphertext = secure_cipher.encrypt(plaintext)
# secure_cipher.decrypt(secure_ciphertext) # ❌ Raises error
puts "Securely deleted: #{plaintext} → #{secure_ciphertext}"
# => "Securely deleted: 4111111111111111 → 2858907702179518"
Using Tweaks
Tweaks provide additional input to the encryption algorithm for enhanced security:
cipher = FF1::Cipher.new(key, 10)
tweak = "user123"
plaintext = "1234567890"
ciphertext = cipher.encrypt(plaintext, tweak)
decrypted = cipher.decrypt(ciphertext, tweak)
Different Radix Values
# For hexadecimal data (radix 16)
hex_cipher = FF1::Cipher.new(key, 16)
hex_data = "ABCDEF123456"
encrypted_hex = hex_cipher.encrypt(hex_data)
# For binary data (radix 2)
binary_cipher = FF1::Cipher.new(key, 2)
binary_data = "1010101" # Must be long enough to meet domain requirements
encrypted_binary = binary_cipher.encrypt(binary_data)
Text Encryption (NEW!)
The gem now supports encrypting arbitrary UTF-8 text while maintaining security properties:
cipher = FF1::Cipher.new(key, 10)
# Encrypt any text including Unicode and special characters
text = "Hello, World! 🌍 Special chars: @#$%^&*()"
encrypted_text = cipher.encrypt_text(text)
decrypted_text = cipher.decrypt_text(encrypted_text)
puts encrypted_text # Returns base64-encoded encrypted data
# => "SGVsbG8sIFdvcmxkISDwn42NIFNwZWNpYWwgY2hhcnM6IEAjJCVeJiooKQ=="
# Text with tweak for additional security
context = "user_session_123"
encrypted_with_context = cipher.encrypt_text(text, context)
decrypted_with_context = cipher.decrypt_text(encrypted_with_context, context)
# Irreversible text encryption for GDPR compliance
irreversible_cipher = FF1::Cipher.new(key, 10, FF1::Modes::IRREVERSIBLE)
sensitive_data = "Personal information to be securely deleted"
irreversibly_encrypted = irreversible_cipher.encrypt_text(sensitive_data)
# Cannot decrypt - data is permanently transformed while maintaining consistency
ActiveRecord Integration (NEW!)
The gem now includes full Rails/ActiveRecord integration for seamless database encryption with GDPR compliance features.
Installation for Rails
Add the gem to your Gemfile:
gem 'ff1'
Run the generator to create the configuration file:
rails generate ff1:install
This creates config/initializers/ff1.rb
with configuration options.
Configuration
Configure FF1 in your Rails initializer:
# config/initializers/ff1.rb
FF1::ActiveRecord.configure do |config|
# Required: Set your encryption key (store securely!)
config.global_key = Rails.application.credentials.ff1_encryption_key
# Or use environment variable:
# config.global_key = ENV['FF1_ENCRYPTION_KEY']&.b
# Optional: Set default encryption mode
config.default_mode = FF1::Modes::REVERSIBLE
# Optional: Customize column name for soft delete tracking
config.deleted_at_column = :deleted_at
end
Basic Model Integration
Include FF1::ActiveRecord
in your models and configure encrypted columns:
class User < ApplicationRecord
include FF1::ActiveRecord
# Encrypt email and phone with reversible encryption (can decrypt)
ff1_encrypt :email, :phone, mode: :reversible
# Encrypt sensitive data with irreversible encryption (cannot decrypt - GDPR compliant)
ff1_encrypt :ssn, :address, :full_name, mode: :irreversible
# Optional: Define scope for active users (alternative to built-in ff1_active)
scope :active, -> { where(deleted_at: nil) }
end
Database Migration
Generate and run migration to add soft delete columns:
rails generate ff1:install User
rails db:migrate
This adds a deleted_at
timestamp column to your users table for tracking soft deletes.
Usage Examples
# Create user - data is automatically encrypted before saving
user = User.create!(
name: 'John Doe',
email: 'john@example.com',
phone: '555-1234',
ssn: '123-45-6789'
)
# Access data - reversible columns decrypt automatically
puts user.email # => 'john@example.com' (decrypted)
puts user.phone # => '555-1234' (decrypted)
puts user.ssn # => '[ENCRYPTED]' (irreversible, cannot decrypt)
# Data is encrypted in the database
User.connection.select_value("SELECT email FROM users WHERE id = #{user.id}")
# => "8224999410799188" (encrypted format)
GDPR Compliant Soft Delete
The integration provides GDPR-compliant "right to be forgotten" functionality:
# Soft delete with irreversible encryption
user.destroy # Encrypts all sensitive data irreversibly, keeps record
# Check deletion status
user.ff1_deleted? # => true (checks if deleted_at is set)
user.ff1_deleted_at # => 2023-01-01 12:00:00 UTC
# Data is now irreversibly encrypted
user.email # => '[ENCRYPTED]'
user.phone # => '[ENCRYPTED]'
user.ssn # => '[ENCRYPTED]'
# Restore record (but data remains encrypted)
user.ff1_restore
user.ff1_deleted? # => false (deleted_at is cleared)
# Find deleted users and restore them
deleted_user = User.ff1_deleted.first
deleted_user.ff1_restore
# True hard delete if needed
user.ff1_hard_destroy!
Query Scopes
Use built-in scopes to query active vs soft-deleted records:
# Active (non-deleted) users only
User.ff1_active.count # => 150
User.ff1_active.where(...) # Chain with other scopes
# Soft-deleted users only
User.ff1_deleted.count # => 25
# All users (active + deleted)
User.ff1_all.count # => 175
# Recently deleted users
User.ff1_recently_deleted(within: 30.days)
# Old deleted users (for purging)
User.ff1_old_deleted(older_than: 1.year)
Bulk Operations
# Bulk soft delete with encryption
User.ff1_destroy_all(status: 'inactive') # Returns count of deleted records
# Purge old soft-deleted records
User.ff1_purge_deleted(older_than: 2.years) # Permanent deletion
# Batch encrypt existing data
User.ff1_encrypt_existing_data!(columns: [:email, :phone])
# Statistics
User.ff1_stats
# => {
# total_records: 175,
# active_records: 150,
# deleted_records: 25,
# encrypted_columns: [:email, :phone, :ssn],
# deletion_rate: 14.29
# }
Advanced Configuration
class User < ApplicationRecord
include FF1::ActiveRecord
# Per-column configuration
ff1_encrypt :email,
mode: :reversible,
key: Rails.application.credentials.user_email_key, # Custom key
radix: 256 # For text data
ff1_encrypt :credit_card_number,
mode: :reversible,
radix: 10 # For numeric data
ff1_encrypt :sensitive_notes,
mode: :irreversible, # Cannot be decrypted
radix: 256
end
Thread Safety
The ActiveRecord integration is thread-safe and caches cipher instances for performance:
# Safe for concurrent access
User.transaction do
User.create!(email: 'user1@example.com')
User.create!(email: 'user2@example.com')
end
Performance Considerations
- Cipher Caching: Ciphers are cached per column configuration for performance
-
Bulk Operations: Use
ff1_find_each
for processing large datasets - Indexing: Encrypted data cannot be indexed for searching - plan accordingly
- Query Performance: Use the provided scopes which include proper indexes
GDPR Compliance Features
The ActiveRecord integration specifically supports GDPR requirements:
-
Right to be Forgotten:
destroy
method irreversibly encrypts sensitive data - Data Minimization: Only configured columns are encrypted
- Audit Trail: Soft delete maintains record for compliance tracking
- Data Portability: Reversible encryption allows data export when legally required
Troubleshooting
Migration Issues
If you encounter errors during migration generation, ensure you're running the correct commands:
# Generate configuration (run once)
rails generate ff1:install
# Generate migration for specific models
rails generate ff1:install User Post Comment
rails db:migrate
Stack Overflow on Destroy
If you encounter "stack level too deep" errors when calling destroy
, ensure your migration has been run and the deleted_at
column exists:
# Check if column exists
User.column_names.include?('deleted_at') # Should return true
# If missing, run the migration
rails generate ff1:install User
rails db:migrate
Restoring Deleted Records
Remember that irreversibly encrypted data cannot be recovered:
# This will restore the record but encrypted data stays encrypted
deleted_user = User.ff1_deleted.first
deleted_user.ff1_restore
# Reversible data might be recoverable if it was encrypted before deletion
puts deleted_user.email # May show original data or '[ENCRYPTED]'
Requirements
Ruby Version:
- Ruby 3.2.0 or higher
Dependencies:
- ActiveRecord ~> 7.0 (for Rails integration)
Key Requirements
The FF1 algorithm supports AES key lengths:
- 128-bit keys (16 bytes)
- 192-bit keys (24 bytes)
- 256-bit keys (32 bytes)
# 128-bit key
key_128 = SecureRandom.bytes(16)
cipher_128 = FF1::Cipher.new(key_128, 10)
# 256-bit key
key_256 = SecureRandom.bytes(32)
cipher_256 = FF1::Cipher.new(key_256, 10)
Domain Size Requirements
For security, the FF1 algorithm requires that radix^length >= 100
. For better security, radix^length >= 1,000,000
is recommended.
Examples:
- Decimal (radix 10): minimum 2 digits, recommended 7+ digits
- Hexadecimal (radix 16): minimum 2 digits, recommended 5+ digits
- Binary (radix 2): minimum 7 bits, recommended 20+ bits
Error Handling
The gem provides comprehensive error handling:
begin
cipher = FF1::Cipher.new(key, 10)
result = cipher.encrypt("123") # Too short
rescue FF1::Error => e
puts "Encryption error: #{e.message}"
end
Common errors:
-
FF1::Error
: Base error class for all FF1-related errors - Invalid key length
- Invalid radix (must be 2-65536)
- Input too short or empty
- Invalid characters for the specified radix
- Domain size too small
Security Considerations
- Key Management: Use cryptographically secure random keys and protect them appropriately
- Domain Size: Ensure your domain size meets the minimum requirements (preferably >= 1,000,000)
- Tweaks: Use tweaks when possible for additional security
- Input Validation: The gem validates inputs, but ensure your data meets the requirements
Algorithm Details
This implementation follows NIST SP 800-38G specification for the FF1 algorithm:
- Uses 10 rounds of a Feistel network
- Supports any radix from 2 to 65, 536
- Uses AES as the underlying block cipher
- Implements proper padding and round function as specified
Thread Safety
The FF1:: Cipher instances are thread-safe for encryption and decryption operations. However, you should not modify the cipher instance (key, radix) from multiple threads simultaneously.
Changelog
Version 1.2.6+
-
BREAKING CHANGE: Simplified soft delete to use only
deleted_at
column - Removed
ff1_deleted
boolean column requirement - Fixed "stack level too deep" error in destroy method
- Updated all scopes to work with
deleted_at
timestamp only - Improved migration generator to create correct class names
- Updated
ff1_deleted?
method to checkdeleted_at.present?
- Enhanced restore functionality and documentation
Previous Versions
- Added Rails/ActiveRecord integration
- Added GDPR compliance features
- Added text encryption support
- Initial FF1 algorithm implementation
Contributing
Bug reports and pull requests are welcome on GitHub.
License
The gem is available as open source under the MIT License.