LeanPool

A lightweight, process-free resource pool for Ruby using concurrent-ruby. Inspired by Elixir's nimble_pool, LeanPool provides efficient resource management without per-resource processes, making it perfect for managing sockets, HTTP connections, ports, and other resources.

Author: Junaid Farooq | Repository: half-blood-labs/lean_pool

• 🚀 Zero Overhead: Direct resource access without data copying between processes
• 🔒 Thread-Safe: Built on concurrent-ruby for reliable thread safety
• ⚡ Efficient: Lazy initialization and smart resource reuse
• 🎯 Simple API: Clean, Ruby-idiomatic interface
• 🔧 Flexible: Works with any resource type (sockets, connections, ports, etc.)
• 🎛️ Resource Selection Strategies: FIFO, LIFO, Random, and LRU (Least Recently Used)
• ⚡ Priority-Based Checkout: High-priority requests get resources first
• ♻️ Resource Recycling: Automatic resource replacement based on usage count or age
• 📡 Event Callbacks: Monitor pool lifecycle with before/after checkout, resource creation, etc.

Add this line to your application's Gemfile:

gem 'lean_pool'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install lean_pool

See examples/real_world_use_cases.rb for comprehensive examples showing:

• Database connection pooling for web applications
• HTTP API client pooling for microservices
• File handle pooling for log processing
• Redis connection pooling with automatic recycling
• Priority-based resource access for background jobs
• How concurrent-ruby enables thread safety
• Performance benefits and use cases

Run the examples:

$ bundle exec ruby examples/real_world_use_cases.rb

require 'lean_pool'

# Create a pool of Redis connections
pool = LeanPool::Pool.new(size: 5) do
  Redis.new(host: "localhost", port: 6379)
end

# Use the pool
pool.checkout do |redis|
  redis.get("my_key")
  redis.set("my_key", "value")
end

LeanPool includes a built-in HTTP connection pool for easy HTTP/HTTPS requests:

require 'lean_pool'

# Create an HTTP pool
http_pool = LeanPool::HTTPPool.new("api.example.com", 443, size: 10, use_ssl: true)

# Perform GET requests
response = http_pool.get("/users")
puts response[:status]  # => 200
puts response[:body]
puts response[:headers]

# Perform POST requests with JSON
response = http_pool.post(
  "/users",
  body: '{"name":"John","email":"john@example.com"}',
  headers: { "Content-Type" => "application/json" }
)

# With custom timeout
response = http_pool.get("/slow-endpoint", timeout: 30.0)

# Check pool statistics
stats = http_pool.stats
# => { size: 10, available: 8, in_use: 2, total: 10 }

# Shutdown when done
http_pool.shutdown

You can also create your own HTTP pool wrapper:

require 'lean_pool'
require 'net/http'

class CustomHTTPPool
  def self.new_pool(host, port, size: 10)
    LeanPool::Pool.new(
      size: size,
      timeout: 5.0,
      lazy: true,
      pool_state: { host: host, port: port }
    ) do |state|
      Net::HTTP.new(state[:host], state[:port])
    end
  end

  def self.get(pool, path, opts = {})
    pool.checkout(timeout: opts[:timeout] || 5.0) do |http|
      http.start unless http.started?
      response = http.get(path)
      
      if response.code == "200"
        { status: response.code.to_i, body: response.body, headers: response.to_hash }
      else
        { remove: true }
      end
    end
  end
end

# Usage
pool = CustomHTTPPool.new_pool("api.example.com", 443)
result = CustomHTTPPool.get(pool, "/users")
puts result[:body]

pool = LeanPool::Pool.new(
  size: 10,
  pool_state: { database: "production", timeout: 30 }
) do |state|
  DatabaseConnection.new(
    database: state[:database],
    timeout: state[:timeout]
  )
end

pool.checkout do |db|
  db.query("SELECT * FROM users")
end

pool = LeanPool::Pool.new(size: 5) { Redis.new }

stats = pool.stats
# => { size: 5, available: 3, in_use: 2, total: 5 }

# Gracefully shutdown the pool
pool.shutdown do |resource|
  resource.close if resource.respond_to?(:close)
end

# Reload the pool (useful after forking)
pool.reload do |resource|
  resource.quit if resource.respond_to?(:quit)
end

LeanPool supports multiple resource selection strategies:

# FIFO (First In, First Out) - default
pool = LeanPool::Pool.new(size: 5, strategy: :fifo) { Redis.new }

# LIFO (Last In, First Out)
pool = LeanPool::Pool.new(size: 5, strategy: :lifo) { Redis.new }

# Random selection
pool = LeanPool::Pool.new(size: 5, strategy: :random) { Redis.new }

# LRU (Least Recently Used)
pool = LeanPool::Pool.new(size: 5, strategy: :lru) { Redis.new }

# Change strategy at runtime
pool.strategy = :lru

High-priority requests get resources first when the pool is exhausted:

pool = LeanPool::Pool.new(size: 1) { Connection.new }

# Low priority (higher number = lower priority)
pool.checkout(priority: 10) do |conn|
  # This will wait if pool is full
end

# High priority (lower number = higher priority)
pool.checkout(priority: 0) do |conn|
  # This will get the resource before lower priority requests
end

Automatically replace resources after a certain number of uses or age:

# Recycle after 100 uses
pool = LeanPool::Pool.new(size: 5, max_uses: 100) { Connection.new }

# Recycle after 1 hour
pool = LeanPool::Pool.new(size: 5, max_age: 3600) { Connection.new }

# Recycle when either condition is met
pool = LeanPool::Pool.new(
  size: 5,
  max_uses: 100,
  max_age: 3600
) { Connection.new }

Monitor pool lifecycle with event callbacks:

pool = LeanPool::Pool.new(size: 5) { Connection.new }

# Before checkout
pool.on(:before_checkout) do |pool|
  puts "Checking out from pool"
end

# After checkout
pool.on(:after_checkout) do |pool, resource|
  puts "Got resource: #{resource}"
end

# Resource created
pool.on(:on_resource_created) do |pool, resource|
  puts "New resource created: #{resource}"
end

# Resource removed
pool.on(:on_resource_removed) do |pool, resource|
  puts "Resource removed: #{resource}"
end

# Pool exhausted
pool.on(:on_pool_exhausted) do |pool|
  puts "Pool is full!"
end

# Multiple callbacks for the same event
pool.on(:before_checkout) { puts "Callback 1" }
pool.on(:before_checkout) { puts "Callback 2" }

Available events:

• :before_checkout - Called before a resource is checked out
• :after_checkout - Called after a resource is successfully checked out
• :before_checkin - Called before a resource is checked in
• :after_checkin - Called after a resource is checked in
• :on_resource_created - Called when a new resource is created
• :on_resource_removed - Called when a resource is removed from the pool
• :on_pool_exhausted - Called when the pool is exhausted (all resources in use)

begin
  pool.checkout(timeout: 2.0) do |resource|
    resource.perform_operation
  end
rescue LeanPool::TimeoutError => e
  puts "Pool is busy, try again later"
rescue LeanPool::ShutdownError => e
  puts "Pool is shutdown"
rescue LeanPool::ResourceError => e
  puts "Failed to create resource: #{e.message}"
end

Use LeanPool when:

• You need direct access to resources (sockets, ports, HTTP connections)
• You want to avoid data copying between processes
• You're managing resources that don't need per-resource processes
• You need high-performance resource pooling

Don't use LeanPool when:

• You're managing processes (use process-based pools instead)
• You need multiplexing (HTTP/2, etc.)
• Resources require per-resource process isolation

• Ruby >= 3.3.0
• concurrent-ruby >= 1.3.0

After checking out the repo, run:

$ bundle install
$ bundle exec rspec

Quick test run:

$ bundle exec rspec

With documentation format:

$ bundle exec rspec --format documentation

Run specific test files:

$ bundle exec rspec spec/lean_pool/pool_spec.rb
$ bundle exec rspec spec/lean_pool/http_pool_spec.rb
$ bundle exec rspec spec/lean_pool/errors_spec.rb

Simulate CI locally:

$ ./.github/workflows/test-local.sh

This script will:

• Check Ruby version
• Install dependencies
• Validate syntax of all Ruby files
• Run RuboCop linting
• Run the full test suite
• Display test summary

Using Rake:

$ bundle exec rake        # Runs tests + linting
$ bundle exec rake spec  # Runs only tests

The project includes comprehensive test coverage with over 222 test cases covering:

• Pool initialization and configuration
• Resource checkout and lifecycle management
• Thread safety and concurrent operations
• Timeout handling and error recovery
• HTTP connection pooling (with WebMock for reliable testing)
• Resource selection strategies (FIFO, LIFO, Random, LRU)
• Priority-based checkout
• Resource recycling (max_uses, max_age)
• Event callbacks
• Integration scenarios and real-world use cases
• Edge cases and boundary conditions

Run the full test suite:

$ bundle exec rspec

• ✅ CI/CD: Automated testing with GitHub Actions (testing on Ruby 3.3, 4.0)
• ✅ Test Coverage: Comprehensive test suite with 222+ test cases
• ✅ Documentation: Complete YARD documentation for all modules
• ✅ Thread Safety: Built on concurrent-ruby for reliable concurrency
• ✅ Production Ready: Suitable for production use
• ✅ Code Quality: RuboCop linting and syntax validation
• ✅ Advanced Features: Priority-based checkout, resource strategies, recycling, and event callbacks

The project uses GitHub Actions for continuous integration:

• Test Matrix: Tests run on Ruby 3.3 and 4.0
• Linting: RuboCop code quality checks
• Syntax Validation: Automatic Ruby syntax checking
• Status Badge:

View the latest CI runs: GitHub Actions

Status: The gem is now published on RubyGems.org! Badges will automatically update to show download counts and build status.

Bug reports and pull requests are welcome on GitHub at https://github.com/half-blood-labs/lean_pool.

Junaid Farooq

• Email: fjunaid252@gmail.com
• GitHub: @half-blood-labs

The gem is available as open source under the terms of the MIT License.

This gem is inspired by nimble_pool from the Elixir ecosystem, adapted for Ruby's concurrency model using concurrent-ruby.