The project is in a healthy, maintained state
deferred_columns leaves the columns you choose out of every query that does not ask for them, and raises instead of querying when code reads them without opting in.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
 Dependencies

Runtime

 Project Readme

Deferred Columns

Build Status License

Leave large columns out of Active Record queries by default.

class Song < ApplicationRecord
  defer_columns :lyrics, :waveform
end

album.songs                        # no lyrics or waveform
album.songs.with_deferred(:lyrics) # includes lyrics in the same query

Why? ๐Ÿค”

An album page only needs a track list, but by default album.songs fetches the lyrics and waveform for every song.

You could use select, but then every query that loads songs needs a list of columns to keep. ignored_columns goes too far: you still want the lyrics on the song page.

With defer_columns, only the query for the song page needs to change. If you forget to select the lyrics, song.lyrics raises rather than quietly running another query.

Installation ๐Ÿ’ฟ

Add this line to your application's Gemfile:

gem 'deferred_columns'

And then run:

$ bundle install

Usage ๐Ÿš€

Loading deferred columns

Use with_deferred to add one or more columns to the ones you'd normally get:

Song.with_deferred(:lyrics).find_by(name: "Oye como va")
album.songs.with_deferred(:lyrics, :waveform)
Song.with_deferred_columns.find(id) # all columns

An unknown column or group raises ArgumentError. If you use select, Active Record loads exactly the columns you name, even on a relation with with_deferred.

Subclasses inherit their parent's deferred columns and can add their own:

class LiveRecording < Song
  defer_columns :crowd_noise
end

LiveRecording.deferred_columns # => ["lyrics", "waveform", "crowd_noise"]

Grouping columns

If you usually read a few columns together, give them a name:

class Album < ApplicationRecord
  defer_columns :liner_notes, :cover_art, as: :booklet
end

Album.with_deferred(:booklet)

You can mix group and column names in a query. A column can belong to more than one group, and declaring a group again adds to it.

Associations

Album.includes(:songs) leaves out the lyrics, just like album.songs. To preload the lyrics too, use an association scope:

class Album < ApplicationRecord
  has_many :songs
  has_many :songs_with_lyrics, -> { with_deferred(:lyrics) }, class_name: "Song"
end

Album.includes(:songs_with_lyrics)

The same scope works with preload, eager_load, and has_many :through. In Song.with_deferred(:lyrics).eager_load(:album), only the songs get the extra column.

Loading into a record you already have

A job might receive a song without its lyrics. Use load_deferred to fetch them without replacing the record:

class RefreshLyricsJob < ApplicationJob
  def perform(song)
    song.load_deferred(:lyrics)
    song.update!(lyrics: LyricsService.normalize(song.lyrics))
  end
end

Without arguments, load_deferred fetches all missing deferred columns. It skips the query for new records or when the columns are already loaded.

For a collection, load the missing columns with one query per model instead of one per record:

DeferredColumns.load(album.songs, :lyrics)

Both forms accept group names and leave values you've already loaded or assigned alone.

Reading deferred columns

song = Song.first
song.has_attribute?(:lyrics) # => false
song.as_json                 # omits lyrics
song.lyrics                  # raises DeferredColumns::NotLoadedError

song[:lyrics] and song.read_attribute(:lyrics) raise the same error. It's a subclass of ActiveModel::MissingAttributeError and has model and column readers.

New records have all their columns. If you override an attribute reader, call super as usual.

Reporting instead of raising

In production you may want to know where you missed a with_deferred without breaking the page:

# config/initializers/deferred_columns.rb
DeferredColumns.on_not_loaded = ->(error) {
  Rails.env.local? ? raise(error) : Rails.error.report(error)
}

If the handler returns, song.lyrics fetches the lyrics and returns them. In a loop that's an extra query per song, so use the report to find the query that needs with_deferred(:lyrics).

The same handler runs if you try to preload lyrics into an association that Active Record considers loaded. By default you get a NotLoadedError if the lyrics are missing. If the handler returns, the preloader fetches them in bulk.

Writing records

You don't need to load the lyrics to update a song, even if you're updating the lyrics themselves:

song = Song.find_by(name: "El Nicoya")
song.update!(composer: "Josรฉ Areas") # leaves lyrics untouched
song.update!(lyrics: "Instrumental")

Validations

class Album < ApplicationRecord
  defer_columns :liner_notes
  validates :name, :liner_notes, presence: true
end

album = Album.find_by(name: "Abraxas")
album.update!(name: "Abraxas (Remastered)") # validates name, skips liner_notes

Attribute validators such as presence and length skip a deferred column unless you loaded or assigned it. On new records they run as usual. In a custom validate method, check has_attribute?(:liner_notes) before reading it.

Reloading

reload remembers which deferred columns you loaded and refreshes those in a second query:

song = Song.with_deferred(:lyrics).find(id)
song.reload.lyrics # fresh lyrics, still no waveform

How it works ๐Ÿ”ง

Active Record builds the default SELECT in Relation#build_select. For models with deferred columns, the gem lists the columns to fetch instead of selecting *. There's no default_scope, and models without deferred columns keep their usual queries. For eager_load, the gem adjusts the column list for each model in the join.

with_deferred keeps the requested columns in a relation extension, so they survive merge and association scopes without changing what count counts. pluck and exists? build their own selects. Raw SQL is unchanged.

Compatibility

  • Ruby 3.1 or newer
  • Active Record 7.2, 8.0, 8.1 and main
  • Tested on SQLite and PostgreSQL

Running tests ๐Ÿงช

bundle install
bin/rspec

To run the tests against a specific version of Rails, set BUNDLE_GEMFILE to one of the files in gemfiles:

BUNDLE_GEMFILE=gemfiles/Gemfile-rails.7.2.x bundle install
BUNDLE_GEMFILE=gemfiles/Gemfile-rails.7.2.x bin/rspec

The tests run on an in-memory SQLite database by default. To run them on PostgreSQL, set DATABASE_URL:

createdb deferred_columns_test
DATABASE_URL=postgres:///deferred_columns_test bin/rspec

Acknowledgements ๐Ÿ™

  • columns_on_demand, which first left columns out of the default SELECT from Relation#build_select

License

The gem is available as open source under the terms of the MIT License.