Recorder

Recorder tracks changes of your Rails models. Each create, update, and destroy on an observed model writes a recorder_revisions row holding an attribute snapshot, the changes, and — when the controller concern is included — the user and IP behind the request.

• PostgreSQL — revisions are stored in jsonb and inet columns
• Ruby and Rails per the table below

Rails 6.1 and 7.0 cannot run on Ruby 3.4 — they require mutex_m, which left the default gems in that release. Every combination in the table is exercised in CI.

Add this line to your application's Gemfile:

gem 'recorder'

And then execute:

$ bundle

Generate the migration for the recorder_revisions table:

$ rails generate recorder:install

The generator writes the migration but does not run it. Run it yourself:

$ rails db:migrate

The generator accepts one option:

• --with_index_by_user_id — adds an index on user_id.

Include Recorder::Observer into the model and configure logging options for it:

class Post < ActiveRecord::Base
  include ::Recorder::Observer

  recorder only: %i[title tags],
    associations: {
      author: { only: %i[full_name] },
      category: { only: %i[name slug] }
    }
end

Recorder supports the following options:

• ignore: [array] - attributes that are ignored on logging. Replaces Recorder.config.ignore for this model rather than adding to it;
• only: [array] - only these attributes are logged, other attributes are ignored. Recorder.config.ignore does not apply, so a listed attribute is logged even if the global list ignores it. Takes precedence over ignore:;
• associations: {hash} (hash) - allows to set what associations will be logged alongside with the model, as a snapshot on create and destroy. For each association you can also set ignore and only options, which follow the same rules; an association given neither falls back to Recorder.config.ignore. Only singular associations, belongs_to and has_one, are recorded: a collection (has_many, has_and_belongs_to_many), or a name that is not an association, is skipped and logged as a warning through the model's logger, once per model and process. On a subclass, a name that only a sibling class declares is skipped without a warning;
• if: Symbol | Proc, unless: Symbol | Proc - a condition the record has to meet, or must not meet, for a change to be recorded. See Conditions;
• changes: Proc | Symbol - extra entries to merge into a revision's changes. A Proc is evaluated on the record, a Symbol names a method on it; both receive the event (:create, :update or :destroy) and return a hash of name => [old, new], or nil for nothing. Anything else raises ArgumentError where recorder is called.

The entries are merged after only: and ignore: have been applied, so those filters never drop them, and a key that matches an attribute replaces that attribute's entry. An update that reports only custom entries still records a revision.

Inside the callback, read saved_changes and saved_change_to_<attribute>?, not <attribute>_changed?: on :create and :update the callback runs from after_create/after_update, where the dirty state has already been reset. On :destroy it runs from after_destroy, where destroy has reset nothing: saved_changes still holds the record's last save, not the deletion.

Recorder::Changeset rebuilds the previous and next versions by assigning each change onto a copy of the record, and skips what it cannot assign: a key that is not an attribute, and any value that is not a two-element [old, new] pair. To display such a key, define previous_<key>/next_<key> on the model's changeset class, and pick a name that Recorder::Changeset does not already answer to.

Option names can be symbols or strings, in recorder and in each association's options, and so can association names. recorder raises ArgumentError for any other option, so a misspelt one fails where it is written, for an option or association given both as a symbol and as a string, and for an association name that is neither. async: and delay: raise too, with a message saying asynchronous recording was removed in 2.0.0.

Model.recorder_options returns the declared options with symbol keys, the association names included, frozen.

A model that needs to decide its options per record can define recorder_options as an instance method. It replaces what was passed to recorder rather than merging with it, so it has to return every option the model needs, with symbol keys; its result is used as it is, unchecked. if: and unless: are the exception: they belong to the callbacks, and an override cannot change them (see Conditions):

def recorder_options
  {ignore: %i[identifier], changes: :extra_changes}
end

if: and unless: decide per record whether a change is recorded, as they do on a Rails callback:

class Post < ActiveRecord::Base
  include ::Recorder::Observer

  recorder ignore: %i[views_count], if: :published?, unless: ->(post) { post.imported? }
end

A Symbol names a method on the record, public or private; a misspelt one raises NoMethodError from the first save, which rolls it back. A Proc takes one argument, the record, or none, in which case it is evaluated on the record. A change is recorded when if: returns a truthy value and unless: a falsy one, and nil means no condition, as on a Rails callback.

recorder raises ArgumentError for any other value: a Proc with other parameters, optional or splat ones included, since Rails would not pass it the record, an Array, or an object that responds to call. For such an object, pass Policy.method(:call).to_proc.

The condition applies to create, update and destroy alike. It runs only while recording is on, so it cannot turn recording back on for a change recorder_disabled! or Recorder.enabled = false skips.

It runs after the change, in the same after_create, after_update and after_destroy callbacks as changes:, so the notes there on saved_changes apply. It sees the record as saved, so a change that makes it false is not recorded: with if: :published?, unpublishing a post writes no revision, and its history ends on the last published state. To record the change that leaves the condition, test the value before the save too:

recorder if: -> { published? || status_previously_was == 'published' }

On destroy, status_previously_was reads the record's last save, and the associated records dependent: :destroy removes are already gone.

To vary the condition by event, destroyed? is true on destroy, and otherwise previously_new_record? tells a create from an update. previously_new_record? stays true when the record that was created is then destroyed, so check destroyed? first. This records a create only for a featured post:

recorder if: -> { destroyed? || !previously_new_record? || featured? }

recorder alone sets the condition. A subclass inherits it, and if: or unless: returned by a recorder_options override have no effect, so an override that builds a new hash keeps it. To vary the condition by subclass, name a method and override that method.

Recorder::Tape.new(record).record_update and its record_create and record_destroy siblings do not check the condition: calling one is the decision to record. They still write nothing while recording is off.

A subclass, STI or not, records under the options its parent passed to recorder and needs no declaration of its own. recorder can be called once per class hierarchy, in the topmost class that records: calling it again, in the same class, a subclass or a parent, raises ArgumentError. So does a declaration that runs twice on one class, such as one in a to_prepare block on a class that is not reloaded. A subclass that needs different options defines recorder_options, public or private, and can build on the parent's with super:

class Bond < Instrument
  def recorder_options
    super.merge(ignore: [*super[:ignore], :coupon])
  end
end

super returns a copy of the parent's options, which the override can change in place or build on. The result is applied as if it had been declared, except for if: and unless:, which an override cannot change. A recorder_options class method that calls super gets the frozen options every subclass shares, so it builds a new hash, such as super.merge(...); one that changes them in place raises FrozenError from the first save that reads it, which rolls that save back. The usual rules hold: only: takes precedence over ignore:, so adding to ignore: has no effect under a parent that declares only:, and an ignore: replaces Recorder.config.ignore, so a subclass adding one under a parent that declares none has to repeat the global list.

Recorder.config do |config|
  config.ignore = %i[created_at updated_at]
end

ignore defaults to [].

ignore applies only to models, and associations, that declare neither only: nor ignore:. A model that declares either uses its own list alone, so to keep the global list and add to it, repeat it:

recorder ignore: [*Recorder.config.ignore, :internal_id]

This reads the global list when the model loads, so configure Recorder in an initializer.

A revision is written from the model's after_create, after_update and after_destroy callbacks, in the same database transaction as the save. The record and its revision commit together, and a save that rolls back leaves no revision behind.

A revision that cannot be written fails the save: it raises, and the save rolls back, along with any transaction around it that does not rescue the error. This holds for save and update as much as for save!, and for destroy. A revision that fails validation, or that a callback on Recorder::Revision halts, raises Recorder::RevisionNotSaved, a subclass of ActiveRecord::RecordNotSaved whose record is the revision. A database error raises as it would from any other write.

Nothing but the error rolls the save back. A transaction of your own that rescues it, or rescues ActiveRecord::RecordNotSaved, commits the record without its revision. Let the error propagate, or re-raise it.

To enable storing of such data as user_id and ip, you need to include Recorder::Rails::ControllerConcern to ApplicationController. Recorder uses request_store to safely store these data on a thread level.

  class ApplicationController < ActionController::Base
    include Recorder::Rails::ControllerConcern
    ...
  end

The concern reads current_user; override recorder_user_id to name a different method. Override recorder_meta to store a hash alongside every revision. A revision without a user is valid — user_id stays nil.

revision.user loads a User model. If your user model has another name, declare the association again in a to_prepare block, so that it picks up the reloaded class in development:

# config/initializers/recorder.rb
Rails.application.config.to_prepare do
  Recorder::Revision.belongs_to :user, class_name: 'Account', optional: true
end

Declaring it on every reload replaces the association each time. Keep optional: true. Without it, belongs_to_required_by_default makes the user required: a change made with no user, or by a user since deleted, writes no revision, and the save goes through without an error.

A revision's action_date is today in the application's time zone, Date.current. To date revisions otherwise, such as an import replaying past changes, set it yourself; it holds for the rest of the request or thread:

Recorder.info = {action_date: Date.new(2020, 1, 1)}

created_at is always when the revision was written. Recorder.info raises ArgumentError for it and for the other columns Recorder writes itself: id, item_type, item_id, event and data.

Recorder::Manager suspends recording for the current request or thread:

class Importer
  include Recorder::Manager

  def call
    recorder_disabled! do
      # nothing recorded in here
    end
  end
end

The block form restores the state it found on the way out, including when the block raises, so a block nested in another leaves recording off. Called without a block, recorder_disabled! stays in effect until recorder_enabled!. That state is cleared when a web request ends, but not reliably between background jobs, where it would carry over to later jobs on the same thread; use the block form there.

Recorder.enabled = false switches recording off for the whole process, every thread included, until Recorder.enabled = true. It suits a one-off script, such as a backfill or a data migration; inside a running app it would stop the trail for every request at once, so use recorder_disabled! there. A change is recorded only while both are on.

Observed models get a revisions association:

revision = post.revisions.ordered_by_created_at.first

revision.event        # "update"
revision.data         # {"attributes" => {...}, "changes" => {...}, "associations" => {...}}
revision.user_id
revision.action_date

data carries a complete attribute snapshot on every event, not only what changed:

• attributes — every attribute of the record, with the values it last loaded from or saved to the database, filtered by the model's only: or ignore: and, failing those, by Recorder.config.ignore. Always present. An attribute assigned and never saved is not recorded: after a create or an update memory and database agree, and a destroy revision holds the row as the record last saw it, not an assignment made since.
• changes — the same filter applied to saved_changes, as name => [old, new], plus any entries from changes:. On destroy, only the entries from changes:. Omitted when that leaves nothing.
• associations — on create and destroy, an attributes snapshot of each association named in associations:, filtered by that association's own only: or ignore:, with the same rule for unsaved assignments. Omitted on update, when no named association is set, and on destroy when the associated record was never saved. An associated record's changes are not recorded: they belong to its own revisions, and replacing a belongs_to target shows up as the foreign key in changes, unless only: or ignore: leaves it out.

The snapshot is the contract, not an accident of the implementation: a revision's attributes are self-contained, so reconstructing a record at a point in time does not mean replaying every prior diff. It is also what keeps a destroy revision useful, since the row it describes is gone. Association snapshots exist only on create and destroy revisions.

An update records a revision only when the record reports a change; create and destroy always record one.

item_type holds the model's polymorphic_name, the value Active Record writes to any polymorphic association, so revisions, includes(:revisions) and Recorder::Revision.where(item: record) all find a record's revisions. For an STI subclass that is the base class: a Bond < Instrument is recorded as "Instrument", revision.item loads it back as a Bond, and the subclass name is in the snapshot's inheritance column (type by default) unless only: or ignore: leaves it out.

Revisions written before 2.0.0 hold the subclass name, so revisions does not find them. Rewriting them is up to the app; this does it for every subclass whose class still exists:

Recorder::Revision.distinct.pluck(:item_type).each do |type|
  model = type.safe_constantize
  next unless model.respond_to?(:polymorphic_name) && model.polymorphic_name != type

  Recorder::Revision.where(item_type: type).update_all(item_type: model.polymorphic_name)
end

#item_changeset wraps data['changes'] in a Recorder::Changeset, which reads the values back as the model's own types:

changeset = revision.item_changeset

changeset.keys                        # attributes that changed
changeset.previous(:title)            # value before the change
changeset.next(:title)                # value after it
changeset.human_attribute_name(:title)
changeset.previous_version            # a copy of the record with the old values

Revisions written before 2.0.0 can also hold an association's changes, reachable the same way. changed_associations lists the associations that hold some, and association_changeset returns nil for one that does not:

revision.changed_associations         # ["author"]
revision.association_changeset('author')

To render a changeset yourself, define PostChangeset — the class is looked up as "#{model}Changeset" — or point at another one with a recorder_changeset_class class method on the model. Otherwise Recorder::Changeset is used.

The gem is under active maintenance and these defects are known:

• recorder_disabled! reaches only the current fiber, not the whole thread: RequestStore keeps its state in fiber-local storage. Code that runs in another fiber inside the block, such as an Enumerator#next body or a task under a fiber scheduler, is recorded.

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

The specs need a PostgreSQL server. Connection settings are read from the environment — RECORDER_DB_HOST, RECORDER_DB_PORT, RECORDER_DB_USERNAME, RECORDER_DB_PASSWORD, RECORDER_DB_NAME — and default to postgres:postgres on localhost:5432 against a recorder_test database, which must already exist.

rake spec runs against whichever Rails version the root Gemfile resolves to. To run against a specific one, use the appraisal gemfiles:

bundle exec appraisal install            # once, to generate gemfiles/
bundle exec appraisal rails-7.2 rake spec

CI runs the suite across the whole Ruby × Rails matrix above, plus bundle exec rubocop. Both must pass before a pull request merges.

To install this gem onto your local machine, run bundle exec rake install.

To release a new version, bump Recorder::VERSION and move the [Unreleased] changelog entries under it in a pull request. Once that merges, tag the merge commit and push the tag:

git fetch origin
git tag vX.Y.Z origin/master
git push origin vX.Y.Z

The Release workflow then runs CI against the tagged commit, checks the tag matches Recorder::VERSION, publishes the gem to rubygems.org through trusted publishing, and creates the GitHub release from the changelog.

Bug reports and pull requests are welcome on GitHub at https://github.com/jetrockets/recorder.

Recorder is maintained by JetRockets.

The gem is available as open source under the terms of the MIT License.