gem skjvs

Very simple yet efficient "string key -> json value" storage, i.e. keys are automatically converted to MD5, values are serialized to JSON.

Persistent file is concurrently accessible from any number of threads or processes. Consistency is gauranteed by file locking. Faster than YAML::Store by magnitudes. You only pay with theoretical MD5 collisions and growing file size on value overwrite.

$ gem install skjvs

Only two methods are supported for now: [] and []=.

store = SKJVS::OneFile.new
store[123] = "2\n3"
assert_equal "2\n3", store[123]

Store file is meant to be accessible from multiple processes at the same time. Though the store file is thread-safe, but store object isn't, so for each thread do own SKJVS::OneFile.new.

Compared to YAML::Store, PStore, gem sequel (sqlite).

Loses to Sequel after skjvs_store.txt reaches 30 MB and corresponding sqlite.db reaches 100 KB.

Not sure why file size difference is so big -- maybe because benchmark is too aritifical and values are duplicating, and sqlite spots it and compacts, idk, didn't debug, didn't profile.
rake benchmark could be done better, could be split into first-fetch/fetch/write, etc. Later.

TODO: compare to https://github.com/ruby/sdbm, https://github.com/ruby/dbm, https://github.com/propublica/daybreak, etc.

The SKJVS::OneFile is optimized for speed. Later the SKJVS::TwoFiles may be implemented for memory optimization.

By current design the store file isn't editing, only growing, so value overwriting creates extra line. You may want to compact the file by yourself (TODO: a tool), but only in between of its usage (for example, restart processes).

To better understand the design and trade-offs of test and benchmark, see automated TLDR:

$ bundle install
$ bundle exec ruby test.rb
$ bundle exec rake benchmark

$ rake -rbundler/gem_tasks release