Neighbor Redis

Nearest neighbor search for Ruby and Redis

Installation

First, install RediSearch. With Docker, use:

docker run -p 6379:6379 redis/redis-stack-server

Add this line to your application’s Gemfile:

gem "neighbor-redis"

And set the Redis client:

Neighbor::Redis.client = RedisClient.config.new_pool

Getting Started

Create an index

index = Neighbor::Redis::HNSWIndex.new("items", dimensions: 3, distance: "l2")
index.create

Add items

index.add(1, [1, 1, 1])
index.add(2, [2, 2, 2])
index.add(3, [1, 1, 2])

Note: IDs are stored and returned as strings (uses less total memory)

Get the nearest neighbors to an item

index.nearest(1, count: 5)

Get the nearest neighbors to a vector

index.search([1, 1, 1], count: 5)

Distance

Supported values are:

l2
inner_product
cosine

Index Types

Hierarchical Navigable Small World (HNSW)

Neighbor::Redis::HNSWIndex.new(
  name,
  initial_cap: nil,
  m: 16,
  ef_construction: 200,
  ef_runtime: 10,
  epsilon: 0.01
)

Flat

Neighbor::Redis::FlatIndex.new(
  name,
  initial_cap: nil,
  block_size: 1024
)

Additional Options

Store vectors as double precision (instead of single precision)

Neighbor::Redis::HNSWIndex.new(name, type: "float64")

Store vectors as JSON (instead of a hash/blob)

Neighbor::Redis::HNSWIndex.new(name, redis_type: "json")

Changing Options

Create a new index to change any index options

Neighbor::Redis::HNSWIndex.new("items-v2", **new_options)

Additional Operations

Add multiple items

index.add_all(ids, embeddings)

Get an item

index.find(id)

Remove an item

index.remove(id)

Remove multiple items

index.remove_all(ids)

Drop the index

index.drop

Example

You can use Neighbor Redis for online item-based recommendations with Disco. We’ll use MovieLens data for this example.

Create an index

index = Neighbor::Redis::HNSWIndex.new("movies", dimensions: 20, distance: "cosine")
index.create

Fit the recommender

data = Disco.load_movielens
recommender = Disco::Recommender.new(factors: 20)
recommender.fit(data)

Store the item factors

index.add_all(recommender.item_ids, recommender.item_factors)

And get similar movies

index.nearest("Star Wars (1977)").map { |v| v[:id] }

See the complete code

Reference

Vector similarity

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

Report bugs
Fix bugs and submit pull requests
Write, clarify, or fix documentation
Suggest or add new features

To get started with development:

git clone https://github.com/ankane/neighbor-redis.git
cd neighbor-redis
bundle install
bundle exec rake test

neighbor-redis

Runtime