GraphQL subscriptions for AnyCable

A (mostly) drop-in replacement for default ActionCable subscriptions adapter shipped with graphql gem but works with AnyCable!

AnyCable is fast because it does not execute any Ruby code. But default subscription implementation shipped with graphql gem requires to do exactly that: re-evaluate GraphQL queries in ActionCable process. AnyCable doesn't support this (it's possible but hard to implement).

See https://github.com/anycable/anycable-rails/issues/40 for more details and discussion.

• Subscription information is stored in Redis database configured to be used by AnyCable. Expiration or data cleanup should be configured separately (see below).
• GraphQL queries for all subscriptions are re-executed in the process that triggers event (it may be web server, async jobs, rake tasks or whatever)

• Should work with ActionCable in development
• Should work without Rails via LiteCable

AnyCable must be configured with redis broadcast adapter (this is default).

Add this line to your application's Gemfile:

gem 'graphql-anycable', '~> 1.0'

And then execute:

$ bundle

Or install it yourself as:

$ gem install graphql-anycable

1. Plug it into the schema (replace from ActionCable adapter if you have one):

   class MySchema < GraphQL::Schema
     use GraphQL::AnyCable, broadcast: true
   
     subscription SubscriptionType
   end

2. Execute query in ActionCable/LiteCable channel.

   class GraphqlChannel < ApplicationCable::Channel
     def execute(data)
       result = 
         MySchema.execute(
           query: data["query"],
           context: context,
           variables: Hash(data["variables"]),
           operation_name: data["operationName"],
         )
   
       transmit(
         result: result.subscription? ? { data: nil } : result.to_h,
         more: result.subscription?,
       )
     end
   
     def unsubscribed
       MySchema.subscriptions.delete_channel_subscriptions(self)
     end
   
     private
   
     def context
       {
         account_id: account&.id,
         channel: self,
       }
     end
   end

   Make sure that you're passing channel instance as channel key to the context.

3. Trigger events as usual:

   MySchema.subscriptions.trigger(:product_updated, {}, Product.first!, scope: account.id)

By default, graphql-anycable evaluates queries and transmits results for every subscription client individually. Of course, it is a waste of resources if you have hundreds or thousands clients subscribed to the same data (and has huge negative impact on performance).

Thankfully, GraphQL-Ruby has added Subscriptions Broadcast feature that allows to group exact same subscriptions, execute them and transmit results only once.

To enable this feature, turn on Interpreter and pass broadcast option set to true to graphql-anycable.

By default all fields are marked as not safe for broadcasting. If a subscription has at least one non-broadcastable field in its query, GraphQL-Ruby will execute every subscription for every client independently. If you sure that all your fields are safe to be broadcasted, you can pass default_broadcastable option set to true (but be aware that it can have security impllications!)

class MySchema < GraphQL::Schema
  use GraphQL::Execution::Interpreter # Required for graphql-ruby before 1.12. Remove it when upgrading to 2.0
  use GraphQL::Analysis::AST # Required for graphql-ruby before 1.12. Remove it when upgrading to 2.0
  use GraphQL::AnyCable, broadcast: true, default_broadcastable: true

  subscription SubscriptionType
end

See GraphQL-Ruby broadcasting docs for more details.

To avoid filling Redis storage with stale subscription data:

1. Set subscription_expiration_seconds setting to number of seconds (e.g. 604800 for 1 week). See configuration section below for details.

2. Execute rake graphql:anycable:clean once in a while to clean up stale subscription data.

   Heroku users should set up use_redis_object_on_cleanup setting to false due to limitations in Heroku Redis.

GraphQL-AnyCable uses anyway_config to configure itself. There are several possibilities to configure this gem:

1. Environment variables:

   GRAPHQL_ANYCABLE_SUBSCRIPTION_EXPIRATION_SECONDS=604800
   GRAPHQL_ANYCABLE_USE_REDIS_OBJECT_ON_CLEANUP=true
   GRAPHQL_ANYCABLE_USE_CLIENT_PROVIDED_UNIQ_ID=false
   GRAPHQL_ANYCABLE_REDIS_PREFIX=graphql

2. YAML configuration files (note that this is config/graphql_anycable.yml, not config/anycable.yml):

   # config/graphql_anycable.yml
   production:
     subscription_expiration_seconds: 300 # 5 minutes
     use_redis_object_on_cleanup: false # For restricted redis installations
     use_client_provided_uniq_id: false # To avoid problems with non-uniqueness of Apollo channel identifiers
     redis_prefix: graphql # You can configure redis_prefix for anycable-graphql subscription prefixes. Default value "graphql"

3. Configuration from your application code:

   GraphQL::AnyCable.configure do |config|
     config.subscription_expiration_seconds = 3600 # 1 hour
     config.redis_prefix = "graphql" # on our side, we add `-` ourselves after the redis_prefix
   end

And any other way provided by anyway_config. Check its documentation!

In situations when you don't set subscription_expiration_seconds, have a lot of inactive subscriptions, and GraphQL::AnyCable::Cleaner does`t help in that, you can do the following actions for clearing subscriptions

1. Set config.subscription_expiration_seconds. After that, the new subscriptions will have TTL
2. Run the script

   redis = GraphQL::AnyCable.redis
   config = GraphQL::AnyCable.config
   
   # do it for subscriptions
   redis.scan_each("graphql-subscription:*") do |key|
     redis.expire(key, config.subscription_expiration_seconds) if redis.ttl(key) < 0
     # or you can just remove it immediately
     # redis.del(key) if redis.ttl(key) < 0
   end

   # do it for channels
   redis.scan_each("graphql-channel:*") do |key|
     redis.expire(key, config.subscription_expiration_seconds) if redis.ttl(key) < 0
     # or you can just remove it immediately
     # redis.del(key) if redis.ttl(key) < 0
   end

Or you can change the redis_prefix in the configuration and then remove all records with the old_prefix For instance:

1. Change the redis_prefix. The default redis_prefix is graphql
2. Run the ruby script, which remove all records with old prefix

  redis.scan_each("graphql-*") do |key|
    redis.del(key)
  end

As in AnyCable there is no place to store subscription data in-memory, it should be persisted somewhere to be retrieved on GraphQLSchema.subscriptions.trigger and sent to subscribed clients. graphql-anycable uses the same Redis database as AnyCable itself.

1. Grouped event subscriptions: graphql-fingerprints:#{event.topic} sorted set. Used to find all subscriptions on GraphQLSchema.subscriptions.trigger.

   ZREVRANGE graphql-fingerprints:1:myStats: 0 -1
   => 1:myStats:/MyStats/fBDZmJU1UGTorQWvOyUeaHVwUxJ3T9SEqnetj6SKGXc=/0/RBNvo1WzZ4oRRq0W9-hknpT7T8If536DEMBg9hyq_4o=
   

2. Event subscriptions: graphql-subscriptions:#{event.fingerptint} set containing identifiers for all subscriptions for given operation with certain context and arguments (serialized in topic). Fingerprints are already scoped by topic.

   SMEMBERS graphql-subscriptions:1:myStats:/MyStats/fBDZmJU1UGTorQWvOyUeaHVwUxJ3T9SEqnetj6SKGXc=/0/RBNvo1WzZ4oRRq0W9-hknpT7T8If536DEMBg9hyq_4o=
   => 52ee8d65-275e-4d22-94af-313129116388
   

3. Subscription data: graphql-subscription:#{subscription_id} hash contains everything required to evaluate subscription on trigger and create data for client.

   HGETALL graphql-subscription:52ee8d65-275e-4d22-94af-313129116388
   => {
     context:        '{"user_id":1,"user":{"__gid__":"Z2lkOi8vZWJheS1tYWcyL1VzZXIvMQ"}}',
     variables:      '{}',
     operation_name: 'MyStats'
     query_string:   'subscription MyStats { myStatsUpdated { completed total processed __typename } }',
   }
   

4. Channel subscriptions: graphql-channel:#{channel_id} set containing identifiers for subscriptions created in ActionCable channel to delete them on client disconnect.

   SMEMBERS graphql-channel:17420c6ed9e
   => 52ee8d65-275e-4d22-94af-313129116388
   

You can grab Redis subscription statistics by calling

    GraphQL::AnyCable.stats

It will return a total of the amount of the key with the following prefixes

    graphql-subscription
    graphql-fingerprints
    graphql-subscriptions
    graphql-channel

The response will look like this

  {
    "total": {
      "subscription":22646,
      "fingerprints":3200,
      "subscriptions":20101,
      "channel": 4900
    }
  }

You can also grab the number of subscribers grouped by subscriptions

    GraphQL::AnyCable.stats(include_subscriptions: true)

It will return the response that contains subscriptions

  {
    "total": {
      "subscription":22646,
      "fingerprints":3200,
      "subscriptions":20101,
      "channel": 4900
    },
    "subscriptions": {
      "productCreated": 11323,
      "productUpdated": 11323
    }
  }

Also, you can set another scan_count, if needed. The default value is 1_000

    GraphQL::AnyCable.stats(scan_count: 100)

We can set statistics data to Yabeda for tracking amount of subscriptions

  # config/initializers/metrics.rb
  Yabeda.configure do
    group :graphql_anycable_statistics do
      gauge :subscriptions_count,  comment: "Number of graphql-anycable subscriptions"
    end
  end

  # in your app
  statistics = GraphQL::AnyCable.stats[:total]

  statistics.each do |key , value|
    Yabeda.graphql_anycable_statistics.subscriptions_count.set({name: key}, value)
  end

Or you can use collect

  # config/initializers/metrics.rb
  Yabeda.configure do
    group :graphql_anycable_statistics do
      gauge :subscriptions_count,  comment: "Number of graphql-anycable subscriptions"
    end
    
    collect do
      statistics = GraphQL::AnyCable.stats[:total]

      statistics.each do |redis_prefix, value|
        graphql_anycable_statistics.subscriptions_count.set({name: redis_prefix}, value)
      end
    end
  end

You can pass custom redis-server URL to AnyCable using ENV variable.

```bash
REDIS_URL=redis://localhost:6379/5 bundle exec rspec
```

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

1. Bump version number in lib/graphql/anycable/version.rb

   In case of pre-releases keep in mind rubygems/rubygems#3086 and check version with command like Gem::Version.new(AfterCommitEverywhere::VERSION).to_s

2. Fill CHANGELOG.md with missing changes, add header with version and date.

3. Make a commit:

   git add lib/graphql/anycable/version.rb CHANGELOG.md
   version=$(ruby -r ./lib/graphql/anycable/version.rb -e "puts Gem::Version.new(GraphQL::AnyCable::VERSION)")
   git commit --message="${version}: " --edit

4. Create annotated tag:

   git tag v${version} --annotate --message="${version}: " --edit --sign

5. Fill version name into subject line and (optionally) some description (list of changes will be taken from CHANGELOG.md and appended automatically)

6. Push it:

   git push --follow-tags

7. GitHub Actions will create a new release, build and push gem into rubygems.org! You're done!

Bug reports and pull requests are welcome on GitHub at https://github.com/Envek/graphql-anycable.

The gem is available as open source under the terms of the MIT License.