Project

spyglasses

0.0
The project is in a healthy, maintained state
spyglassesorchestra-code/spyglasses-rubyHomepageDocumentationSource CodeBug TrackerWiki
Spyglasses provides comprehensive AI agent detection and management capabilities for Ruby web applications, including Rails, Sinatra, and other Rack-based frameworks.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
 Popularity
Downloads
502
Stars
0
Forks
0
Watchers
0
 Releases
Current version
1.1.3
Total releases
6
First release
2025-06-24
Latest release
2025-06-26
 Development
Primary Language
Ruby
Licenses
MIT
Average date of last 50 commits
2025-06-24
Reverse Dependencies
0

 Dependencies

Development

bundler
>= 2.0
rack-test
~> 2.0
rake
>= 13.0
rspec
~> 3.0
rubocop
~> 1.0
simplecov
~> 0.21
webmock
~> 3.0
yard
~> 0.9

Runtime

json
>= 2.0
rack
>= 2.0
 Project Readme

Spyglasses Ruby Gem

Gem Version Ruby

AI Agent Detection and Management for Ruby web applications. Spyglasses provides comprehensive AI agent detection and management capabilities for Ruby web applications, including Rails, Sinatra, and other Rack-based frameworks.

Features

  • 🤖 AI Bot Detection: Automatically detect AI agents like GPTBot, ClaudeBot, ChatGPT-User, and more
  • 🧠 AI Referrer Detection: Track traffic from AI platforms like ChatGPT, Claude, Perplexity
  • 🚫 Flexible Blocking: Configure blocking rules via the Spyglasses platform
  • 📊 Request Logging: Non-blocking request logging to Spyglasses collector
  • ⚡ High Performance: Minimal overhead with pattern caching and background processing
  • 🔧 Easy Integration: Drop-in Rack middleware for any Ruby web framework
  • 🛡️ Thread Safe: Built for concurrent Ruby applications

Installation

Add this line to your application's Gemfile:

gem 'spyglasses'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install spyglasses

Quick Start

1. Get Your API Key

Sign up at spyglasses.io to get your API key.

2. Set Environment Variable

export SPYGLASSES_API_KEY=your_api_key_here

3. Add Middleware

Rails

Add to your config/application.rb:

# config/application.rb
class Application < Rails::Application
  config.middleware.use Spyglasses::Middleware
end

Or with options:

# config/application.rb
class Application < Rails::Application
  config.middleware.use Spyglasses::Middleware, {
    api_key: ENV['SPYGLASSES_API_KEY'],
    debug: Rails.env.development?,
    exclude_paths: ['/admin', '/internal']
  }
end

Sinatra

require 'sinatra'
require 'spyglasses'

use Spyglasses::Middleware, api_key: ENV['SPYGLASSES_API_KEY']

get '/' do
  'Hello World!'
end

Rack Application

# config.ru
require 'spyglasses'

use Spyglasses::Middleware, api_key: ENV['SPYGLASSES_API_KEY']

app = lambda do |env|
  [200, {'Content-Type' => 'text/plain'}, ['Hello World!']]
end

run app

Configuration

Environment Variables

Variable Description Default
SPYGLASSES_API_KEY Your Spyglasses API key Required
SPYGLASSES_DEBUG Enable debug logging false
SPYGLASSES_AUTO_SYNC Auto-sync patterns on startup true
SPYGLASSES_CACHE_TTL Pattern cache TTL in seconds 86400 (24 hours)
SPYGLASSES_COLLECT_ENDPOINT Custom collector endpoint https://www.spyglasses.io/api/collect
SPYGLASSES_PATTERNS_ENDPOINT Custom patterns endpoint https://www.spyglasses.io/api/patterns

Middleware Options

use Spyglasses::Middleware, {
  api_key: 'your-api-key',           # API key (overrides env var)
  debug: true,                       # Enable debug logging
  auto_sync: true,                   # Auto-sync patterns
  platform_type: 'rails',           # Platform identifier
  exclude_paths: [                   # Paths to exclude from monitoring
    '/admin',
    '/api/internal',
    /\.json$/                        # Regex patterns supported
  ],
  collect_endpoint: 'https://...',   # Custom collector endpoint
  patterns_endpoint: 'https://...'   # Custom patterns endpoint
}

Global Configuration

# config/initializers/spyglasses.rb (Rails)
Spyglasses.configure do |config|
  config.api_key = ENV['SPYGLASSES_API_KEY']
  config.debug = Rails.env.development?
  config.platform_type = 'rails'
  config.exclude_paths = ['/admin', '/internal']
end

Usage Examples

Manual Detection

require 'spyglasses'

# Configure the client
client = Spyglasses::Client.new(
  Spyglasses::Configuration.new.tap do |config|
    config.api_key = 'your-api-key'
  end
)

# Detect bots
result = client.detect_bot('GPTBot/1.0')
puts "Bot detected: #{result.is_bot}" # => true
puts "Should block: #{result.should_block}" # => depends on your settings

# Detect AI referrers
result = client.detect_ai_referrer('https://chat.openai.com/')
puts "AI referrer: #{result.source_type}" # => 'ai_referrer'

# Combined detection
result = client.detect('Mozilla/5.0', 'https://chat.openai.com/')
puts "Source type: #{result.source_type}" # => 'ai_referrer'

Custom Rack Middleware

class MyCustomMiddleware
  def initialize(app)
    @app = app
    @spyglasses = Spyglasses::Client.new
  end

  def call(env)
    request = Rack::Request.new(env)
    
    # Detect using Spyglasses
    result = @spyglasses.detect(request.user_agent, request.referrer)
    
    if result.is_bot && result.should_block
      return [403, {}, ['Forbidden']]
    end
    
    # Log the request if something was detected
    if result.source_type != 'none'
      @spyglasses.log_request(result, {
        url: request.url,
        user_agent: request.user_agent,
        # ... other request info
      })
    end
    
    @app.call(env)
  end
end

Blocking Configuration

Blocking rules are configured through the Spyglasses platform dashboard, not in code:

  1. Global AI Model Trainer Blocking: Block all AI training bots (GPTBot, ClaudeBot, etc.)
  2. Custom Block Rules: Block specific categories, subcategories, or individual patterns
  3. Custom Allow Rules: Create exceptions for specific bots

The middleware automatically loads and applies these settings.

Default Patterns

The gem includes default patterns for common AI agents:

AI Assistants (Usually Allowed)

  • ChatGPT-User/* - OpenAI ChatGPT user requests
  • Claude-User/* - Anthropic Claude user requests
  • Perplexity-User/* - Perplexity AI user requests
  • Gemini-User/* - Google Gemini user requests

AI Model Training Crawlers (Can be Blocked)

  • GPTBot/* - OpenAI training crawler
  • ClaudeBot/* - Anthropic training crawler
  • CCBot/* - Common Crawl bot
  • meta-externalagent/* - Meta training crawler
  • Applebot-Extended/* - Apple training crawler

AI Referrers

  • chat.openai.com, chatgpt.com - ChatGPT
  • claude.ai - Claude
  • perplexity.ai - Perplexity
  • gemini.google.com - Gemini
  • copilot.microsoft.com - Microsoft Copilot

Framework Integration

Rails Integration

For Rails applications, you can also create an initializer:

# config/initializers/spyglasses.rb
Spyglasses.configure do |config|
  config.api_key = Rails.application.credentials.spyglasses_api_key
  config.debug = Rails.env.development?
  config.platform_type = 'rails'
  config.exclude_paths = [
    '/rails/active_storage',
    '/admin',
    /^\/api\/internal/
  ]
end

# Add middleware
Rails.application.config.middleware.use Spyglasses::Middleware

Sinatra Integration

# app.rb
require 'sinatra'
require 'spyglasses'

configure do
  Spyglasses.configure do |config|
    config.api_key = ENV['SPYGLASSES_API_KEY']
    config.platform_type = 'sinatra'
  end
  
  use Spyglasses::Middleware
end

Testing

Add to your test helper:

# spec/spec_helper.rb or test/test_helper.rb
require 'spyglasses'

# Disable API calls in tests
Spyglasses.configure do |config|
  config.api_key = nil
  config.auto_sync = false
end

Run the test suite:

$ bundle exec rspec

Development

After checking out the repo, run:

$ bin/setup      # Install dependencies
$ rake spec      # Run tests  
$ rake console   # Interactive console
$ rake check     # Run all checks (tests + linting)

Performance

The Spyglasses middleware is designed for high-performance applications:

  • Pattern Caching: Regex patterns are compiled and cached
  • Background Logging: API calls are made in background threads
  • Minimal Overhead: Typical overhead is <1ms per request
  • Smart Exclusions: Static assets and health checks are automatically excluded

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/spyglasses/spyglasses-ruby.

License

The gem is available as open source under the terms of the MIT License.

Support