seo_geo

SEO and GEO for Rails in one engine: one initializer and one layout line.

seo_geo handles SEO (search engines) and GEO, Generative Engine Optimization: being found, read and quoted by AI search and assistants such as ChatGPT, Claude, Perplexity and Gemini. It is not a geolocation, GeoIP or maps library.

Rails apps usually rewrite the same SEO code by hand, and the copies drift apart. seo_geo replaces that code with one configuration file. It renders the head (title, description, canonical URL, hreflang, Open Graph, JSON-LD). It serves robots.txt, sitemap.xml and llms.txt per host, and Markdown versions of your pages for agents. It notifies IndexNow and audits the result. All of this follows one crawler policy and one copy source per page, so the FAQ a crawler reads in JSON-LD is the FAQ a person sees.

• Head in one line — <%= seo_geo_head %> emits exactly one <title> (through title_template), a truncated meta description, a canonical URL folded onto canonical_origin, hreflang + x-default, robots noindex outside production, search-console verification metas, Open Graph/Twitter tags, a <link rel="alternate" type="text/markdown"> and a JSON-LD @graph (CSP nonce aware). seo_geo_html_lang fills <html lang>.
• Per-request values — seo title:, description:, image:, noindex:, index:, canonical:, type:, json_ld: in controllers; SeoGeo::JsonLd checks the properties Google requires for each type.
• One copy source per page — SeoGeo.page(:home) / seo_geo_page(:home) reads config/seo_geo/pages/*.yml, I18n or Ruby. The same copy feeds the head, the FAQPage JSON-LD, the visible FAQ (seo_geo_faq) and Inertia props (Page#props).
• Crawler fallback body — seo_geo_fallback_body(:home) renders that copy as static HTML for crawlers that do not run JavaScript on Inertia/SPA pages; window.SeoGeo.mounted() hides it for people.
• Purpose-based crawler policy — crawlers.training, crawlers.search and crawlers.user (user-triggered fetches such as ChatGPT-User) are :allow or :deny. That one policy drives the robots.txt groups and the Content-Signal line and header, plus AIPREF Content-Usage (experimental).
• GET /robots.txt per host — RFC 9309 groups that repeat every host Disallow. Undeclared hosts fail closed (Disallow: /), and so do non-production environments.
• GET /sitemap.xml — static paths and lazy collections (add -> { Post.published }, path: ...), a sitemap index above 50,000 URLs, never an invented lastmod.
• GET /llms.txt and /llms-full.txt — generated from the page copy and the sitemap. Both use the same publication rule as sitemap.xml, so robots-blocked and noindex pages stay out.
• Markdown for agents — include SeoGeo::Markdownable answers Accept: text/markdown and /page.md URLs with YAML frontmatter, the main content and JSON-LD, plus Vary: Accept.
• IndexNow — SeoGeo::IndexNow.submit(urls), SeoGeo::IndexNowJob with retries, the model macro seo_geo_indexnow on: :publish, url: ->(post) { post_url(post) }, and the key file at /<key>.txt.
• OG images — GET /og/:key.png with your own backend (or the optional ferrum card renderer), cached, with a bounded number of concurrent renders and a fallback image.
• rake seo_geo:audit — fetches every host's robots.txt, sitemap.xml, llms.txt and listed pages in-process and reports title, canonical, noindex, JSON-LD, FAQ, Markdown and shadowing problems. Exits 1 on errors; FORMAT=json for CI.
• rails g seo_geo:install — initializer, sample page copy and the engine mount.
• Test helpers — SeoGeo::Testing Minitest assertions and RSpec matchers (require "seo_geo/rspec") built on the audit's checks; with_seo_geo_environment("production") { ... } / SeoGeo::Testing.with_environment to see production robots.txt and sitemaps from a test.
• SeoGeo::BotVerifier (opt-in) — reverse DNS and published IP ranges tell you whether a request that claims to be Googlebot or GPTBot really is. The gem itself never uses the User-Agent for access decisions.

# Gemfile
gem "seo_geo"

• Version 0.1.0, released to RubyGems on 2026-10-14.
• CI runs Ruby 3.2, 3.3 and 3.4 × Rails 7.2, 8.0 and 8.1 (every cell), each against an ERB app and an Inertia-style app.
• Runtime dependencies: railties, actionpack, meta-tags ~> 2.24 and reverse_markdown ~> 3.0. ferrum (OG card rendering) and ActiveJob (SeoGeo::IndexNowJob) are optional.
• Piloted against a real Rails + Inertia (Svelte) app's codebase (four hostnames folded onto one canonical origin): its full request spec suite and rake seo_geo:audit pass in a production-like configuration. Not yet deployed to production.
• Every code example in this README and in AGENTS.md lives in docs/examples/ and runs in the test suite (test/docs/), together with the output it claims.

bundle add seo_geo
bin/rails g seo_geo:install https://www.example.com

The generator writes config/initializers/seo_geo.rb and config/seo_geo/pages/home.yml. It also mounts the engine as the last route, so your own routes win where paths overlap. A minimal configuration:

# config/initializers/seo_geo.rb
SeoGeo.configure do |seo|
  seo.site.name = "Example"
  seo.site.description = "Example helps small teams plan their week."
  seo.canonical_origin = "https://www.example.com"
  seo.canonical_aliases = ["example.com"]           # folded onto www.example.com
  seo.host "www.example.com", disallow: ["/admin"]  # every crawler group repeats this
  seo.crawlers.training = :deny                     # GPTBot, ClaudeBot, CCBot, ... => Disallow: /
end

SeoGeo.sitemap.draw do
  add "/", priority: 1.0
  add "/pricing"
end

SeoGeo.config.llms_txt.page :home, path: "/"

# config/routes.rb
Rails.application.routes.draw do
  root "pages#home"
  get "/pricing", to: "pages#pricing"

  mount SeoGeo::Engine, at: "/"  # last, so your own routes win where paths overlap
end

One layout line replaces your <title>, meta description, canonical link and JSON-LD:

<%# app/views/layouts/application.html.erb %>
<!DOCTYPE html>
<html lang="<%= seo_geo_html_lang %>">
  <head>
    <meta charset="utf-8">
    <%= seo_geo_head %>
  </head>
  <body>
    <main><%= yield %></main>
  </body>
</html>

Page copy lives in one file per page. title and description feed the head; faq feeds both the visible FAQ and the FAQPage JSON-LD:

# config/seo_geo/pages/home.yml — one copy source for the head, the visible FAQ and FAQPage JSON-LD
en:
  title: Plan your week in five minutes
  description: Example turns a messy task list into a weekly plan your whole team can see, in five minutes.
  heading: Plan your week in five minutes
  intro: Example turns a messy task list into a weekly plan your whole team can see.
  faq:
    - question: Is Example free?
      answer: Teams of up to three people use Example for free.

# app/controllers/pages_controller.rb
class PagesController < ApplicationController
  include SeoGeo::Markdownable  # Accept: text/markdown and /pricing.md

  def home
    @page = seo_geo_page(:home)  # title, description and FAQPage JSON-LD from home.yml
  end

  def pricing
    seo title: "Pricing", description: "Free for up to three people; paid plans for larger teams, billed monthly."
  end
end

<%# app/views/pages/home.html.erb %>
<h1><%= @page.heading %></h1>
<p><%= @page.intro %></p>
<%= seo_geo_faq(@page) %>

<%# app/views/pages/pricing.html.erb %>
<h1>Pricing</h1>
<p>Free for up to three people. Paid plans start at $8 per user per month.</p>

In production, GET /pricing on example.com (an alias) renders this head (excerpt):

<title>Pricing | Example</title>
<meta name="description" content="Free for up to three people; paid plans for larger teams, billed monthly.">
<link rel="canonical" href="https://www.example.com/pricing">
<link rel="alternate" href="https://www.example.com/pricing.md" type="text/markdown">
<meta property="og:locale" content="en_US">

GET /robots.txt on www.example.com: search and user-triggered fetchers share one rule block, and denied training crawlers get Disallow: / plus every host rule:

User-agent: *
User-agent: Googlebot
User-agent: OAI-SearchBot
User-agent: ChatGPT-User
Allow: /.well-known/
Disallow: /admin
Content-Signal: search=yes, ai-input=yes, ai-train=no

User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
Allow: /.well-known/
Disallow: /
Disallow: /admin

Sitemap: https://www.example.com/sitemap.xml

A host you did not declare fails closed:

User-agent: *
Allow: /.well-known/
Disallow: /

GET /llms.txt links the Markdown form of each listed page:

# Example

> Example helps small teams plan their week.

## Pages

- [Plan your week in five minutes](https://www.example.com/index.md): Example turns a messy task list into a weekly plan your whole team can see, in five minutes.

An agent asking for Markdown gets frontmatter, the main content and Vary: Accept:

curl -s -H "Host: www.example.com" -H "Accept: text/markdown" http://localhost:3000/pricing
curl -s -H "Host: www.example.com" http://localhost:3000/pricing.md

content-type: text/markdown; charset=utf-8
vary: Accept
content-signal: search=yes, ai-input=yes, ai-train=no
---
title: Pricing | Example
description: Free for up to three people; paid plans for larger teams, billed monthly.
canonical: https://www.example.com/pricing
locale: en
---
# Pricing
Free for up to three people. Paid plans start at $8 per user per month.

Then check the whole site. The audit requests every host in-process, as production would serve it, even in development: it switches seo_geo to the first indexable environment and, outside production, adds the configured hosts to config.hosts for that run only.

bin/rails seo_geo:audit            # FORMAT=json HOSTS=www.example.com MAX_PAGES=200

seo_geo audit: temporarily allowing www.example.com, example.com in config.hosts
seo_geo audit: 0 error(s), 0 warning(s) (1 audited as production, 3 pages, 1 llms_links)

On a real app warnings appear in the same line, for example seo_geo audit: 0 error(s), 2 warning(s) (1 audited as production, 7 pages, 6 llms_links), followed by one line per finding. Calling SeoGeo::Audit::Runner from bin/rails runner needs SEO_GEO_AUDIT=1 set before boot for the same host allowance.

Inertia sends an empty <div id="app">, and crawlers that do not run JavaScript (GPTBot, ClaudeBot, PerplexityBot, Yeti) would see nothing. Pass the page copy as props for people, and render the same copy as a fallback body for crawlers:

# app/controllers/pages_controller.rb (inertia_rails)
class PagesController < ApplicationController
  include SeoGeo::Markdownable
  seo_geo_markdown only: :home, page: :home  # Markdown from the page copy, not the empty #app div

  def home
    render inertia: "Home", props: { seo: seo_geo_page(:home).props }
  end
end

<%# app/views/layouts/application.html.erb (Inertia) %>
<!DOCTYPE html>
<html lang="<%= seo_geo_html_lang %>">
  <head>
    <meta charset="utf-8">
    <%= seo_geo_head %>
  </head>
  <body>
    <%= yield %>
    <% if (page = seo_geo_state.page) %>
      <%= seo_geo_fallback_body(page) %>
    <% end %>
  </body>
</html>

// app/frontend/entrypoints/application.js — after createInertiaApp(...) has mounted:
window.SeoGeo?.mounted() // hides the crawler fallback body for people running JavaScript

The server HTML then carries the copy inside <main data-seo-fallback>, next to the props:

<main data-seo-fallback>
<h1>Plan your week in five minutes</h1>
<p>Example turns a messy task list into a weekly plan your whole team can see.</p>
<h3>Is Example free?</h3>
<p>Teams of up to three people use Example for free.</p>

The seo prop is Page#props: key, locale, title, description, heading, intro, features ([{title:, description:}]), faq ([{question:, answer:}]) and any extra YAML keys. Render the visible FAQ from seo.faq; that keeps it identical to the FAQPage JSON-LD. Extra crawler-only HTML (links to sign-up or legal pages) goes in a block, seo_geo_fallback_body(page) { ... }, or in content_for(:seo_geo_fallback_extra); both land at the end of <main>. The head snippet hides the fallback while JavaScript runs and shows it again if mounted() is not called within five seconds. The tests run this example with a stand-in for inertia_rails' render inertia:; seo_geo does not depend on inertia_rails.

Every key, default and validation rule is in docs/DESIGN.md § Configuration. The ones most apps touch:

Sitemap sources: SeoGeo.sitemap.draw { add "/pricing"; add -> { Post.published }, path: ->(post) { "/posts/#{post.slug}" }, lastmod: :updated_at }. Pass collections as a lambda, so boot loads no model and a development reload never keeps a stale class. Records are read with find_each (select must include id, order is ignored); batch: false keeps the scope's order.

• No rich-result promises. It emits FAQPage JSON-LD because AI answers and search engines read it, but since 2023 Google shows FAQ rich results only for a few authoritative sites and no longer shows HowTo rich results. The gem never adds markup for content that is not on the page.
• The User-Agent is never identity. robots.txt is the only place the gem reads crawler names, Markdown is chosen by the Accept header, and every client gets the same public content. The audit warns when app code branches on request.user_agent. When a bot must be trusted, use SeoGeo::BotVerifier and require verified == true.
• llms.txt is generated, not promoted. Its effect is unproven. Evil Martians measured ~268,000 agent requests over two months (2026): /llms.txt was fetched ~660 times and /llms-full.txt ~110 times, and only ~37 of those fetches came from named AI assistants (source). In the same data, ChatGPT-User made 73% of agent requests and read HTML, while Claude Code asked for Accept: text/markdown 76% of the time. seo_geo therefore puts server HTML first and Markdown negotiation second, and keeps llms.txt cheap.
• No invented data. No Time.now as lastmod, and no prices, ratings or counts it was not given.
• No geolocation. GEO here is Generative Engine Optimization.

bundle exec rake test                                    # unit, docs, and app tests against both dummy apps
BUNDLE_GEMFILE=gemfiles/rails_7_2.gemfile bundle exec rake test

Design and module boundaries: docs/DESIGN.md. Changes: CHANGELOG.md. Guidance for coding agents: AGENTS.md.

MIT. See LICENSE.txt.