logstash-filter-ipinfo-db-zuin

A Logstash filter plugin that looks up the country and ASN of an IP address in the ipinfo.io free country and ASN database (.mmdb), locally, without calling the ipinfo API per event.

This is a fork of logstash-filter-ipinfo-db. The gem is renamed, but the filter is still called ipinfo-db, so existing pipeline configs keep working. Do not install both gems in the same Logstash.

The license is Apache 2.0.

bin/logstash-plugin install logstash-filter-ipinfo-db-zuin

filter {
  ipinfo-db {
    source => "ip"
    token => "${IPINFO_TOKEN}"
  }
}

On first start, the plugin downloads the database with your ipinfo.io token to <path.data>/plugins/filters/ipinfo-db/country_asn.mmdb (for example /var/lib/logstash/... for deb/rpm installs). Store the token in an environment variable or the Logstash keystore:

bin/logstash-keystore add IPINFO_TOKEN

The lookup result is written to the ipinfo-db field. For 8.8.8.8 it looks like this (illustrative):

"ipinfo-db": {
  "country": "US",
  "country_name": "United States",
  "continent": "NA",
  "continent_name": "North America",
  "asn": "AS15169",
  "as_name": "Google LLC",
  "as_domain": "google.com"
}

Private IPv4 addresses are cheap to look up. When it loads the database, the plugin indexes the IPv4 /16 blocks that hold no data (private, reserved and unused ranges), which takes about a second, and answers lookups in them without searching the database. You don't need a cidr or ruby filter in front of ipinfo-db to skip private IPv4 addresses.

The plugin does not update the database automatically. To refresh it, delete the file and restart Logstash (needs token, to download it again), or replace the file with a newer copy and restart or reload the pipeline. See docs/index.asciidoc for the full reference.

You need JRuby with Bundler, and a local Logstash for logstash-core. Set LOGSTASH_PATH if it is not in /usr/share/logstash (see Gemfile).

bundle install
bundle exec rspec

Run them with Logstash's JRuby and JDK, and pass the JVM flags that bin/logstash normally adds itself (the JVM_FLAGS list in verify/verify.sh) through JAVA_OPTS. Without them the pipeline specs fail with IllegalAccessError on jdk.compiler. The easiest way to get this right is verify/verify.sh (below), which runs the suite inside the official Logstash image.

The default suite runs offline against spec/fixtures/ipinfo-test.mmdb, a tiny synthetic database (documentation IP ranges only) generated by spec/fixtures/generate_ipinfo_test_mmdb.py. The live test, which downloads the real database from ipinfo.io, is tagged integration and excluded by default:

IPINFO_TOKEN=... bundle exec rspec --tag integration

verify/verify.sh builds the gem, installs it into the official Logstash Docker image, runs real pipelines (download as a non-root user with working directory /, lookups, restart without token, missing-token error) and the RSpec suite on Logstash's JRuby. It needs Docker, Ruby and Python 3:

IPINFO_TOKEN=... verify/verify.sh 9.5.5

Without IPINFO_TOKEN, the checks that download the live database (first start, restart and the integration spec) are reported as SKIP; the install check, the missing-token error and the offline specs still run.

Logs are written to verify/out/. GitHub Actions (.github/workflows/verify.yml) runs the same script against Logstash 8.19.24 and 9.5.5 on pushes and pull requests to main, reading the token from the IPINFO_TOKEN repository secret. Pull requests from forks don't get the secret, so they run the offline checks only.

Build and install the gem:

gem build logstash-filter-ipinfo-db-zuin.gemspec
bin/logstash-plugin install /path/to/logstash-filter-ipinfo-db-zuin-<version>.gem

Or add it to the Logstash Gemfile from your local checkout, then run bin/logstash-plugin install --no-verify:

gem "logstash-filter-ipinfo-db-zuin", :path => "/path/to/logstash-filter-ipinfo-db"

1. Set s.version in logstash-filter-ipinfo-db-zuin.gemspec, and add a ## <version> section at the top of CHANGELOG.md (rename ## Unreleased if there is one).
2. Merge to main.

.github/workflows/tag.yml sees the new gemspec version and pushes v<version>. That tag starts .github/workflows/release.yml, which checks that the tag, gemspec and newest CHANGELOG section agree, runs the verification above, publishes the gem to RubyGems via trusted publishing, and creates the GitHub Release. Versions with a letter (e.g. 0.4.0.rc1) are published as pre-releases.

Open an issue at https://github.com/reflective-technology/logstash-filter-ipinfo-db/issues, or ask in the Logstash discussion forum.