logstash-filter-phpipam

A Logstash filter that looks up an IP-address, and returns results from phpIPAM

Redis is required for this plugin to work.

You can install it using most your distributions package manager.

You can install it with apt:

sudo apt install redis

This plugin can be installed using the logstash-plugin command in $LOGSTASH_HOME:

${LOGSTASH_HOME:-/usr/share/logstash}/bin/logstash-plugin install logstash-filter-phpipam

Redis is primarily used as a LRU cache, and can be configured in a lot of ways.

The two highly recommended settings to set, are maxmemory and maxmemory-policy:

• maxmemory can be tested using the redis-cli --lru-test command
• maxmemory-policy I would set to allkeys-lru

The above settings would limit the memory that Redis can use. The limit should be high enough to contain almost all keys.
If the limit is reached--and maxmemory-policy is set to allkeys-lru--the least accessed keys would be evicted first.

Every key will also have an expiration time of cache_fresshness (default 24 hours) associated, meaning that every key will live for 24 hours maximum.
This is to prevent cached data to become stale, and always keep (approximately) up-to-date data from phpIPAM.

Here are some stats from my production environment:

keys       mem      clients blocked requests            connections
10516      2.87M    12      0       8047901 (+0)        26

app_id can be found in phpIPAM: Administration -> API
It's recommended to use SSL when accessing the app_id in phpIPAM.

By default the lon and lat are mapped as normal floats, NOT geo-points!

To use the latitude and longtitude in Kibana Maps, you either need to:

1. Preload mappings yourself
2. Use preloaded mappings from something like Filebeat (7.0+)

For option 2, if you use the default target of phpipam, you can do something like this, after the phpipam filter:

mutate {
  rename => {
    "[phpipam][location][location]" => "[geo][location]"
  }
}

This example...

phpipam {
  host     => "https://phpipam.local.domain"
  app_id   => "logstash"
  username => "username"
  password => "password"
  source   => "[source][ip]"
  target   => "[source][phpipam]"
}

...would produce:

"source" => {
  "phpipam" => {
    "subnet" => {
      "network"    => "172.16.1.0",
      "bitmask"    => 24,
      "netmask"    => "255.255.255.0",
      "section_id" => 1,
      "wildcard"   => "0.0.0.255",
      "id"         => 1
    },
    "ip" => {
      "description" => "This is my test IP",
      "hostname"    => "test.domain.local",
      "id"          => 1,
      "note"        => "This switch is in test!",
      "address"     => "172.16.1.10",
      "mac"         => "aa:bb:cc:dd:ee:ff"
      "owner"       => "Testing Team"
    },
    "vlan" => {
      "name"      => "TestVLAN",
      "number"    => 100,
      "id"        => 1,
      "domain_id" => 1,
    },
    "device" => {
      "name"        => "test.domain.local",
      "description" => "Juniper Switch",
      "type"        => "Switch",
      "id"          => 1
    },
    "location" => {
      "name"     => "Null Island",
      "id"       => 1,
      "location" => {
        "lat" => 0.0,
        "lon" => 0.0
      },
      "address" => "Null Island, Atlantic Ocean"
    }
  }
}

Provided that all that information is entered in phpIPAM.

Empty values in phpIPAM will not be pulled, therefore the output can vary, depending on the information gathered from the IP-address.