This repository contains the Fluentd MacOs unified logs input Plugin.
Install this gem when setting up fluentd:
gem install fluent-plugin-macos-log
This is a process execution input plugin for Fluentd that periodically executes external
log show command and parses log events into Fluentd's core system.
Each execution alters
end time input parameters of log utility to slowly iterates over log data. The iteration
interval can be configured by user, but cannot be lower than 1s.
There are multiple configurations one can use:
Uses human-readable output of the command. The process output is parsed using
and logic, which combines multiple lines together. The parameter
log_line_start defines regular expresion, which matches the
beginning of line. Anything in between will be merged into single log entry. Although the parser is
regexp, user can select any other supported parser.
To configure this in fluentd:
<source> @type macoslog tag macos pos_file /path/to/position/file run_interval 10s </source>
Use when more detailed output is required. It uses
ndjson style of
log command, which is then parsed by json parser.
To configure this in fluentd:
<source> @type macoslog style ndjson tag macos pos_file last-starttime.log run_interval 10s <parse> @type json time_type string time_key timestamp time_format %Y-%m-%d %H:%M:%S.%L%z </parse> </source>
This plugin inherits Fluentd's standard input parameters.
The command used by default is
log show --style default --start @%s --end @%s in order to combine multiple lines and iterate over
each period of time. Notice the
end parameters use
@, which is notation for unix timestamp format, used by plugin.
Optionally the plugin uses position file, where it records last processed timestamp. Whenever the
restarts the plugin picks up from the last position. When no position files is used the plugin starts from current time
and keeps last position only in memory.
Optionally one can configure any
predicate to filter required logs.
command- external command to be executed for each interval. The command's first parameter noted ruby's
%sas start unix timestamp and the second
%sfor end timestamp. Default:
log show --style default --start @%s --end @%s
predicate- log filter predicate as per Apple's documentation. Default:
levels- Controls what logging levels will be shown. Supported by
- [no-]backtrace Control whether backtraces are shown
- [no-]debug Control whether "Debug" events are shown
- [no-]info Control whether "Info" events are shown
- [no-]loss Control whether message loss events are shown
- [no-]signpost Control whether signposts are shown
style- Controls style of logging tool output.
ndjson- single lined json format output. When used, the json parser must be configured.
connect_mode- Control target IO:
read: Read logs from stdio
read_with_stderr: Read logs from stdio and stderr (mainly for debug).
parsersection - Refer these for more details about parse section. Default
tag- The tag of the output events.
run_interval- The interval time between periodic program runs.
max_age- The time base max age of logs to process. Default
pos_file- Fluentd will record the position it last read from external command. Don't share pos_file between in_macoslog configurations. It causes unexpected behavior e.g. corrupt pos_file content.
log_line_start- Regexp of start of the log to combine multiline logs. Default:
log_header_lines- Number of header lines to skip when parsing. When
ndjsonstyle used the parameter refers to number of footer lines to be skipped. Default:
One can configure own parser:
<source> @type macoslog tag macos pos_file /path/to/position/file run_interval 10s <parse> @type tsv keys avg1,avg5,avg15 delimiter " " </parse> </source>
Example configuration for sending logs over to Loggly. The input plugin collects unified logs with filter
process == "sharingd"
10s while recording position in file
It uses output fluent-plugin-loggly configured in buffer mode.
<source> @type macoslog predicate process == "sharingd" tag macos pos_file /path/to/position/file run_interval 10s </source> <match macos> type loggly_buffered loggly_url https://logs-01.loggly.com/bulk/xxx-xxxx-xxxx-xxxxx-xxxxxxxxxx output_include_time true time_precision_digits 3 buffer_type file buffer_path /path/to/buffer/file flush_interval 10s </match>
This plugin is targeting Ruby 2.6 and Fluentd v1.0, although it should work with older versions of both.
We have a Makefile to wrap common functions and make life easier.
To install fluentd on MacOs use following ruby environment.
brew install rbenv ruby-build echo 'if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi' >> ~/.zshrc source ~/.zshrc rbenv install 2.6.3 rbenv global 2.6.3 gem install fluentd --no-doc
Install latest bundler
gem install bundler
Release in RubyGems
To release a new version, update the version number in the GemSpec and then, run:
Bug reports and pull requests are welcome on GitHub at: https://github.com/loggly/fluent-plugin-macos-log
Please open an issue, we'd love to hear from you.