Project

logflume

0.0
No commit activity in last 3 years
No release in over 3 years
logflumeshadowbq/logflumeHomepageDocumentationSource CodeBug TrackerWiki
A library to continually dump the contents of new logfiles into a POSIX FIFO pipe
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Popularity
Downloads
24,827
Stars
1
Forks
0
Watchers
2
 Releases
Current version
0.0.11
Total releases
11
First release
2015-02-09
Latest release
2015-04-02
 Issues
Open Issues
1
Closed Issues
0
Total Issues
1
Issue Closure Rate
0%
 Development
Primary Language
Ruby
Licenses
MIT
Average date of last 50 commits
2015-03-13
Reverse Dependencies
0

 Dependencies

Development

codeclimate-test-reporter
>= 0
fivemat
>= 0
pry
>= 0
rake
>= 0
rspec
~> 3.1
simplecov
>= 0

Runtime

directory_watcher
~> 1.5.1
mkfifo
>= 0.0.1
ruby-fifo
= 0.0.1
 Project Readme

logflume

Gem Version Gem

A library to continually dump the contents of new logfiles into a POSIX FIFO pipe

Build Status

Build Status Code Climate Test Coverage GitHub tag

FEATURES/PROBLEMS:

A library to continually dump the contents of new logfiles into a POSIX FIFO pipe. This function can mimic what syslog-ng does with FIFO exports. This library is generally dependant on POSIX pipes, so it will not act kindly in Windows ENV.

INSTALL:

  • sudo gem install logflume

EXAMPLE IMPLEMENTATION:

This is a single blocking logflume. We can call this worker.rb.

You need three posix shells for this to properly work

  • Ruby execution shell for worker.rb
  • A process to cat '/tmp/logflume.conveyor.fifo'
  • A shell/process to send SIG controls (INT,QUIT,TERM)

worker.rb Source

#!/usr/bin/env ruby
require "rubygems"
$:.unshift File.join(File.dirname(__FILE__), *%w[. lib])

require "logflume"


flume = Logflume::Flume.new
flume.dir=File.expand_path(File.join(File.dirname(__FILE__), './spec/data/flume'))
flume.glob='*.log'
flume.blocking=true
flume.pipe='/tmp/logflume.conveyor.fifo'
flume.logger = Logger.new(STDOUT)
flume.logger.level = Logger::INFO
flume.start

gets

CAT - Dump the FIFO pipe

[shadowbq@fn-pound](~)$ cat /tmp/logflume.conveyor.fifo
2015-02-04 13:17:16 startup archives unpack
2015-02-04 13:17:16 install libjpeg-turbo-progs:amd64 <none> 1.3.0-0ubuntu2
2015-02-04 13:17:16 status half-installed libjpeg-turbo-progs:amd64 1.3.0-0ubuntu2
2015-02-04 13:17:17 status triggers-pending man-db:amd64 2.6.7.1-1ubuntu1
2015-02-04 13:17:17 status unpacked libjpeg-turbo-progs:amd64 1.3.0-0ubuntu2
2015-02-04 13:17:17 status unpacked libjpeg-turbo-progs:amd64 1.3.0-0ubuntu2
2015-02-04 13:17:17 install libjpeg-progs:amd64 <none> 8c-2ubuntu8
2015-02-04 13:17:17 status half-installed libjpeg-progs:amd64 8c-2ubuntu8
2015-02-04 13:17:17 status unpacked libjpeg-progs:amd64 8c-2ubuntu8
2015-02-04 13:17:17 status unpacked libjpeg-progs:amd64 8c-2ubuntu8
2015-02-04 13:17:17 install jhead:amd64 <none> 1:2.97-1
2015-02-04 13:17:17 status half-installed jhead:amd64 1:2.97-1
2015-02-04 13:17:17 status unpacked jhead:amd64 1:2.97-1
2015-02-04 13:17:17 status unpacked jhead:amd64 1:2.97-1
2015-02-04 13:17:17 install libimage-exiftool-perl:all <none> 9.46-1

Example worker.rb Output

$ ruby worker.rb
I, [2015-02-09T00:31:22.389096 #20712]  INFO -- : new File found => /home/shadowbq/sandbox/github-shadowbq/logflume/spec/data/flume/dpkg.log
I, [2015-02-09T00:31:22.395498 #20712]  INFO -- : new File found => /home/shadowbq/sandbox/github-shadowbq/logflume/spec/data/flume/test.log
(.. RCVR SIGQUIT see below ..)
Terminating...

Control Signal Hooks

$ kill -l
 1) SIGHUP	 2) SIGINT	 3) SIGQUIT	 4) SIGILL	 5) SIGTRAP [..]

$ ps aux |grep ruby
shadowbq 20712  0.4  0.0 262508 14836 pts/0    Sl+  00:30   0:04 ruby worker.rb

$ kill -3 20712

Troubleshooting

If mkfifo gem fails to load:

`rescue in <top (required)>': No such file to load, please install 'win32/pipe' or 'mkfifo' (LoadError)

check the permissions on the lib of the gems directory

Ala..

sudo chmod a+rx /usr/local/lib/ruby/gems/2.0/gems/mkfifo-0.0.1/lib

LICENSE:

(The MIT License)