Repository is archived
No release in over 3 years
Low commit activity in last 3 years
There's a lot of open issues
This gem is a logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/plugin install gemname. This gem is not a stand-alone program
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Dependencies

Development

Runtime

<= 2.99, >= 1.60
~> 0.0.22
 Project Readme

Logstash Input for CloudWatch Logs

Gem Downloads Software License Build Status

Stream events from CloudWatch Logs.

Update July 2023

Hello, this project has not received any maintenance since May of 2018. It is no longer a path I am using professionally, and I can not devote the time to maintaining it.

Purpose

Specify an individual log group or array of groups, and this plugin will scan all log streams in that group, and pull in any new log events.

Optionally, you may set the log_group_prefix parameter to true which will scan for all log groups matching the specified prefix(s) and ingest all logs available in all of the matching groups.

Usage

Parameters

Parameter Input Type Required Default
log_group string or Array of strings Yes
log_group_prefix boolean No false
start_position beginning, end, or an Integer No beginning
sincedb_path string No $HOME/.sincedb*
interval number No 60
aws_credentials_file string No
access_key_id string No
secret_access_key string No
session_token string No
region string No us-east-1
codec string No plain

start_position

The start_position setting allows you to specify where to begin processing a newly encountered log group on plugin boot. Whether the group is 'new' is determined by whether or not the log group has a previously existing entry in the sincedb file.

Valid options for start_position are:

  • beginning - Reads from the beginning of the group (default)
  • end - Sets the sincedb to now, and reads any new messages going forward
  • Integer - Number of seconds in the past to begin reading at

Logstash Default config params

Other standard logstash parameters are available such as:

  • add_field
  • type
  • tags

Example

input {
    cloudwatch_logs {
        log_group => [ "/aws/lambda/my-lambda" ]
        access_key_id => "AKIAXXXXXX" 
        secret_access_key => "SECRET"
    }
}

Development

The default logstash README which contains development directions and other information has been moved to DEVELOPER.md.

Contributing

All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.

Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.

It is more important to the community that you are able to contribute.

For more information about contributing, see the CONTRIBUTING file.