No commit activity in last 3 years
No release in over 3 years
Formats Avro files for other file output plugins.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
 Dependencies

Development

~> 1.0
>= 10.0
 Project Readme

Avro formatter plugin for Embulk

Avro formatter plugin for Embulk.

Overview

  • Plugin type: formatter

Support avro types

Support all avro basic types.

  • string
  • int
  • long
  • float
  • double
  • boolean
  • enum
  • fixed
  • array
  • map
  • record

But typecasting is restricted by embulk column type. See. AvroValueConverters,

Configuration

  • avsc: avro schema (avsc) filepath (string, required)
  • codec: avro codec type (enum: deflate, bzip2, xz, snappy, optional)
  • compression_level: avro codec compression level (integer, optional, for only deflate and xz codec)
  • skip_error_record: If you want to skip error record, set true (boolean, default: false)

Example

out:
  type: file
  path_prefix: ./out_
  file_ext: avro
  formatter:
    type: avro
    avsc: schema.avsc
    skip_error_record: true

Build

$ ./gradlew gem  # -t to watch change of files and rebuild continuously