Project

embulk-filter-expand_json_array

0.0
No commit activity in last 3 years
No release in over 3 years
embulk-filter-expand_json_arrayainoya/embulk-filter-expand_json_arrayHomepageDocumentationSource CodeBug TrackerWiki
Expand Json Array
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
 Popularity
Downloads
23,224
Stars
2
Forks
1
Watchers
3
 Releases
Current version
0.1.1
Total releases
2
First release
2016-10-12
Latest release
2016-10-12
 Issues
Open Issues
1
Closed Issues
0
Total Issues
1
Issue Closure Rate
0%
 Development
Primary Language
Ruby
Licenses
MIT
Average date of last 50 commits
2016-10-12
Reverse Dependencies
0

 Dependencies

Development

bundler
>= 1.10.6
embulk
>= 0.8.13
rake
>= 10.0

Runtime

jsonpath
>= 0.5.8
 Project Readme

Expand Json Array filter plugin for Embulk Gem Version

Overview

  • Plugin type: filter

This is a embulk plugin for expanding record which has json column includes N-size array to N records.

For example, plugin expands below record which has json column json_value includes size-3 array value as latest_receipt_info to 3 records.

Before:

+-------------------------+---------+-------------+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|          time:timestamp | id:long | name:string |       score:double |                                                                                                                                                                                                                                 json_payload:string |
+-------------------------+---------+-------------+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 2015-10-11 00:00:00 UTC |       0 |   Avis Lind | -3256.869635206057 | {"phone_numbers":"1-276-220-7263","app_id":0,"point":-1601.6890336884562,"created_at":"2015-10-07 20:23:57 +0900","profile":{"like_words":["maiores","eum","aut"],"anniversary":{"voluptatem":"dolor","et":"ullam"}},"latest_receipt_info":[1,2,3]} |
+-------------------------+---------+-------------+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Filtered:

+-------------------------+---------+-------------+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+
|          time:timestamp | id:long | name:string |       score:double |                                                                                                                                                                                                                                 json_payload:string | latest_receipt_info:string |
+-------------------------+---------+-------------+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+
| 2015-10-11 00:00:00 UTC |       0 |   Avis Lind | -3256.869635206057 | {"phone_numbers":"1-276-220-7263","app_id":0,"point":-1601.6890336884562,"created_at":"2015-10-07 20:23:57 +0900","profile":{"like_words":["maiores","eum","aut"],"anniversary":{"voluptatem":"dolor","et":"ullam"}},"latest_receipt_info":[1,2,3]} |                          1 |
| 2015-10-11 00:00:00 UTC |       0 |   Avis Lind | -3256.869635206057 | {"phone_numbers":"1-276-220-7263","app_id":0,"point":-1601.6890336884562,"created_at":"2015-10-07 20:23:57 +0900","profile":{"like_words":["maiores","eum","aut"],"anniversary":{"voluptatem":"dolor","et":"ullam"}},"latest_receipt_info":[1,2,3]} |                          2 |
| 2015-10-11 00:00:00 UTC |       0 |   Avis Lind | -3256.869635206057 | {"phone_numbers":"1-276-220-7263","app_id":0,"point":-1601.6890336884562,"created_at":"2015-10-07 20:23:57 +0900","profile":{"like_words":["maiores","eum","aut"],"anniversary":{"voluptatem":"dolor","et":"ullam"}},"latest_receipt_info":[1,2,3]} |                          3 |
+-------------------------+---------+-------------+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------+

Configuration

  • json_column_name: column name of json value in record (string , required)
  • root: pointer of array value in json, specified as jsonpath(string , required)
  • expanded_column_name: column name after parsing target array value in json (string , required)

Example

filters:
  - type: expand_json_array
    json_column_name: json_payload
    root: "$.latest_receipt_info"
    expanded_column_name: latest_receipt_info

Build

$ rake

Reference