No commit activity in last 3 years
No release in over 3 years
There's a lot of open issues
Embulk executor plugin to run plugins on remote server.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
 Dependencies

Development

~> 1.0
>= 10.0
 Project Readme

Remote server executor plugin for Embulk

Embulk executor plugin to run Embulk tasks on remote servers.

Overview

  • Plugin type: executor

Notes

  • It's still very experimental version, so might change its spec without notification.
  • It might have some issues or bugs. I would appreciate it if you use this and give me reports/feedback!

Configuration

  • hosts: List of remote servers (hostname or hostname:port, default port is 30001). If not specified, the executor runs as the local mode, which starts an Embulk server on its own process (array of string)
  • timeout_seconds: Timeout seconds of the whole execution (integer, default: 3600)
  • use_tls: Enable to connect server over TLS (boolean, default: false)
  • cert_p12_file: Information of a PKCS12 file used as your client certificate. It would be needed when client authentication is enabled on Embulk server.
    • path: Path of the file (string, required)
    • password: Password of the file (string, required)
  • ca_cert_path: Path of a CA certificate file. It would be needed when Embulk server uses a certificate signed by an unknown CA.

Example

exec:
  type: remoteserver
  hosts:
    - embulk-server1.local
    - embulk-server2.local:30002
  timeout_seconds: 86400

Embulk server

The server receives requests from client (Embulk) and run Embulk tasks, then returns results to client. It communicates with clients via TCP 30001 port.

Running Embulk server as a Docker container

The image is hosted by DockerHub.
You can try running Embulk server by the following command.

$ docker run --rm -p 30001:30001 kamatama41/embulk-executor-remoteserver

Configure Embulk server

There are some environment variables to configure the server

  • BIND_ADDRESS: Bind address of the server (default: 0.0.0.0)
  • PORT: Port number to listen (default: 30001)
  • USE_TLS: Try to connect to client via TLS if true (default: false)
  • REQUIRE_TLS_CLIENT_AUTH: Require client authentication if true (default: false)
  • CERT_P12_PATH, CERT_P12_PASSWORD: Path and password of the PKCS12 file for server certificate. Ignored unless both is set.
  • CA_CERT_PATH: Path of the CA certificate file. It would be needed when client authentication is enabled and client certificate is signed by an unknown CA.

Build

$ ./gradlew gem  # -t to watch change of files and rebuild continuously