Remote server executor plugin for Embulk
Embulk executor plugin to run Embulk tasks on remote servers.
Overview
- Plugin type: executor
Notes
- It's still very experimental version, so might change its spec without notification.
- It might have some issues or bugs. I would appreciate it if you use this and give me reports/feedback!
Configuration
-
hosts: List of remote servers (
hostnameorhostname:port, default port is30001). If not specified, the executor runs as the local mode, which starts an Embulk server on its own process (array of string) -
timeout_seconds: Timeout seconds of the whole execution (integer, default:
3600) -
use_tls: Enable to connect server over TLS (boolean, default:
false) -
cert_p12_file: Information of a PKCS12 file used as your client certificate. It would be needed when client authentication is enabled on Embulk server.
- path: Path of the file (string, required)
- password: Password of the file (string, required)
- ca_cert_path: Path of a CA certificate file. It would be needed when Embulk server uses a certificate signed by an unknown CA.
Example
exec:
type: remoteserver
hosts:
- embulk-server1.local
- embulk-server2.local:30002
timeout_seconds: 86400Embulk server
The server receives requests from client (Embulk) and run Embulk tasks, then returns results to client. It communicates with clients via TCP 30001 port.
Running Embulk server as a Docker container
The image is hosted by DockerHub.
You can try running Embulk server by the following command.
$ docker run --rm -p 30001:30001 kamatama41/embulk-executor-remoteserverConfigure Embulk server
There are some environment variables to configure the server
-
BIND_ADDRESS: Bind address of the server (default:0.0.0.0) -
PORT: Port number to listen (default:30001) -
USE_TLS: Try to connect to client via TLS iftrue(default:false) -
REQUIRE_TLS_CLIENT_AUTH: Require client authentication iftrue(default:false) -
CERT_P12_PATH,CERT_P12_PASSWORD: Path and password of the PKCS12 file for server certificate. Ignored unless both is set. -
CA_CERT_PATH: Path of the CA certificate file. It would be needed when client authentication is enabled and client certificate is signed by an unknown CA.
Build
$ ./gradlew gem # -t to watch change of files and rebuild continuously