buildkite-sdk

A multi-language SDK for Buildkite managed with Nx. 🪁

Consumes the Buildkite pipeline schema and generates packages for TypeScript, Python, Go, and Ruby!

Prerequisites

For development, you'll need current versions of the following tools:

Node.js, Python, Go, Ruby
For Python: uv, Black
For Ruby: Bundler

See .tool-versions for details. If you're on a Mac, and you use Homebrew, you can run brew bundle and asdf install to get all you need:

brew bundle
asdf install

If you hit any rough edges during development, please file an issue. Thanks!

Useful commands

# Install dependencies.
npm install

# Test all SDKs and apps.
npm test

# Build all SDKs (and write them to ./dist/sdks).
npm run build

# Build all SDK docs (and write them to ./dist/docs).
npm run docs

# Serve the docs locally (which builds them implicitly).
npm run docs:serve

# Run all apps (which writes JSON and YAML pipelines to ./out).
npm run apps

# Watch all projects for changes (which rebuilds the docs and SDKs and re-runs all apps).
npm run watch

# Launch web servers for all docsets and watch all projects for changes. (Requires reload.)
npm run dev

# Format all SDK code.
npm run format

# Publish to npm, PyPi pkg.go.dev, and RubyGems.
npm run publish

# Clear away build and test artifacts.
npm run clean

Installing and using the SDKs

The easiest way to use the SDK is to install the appropriate package for your language of choice, import the library into your program, assemble your pipeline steps programmatically, and serialize the pipeline to JSON or YAML, passing the output to buildkite-agent pipeline upload.

For example, if your language of choice were Ruby:

gem install cnunciato-buildkite

# In ~/.buildkite/pipeline.rb:
require "buildkite"

pipeline = Buildkite::Pipeline.new

pipeline.add_command_step(
  label: "some-label",
  command: "echo 'Hello, World!'"
)

puts pipeline.to_json

# In your pipeline's Settings > Steps:
steps:
    - label: ":pipeline: Generate pipeline"
      command: ruby .buildkite/pipeline.rb | buildkite-agent pipeline upload

This repository uses this approach to ship the Buildkite SDK ... with the Buildkite SDK!

See below for more examples.

Node.js

Install the package:

npm install @cnunciato/buildkite-sdk

Use it in your program:

const { Pipeline } = require("@cnunciato/buildkite-sdk");

const pipeline = new Pipeline();

pipeline.addStep({
    command: "echo 'Hello, world!'",
});

console.log(pipeline.toJSON());
console.log(pipeline.toYAML());

Python

Install the package:

uv add cnunciato-buildkite-sdk

Use it in your program:

from buildkite_sdk import Pipeline

pipeline = Pipeline()
pipeline.add_command_step({"command": "echo 'Hello, world!'"})

print(pipeline.to_json())
print(pipeline.to_yaml())

Go

Install the package:

go get github.com/cnunciato/buildkite-sdk/sdk/go

Use it in your program:

package main

import (
	"fmt"
	"github.com/cnunciato/buildkite-sdk/sdk/go/sdk/buildkite"
)

func main() {
	pipeline := buildkite.Pipeline{}
	command := "echo 'Hello, world!"

	pipeline.AddCommandStep(buildkite.CommandStep{
		Command: &buildkite.CommandUnion{
			String: &command,
		},
	})

	fmt.Println(pipeline.ToJSON())
	fmt.Println(pipeline.ToYAML())
}

Ruby

Install the package:

gem install cnunciato-buildkite

Use it in your program:

require "buildkite"

pipeline = Buildkite::Pipeline.new

pipeline.add_command_step(
  label: "some-label",
  command: "echo 'Hello, World!'"
)

puts pipeline.to_json
puts pipeline.to_yaml

Publishing new versions

All SDKs version on the same cadence. To publish a new version (of all SDKs), follow these steps:

Commit all pending changes. We want the release commit to be "clean" (i.e., to consist only of changes related to the release itself.)
Update the FROM and TO versions in the release:all task in ./project.json.
Leaving that single change uncommitted, make sure you've exported a GitHub access token (as GITHUB_TOKEN -- see below) with push access to main branch of the repository, then run the release script:
```
npm run release
```
This script:
- Updates the version numbers in all affected files
- Rebuilds all SDKs
- Commits all changes (e.g., to version files, lockfiles, and anything else under ./sdk)
- Adds two new tags to mark the release (v0.0.0 and sdk/go/v0.0.0)
- Pushes the commit and tags to GitHub, triggering the publish task
- Creates a new GitHub release
If for some reason the Buildkite publish job doesn't finish successfully, you can run some or all publish tasks from your local machine by exporting the applicable environment variables (again, see below), then running:
```
npm run clean
npm run build
npm run publish                # To publish all packages
npx nx publish sdk/typescript  # To publish only the Node.js package
npx nx publish sdk/python      # To publish only the Python package
npx nx publish sdk/go          # To publish only the Go package
npx nx publish sdk/ruby        # To publish only the Ruby package
```
Once the publish job completes, verify the releases at their respective URLs:

Required environment variables

The following environment variables are required for releasing and publishing:

GITHUB_TOKEN for creating GitHub releases
NPM_TOKEN for publishing to npm
PYPI_TOKEN fror publishing to PyPI
RUBYGEMS_API_KEY for publishing to RubyGems