Project

tasker-rb

0.0
No release in over 3 years
Ruby FFI bindings for tasker-core, providing 10-100x performance improvements for workflow orchestration, dependency resolution, and state management. This gem enables Rails applications using the Tasker engine to leverage Rust's performance for computationally intensive orchestration operations while maintaining Ruby's flexibility for business logic.
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
 Dependencies

Runtime

~> 3.3
~> 2.8
~> 1.8
~> 2.12.2
~> 2.4, >= 2.4.0
~> 1.6
~> 0.9.39
 Project Readme

Tasker Core

CI GitHub GitHub release (latest SemVer) Tests

High-performance Rust workflow orchestration with PostgreSQL-native messaging. Tasker Core provides DAG-based task execution, event-driven coordination, and comprehensive state management for complex workflows. Supports PGMQ (default, single-dependency) or RabbitMQ (high-throughput) backends.

Status: Alpha Ready | Version: 0.1.0


Why Tasker Core?

  • DAG-Based Workflows - Define complex workflows as directed acyclic graphs with dependencies
  • Flexible Messaging - PGMQ (PostgreSQL-native, zero extra dependencies) or RabbitMQ (high-throughput)
  • Event-Driven - Real-time step discovery with push notifications and polling fallback
  • Multi-Language Workers - Rust native, Ruby via FFI, Python via PyO3, TypeScript FFI
  • Production-Ready (Alpha) - Circuit breakers, health monitoring, zero race conditions, 50+ metrics

Ideal For

  • Order fulfillment workflows with inventory, payment, and shipping coordination
  • Payment processing pipelines with retry logic and idempotency
  • ETL pipelines with complex dependencies and parallel execution
  • Microservices orchestration across distributed systems

Not Ideal For

  • Simple cron jobs (use native cron)
  • Single-step operations (overhead not justified)
  • Sub-millisecond requirements (~10-20ms architectural overhead)

Quick Start

Prerequisites

  • Rust 1.75+ | PostgreSQL 17+ (uuidv7 support) | Docker (recommended)
  • Optional: RabbitMQ (for high-throughput messaging)

Get Running

# Start PostgreSQL (includes PGMQ extension)
docker-compose up -d postgres

# Run migrations
export DATABASE_URL="postgresql://tasker:tasker@localhost/tasker_rust_test"
cargo sqlx migrate run

# Start orchestration server
docker-compose --profile server up -d

# Create a task
curl -X POST http://localhost:8080/v1/tasks \
  -H "Content-Type: application/json" \
  -d '{"template_name": "linear_workflow", "namespace": "example"}'

# Check health
curl http://localhost:8080/health

Full Guide: docs/guides/quick-start.md


Architecture

┌─────────────────────────────────────────────────────────────┐
│                     Client Applications                     │
│                   (REST API / CLI / SDK)                    │
└──────────────────────────┬──────────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────────┐
│              Tasker Orchestration Server                    │
│    Task Init → Step Discovery → Enqueueing → Finalization  │
└──────────┬──────────────────────────────────┬───────────────┘
           │                                  │
           ▼                                  ▼
┌──────────────────────┐          ┌──────────────────────────┐
│  PostgreSQL + Broker │◄────────►│  Namespace Worker Pools  │
│ (PGMQ or RabbitMQ)   │          │  (Horizontal Scaling)    │
└──────────────────────┘          └──────────────────────────┘

Key Patterns: Dual state machines (12 task + 9 step states), event-driven with polling fallback, autonomous workers, PostgreSQL as source of truth. Messaging via PGMQ (default) or RabbitMQ.

Deep Dive: Architecture Documentation


Workspace Structure

Crate Purpose
tasker-pgmq PGMQ wrapper with atomic notify (PostgreSQL backend)
tasker-shared Core types, state machines, messaging abstraction
tasker-orchestration Task coordination, REST API
tasker-worker Step execution, FFI layer
tasker-client REST client library
tasker-ctl CLI binary
workers/rust Native Rust workers
workers/ruby Ruby FFI bindings
workers/python Python FFI bindings
workers/typescript TypeScript FFI bindings

Performance

Benchmarked in 10-instance cluster (2 orchestration, 2 each worker type). Each step involves ~19 DB operations, 2 message queue round-trips, and full state machine lifecycle.

Metric Target Actual
Task initialization API (p99) < 100ms 17.7ms
SQL orchestration functions < 3ms 1.75-2.93ms
4-step linear workflow (P50) < 500ms 257ms
7-step complex DAG (P50) < 800ms 382ms
FFI workers (Ruby/Python/TS) < 800ms 315ms (+23% vs Rust)
1000-row batch processing (P50) < 1000ms 363ms (2,700 rows/sec)

Horizontal scaling for orchestration and workers. SQL performance at sub-3ms with 5K+ tasks. All FFI languages within 3ms of each other (framework-dominated overhead).

Full Benchmarks: docs/benchmarks/README.md


Documentation

All documentation is organized in the Documentation Hub:

Section Description
Quick Start Get running in 5 minutes
Architecture System design, state machines, event systems
Guides Workflows, batch processing, configuration
Workers Ruby, Python, TypeScript, Rust handler development
Operations Deployment, monitoring, tuning
Principles Design philosophy and tenets
Decisions Architecture Decision Records (ADRs)

AI Assistant Context: CLAUDE.md | Development Guide: docs/development/


Development

# Build and test (always use --all-features)
cargo build --all-features
cargo test --all-features

# Lint
cargo fmt && cargo clippy --all-targets --all-features

# Run server
cargo run --bin tasker-server

See CLAUDE.md for complete development context.


Contributing

  1. Review CLAUDE.md for project context
  2. Run tests: cargo test --all-features
  3. Format and lint before PR
  4. See Documentation Hub for documentation structure

License

MIT License - see LICENSE for details.


Production-ready workflow orchestration at scale.

Quick Start | Documentation | Examples