Tasker Core
High-performance Rust workflow orchestration with PostgreSQL-native messaging. Tasker Core provides DAG-based task execution, event-driven coordination, and comprehensive state management for complex workflows. Supports PGMQ (default, single-dependency) or RabbitMQ (high-throughput) backends.
Status: Alpha Ready | Version: 0.1.0
Why Tasker Core?
- DAG-Based Workflows - Define complex workflows as directed acyclic graphs with dependencies
- Flexible Messaging - PGMQ (PostgreSQL-native, zero extra dependencies) or RabbitMQ (high-throughput)
- Event-Driven - Real-time step discovery with push notifications and polling fallback
- Multi-Language Workers - Rust native, Ruby via FFI, Python via PyO3, TypeScript FFI
- Production-Ready (Alpha) - Circuit breakers, health monitoring, zero race conditions, 50+ metrics
Ideal For
- Order fulfillment workflows with inventory, payment, and shipping coordination
- Payment processing pipelines with retry logic and idempotency
- ETL pipelines with complex dependencies and parallel execution
- Microservices orchestration across distributed systems
Not Ideal For
- Simple cron jobs (use native cron)
- Single-step operations (overhead not justified)
- Sub-millisecond requirements (~10-20ms architectural overhead)
Quick Start
Prerequisites
- Rust 1.75+ | PostgreSQL 17+ (uuidv7 support) | Docker (recommended)
- Optional: RabbitMQ (for high-throughput messaging)
Get Running
# Start PostgreSQL (includes PGMQ extension)
docker-compose up -d postgres
# Run migrations
export DATABASE_URL="postgresql://tasker:tasker@localhost/tasker_rust_test"
cargo sqlx migrate run
# Start orchestration server
docker-compose --profile server up -d
# Create a task
curl -X POST http://localhost:8080/v1/tasks \
-H "Content-Type: application/json" \
-d '{"template_name": "linear_workflow", "namespace": "example"}'
# Check health
curl http://localhost:8080/healthFull Guide: docs/guides/quick-start.md
Architecture
┌─────────────────────────────────────────────────────────────┐
│ Client Applications │
│ (REST API / CLI / SDK) │
└──────────────────────────┬──────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Tasker Orchestration Server │
│ Task Init → Step Discovery → Enqueueing → Finalization │
└──────────┬──────────────────────────────────┬───────────────┘
│ │
▼ ▼
┌──────────────────────┐ ┌──────────────────────────┐
│ PostgreSQL + Broker │◄────────►│ Namespace Worker Pools │
│ (PGMQ or RabbitMQ) │ │ (Horizontal Scaling) │
└──────────────────────┘ └──────────────────────────┘
Key Patterns: Dual state machines (12 task + 9 step states), event-driven with polling fallback, autonomous workers, PostgreSQL as source of truth. Messaging via PGMQ (default) or RabbitMQ.
Deep Dive: Architecture Documentation
Workspace Structure
| Crate | Purpose |
|---|---|
tasker-pgmq |
PGMQ wrapper with atomic notify (PostgreSQL backend) |
tasker-shared |
Core types, state machines, messaging abstraction |
tasker-orchestration |
Task coordination, REST API |
tasker-worker |
Step execution, FFI layer |
tasker-client |
REST client library |
tasker-ctl |
CLI binary |
workers/rust |
Native Rust workers |
workers/ruby |
Ruby FFI bindings |
workers/python |
Python FFI bindings |
workers/typescript |
TypeScript FFI bindings |
Performance
Benchmarked in 10-instance cluster (2 orchestration, 2 each worker type). Each step involves ~19 DB operations, 2 message queue round-trips, and full state machine lifecycle.
| Metric | Target | Actual |
|---|---|---|
| Task initialization API (p99) | < 100ms | 17.7ms |
| SQL orchestration functions | < 3ms | 1.75-2.93ms |
| 4-step linear workflow (P50) | < 500ms | 257ms |
| 7-step complex DAG (P50) | < 800ms | 382ms |
| FFI workers (Ruby/Python/TS) | < 800ms | 315ms (+23% vs Rust) |
| 1000-row batch processing (P50) | < 1000ms | 363ms (2,700 rows/sec) |
Horizontal scaling for orchestration and workers. SQL performance at sub-3ms with 5K+ tasks. All FFI languages within 3ms of each other (framework-dominated overhead).
Full Benchmarks: docs/benchmarks/README.md
Documentation
All documentation is organized in the Documentation Hub:
| Section | Description |
|---|---|
| Quick Start | Get running in 5 minutes |
| Architecture | System design, state machines, event systems |
| Guides | Workflows, batch processing, configuration |
| Workers | Ruby, Python, TypeScript, Rust handler development |
| Operations | Deployment, monitoring, tuning |
| Principles | Design philosophy and tenets |
| Decisions | Architecture Decision Records (ADRs) |
AI Assistant Context: CLAUDE.md | Development Guide: docs/development/
Development
# Build and test (always use --all-features)
cargo build --all-features
cargo test --all-features
# Lint
cargo fmt && cargo clippy --all-targets --all-features
# Run server
cargo run --bin tasker-serverSee CLAUDE.md for complete development context.
Contributing
- Review CLAUDE.md for project context
- Run tests:
cargo test --all-features - Format and lint before PR
- See Documentation Hub for documentation structure
License
MIT License - see LICENSE for details.
Production-ready workflow orchestration at scale.