IONOS Spectre

Describe Tests. Analyse Results. Understand What Happened.

Spectre is a DSL and command line tool for test automation.

It is written in Ruby and inspired by the Unit-Test framework rspec.

This framework focuses on API behavior testing, rapid and flexible test development.

Code is documentation

The framework is designed for non-developers and to provide easy to read tests. When writing and reading tests, you should immediately understand what is going on. This helps to debug test subjects and to better understand what and how it is tested.

Spectre core only contains logic to run, log and report tests. For more functionality like HTTP requests, SSH command, database access,... see ionos-spectre.

See minimal example module for a minimal module example and descriptions about how to create custom modules.

There is a Dockerfile to build spectre Docker images. To build a Docker image run

docker build -t spectre .

and then run with

docker run -t --rm -v "path/to/specs:/spectre" spectre [command] [options]

Ruby has to be installed on your system.

See ruby-lang.org for installation instructions.

Also consider to use a Ruby version manager like https://github.com/rbenv/rbenv or https://rvm.io/

To install Ruby on windows, just use winget

# Search for available Ruby versions
winget search ruby

# Install Ruby with dev kit
winget install RubyInstallerTeam.RubyWithDevKit.3.4

Spectre is available as a Ruby gem from the GitHub packages repository. See https://rubygems.pkg.github.com/ionos-spectre for instruction.

It is recommended to create a Gemfile and install spectre-core with bundler.

For more information about bundler see https://bundler.io

You can also clone this repository and run

rake install

To test, if the tool is working, try one of the following commands.

# When using bundler
bundle exec spectre -h

# otherwise
spectre -h

There is a more detailed documentation of available methods and functions via bundle exec rdoc. The generated files are located at doc/.

This documentation will be hosted and available in the future.

Create a new project structure by executing

spectre init

This will create multiple empty directories and a spectre.yml config file.

You can see the set values of an spectre config, including the environment file by executing

spectre show

See Spectre::CONFIG for available options.

All options can also be overridden with the command line argument -p or --property

spectre -p spec_patterns=some_specific.spec.rb -p mixin_patterns=**/*.my_mixins.rb

You can also create a global spectre config file with the options above. Create a spectre.yml file in your users .config directory (~/config/spectre.yml) and set the options which shall be used for all projects on your computer.

To write automated tests, just open an editor of your choice and create a file named, for example spooky.spec.rb in the specs folder. Specs are structured in three levels. The subject, the root context, defined by the keyword describe, the actual specification defined by the it keyword and one or more assertions or expectations described by assert or expect. A subject can contain one or more contexts and/or specs.

See spec example for an example and more detailed descriptions on how specs are structured.

To list specs execute

spectre list

The output looks like this

[ghostbuster-1] Ghostbuster accepts emergency calls #emergency #call #failed #expect #assert
[ghostbuster-2] Ghostbuster while preparing lookups Zuul #entity #location #env
[ghostbuster-3] Ghostbuster while preparing lookups Dream Ghost #entity #location #env
[ghostbuster-4] Ghostbuster hunts at the Sedgewick Hotel #ghosts #success #expect #assert
[ghostbuster-5] Ghostbuster at midnight captures some ghosts #emergency #ghosts #error #group #mixin
[firehouse-1] Firehouse is the home of the Ladder 8 company #trivia
[firehouse-2] Firehouse has a functioning containment unit #fails #observe #expect #assert

The name in the brackets is an identifier for a spec. This can be used to run only specific specs.

spectre -s ghostbuster-1

Note Note that this ID can change, when more specs have been added.

Spec tags are listed with #. Those tags can be used to filter specs.

Run specs with one of the listed tags

spectre -t emergency,entity

Run specs containing all listed tags

spectre -t emergency+entity

Run specs not containing specific tags

spectre -t !ghosts

You can also use a combination of all

spectre -t emergency,trivia+entity,success+!ghosts

Your project could consist of hundreds and thousand of specs. In order to easier maintain your project, it is recommended to place specs of a subject in different *.spec.rb files and folders, grouped by a specific context. A subject can be described in multiple files.

For example, when writing specs for a REST API, the specs could be grouped by the APIs resources in different folders, and their operations in different files.

Specs of a RPC API can be grouped by its functions.

Let's asume we have a Hollow API which operates on two resources ghosts and monsters. Each resource can be created, read, updated and deleted. The project structure could then look something like this:

hollow_webapi
+-- environments
|   +-- development.env.rb
|   +-- development.env.secret.rb
|   +-- staging.env.rb
|   +-- staging.env.secret.rb
|   +-- production.env.rb
|   +-- production.env.secret.rb
+-- logs
+-- specs
|   +-- ghosts
|   |   +-- create.spec.rb
|   |   +-- read.spec.rb
|   |   +-- update.spec.rb
|   |   +-- delete.spec.rb
|   |   +-- spook.spec.rb
|   +-- monsters
|       +-- create.spec.rb
|       +-- read.spec.rb
|       +-- update.spec.rb
|       +-- delete.spec.rb
+-- spectre.yaml

In this case the resources can be defined as subjects.

You can define reusable specs by using mixins. Create a .mixin.rb file in the mixin directory (default: mixins)

See mixin example for examples and usage description.