AU PS Inferno Test Kit

This is an Inferno test kit for the AU PS Implementation Guide

Currently available versions:

1. 1.0.0

You can test your FHIR server using this link https://inferno.hl7.org.au/test-kits/au-ps

It is highly recommended that you use Docker to run these tests. This test kit requires at least 10 GB of memory are available to Docker.

1. Clone this repo.
2. Run the setup script to initialize the database

make setup

3. Run project

make run

4. Navigate to http://localhost. The AU PS test suite will be available.

The kit carries one suite per AU PS IG version, and several versions can be loaded side by side. Each version is made of:

• a generator config in the repository root, config.<key>.json, whose ig block names the IG package archive and whose kit.suite_file names the suite class;
• generated metadata in lib/au_ps_inferno/generated/<key>/ (metadata.yaml for core IG metadata, composition_metadata.yaml for Composition-specific metadata);
• a suite class in lib/au_ps_inferno/suite/ with its own IG_VERSION constant, registered in lib/au_ps_inferno.rb.

For a released IG the key is the IG version (1.0.0). The suite id is derived from the IG version by AUPSTestKit::SuiteVersion.suite_id_for: the release digits are joined and a pre-release label is appended in snake case, so 1.0.0 becomes au_ps_v100.

All versions share the group and test classes under lib/au_ps_inferno/suite/. AUPSTestKit::AUPSSuiteDefinition assembles a suite from them and passes the version's metadata path and IG version down to every group and test through Inferno's config options, so a shared test reads the metadata of the suite it runs in. Inferno prefixes every group and test id with the id of its suite, so each version's ids are its own.

bundle exec rake au_ps:generate          # every released version
bundle exec rake 'au_ps:generate[1.0.0]' # one version

The generator reads the version's package archive, writes lib/au_ps_inferno/generated/<key>/metadata.yaml and composition_metadata.yaml, and sets IG_VERSION in that version's suite class to the version declared in the package's package.json. Other versions are left untouched. make generate_and_fix (used by the Generate Suite workflow) regenerates every version inside Docker and opens a pull request.

Run the Sync IG Package workflow (scripts/sync_ig_and_generate_suite.rb). When the FHIR package registry has an AU PS release newer than the newest released version the kit carries, it adds the new release alongside the existing ones: it downloads the package into lib/au_ps_inferno/igs/, writes config.<version>.json, scaffolds lib/au_ps_inferno/suite/au_ps_v<digits>.rb, registers it in lib/au_ps_inferno.rb, generates its metadata and opens a pull request. Existing suites, their ids and their sessions are not changed.

lib/au_ps_inferno/suite/au_ps_ci_build.rb tracks the continuous-integration build of the IG (https://build.fhir.org/ig/hl7au/au-fhir-ps/). It is carried in the gem but registered only when the environment sets INFERNO_CI_BUILD_SUITES=true, which the development deployment does and production does not. The tooling follows the AU Core kit's ci-build suite, so both kits read the same way.

• Its key is ci-build: config config.ci-build.json (with a ci_build section recording the package and manifest URLs and the package date), metadata lib/au_ps_inferno/generated/ci-build/.
• Its suite id is fixed, au_ps_ci_build, whatever the CI package version, so sessions and the places that list the suite by id survive a CI version bump. The title shows the current package version.
• Its validator loads hl7.fhir.au.ps#current. The HL7 validator resolves #current from build.fhir.org itself and refreshes its package cache when the CI build changes; a running validator keeps the IG it already loaded until it restarts. Because #current can move ahead of the suite, the CI suite validates bundles against the unversioned AU PS profile canonicals.

bundle exec rake au_ps:ci_build:check            # has the CI build changed since the suite was generated?
bundle exec rake au_ps:ci_build:download         # fetch package.tgz into lib/au_ps_inferno/igs/ci-build.tgz
bundle exec rake 'au_ps:ci_build:refresh'        # regenerate if the CI build changed
bundle exec rake 'au_ps:ci_build:refresh[force]' # regenerate regardless (make generate_ci_build)

A refresh takes the version and date from the downloaded package's own package.json, refuses a package that is not a -ci-build version, and puts the config back if generation fails. The package is not committed (it is in .gitignore, .dockerignore and excluded from the gem), so rake au_ps:generate without a key skips the CI build, and rake 'au_ps:generate[ci-build]' refuses a local package the config does not describe.

The Refresh CI Build Suite workflow runs the refresh daily at 19:00 UTC and on demand (with a force input), and opens or updates the automated/refresh-ci-build-suite pull request (label automated-pr). It opens the PR with a token from the kit automation GitHub App (org variable KIT_AUTOMATION_APP_CLIENT_ID, org secret KIT_AUTOMATION_APP_PRIVATE_KEY) so Automated Quality Control runs on it; until those exist it falls back to GITHUB_TOKEN, whose pushes start no workflows, and the reviewer runs make tests locally. After a generator change, run the workflow with force so the ci-build metadata is regenerated in the new format too.

This repository contains both the source code of the tests generator and the generated tests themselves. Even a small change in the generator source causes a huge amount of changes in the generated tests. As a result, when a pull request contains both changes in the generator and changes in the generated files, it is almost impossible to review. Furthermore, there is no sense in reviewing the generated files at all, they are just artifacts that this repo produces. They are placed in the same repo under source control just for simplicity reasons. So, the development process should look like this. When you change the source code of the generator and create a pull request, you SHALL NOT add generated tests in this pull request. Once the code review is done, a person who merged the changes SHALL run the generator and update generated tests. It may be a direct commit to the master branch.

When we would like to issue a new release, you need to update VERSION (the gem version, e.g. '0.0.2') in lib/au_ps_inferno/version.rb. The IG version each suite targets is the IG_VERSION constant in that suite's class (see Suite versions).

Then you need to create a tag for this version. The tag name should start with v and then contain a numeric version like this v0.0.1 Once a tag is created, you need to create a GitHub release for this newly published version. The release creation triggers the pipeline that deploys a new version to the cloud environment.

1. Discuss an issue in chat.fhir.org If you have a question, feature request, or proposed change, the best place to start is Zulip i.e. the https://chat.fhir.org/#narrow/stream/179173-australia/topic/Inferno.20Test.20Kit.20feedback.20and.20queries

If you're unable to find an open request, please create a GitHub to: contribute Test suites or Code to the repository: state your details and the nature of the changes to be contributed suggest improvements or enhancements to the project

We appreciate your contributions to improving this test suite. If you encounter any issues or have suggestions for enhancements, please follow the steps below to report them:

For feedback about a specific hosted test run, use the Inferno feedback form, which can publish a reviewed report without GitHub sign-in. For kit bugs and test requests, choose the matching repository issue form.

1. Search for Existing Issues: Before submitting a new issue, please check the Issues section to see if the problem or suggestion has already been reported. If you find an existing issue, you can add your comments or additional information to it.
2. Open a New Issue: If you do not find a similar issue, you can open a new one. Click on the New Issue button and provide details, e.g., for a bug report:

Title: A brief and descriptive title for the issue.
Description: A detailed description of the issue, including:
1. Steps to reproduce the issue.
2. Expected and actual behavior.
3. Screenshots or another related information (if applicable).

3. Labeling: Help us categorize the issue by adding relevant labels (e.g., bug, enhancement, question). This helps us prioritize and address the issues more efficiently.

To support effective issue resolution, reporters may be engaged in the review process to help confirm that resolutions address their concerns. This can involve notifying the reporter when a fix is implemented and inviting them to validate the solution or provide feedback before the issue is formally closed.

In addition to reporting issues on GitHub, you can also ask questions or report issues on Zulip (chat.fhir.org).

• Specific topic for HL7 AU Inferno Test Kit feedback and queries: australia > Inferno Test Kit feedback and queries
• General: Inferno channel

If you would like to contribute to hl7au/au-ps-inferno, here’s how:

• Open a GitHub issue to discuss your plans to help avoid duplication of effort, align and prioritise your contributions.
• Join the fortnightly HL7 AU Infrastructure and Tooling Community Meetings (register here) where we discuss and triage issues. Feel free to add your issue to the meeting agenda and we'll aim to discuss your issue/ proposed contribution when you are present at the meeting.
• Use Zulip to connect with the team and community asynchronously:
  • Specific topic for HL7 AU Inferno Test Kit feedback and queries: australia > Inferno Test Kit feedback and queries
  • General: Inferno channel

1. Fork this repository.
2. Create a branch and use the GitHub issue number followed by a meaningful description as the branch name for your contribution.
3. Make your contributions/ changes. Please DO NOT add generated tests as instructed in the Development Workflow section of this README.
4. Submit a pull request (PR) for review.
5. Once the PR has been reviewed, feedback addressed collaboratively, and approved (by the designated HL7 AU project facilitator or their delegate - refer to the HL7 AU Project Registry), it may be merged into the main branch.

1. Pre-requisites
2. Validator instructions
3. Changelog