Test Guidelines

Every project should provide for each test complexity scope a dedicated project command, if there are such tests present. The commandbin/text.everythingcan be created in order to execute every test. Every test should be annotated with a complexity scope, so it easy to execute all relevant tests. Other test scopes and tags can optionally be used as well, but the complexity scope must be present in every test.

  1. bin/test: Default test complexity scope, which should be minimal or extensive. The default test scope should never bebin/test.side.effectsin order to prevent damaging systems.
  2. bin/test.minimal: Minimal test should always be tested and run in CI.
  3. bin/test.extensive:Extensive test should test the projects core functionality as much as reasonable.
  4. bin/test.side.effects:Side effect tests check side effects, that may modify the host system in a unreasonable way (i.e. install and uninstall a software package in order to test installation and deletion).

It can be helpful to create abin/test.everythingcommand, in order to easily ensure, that every test is executed.