ORCA's primary use case is in a continuous integration workflow, running against pull requests and commits. It provides several scripts in bin/travis corresponding to Travis CI phases:
before_install.shprepares the environment and installs ORCA.install.shcreates the test fixture and places the system under test (SUT).before_script.shdisplays details about the fixture for debugging purposes.script.shruns static analysis and automated tests.before_cache.shis reserved for future use.after_success.shis reserved for future use.after_failure.shdisplays debugging information in case of job failure.after_script.shis reserved for future use.
See example/.travis.yml for an example Travis CI configuration. Features are explained in the comments.
For more complex testing needs, ORCA commands can be invoked directly. See this this example from Lightning.
See also Continuous integration.
ORCA can also be installed and run locally for testing and development. Follow these steps to set it up:
-
Choose a directory to contain your package(s), e.g.:
PARENT_DIR="$HOME/Projects" -
Clone ORCA and your package(s) each into the directory, e.g.:
git clone git@github.com:acquia/orca.git "${PARENT_DIR}/orca" git clone git@github.com:acquia/EXAMPLE.git "${PARENT_DIR}/EXAMPLE"
-
Install ORCA with Composer, e.g.:
composer install --no-dev --working-dir="${PARENT_DIR}/orca" -
Optionally add command autocompletion to your shell:
# Bash: bash $(path/to/orca _completion --generate-hook) # Zsh: source <(path/to/orca _completion --generate-hook)
Invoke ORCA from the terminal (bin/orca). Use the --help command option to learn more about the various commands or see how they're used in bin/travis/script. Use the fixture:run-server command to run the web server for local development.
ORCA uses tags (for Behat) and groups (for PHPUnit) to determine which tests to run when, as depicted in the table below, where black indicates a test's being included and white indicates its being ignored:
| (Default) | orca_public |
orca_ignore |
|
|---|---|---|---|
| Isolated tests (own) | ⚫ | ⚫ | ⚪ |
| Integrated tests (own) | ⚫ | ⚫ | ⚪ |
| Integrated tests (others') | ⚪ | ⚫ | ⚪ |
The default behavior is to run a test only when the package providing it is the SUT--not when it is merely included in another package's test fixture. Any test not designated public or ignored is so treated. Such tests are referred to as "private tests". This should be considered the correct choice for most tests--particularly for features that involve little or no risk of conflict with other Acquia packages, including isolated unit tests by definition.
Public tests (orca_public) are always run, including when testing packages other than the one providing them. For example, a public test provided by Lightning API will also be run during tests of Acquia Commerce Manager, Acquia Lift, and the rest. Public tests thus lengthen builds for all Acquia packages and should be used judiciously. Reserve them for high value features with meaningful risk of being broken by other packages, and make them as fast as possible.
Ignored tests (orca_ignore) are "ignored" and never run by ORCA. Tests should be ignored when they depend upon setup or preconditions that ORCA doesn't provide, such as a fixture with unique dependencies or a database populated by SQL dump. Once ignored, such tests can be scripted to run apart from ORCA after custom setup. In practice, it should rarely be necessary to ignore a test, as most setup and teardown can be accomplished through Behat hooks and PHPUnit template methods.
README | Understanding ORCA | Getting Started | Advanced Usage | Project Glossary | FAQ | Contribution Guide