5.1.1.6.2 Process for testing and evaluating the effect of changes to the repository's critical processes

From aptrust
Jump to: navigation, search


5.1.1.6.2 Process for testing and evaluating the effect of changes to the repository's critical processes
Status Ready for review
Compliance Rating Fully compliant
Responsible APTrust Staff

The repository shall have a process for testing and evaluating the effect of changes to the repository’s critical processes.

Supporting Text

This is necessary in order to protect the integrity of the repository's critical processes such that they continue in their ability to meet the repository's mandatory requirements.

Examples for Meeting the Requirement

Documented testing procedures; documentation of results from prior tests and proof of changes made as a result of tests; analysis of the impact of a process change.

Discussion

Changes to critical systems should be, where possible, pre-tested separately, the expected behaviors documented, and roll-back procedures prepared. After changes, the systems should be monitored for unexpected and unacceptable behavior. If such behavior is discovered the changes and their consequences should be reversed. Whole-system testing or unit testing can address this requirement; complex safety-type tests are not required. Testing can be very expensive, but there should be some recognition of the fact that a completely open regime where no changes are ever evaluated or tested will have problems.

Evidence Provided

APTrust staff adheres to a Software Development Styleguide that includes the use of unit tests and continuous integration testing. Changes to our software and repository are tested by developers with localized integration tests and by automated tools (Travis-CI) before being put in place. Every commit to our codebase triggers automated unit tests. Integration tests are run locally (as of June 2018) but are intended to be automated once a move to a Dockerized infrastructure has been completed. Integration tests mock all of the repositories services to test ingest and restore workflows.

APTrust maintains a test/demo environment that mirrors the production environment. Changes to the repositories software are deployed there first to assure that changes won’t adversely affect the production system.