Verification for safety-related applications
The term “verification” often pops up when we’re developing safety-related systems for the rail sector. But what exactly does verification entail? Who’s responsible for it? When and where is verification required? We’ll shed some light on these issues and take a look at how verification works by creating a security requirement specification.
Figure 1: Tracks at Zurich Central Station (source: https://pixabay.com)
What is verification? – A definition
The term “verification” is used in different ways and in a range of fields. As such, our first, important step is to provide some clarity on how we’ll use the term here. In this article, we’ll use the definition of “verification” as set out in the functional safety standards produced by CENELEC (the European Committee for Electrotechnical Standardization), such as the EN 50126:2017 and EN 50129:2019 standards for railway applications:
“Verification is the provision of confirmation, through the provision of objective evidence, that specified requirements have been met.”
More specifically, verification allows us to demonstrate that our product (or safety requirements specification, in this case) meets all of its requirements, whether they be functional or non-functional (e.g. normative) requirements. Verification also confirms that all pre-defined processes have been adhered to.
When is verification performed?
Verification is performed at different stages of the development process and entails checking that the system, and/or components thereof, comply with the requirements defined at the start of that particular phase.
Phase-end verification in the development life cycle for safety-related rail systems
Process standard EN 50126-1:2017 sets out an all-purpose approach to verification that divides the system development life cycle into twelve distinct phases. Verification takes place at the end of each respective phase, as represented by the red arrows in the diagram below.
Figure 2: Life cycle (own diagram, based on Figure 7 from EN 50126-1:2017)
The aim of this approach to verification is to check and prove that all pre-defined system requirements have been met at the end of each individual phase on the one hand, and on the other hand to ascertain that all work carried out during that phase was performed in compliance with the requirements of the applicable standards and regulations.
Duties of the verifier and separation between the verifier and verification
Duties
As outlined above, the verifier verifies that the pre-defined requirements for each individual phase of the development life cycle have been met. The verifier doesn’t necessarily have to be the same person as the one who conducts the technical review on the outcomes of the work; Instead, the verifier assesses the appropriateness (completeness, consistency, accuracy, relevance, and traceability) of the review documents and compares them with the pre-defined objectives of the verification.
Verification is typically carried out at the end of a development phase and documented in a verification report.
The verifier plans their activities in a verification plan, in which they define aspects such as the methods to be used for the verification and/or the requirements to be checked for. The applicable input and output criteria are also described in the plan.
Separation of roles
The way in which a project’s roles and responsibilities are divided essentially depends on the Safety Integrity Level (SIL) requirements of the system being developed. As a general rule, it should be noted that the verifier may not be involved in any other activity during a given phase in the development life cycle. They may, however, identify missing security requirements and influence the specification of system requirements in that way. In these instances, the specified requirements must then be reviewed by another suitably qualified individual.
Figure 3: Two possible role distribution models for an SIL4 development project (own diagram, based on Figure 5 from EN 50129:2019)
The illustration above shows two possible role distribution models indicating how the project organisation has to be structured for an SIL4-rated project. What’s evident here is that the verifier may in principle be subordinate to the project manager, provided that the same person doesn’t also perform validation-related tasks.
The verification process
Step one: Define document requirements
The first step is to define the requirements for project documents, ideally before the actual documents are created. It makes sense to create inspection checklists per artifact type at this juncture. These can be used by document creators to help design their documents correctly, as well as by verifiers, who develop their verification plans based on said checklists.
Moreover, these checklists can function as solid evidence that the requirements for the individual documents have been taken into account and met accordingly. This type of proof can easily be verified by independent assessors and thereby constitutes a solid factual basis for assessments.
Verification using the example of a security requirement specification
To give you a better idea of the work that verification entails, let’s take a closer look at how the “system requirements specification” phase is verified. Let’s assume that a security requirement specification with safety-related functions rated at Safety Integrity Level 4 (SIL4), among other things, will be written for our system during this phase.
The safety requirements specification contributes to creating a comprehensive and clearly defined set of requirements for subsequent phases of the development life cycle.
The verifier specifies the entry criteria, incoming documents, outgoing documents and verification activitiesinvolved in this phase in the verification plan.
Figure 4: Activities and relevant documents for verifying a phase (own diagram)
By this point, the designer will already have created a security requirement specification that will have since been professionally reviewed by a second designer.
The phase concludes once the activities defined in the verification plan have been carried out and the outgoing documents have been reviewed. The inspection checklists mentioned above are highly recommended for completing these tasks. This makes the verification process transparent and easy to follow, which also makes life easier for the assessor who has to prepare the safety report.
In our example, the verifier verifies that the technical review has been carried out and uses the checklist to check that the review document meets the requirements set forth in the standards. Checks are also performed to ensure that the measures and techniques specified in Appendix E of EN 50129:2019 were applied accordingly for the document in question.
In the case of our example of a safety requirement specification for an SIL4-rated project, the standard permits the use of a combination of structured specifications and illustrative diagrams (e.g. block diagrams).
And finally: the verification report
As mentioned previously, verification is generally performed at the end of each individual development phase. That said, it is advisable to verify results on an ongoing basis so that deviations can be identified early on, particularly in the case of complex projects. One way to do this would be to create and complete partial verification reports, for example. The verification report created at the end of the corresponding development phase could then refer back to any existing partial reports.
The phase-end verification report is where the verifier records the current status of all results and outcomes that need to be inspected and determines which objectives have been achieved during that phase. Completing this report is also a prerequisite for concluding that particular phase of the corresponding life cycle, as it provides information on the extent to which the activities, results and outcomes required for that phase have been completed.