Why Functional Safety testing is so important and how it’s done

Nowadays we have connected cars, which means everything in our vehicle is a huge system of systems. Even a single failure that may seem unimportant at first, in the end can cause serious injuries to a driver or the passengers. That’s why testing on each level of development is so important. Especially when we consider Functional Safety – ASIL functions that are critical for our safety and health.  

Testing in automotive takes the entire right side of the V-model. Great importance is attached to testing due to the requirements of ASPICE level requested by OEMs. Assuming that ASPICE processes are defined and well known within an organisation and there is not too much tailoring, then the testing process should be performed correctly.  Its correctness is also managed by the quality departments and verified during the ASPICE audits and assessments. Looks simple and familiar to everyone who is involved in the development processes in automotive, right? 

But where is a place for Functional Safety testing there? Is there any difference if there is an ASIL marked requirement for a test? Maybe it’s just an additional attribute for a requirement so we can just skip that, and nobody will care? Do we really have to change our standard testing approach that “always worked and nothing happened”?  

Yes, we have to. 

The importance of a well-defined process for Functional Safety testing 

I often had an impression that Functional Safety testing is somewhat underrated and treated as a necessary evil. Meanwhile, despite that it is required by  ISO standard, checked and verified during Functional Safety audits and assessments, it can bring us many benefits as testers.  

Some time ago I’ve heard such a saying: “Only when you pay for the mistake you realize its price”. In my opinion, it may be thought-provoking especially regarding the safety functions where the “price” may be the highest.  

ASIL functions are crucial for our health and life and during the development of embedded systems they have the highest priority. ISO26262 defines a lot of restrictions for their implementation as well as for testing. A well-defined process for Functional Safety testing will let us properly and completely verify implementation, operation and safety mechanisms specified for safety functionalities. 

ISO26262 does not define a completely new testing strategy, it’s rather a supplement of a well-known approach from ASPICE and ISTQB:  

For each test level (Unit tests, SW integration tests, SW Qualification tests, System integration tests, System Qualification tests) ISO26262 defines methods, which should be used to prepare a test environment, deriving test cases, perform functional and non-functional testing.  

How do we approach Functional Safety Testing at Spyrosoft?  

I would call our Functional Safety testing approach at Spyrosoft “complex”. The FuSa testing process is always described in our test plans. Our starting point is to define all methods and their combinations that we are planning to use for a specific project. It depends on the ASIL level, what part of development, system, architecture and type of testing we are responsible for and available tooling. We are using our own templates created according to ISO26262 recommendations. 

For each dedicated method listed in ISO, we start with a justification to see if it’s applicable for our system or not. There is always an explanation when something is not applicable if recommended by standard and why such a combination of methods was chosen. The next step is defining the implementation strategy of a specific method. We also specify the work product (an output of method usage) and its verification strategy. 

Here’s how the summary looks like: 

Using our own documents and scripts (evaluated and qualified with respect to ISO26262) we can ensure full traceability required by ASPICE and ISO26262 process compliance regarding test preparation, performance and reporting.  

We are aware that FuSa testing can be very useful during the HW and SW component qualification whereas fault injection (the process of adding failures in the system intentionally for verification if detection and recovery work as expected) is very useful to check if the implemented safety mechanisms are working.  

At Spyrosoft we have possibilities to use many types of fault injection like injections via XCP to SUT (Software Under Test) and HW failure stimulations – contact us for more details.