Fault-based Conformance Testing in Practice

Publication Type:

Journal Article


International Journal of Software and Informatics, Institute of Software, Chinese Academy of Sciences., Volume 3, Issue 2-3, p.375–411 (2009)




Conforming to protocol specifications is a critical issue in modern distributed
software systems. Nowadays, complex service infrastructures, such as Voice-over-IP systems,
are usually built by combining components of different vendors. If the components do not
correctly implement the various protocol specifications, failures will certainly occur. In the
case of emergency calls this may be even life-threatening. Functional black-box conformance
testing, where one checks the conformance of the implemented protocol to a specification
becomes therefore a major issue. In this work, we report on our experiences and findings when
applying fault-based conformance testing to an industrial application. Besides a discussion
on modeling and simplifications we present a technique that prevents an application from
implementing particular faults. Faults are modeled at the level of the specification. We show
how such a technique can be adapted to specifications with large state spaces and present
results obtained when applying our technique to the Session Initiation Protocol and to the
Conference Protocol. Finally, we compare our results to random and scenario based testing.