Symbolic knowledge extraction from trained neural networks: a new approach

Abstract

In this report, we investigate the problem of symbolic knowledge extraction from trained neural networks, and present a new extraction method. Although neural networks have shown very good performance in terms of learnability, generalizability and speed in many application domains, one of their main drawbacks lies in their incapacity to provide an explanation to the underlying reasoning mechanisms that justify a given answer. As a result, their use in many application areas, for instance in safety-critical domains, has become limited. The so called "explanation capability" of neural networks can be achieved by the extraction of symbolic knowledge from it, using "rules extraction" methods. We start by discussing some of the main problems of knowledge extraction methods. In an attempt to ameliorate these problems, we identify, in the case of neural networks, a partial ordering on the input vector space. A number of pruning rules and simplification rules that interact with this ordering are defined. These rules are used in our extraction algorithm in order to help reduce the input vector search space during a pedagogical knowledge extraction from trained networks. They are also very useful in helping to reduce the number of rules extracted, which provides clarity and readability to the rule set. We show that, in the case of regular networks, the extraction algorithm is sound and complete. We proceed to extend the extraction algorithm to the class of non-regular networks, the general case. We identify that non-regular networks contain regularities in their subnetworks. As a result, the underlying extraction method for regular networks can be applied, but now in decomposable fashion. The problem, however, is how to combine the set of rules extracted from each subnetwork into the final rule set. We propose a solution to this problem in which we are able to keep the soundness of the extraction algorithm, although we have to drop completeness.