Training improvement

This use case example will help to understand what an association bias is, how to detect it and finally how to overcome the issue with Saimple.

Saimple : Detecting a learning bias

The Saimple tool developed by Numalis allows to improve the training of neural networks. Biases are common in the training of artificial intelligence algorithms. Although such potential problems are known, it is not easy to detect them.

1. Use case presentation

The objective is to create and train a classification model with an association bias, i.e. a model that gives more or less correct results but whose decision making is not based solely on the characteristic representing the class.

For this purpose, a dataset containing numerous images of wasps and bees was used. Two classes will therefore be represented: "bee" and "wasp". The images in the dataset are divided into two categories for each class:

  • Photos taken in nature (on flowers, leaves, etc)

  • Photos taken in a laboratory or in a non-natural environment (jar, plate, etc)

2. Detecting a learning bias

1st experimentation :

The first experiment will be carried out by training the model on "biased" data. The images of bees and wasps being classified in two different categories, the aim will be to train the model on only one of the categories of each class. Thus, the bee images used will be in a natural environment and the wasp images will be in a laboratory environment. The objective is to make the model base its decision on the environment rather than on the insect.

The biased dataset therefore contains images of bees in a natural environment and images of wasps in a laboratory-type environment, divided into 2469 images of bees and 2816 images of wasps.

Results :

The performance of the model is tested on 4 images: :

  • A bee on a natural background

  • A bee on a laboratory background

  • A wasp on a natural background

  • A wasp on a laboratory background

We observe that the model classifies well the images on which it has been trained (bee in natural environment and wasp in laboratory environment) but it does not classify correctly the others. Indeed, if the model takes into account the environment during its training rather than the insect, then when it evaluates an image of a wasp in a natural environment, it will identify it as a bee.

Predicted class and actual class as a function of the network input image

Here are the relevance data obtained by SAIMPLE, with the 4 images presented above :

Relevance of the biased model on the 4 reference images

The use of the model on Saimple is indicative of poor learning and the presence of bias. The areas taken into account by the neural network to decide on the class of the insect are visually represented thanks to the Saimple relevance function. We observe that on an evaluation that is performed with an image of a wasp in a natural environment, the dominance will be very high for the class "bee" and the relevance shows that the model is not only based on the insect, but also on what surrounds it (e.g. a flower or a background). There are no relevant elements that are taken into account. The entire body of the insects is selected, as well as parts of the landscape such as flowers. The association bias is clearly visible.

3. Eliminating association bias

2nd experimentation

The aim here is to train the model without bias. All data available will therefore be used and the model will be saved at each training epoch, in order to be able to follow the evolution of the training and the performance of the model. The unbiased dataset has been balanced and now contains 3183 images for each class: bees and wasps in laboratory and natural environments.

Results

Firstly, at the end of the training phase, we obtain a percentage of correct classifications of 80%, for the 30th epoch. This second model shows less accurate results compared to the previous one on the validation data. It is normal that the model has lost in accuracy since it is now based on the real characteristics of the insect and not simply on the environment in which it is found. This new model does however make the right classifications for the 4 test images presented above.

Relevance of the unbiased model on the 4 reference images

We can see thanks to Saimple that the model focuses more on the body of the insect, we do not notice anymore signs of the bias. Another interesting point can be observed in the image of the bee in the laboratory. Indeed, we can see that many points of relevance are placed on the petiole of the bee. Knowing that the wasp petiole is much thinner than the bee one, the fact that the model focuses on this point proves that the model has learned fundamental characteristics to distinguish these two insects.

4. Improvement through training

Thanks to Saimple, it is possible to compare the results of the model according to the epochs. Three different states of the model are evaluated on the same image, corresponding to weak, medium and strong training (respectively 1 epoch, 15 epochs, 30 epochs). The image used is an image of a wasp on a natural background, which the biased model therefore confuses with a bee.

Evolution of relevance over time

The relevance results obtained with Saimple ensure that the model has learned correctly by checking which elements are important for the classification. The results show that the model is not sufficiently trained at epochs 1 and 15 because parts of the flower are still taken into account and the dominant class is in conflict with other classes, dominance cannot be ensured. Indeed, a class is dominant if its dominance score is always higher than the dominance score of all other classes.

5. Conclusion

Saimple therefore makes it easy to detect an association bias, which is very difficult to detect without the use of such a tool. Indeed, one cannot know in advance what the model will be based on. In the case of an association bias, it is the training data that introduces the bias and it is impossible to imagine that a human operator could check the data of an entire dataset.

In this use case, Saimple allowed not only to detect the bias, but also to correct it, and finally to ensure that the corrected model was no longer biased.

Finally, Saimple makes it possible to follow the evolution of the training of the model over time. This makes it possible to know after how long the model has been sufficiently trained, thus avoiding problems of under- or over-fitting.

Numalis

We are a French innovative software editor company providing tools and services to make your neural networks reliable and explainable.

Contact us

Follow us