Using Convolutional Block Attention Module for Interpreting the Results of Artificial Neural Networks Operation in Handwritten Signature Identification
https://doi.org/10.30764/1819-2785-2025-2-65-81
Abstract
This study explores potential application of a Siamese Convolutional Neural Network (SNN) integrated with a Convolutional Block Attention Module (CBAM) in the field of handwritten signature identification. One well-known obstacle to the introduction of artificial neural networks into forensic expert practice is the low level of their interpretability.
This limitation makes it difficult for experts to determine what patterns were identified by the neural network algorithm, and which ones of them form the basis of its forecast. Furthermore, in most contemporary studies on handwriting analysis, specialists tend to use the “classical” authorship attribution approach, treating the task as a particular case of classification.
However, in authors’ view, this method often leads to Type II errors making the use of classification algorithms for identification purposes unacceptable. As an alternative, the authors propose to focus on the SNN architecture. To support these claims, a series of experiments were conducted as part of this study, demonstrating that modern mechanisms of attention – particularly CBAM – can partially interpret the neural network results. Additionally, the use of SNN helps to minimize the number of Type II errors compared to the traditional classification-based approach.
About the Author
V. A. Mishchuk
Peoples’ Friendship University of Russia named after Patrice Lumumba
Russian Federation
Russian Federation
Mishchuk Vsevolod Aleksandrovich – PhD student of the Department of Forensic Science, Institute of Law
Moscow 117198
References
1. Fedorenko V.A., Sorokina K.O., Giverts P.V. Multigroup classification of Firing Pin Marks with the Use of a Fully Connected Neural Network. Information Technology and Computing Systems. 2022. No. 3. P. 43–57. (In Russ.). https://doi.org/10.14357/20718632220305
2. Fedorenko V.A., Sorokina K.O., Giverts P.V. The Classification of Firing Pin Impressions Using the Convolutional Neural Network (CNN). Information Technology and Computing Systems. 2024. No. 1. P. 87–96. (In Russ.). https://doi.org/10.14357/20718632240109
3. Rossinskaya E.R. Neural Networks in Forensic Expertology and Expert Practice: Problems and Prospects. Courier of Kutafin Moscow State Law University (MSAL). 2024. Vol. 115. No. 3. P. 21–33. (In Russ.). https://doi.org/10.17803/2311-5998.2024.115.3.021-033
4. Kokin A.V., Denisov Yu.D. Artificial Intelligence in Criminalistics and Forensic Examination: Issues of Legal Personality and Algorithmic Bias. Theory and Practice of Forensic Science. 2023. Vol. 18. No. 2. P. 30–37. (In Russ.). https://doi.org/10.30764/1819-2785-2023-2-30-37
5. Andreeva O.I., Ivanov V.V., Nesterov A.Y., Trubnikova T.V. Facial Recognition Technologies in Criminal Proceedings: Problems of Grounds for the Legal Regulation of Using Artificial Intelligence. Tomsk State University Journal. 2019. No. 449. P. 201–212. (In Russ.). https://doi.org/10.17223/15617793/449/25
6. Fundamentals of Forensic Expertology: Educational and Methodological Guide. Moscow: RFCFS, 2023. 384 p. (In Russ.). https://doi.org/10.30764/978-5-91133-267-9-2023
7. Ghorbani A., Abid A., Zou J. Interpretation of Neural Networks Is Fragile. Proceedings of the AAAI Conference on Artificial Intelligence. 2019. Vol. 33. No. 1. P. 3681–3688.
8. Das A., Agrawal H., Zitnick L., Parikh D., Batra D. Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions? Computer Vision and Image Understanding. 2017. Vol. 163. P. 90–100.
9. Ustinov V.V. On Possibility of Objectivization of Handwriting Expert Examination of Handwritten Requisites. Issues of Expert Practice. 2019. No. S1. P. 663–668. (In Russ.).
10. Meshcheryakov V.A., Butov V.V. Evaluation of the Results of the Identifical Examination of Script/ Writing Due to Modern Information Technologies. The Bulletin of Voronezh Institute of the Ministry of Internal Affairs of Russia. 2017. No. 2. P. 40–46. (In Russ.).
11. Zhuravel A.A., Troshko N.V., Edzhubov L.G. Using the Generalized Portrait Algorithm for Pattern Recognition in Forensic Handwriting Examination Legal Cybernetics. 1970. P. 212–227. (In Russ.).
12. Woo S., J. Park, Lee J.Y., Kweon I.S. CBAM: Convolutional Block Attention Module. Proceedings of the European Conference on Computer Vision (ECCV). 2018. P. 3–19.
13. Xue G., Liu S., Gong D., Ma Y. ATP-DenseNet: A Hybrid Deep Learning-based Gender Identification of Handwriting. Neural Computing and Applications. 2021. Vol. 33. P. 4611–4622.
14. Bobovkin M.V., Didenko O.A., Nesterov A.E. Computer Simulation of Separate and Comparative Study of Private Characteristics Based on the ‘Frosya’ Software Complex. Forensic Sciences. 2021. No. 3 (67). P. 62–71. (In Russ.). https://doi.org/10.25724/VAMVD.UXYZ
15. Zhang X.Y., Xie G.S., Liu C.L., Bengio Y. End-to-End Online Writer Identification with Recurrent Neural Network. IEEE Transactions on Human-Machine Systems. 2016. Vol. 47. No. 2. P. 285– 292.
16. Kao H.H., Wen C.Y. An Offline Signature Verification and Forgery Detection Method Based on a Single Known Sample and an Explainable Deep Learning Approach. Applied Sciences. 2020. Vol. 10. No. 11. 3716. https://doi.org/10.3390/app10113716
17. Simonyan K., Vedaldi A., Zisserman A. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv. 2013. https://doi.org/10.48550/arXiv.1312.6034
18. Selvaraju R.R., Cogswell M., Das A., Vedantam R., Parikh D., Batra D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the IEEE International Conference on Computer Vision. 2017. P. 618–626.
19. Droby A., Rabaev I., Shapira D.V., Kurar Barakat B., El-Sana J. Digital Hebrew Paleography: Script Types and Modes. Journal of Imaging. 2022. Vol. 8. No. 5. P. 143.
20. Marcinowski M. Top Interpretable Neural Network for Handwriting Identification. Journal of Forensic Sciences. 2022. Vol. 67. No. 3. P. 1140–1148.
21. Harralson H.H., Miller L.S. Huber and Headrick’s Handwriting Identification: Facts and Fundamentals. CRC press, 2017. 420 p.
22. Bromley J., Guyon I., LeCun Y., Säckinger E., Shah R. Signature Verification using a «Siamese» Time Delay Neural Network. Advances in Neural Information Processing Systems. 1993. Vol. 6. P. 669–686.
23. Dey S., Dutta A., Toledo J.I., Ghosh S.K., Llados J., Pal U. SigNet: Convolutional Siamese Network for Writer Independent Offline Signature Verification. arXiv. 2017. https://doi.org/10.48550/arXiv.1707.02131
24. Bakhteev D.V. Criminalistic Conclusions on Signature Forgery Process During Building an Offline Signature Verification Intellectual System. Nowa Kodyfikacja Prawa Karnego. 2021. No. 60. P. 9–15.
25. P. Chopra Making Your Neural Network Say “I Don’t Know” – Bayesian NNs using Pyro and PyTorch. Medium. https://towardsdatascience.com/making-your-neural-network-say-i-dont-know-bayesian-nns-using-pyro-and-pytorch-b1c24e6ab8cd
26. Bahdanau D., Cho K., Bengio Y. Neural Machine Translation by Jointly Learning to Align and Translate. arXiv. 2014. https://doi.org/10.48550/arXiv.1409.0473
27. Bakhteev D.V., Sudarikov R. NSP Dataset and Offline Signature Verification. Quality of Information and Communications Technology: 13th International Conference, QUATIC 2020 (Faro, Portugal, September 9–11, 2020). Proceedings 13. Springer International Publishing, 2020. P. 41–49.
Review
For citations:
Mishchuk V.A. Using Convolutional Block Attention Module for Interpreting the Results of Artificial Neural Networks Operation in Handwritten Signature Identification. Theory and Practice of Forensic Science. 2025;20(2):65-81. (In Russ.) https://doi.org/10.30764/1819-2785-2025-2-65-81
Views:
28
Email this article 