17th AIAI 2021, 25 - 27 June 2021, Greece

Evaluating mental patients utilizing video analysis of facial expressions

M Tziomaka, Athanasios Kallipolitis, P Tsanakas, Ilias Maglogiannis


  The objective of this work is to put in numbers the degree of symptoms severity based on the social behavior and cognitive functioning of mental patients when conducting a routine conversation with their attending doctor. Examination of patient’s facial expression manifestations can be a key indicator towards the quantization of cognitive impairment in respect to receiving external emotion expressions. Recent advancements in computer vision machine and deep learn-ing techniques allow the evaluation and recognition of temporal emotional sta-tus through facial expressions. In this context, the paper studies the application of these techniques for the automated recognition of Positive and Negative Syn-drome Scale (PANSS) indicators by means of extracting features from patients’ facial expressions during video teleconferences. The paper discusses the tech-nical details of the implementations of a video classification methodology for the prediction of schizophrenia symptoms’ severity, introduces a novel ap-proach for the interpretation of video classification results and presents initial results where it is demonstrated that the proposed automated techniques can classify to a certain extend specific PANSS indicators.  

*** Title, author list and abstract as seen in the Camera-Ready version of the paper that was provided to Conference Committee. Small changes that may have occurred during processing by Springer may not appear in this window.