Multi-Modal Framework for Autism Severity Assessment Using Spatio-Temporal Graph Transformers

Diagnosing Autism Spectrum Disorder (ASD) remains challenging, as it often relies on subjective evaluations and traditional methods using fMRI data. This paper proposes an innovative multi-modal framework that leverages spatiotemporal graph transformers to assess ASD severity using skeletal and optical flow data from the MMASD dataset. Our approach captures movement synchronization between children with ASD and therapists during play therapy interventions. The framework integrates a spatial encoder, a temporal transformer, and an I3D network for comprehensive motion analysis. Through this multi-modal approach, we aim to deliver reliable ASD severity scores, enhancing diagnostic accuracy and offering a scalable, robust alternative to traditional techniques.