STEM-Net: An Explainable Multi-Task Hybrid Architecture Combining Tabular Transformers, Gradient-Boosted Trees and Ensemble Learning for the Simultaneous Prediction of Eight Psychological and Academic Problems in High-School Students
Abstract
Psychological and academic problems in students rarely occur in isolation: depression, anxiety, low self-esteem and addiction proneness tend to appear together, and modelling each of them separately wastes the signal they share. This study introduces the STEM-Net framework. Data were collected from 1,713 high-school students using eight validated questionnaires; after synthetic minority oversampling (SMOTE) the set grew to 5,139 records, and labels were coded at several risk levels based on the official cut-off points of each instrument. This multi-task hybrid architecture predicts all eight problems simultaneously, each at the risk levels defined by its own instrument. The base layer contains five heterogeneous learners in three branches: a multi-task tabular transformer whose attention body is shared across the eight targets, gradient-boosted tree models trained per target, and a pre-trained network. Before fine-tuning, the body of the deep branch is pre-trained on roughly eighty-eight thousand public respondents to two psychometric instruments, and the outputs of the five learners are combined through an automatic choice between stacking and simple averaging. To prevent item-to-score leakage, the items of the target scale are removed from the feature space of that task. On the test set STEM-Net reaches a macro-average accuracy of 0.9277, an F1 of 0.9034 and a ROC-AUC of 0.9844. A four-method explainability analysis shows that in every one of the eight tasks the most informative source is another one of the scales, and that addiction proneness and suicidal ideation are mutually predictive. Building on these features, a web-based screening system with a short questionnaire was designed that reports, for each student, the predicted risk level together with the features that most influenced it.
Downloads
References
World Health Organization, "Mental health of adolescents [Fact sheet]," 2024. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/adolescent-mental-health.
R. Qasrawi, S. P. Vicuna Polo, D. Abu Al-Halawa, S. Hallaq, and Z. Abdeen, "Assessment and prediction of depression and anxiety risk factors in schoolchildren: Machine learning techniques performance analysis," JMIR Formative Research, vol. 6, no. 8, p. e32736, 2022, doi: 10.2196/32736.
C. Romero and S. Ventura, "Educational data mining and learning analytics: An updated survey," WIREs Data Mining and Knowledge Discovery, vol. 10, no. 3, p. e1355, 2020, doi: 10.1002/widm.1355.
R. C. Kessler, P. Berglund, O. Demler, R. Jin, K. R. Merikangas, and E. E. Walters, "Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication," Archives of General Psychiatry, vol. 62, no. 6, pp. 593-602, 2005, doi: 10.1001/archpsyc.62.6.593.
S. Ruder, "An overview of multi-task learning in deep neural networks," arXiv, 2017, doi: 10.48550/arXiv.1706.05098.
B. Saylam and Ö. Durmaz İncel, "Multitask learning for mental health: Depression, anxiety, stress (DAS) using wearables," Diagnostics, vol. 14, no. 5, p. 501, 2024, doi: 10.3390/diagnostics14050501.
V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, and G. Kasneci, "Deep neural networks and tabular data: A survey," IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 6, pp. 7499-7519, 2024, doi: 10.1109/TNNLS.2022.3229161.
Y. Gorishniy, I. Rubachev, V. Khrulkov, and A. Babenko, "Revisiting deep learning models for tabular data," 2021, vol. 34, pp. 18932-18943, doi: 10.48550/arXiv.2106.11959.
N. Hollmann, S. Müller, K. Eggensperger, and F. Hutter, "TabPFN: A transformer that solves small tabular classification problems in a second," in International Conference on Learning Representations, 2023, doi: 10.48550/arXiv.2207.01848.
D. H. Wolpert, "Stacked generalization," Neural Networks, vol. 5, no. 2, pp. 241-259, 1992, doi: 10.1016/S0893-6080(05)80023-1.
S. M. Lundberg and S. I. Lee, "A unified approach to interpreting model predictions," 2017, vol. 30, pp. 4765-4774. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html.
M. Sundararajan, A. Taly, and Q. Yan, "Axiomatic attribution for deep networks," in Proceedings of the 34th International Conference on Machine Learning, 2017, vol. 70, pp. 3319-3328. [Online]. Available: http://proceedings.mlr.press/v70/sundararajan17a.html.
T. Chen and C. Guestrin, "XGBoost: A scalable tree boosting system," in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 785-794, doi: 10.1145/2939672.2939785.
M. Rosenberg, Society and the adolescent self-image. Princeton University Press, 1965.
S. H. Lovibond and P. F. Lovibond, Manual for the Depression Anxiety Stress Scales, 2nd ed. Psychology Foundation of Australia, 1995.
A. T. Beck, R. A. Steer, and G. K. Brown, Manual for the Beck Depression Inventory-II. Psychological Corporation, 1996.
N. B. Epstein, L. M. Baldwin, and D. S. Bishop, "The McMaster Family Assessment Device," Journal of Marital and Family Therapy, vol. 9, no. 2, pp. 171-180, 1983, doi: 10.1111/j.1752-0606.1983.tb01497.x.
A. T. Beck, M. Kovacs, and A. Weissman, "Assessment of suicidal intention: The Scale for Suicide Ideation," Journal of Consulting and Clinical Psychology, vol. 47, no. 2, pp. 343-352, 1979, doi: 10.1037/0022-006X.47.2.343.
Y. Zargar, "Construction and validation of the Iranian Addiction Potential Scale," 2006.
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, "SMOTE: Synthetic Minority Over-sampling Technique," Journal of Artificial Intelligence Research, vol. 16, pp. 321-357, 2002, doi: 10.1613/jair.953.
F. Pedregosa et al., "Scikit-learn: Machine learning in Python," Journal of Machine Learning Research, vol. 12, pp. 2825-2830, 2011. [Online]. Available: http://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf?source=post_page.
Downloads
Published
Submitted
Revised
Accepted
Issue
Section
License
Copyright (c) 2025 AbdolRahim Papi (Author); Maseud Rahgozar; Habibollah Arasteh Rad, Arshia Badi, Mohammad Hossein Rezvani (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
