1 First Department of Neurology, Faculty of Medicine, Masaryk University, Pekarska 53, 602 00 Brno, Czech Republic; Institute of Scientific Instruments of the CAS, v. v. i., Kralovopolska 147, 612 00 Brno, Czech Republic; Department of Neurology, Duke University School of Medicine, 2424 Erwin Rd, Durham, NC 27705, the United States of America.
2 First Department of Neurology, Faculty of Medicine, Masaryk University, Pekarska 53, 602 00 Brno, Czech Republic; Institute of Scientific Instruments of the CAS, v. v. i., Kralovopolska 147, 612 00 Brno, Czech Republic.
3 Institute of Scientific Instruments of the CAS, v. v. i., Kralovopolska 147, 612 00 Brno, Czech Republic; International Clinical Research Center, St. Anne's University Hospital, Pekarska 53, 602 00 Brno, Czech Republic.
4 International Clinical Research Center, St. Anne's University Hospital, Pekarska 53, 602 00 Brno, Czech Republic.
5 Brno Epilepsy Center, Department of Neurology, St. Anne's University Hospital, member of ERN-EpiCARE, Faculty of Medicine, Masaryk University, Pekarska 53, 602 00 Brno, Czech Republic.
6 Center for Sleep Medicine, Lyon University Hospital, Lyon 1 University, 103 Grande Rue de la Croix-Rousse, 69004 Lyon, France; Lyon Neuroscience Research Center, CH Le Vinatier - Batiment 462 - Neurocampus, 95 Bd Pinel, 69500 Lyon, France.
7 Multimodal Functional Imaging Lab, Department of Physics and Concordia School of Health, Concordia University and Biomedical Engineering Department, McGill University, Montreal Neurological Hospital, Concordia University, 7141 Sherbrooke Street West, Montreal, QC H4B 1R6.
8 Montreal Neurological Institute, McGill University, 3801 Rue University, Montreal, QC H3A 2B4, Quebec, Canada.
9 Institute of Scientific Instruments of the CAS, v. v. i., Kralovopolska 147, 612 00 Brno, Czech Republic.
10 Brno Epilepsy Center, Department of Neurology, St. Anne's University Hospital, member of ERN-EpiCARE, Faculty of Medicine, Masaryk University, Pekarska 53, 602 00 Brno, Czech Republic; Behavioral and Social Neuroscience Research Group, CEITEC Central European Institute of Technology, Masaryk University, Zerotinovo nám 617/9, 601 77 Brno, Czech Republic.
11 Institute of Scientific Instruments of the CAS, v. v. i., Kralovopolska 147, 612 00 Brno, Czech Republic. Electronic address: petr.klimes@isibrno.cz.
12 Montreal Neurological Hospital, McGill University, 3801 Rue University, Montreal, QC H3A 2B4, Quebec, Canada; Department of Neurology, Duke University Medical School and Department of Biomedical Engineering, Pratt School of Engineering, 2424 Erwin Road, Durham, NC 27705, the United States of America. Electronic address: birgit.frauscher@duke.edu.

Introduction: Precise localization of the epileptogenic zone is critical for successful epilepsy surgery. However, imbalanced datasets in terms of epileptic vs. normal electrode contacts and a lack of standardized evaluation guidelines hinder the consistent evaluation of automatic machine learning localization models.

Methods: This study addresses these challenges by analyzing class imbalance in clinical datasets and evaluating common assessment metrics. Data from 139 drug-resistant epilepsy patients across two Institutions were analyzed. Metric behaviors were examined using clinical and simulated data.

Results: Complementary use of Area Under the Receiver Operating Characteristic (AUROC) and Area Under the Precision-Recall Curve (AUPRC) provides an optimal evaluation approach. This must be paired with an analysis of class imbalance and its impact due to significant variations found in clinical datasets.

Conclusions: The proposed framework offers a comprehensive and reliable method for evaluating machine learning models in epileptogenic zone localization, improving their precision and clinical relevance.

Significance: Adopting this framework will improve the comparability and multicenter testing of machine learning models in epileptogenic zone localization, enhancing their reliability and ultimately leading to better surgical outcomes for epilepsy patients.