4th BioHackathon Germany - Implementing AI in Meta-analysis: Can domain specific paper selection and data extraction be done effectively?

Project's leads / contact persons:

- Madhuri Paul This email address is being protected from spambots. You need JavaScript enabled to view it.
- Thomas Döring This email address is being protected from spambots. You need JavaScript enabled to view it.
- Wopke van der Werf This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract:

Scientists have known for centuries that a single study will not resolve a major issue—sometimes not even a minor one (E. Hunter et al., 1986). Therefore, meta-analysis, a systematic approach to synthesizing data from multiple studies (Papakostidis & Giannoudis, 2023), has become a cornerstone in evidence-based research. It helps the researcher to draw a more robust conclusion than any individual study. Particularly in agricultural science, meta-analysis proves to be a powerful statistical method (Ding et al., 2023), as to draw an overall conclusion and see the whole picture, we need to go beyond any singular year or location (Yu et al., 2016).However, in recent days, the number of publications has grown exponentially. A simple search in the Web of Science Core Collection for "intercrop" with other synonyms yielded 10,276 articles (Paul, 2025), which creates an unmanageable workload for researchers to perform meta-analysis. To deal with the exponential growth of scientific literature, as well as to perform more meta-analyses, Artificial Intelligence (AI) and Machine Learning (ML) could be the potential answer.

An authenticated meta-analysis needs a systematic, reproducible, and transparent search process. To select all relevant studies, we need more than a simple keyword search string. For the final selection, different other inclusion and exclusion criteria need to be considered. For example, scientific literature is heterogeneous in structure and terminology. Variations in data collection methods, reporting standards, and measurement tools across studies can introduce inconsistencies. There might be multiple studies by the same authors using similar data sets, using unsuitable variable measurements, or not reporting usable effect sizes—this makes it difficult for AI systems to generalize. Another major challenge is domain specificity. A model trained on medical literature may not perform well on agriculture or social science articles due to differences in terminology and study design. Thus, customization and fine-tuning of models are often required for each domain. Therefore, although there are some meta-analysis software tools available in the market (Yadav, 2024), their usability and limitations need to be checked. Particularly, they need to be validated against real extracted data, like the dataset we obtained from our personal work (Paul, 2025).

To do a domain-specific meta-analysis with an AI model, a multidisciplinary collaboration between statisticians, software developers, and domain experts is essential. Different competence challenges may arise when researchers from different fields attempt to use meta-analysis software. A shared understanding of domain-specific knowledge, statistical principles and methodologies, as well as expertise in ML models, is necessary to create successful domain-specific meta-analysis software. Therefore, it is important to initiate such a complex topic in the presence of different experts together in the same place and at the same time—like the opportunity offered by the BioHackathon by ELIXIR in Germany.

References

Ding, W., Li, J., Ma, H., Wu, Y., & He, H. (2023). Science Mapping of Meta-Analysis in Agricultural Science. Information,14(11), 611. https://doi.org/10.3390/info14110611

E. Hunter, J., L. Schmidt, F., & B. Jackson, G. (1986). Meta-Analysis: Cumulating Research Findings Across StudiesSage Publications: Beverly Hills, 1982, 176 pp. Educational Researcher, 15(8), 20–21. https://doi.org/10.3102/0013189x015008020

Papakostidis, C., & Giannoudis, P. V. (2023). Meta-analysis. What have we learned? Injury, 54, S30–S34. https://doi.org/10.1016/j.injury.2022.06.012

Paul, M. (2025). Intercropping in Europe: A meta-analysis. ARPHA Conference Abstracts, 8. https://doi.org/10.3897/aca.8.e152342

Yadav, S. (2024). Challenges and Concerns in the Utilization of Meta-Analysis Software: Navigating the Landscape of Scientific Synthesis. Cureus. https://doi.org/10.7759/cureus.53322

Yu, Y., Stomph, T.-J., Makowski, D., Zhang, L., & van der Werf, W. (2016). A meta-analysis of relative crop yields in cereal/legume mixtures suggests options for management. Field Crops Research, 198, 269–279. https://doi.org/10.1016/j.fcr.2016.08.001