The landscape of ontologies in materials science and engineering: A survey and evaluation

Ontologies are widely used in materials science to describe experiments, processes, material properties, and experimental and computational workflows. Numerous online platforms are available for accessing and sharing ontologies in Materials Science and Engineering (MSE). Additionally, several surveys of these ontologies have been conducted. However, these studies often lack comprehensive analysis and quality control metrics. This paper provides an overview of ontologies used in Materials Science and Engineering to assist domain experts in selecting the most suitable ontology for a given purpose. Sixty selected ontologies are analyzed and compared based on the requirements outlined in this paper. Statistical data on ontology reuse and key metrics are also presented. The evaluation results provide valuable insights into the strengths and weaknesses of the investigated MSE ontologies. This enables domain experts to select suitable ontologies and to incorporate relevant terms from existing resources.

OAEI Machine Learning Dataset for Online Model Generation

Ontology and knowledge graph matching systems are evaluated annually by the Ontology Alignment Evaluation Initiative (OAEI). More and more systems use machine learning-based approaches, including large language models. The training and validation datasets are usually determined by the system developer and often a subset of the reference alignments are used. This sampling is against the OAEI rules and makes a fair comparison impossible. Furthermore, those models are trained offline (a trained and optimized model is packaged into the matcher) and therefore the systems are specifically trained for those tasks. In this paper, we introduce a dataset that contains training, validation, and test sets for most of the OAEI tracks. Thus, online model learning (the systems must adapt to the given input alignment without human intervention) is made possible to enable a fair comparison for ML-based systems. We showcase the usefulness of the dataset by fine-tuning the confidence thresholds of popular systems.