Evidence-Driven Decision Support for AI Model Selection in Research Software Engineering

The rapid proliferation of artificial intelligence (AI) models and methods presents growing challenges for research software engineers and researchers who must select, integrate, and maintain appropriate models within complex research workflows. Model selection is often performed in an ad hoc manner, relying on fragmented metadata and individual expertise, which can undermine reproducibility, transparency, and overall research software quality. This work proposes a structured and evidence-driven approach to support AI model selection that aligns with both technical and contextual requirements. We conceptualize AI model selection as a Multi-Criteria Decision-Making (MCDM) problem and introduce an evidence-based decision-support framework that integrates automated data collection pipelines, a structured knowledge graph, and MCDM principles. Following the Design Science Research methodology, the proposed framework (ModelSelect) is empirically validated through 50 real-world case studies and comparative experiments against leading generative AI systems. The evaluation results show that ModelSelect produces reliable, interpretable, and reproducible recommendations that closely align with expert reasoning. Across the case studies, the framework achieved high coverage and strong rationale alignment in both model and library recommendation tasks, performing comparably to generative AI assistants while offering superior traceability and consistency. By framing AI model selection as an MCDM problem, this work establishes a rigorous foundation for transparent and reproducible decision support in research software engineering. The proposed framework provides a scalable and explainable pathway for integrating empirical evidence into AI model recommendation processes, ultimately improving the quality and robustness of research software decision-making.

Understanding User Intent Modeling for Conversational Recommender Systems: A Systematic Literature Review

Context: User intent modeling is a crucial process in Natural Language Processing that aims to identify the underlying purpose behind a user's request, enabling personalized responses. With a vast array of approaches introduced in the literature (over 13,000 papers in the last decade), understanding the related concepts and commonly used models in AI-based systems is essential. Method: We conducted a systematic literature review to gather data on models typically employed in designing conversational recommender systems. From the collected data, we developed a decision model to assist researchers in selecting the most suitable models for their systems. Additionally, we performed two case studies to evaluate the effectiveness of our proposed decision model. Results: Our study analyzed 59 distinct models and identified 74 commonly used features. We provided insights into potential model combinations, trends in model selection, quality concerns, evaluation measures, and frequently used datasets for training and evaluating these models. Contribution: Our study contributes practical insights and a comprehensive understanding of user intent modeling, empowering the development of more effective and personalized conversational recommender systems. With the Conversational Recommender System, researchers can perform a more systematic and efficient assessment of fitting intent modeling frameworks.

A Decision Model for Decentralized Autonomous Organization Platform Selection: Three Industry Case Studies

Decentralized autonomous organizations as a new form of online governance arecollections of smart contracts deployed on a blockchain platform that intercede groupsof people. A growing number of Decentralized Autonomous Organization Platforms,such as Aragon and Colony, have been introduced in the market to facilitate thedevelopment process of such organizations. Selecting the best fitting platform ischallenging for the organizations, as a significant number of decision criteria, such aspopularity, developer availability, governance issues, and consistent documentation ofsuch platforms, should be considered. Additionally, decision-makers at theorganizations are not experts in every domain, so they must continuously acquirevolatile knowledge regarding such platforms and keep themselves updated.Accordingly, a decision model is required to analyze the decision criteria usingsystematic identification and evaluation of potential alternative solutions for adevelopment project. We have developed a theoretical framework to assist softwareengineers with a set of Multi-Criteria Decision-Making problems in software production.This study presents a decision model as a Multi-Criteria Decision-Making problem forthe decentralized autonomous organization platform selection problem. Weconducted three industry case studies in the context of three decentralizedautonomous organizations to evaluate the effectiveness and efficiency of the decisionmodel in assisting decision-makers.