Tasks People Prompt: A Taxonomy of LLM Downstream Tasks in Software Verification and Falsification Approaches

Prompting has become one of the main approaches to leverage emergent capabilities of Large Language Models [Brown et al. NeurIPS 2020, Wei et al. TMLR 2022, Wei et al. NeurIPS 2022]. During the last year, researchers and practitioners have been playing with prompts to see how to make the most of LLMs. By homogeneously dissecting 80 papers, we investigate in deep how software testing and verification research communities have been abstractly architecting their LLM-enabled solutions. More precisely, first, we want to validate whether downstream tasks are an adequate concept to convey the blueprint of prompt-based solutions. We also aim at identifying number and nature of such tasks in solutions. For such goal, we develop a novel downstream task taxonomy that enables pinpointing some engineering patterns in a rather varied spectrum of Software Engineering problems that encompasses testing, fuzzing, debugging, vulnerability detection, static analysis and program verification approaches.

Empirical Evaluation of Project Scheduling Algorithms for Maximization of the Net Present Value

This paper presents an empirical performance analysis of three project scheduling algorithms dealing with maximizing projects' net present value with unrestricted resources. The selected algorithms, being the most recently cited in the literature, are: Recursive Search (RS), Steepest Ascent Approach (SAA) and Hybrid Search (HS). The main motivation for this research is the lack of knowledge about the computational complexities of the RS, SAA, and HS algorithms, since all studies to date show some gaps in the analysis. Furthermore, the empirical analysis performed to date does not consider the fact that one algorithm (HS) uses a dual search strategy, which markedly improved the algorithm's performance, while the others don't. In order to obtain a fair performance comparison, we implemented the dual search strategy into the other two algorithms (RS and SAA), and the new algorithms were called Recursive Search Forward-Backward (RSFB) and Steepest Ascent Approach Forward-Backward (SAAFB). The algorithms RSFB, SAAFB, and HS were submitted to a factorial experiment with three different project network sampling characteristics. The results were analyzed using the Generalized Linear Models (GLM) statistical modeling technique that showed: a) the general computational costs of RSFB, SAAFB, and HS; b) the costs of restarting the search in the spanning tree as part of the total cost of the algorithms; c) and statistically significant differences between the distributions of the algorithms' results.