A Practical Guide to Fine-tuning Language Models with Limited Data

Employing pre-trained Large Language Models (LLMs) has become the de facto standard in Natural Language Processing (NLP) despite their extensive data requirements. Motivated by the recent surge in research focused on training LLMs with limited data, particularly in low-resource domains and languages, this paper surveys recent transfer learning approaches to optimize model performance in downstream tasks where data is scarce. We first address initial and continued pre-training strategies to better leverage prior knowledge in unseen domains and languages. We then examine how to maximize the utility of limited data during fine-tuning and few-shot learning. The final section takes a task-specific perspective, reviewing models and methods suited for different levels of data scarcity. Our goal is to provide practitioners with practical guidelines for overcoming the challenges posed by constrained data while also highlighting promising directions for future research.

Spotlight on nerves: Portable multispectral optoacoustic imaging of peripheral nerve vascularization and morphology

Various morphological and functional parameters of peripheral nerves and their vascular supply are indicative of pathological changes due to injury or disease. Based on recent improvements in optoacoustic image quality, we explore the ability of multispectral optoacoustic tomography, in tandem with ultrasound imaging (OPUS), to investigate the vascular environment and morphology of peripheral nerves in vivo in a pilot study on healthy volunteers. We showcase the unique ability of optoacoustic imaging to visualize the vasa nervorum by observing intraneurial vessels in healthy nerves in vivo for the first time. In addition, we demonstrate that the label-free spectral optoacoustic contrast of the perfused connective tissue of peripheral nerves can be linked to the endogenous contrast of haemoglobin and collagen. We introduce metrics to analyze the composition of tissue based on its optoacoustic contrast and show that the high-resolution spectral contrast reveals specific differences between nervous tissue and reference tissue in the nerve's surrounding. We discuss how this showcased extraction of peripheral nerve characteristics using multispectral optoacoustic and ultrasound imaging can offer new insights into the pathophysiology of nerve damage and neuropathies, for example, in the context of diabetes.