Simple is Better and Large is Not Enough: Towards Ensembling of Foundational Language Models

Foundational Language Models (FLMs) have advanced natural language processing (NLP) research. Current researchers are developing larger FLMs (e.g., XLNet, T5) to enable contextualized language representation, classification, and generation. While developing larger FLMs has been of significant advantage, it is also a liability concerning hallucination and predictive uncertainty. Fundamentally, larger FLMs are built on the same foundations as smaller FLMs (e.g., BERT); hence, one must recognize the potential of smaller FLMs which can be realized through an ensemble. In the current research, we perform a reality check on FLMs and their ensemble on benchmark and real-world datasets. We hypothesize that the ensembling of FLMs can influence the individualistic attention of FLMs and unravel the strength of coordination and cooperation of different FLMs. We utilize BERT and define three other ensemble techniques: {Shallow, Semi, and Deep}, wherein the Deep-Ensemble introduces a knowledge-guided reinforcement learning approach. We discovered that the suggested Deep-Ensemble BERT outperforms its large variation i.e. BERTlarge, by a factor of many times using datasets that show the usefulness of NLP in sensitive fields, such as mental health.

A Community-Aware Framework for Social Influence Maximization

We consider the Influence Maximization (IM) problem: 'if we can try to convince a subset of individuals in a social network to adopt a new product or innovation, and the goal is to trigger a large cascade of further adoptions, which set of individuals should we target'? Formally, it is the task of selecting $k$ seed nodes in a social network such that the expected number of influenced nodes in the network (under some influence propagation model) is maximized. This problem has been widely studied in the literature and several solution approaches have been proposed. However, most simulation-based approaches involve time-consuming Monte-Carlo simulations to compute the influence of the seed nodes in the entire network. This limits the applicability of these methods on large social networks. In the paper, we are interested in solving the problem of influence maximization in a time-efficient manner. We propose a community-aware divide-and-conquer strategy that involves (i) learning the inherent community structure of the social network, (ii) generating candidate solutions by solving the influence maximization problem for each community, and (iii) selecting the final set of individuals from the candidate solutions using a novel progressive budgeting scheme. We provide experiments on real-world social networks, showing that the proposed algorithm outperforms the simulation-based algorithms in terms of empirical run-time and the heuristic algorithms in terms of influence. We also study the effect of the community structure on the performance of our algorithm. Our experiments show that the community structures with higher modularity lead the proposed algorithm to perform better in terms of run-time and influence.