Predicting Visual Importance Across Graphic Design Types

This paper introduces a Unified Model of Saliency and Importance (UMSI), which learns to predict visual importance in input graphic designs, and saliency in natural images, along with a new dataset and applications. Previous methods for predicting saliency or visual importance are trained individually on specialized datasets, making them limited in application and leading to poor generalization on novel image classes, while requiring a user to know which model to apply to which input. UMSI is a deep learning-based model simultaneously trained on images from different design classes, including posters, infographics, mobile UIs, as well as natural images, and includes an automatic classification module to classify the input. This allows the model to work more effectively without requiring a user to label the input. We also introduce Imp1k, a new dataset of designs annotated with importance information. We demonstrate two new design interfaces that use importance prediction, including a tool for adjusting the relative importance of design elements, and a tool for reflowing designs to new aspect ratios while preserving visual importance. The model, code, and importance dataset are available at https://predimportance.mit.edu .

Development and application of a machine learning supported methodology for measurement and verification (M&V) 2.0

The foundations of all methodologies for the measurement and verification (M&V) of energy savings are based on the same five key principles: accuracy, completeness, conservatism, consistency and transparency. The most widely accepted methodologies tend to generalise M&V so as to ensure applicability across the spectrum of energy conservation measures (ECM's). These do not provide a rigid calculation procedure to follow. This paper aims to bridge the gap between high-level methodologies and the practical application of modelling algorithms, with a focus on the industrial buildings sector. This is achieved with the development of a novel, machine learning supported methodology for M&V 2.0 which enables accurate quantification of savings. A novel and computationally efficient feature selection algorithm and powerful machine learning regression algorithms are employed to maximise the effectiveness of available data. The baseline period energy consumption is modelled using artificial neural networks, support vector machines, k-nearest neighbours and multiple ordinary least squares regression. Improved knowledge discovery and an expanded boundary of analysis allow more complex energy systems be analysed, thus increasing the applicability of M&V. A case study in a large biomedical manufacturing facility is used to demonstrate the methodology's ability to accurately quantify the savings under real-world conditions. The ECM was found to result in 604,527 kWh of energy savings with 57% uncertainty at a confidence interval of 68%. 20 baseline energy models are developed using an exhaustive approach with the optimal model being used to quantify savings. The range of savings estimated with each model are presented and the acceptability of uncertainty is reviewed. The case study demonstrates the ability of the methodology to perform M&V to an acceptable standard in challenging circumstances.

Learning Visual Importance for Graphic Designs and Data Visualizations

Knowing where people look and click on visual designs can provide clues about how the designs are perceived, and where the most important or relevant content lies. The most important content of a visual design can be used for effective summarization or to facilitate retrieval from a database. We present automated models that predict the relative importance of different elements in data visualizations and graphic designs. Our models are neural networks trained on human clicks and importance annotations on hundreds of designs. We collected a new dataset of crowdsourced importance, and analyzed the predictions of our models with respect to ground truth importance and human eye movements. We demonstrate how such predictions of importance can be used for automatic design retargeting and thumbnailing. User studies with hundreds of MTurk participants validate that, with limited post-processing, our importance-driven applications are on par with, or outperform, current state-of-the-art methods, including natural image saliency. We also provide a demonstration of how our importance predictions can be built into interactive design tools to offer immediate feedback during the design process.