Abstract:Generalized additive models (GAM) have been successfully applied to high dimensional data analysis. However, most existing methods cannot simultaneously estimate the link function, the component functions and the variable interaction. To alleviate this problem, we propose a new sparse additive model, named generalized sparse additive model with unknown link function (GSAMUL), in which the component functions are estimated by B-spline basis and the unknown link function is estimated by a multi-layer perceptron (MLP) network. Furthermore, $\ell_{2,1}$-norm regularizer is used for variable selection. The proposed GSAMUL can realize both variable selection and hidden interaction. We integrate this estimation into a bilevel optimization problem, where the data is split into training set and validation set. In theory, we provide the guarantees about the convergence of the approximate procedure. In applications, experimental evaluations on both synthetic and real world data sets consistently validate the effectiveness of the proposed approach.
Abstract:The performance of existing underwater object detection methods degrades seriously when facing domain shift problem caused by complicated underwater environments. Due to the limitation of the number of domains in the dataset, deep detectors easily just memorize a few seen domain, which leads to low generalization ability. Ulteriorly, it can be inferred that the detector trained on as many domains as possible is domain-invariant. Based on this viewpoint, we propose a domain generalization method from the aspect of data augmentation. First, the style transfer model transforms images from one source domain to another, enriching the domain diversity of the training data. Second, interpolating different domains on feature level, new domains can be sampled on the domain manifold. With our method, detectors will be robust to domain shift. Comprehensive experiments on S-UODAC2020 datasets demonstrate that the proposed method is able to learn domain-invariant representations, and outperforms other domain generalization methods. The source code is available at https://github.com/mousecpn.
Abstract:Learning with Fredholm kernel has attracted increasing attention recently since it can effectively utilize the data information to improve the prediction performance. Despite rapid progress on theoretical and experimental evaluations, its generalization analysis has not been explored in learning theory literature. In this paper, we establish the generalization bound of least square regularized regression with Fredholm kernel, which implies that the fast learning rate O(l^{-1}) can be reached under mild capacity conditions. Simulated examples show that this Fredholm regression algorithm can achieve the satisfactory prediction performance.