查看更多>>摘要：This paper addresses a new interpretation of the traditional optimization method in reinforcement learning (RL) as optimization problems using reverse Kullback-Leibler (KL) divergence, and derives a new optimization method using forward KL divergence, instead of reverse KL divergence in the optimization problems. Although RL originally aims to maximize return indirectly through optimization of policy, the recent work by Levine has proposed a different derivation process with explicit consideration of optimality as stochastic variable. This paper follows this concept and formulates the traditional learning laws for both value function and policy as the optimization problems with reverse KL divergence including optimality. Focusing on the asymmetry of KL divergence, the new optimization problems with forward KL divergence are derived. Remarkably, such new optimization problems can be regarded as optimistic RL. That optimism is intuitively specified by a hyperparameter converted from an uncertainty parameter. In addition, it can be enhanced when it is integrated with prioritized experience replay and eligibility traces, both of which accelerate learning. The effects of this expected optimism was investigated through learning tendencies on numerical simulations using Pybullet. As a result, moderate optimism accelerated learning and yielded higher rewards. In a realistic robotic simulation, the proposed method with the moderate optimism outperformed one of the state-of-the-art RL method. (C) 2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：Recently, deep convolutional neural networks (CNNs) have been widely explored in single image super-resolution (SISR) and achieved remarkable progress. However, most of the existing CNN-based SISR networks with a single-stream structure fail to make full use of the multi-scale features of low resolution (LR) image. While those multi-scale SR models often integrate the information with different receptive fields by means of linear fusion, which leads to the redundant feature extraction and hinders the reconstruction performance of the network. To address both issues, in this paper, we propose a non-linear perceptual multi-scale network (NLPMSNet) to fuse the multi-scale image information in a non-linear manner. Specifically, a novel non-linear perceptual multi-scale module (NLPMSM) is developed to learn more discriminative multi-scale feature correlation by using high-order channel attention mechanism, so as to adaptively extract image features at different scales. Besides, we present a multi-cascade residual nested group (MC-RNG) structure, which uses a global multi-cascade mechanism to organize multiple local residual nested groups (LRNG) to capture sufficient non local hierarchical context information for reconstructing high-frequency details. LRNG uses a local residual nesting mechanism to stack NLPMSMs, which aims to form a more effective residual learning mechanism and obtain more representative local features. Experimental results show that, compared with the state-of-the-art SISR methods, the proposed NLPMSNet performs well in both quantitative metrics and visual quality with a small number of parameters. (C) 2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：In this paper, an event-triggered integral reinforcement learning (IRL) algorithm is developed for the nonzero-sum game problem with asymmetric input saturation. First, for each player, a novel non quadratic value function with a discount factor is designed, and the coupled Hamilton-Jacobi equation that does not require a complete knowledge of the game is derived by using the idea of IRL. Second, the execution of each player is based on the event-triggered mechanism. In the implementation, an adaptive dynamic programming based learning scheme using a single critic neural network (NN) is developed. Experience replay technique is introduced into the classical gradient descent method to tune the weights of the critic NN. The stability of the system and the elimination of Zeno behavior are proved. Finally, simulation experiments verify the effectiveness of the event-triggered IRL algorithm. (C) 2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：Attributed graph clustering is challenging as it needs to effectively combine both graph structure and node feature information to accomplish node clustering. Recent studies mostly adopt graph neural networks to learn node embeddings, then apply traditional clustering methods to obtain clusters. However, their node embeddings are not specifically designed for clustering. Moreover, most of their loss functions only rely on either structure or feature information, making both kinds of information not fully retained in node embeddings. In this paper, we propose a multi-task embedding learning method (MTEL) for attributed graph clustering, which constructs two prediction tasks in terms of structure and feature based adjacency matrices respectively. To make the node embeddings helpful for the downstream clustering, in each task, we predict the minimum hop number between each pair of nodes in the adjacency matrix, so that the correlation degrees among nodes can be encoded into node embeddings. To improve the performance of the prediction task, we regularize the model parameters in these two tasks via l(2,1) norm, through which the model parameters can be jointly learned. Experiments on real attributed graphs show that MTEL is superior for attributed graph clustering over state-of-the-art methods. (C) 2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：We present a novel artificial cognitive mapping system using generative deep neural networks, called variational autoencoder/generative adversarial network (VAE/GAN), which can map input images to latent vectors and generate temporal sequences internally. The results show that the distance of the predicted image is reflected in the distance of the corresponding latent vector after training. This indicates that the latent space is self-organized to reflect the proximity structure of the dataset and may provide a mechanism through which many aspects of cognition are spatially represented. The present study allows the network to internally generate temporal sequences that are analogous to the hippocampal replay/pre-play ability, where VAE produces only near-accurate replays of past experiences, but by introducing GANs, the generated sequences are coupled with instability and novelty. (C)& nbsp;2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：Deep learning (DL) and reinforcement learning (RL) methods seem to be a part of indispensable factors to achieve human-level or super-human AI systems. On the other hand, both DL and RL have strong connections with our brain functions and with neuroscientific findings. In this review, we summarize talks and discussions in the "Deep Learning and Reinforcement Learning"session of the symposium, International Symposium on Artificial Intelligence and Brain Science. In this session, we discussed whether we can achieve comprehensive understanding of human intelligence based on the recent advances of deep learning and reinforcement learning algorithms. Speakers contributed to provide talks about their recent studies that can be key technologies to achieve human-level intelligence. (c) 2022 Published by Elsevier Ltd.
查看更多>>摘要：Recent years deep learning-based methods incorporating facial prior knowledge for face super resolution (FSR) are advancing and have gained impressive performance. However, some important priors such as facial landmarks are not fully exploited in existing methods, leading to noticeable artifacts in the resultant SR face images especially under large magnification. In this paper, we propose a novel multi-level landmark-guided deep network (MLGDN) for FSR. More specifically, to fully exploit the dependencies between low and high resolution images and to reduce network parameters as well as capture more reliable feature representation, we introduce a recursive back-projection network with a particular feedback mechanism for coarse-to-fine FSR. Furthermore, we incorporate an attention fusion module in the front of backbone network to strengthen face components and a feature modulation module to refine features in the middle of backbone network. By this way, the facial landmarks extracted from face images can be fully shared by the modules in different levels, which benefit to produce more faithful facial details. Both quantitative and qualitative performance evaluations on two benchmark databases demonstrate that the proposed MLGDN can achieve more impressive SR results than other state-of-the-art competitors. Code will be available at https://github. com/zhuangcheng31/MLG_Face.git/(C) 2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：Multivariate time series forecasting remains a challenging task because of its nonlinear, non-stationary, high-dimensional, and spatial-temporal characteristics, along with the dependence between variables. To address this limitation, we propose a novel method for multivariate time series forecasting based on nonlinear spiking neural P (NSNP) systems and non-subsampled shearlet transform (NSST). A multivariate time series is first converted into the NSST domain, and then NSNP systems are automatically constructed, trained, and predicted in the NSST domain. Because NSNP systems are used as nonlinear prediction models and work in the NSST domain, the proposed prediction method is essentially a multiscale transform (MST)-based prediction method. Therefore, the proposed prediction method can process nonlinear and non-stationary time series, and the dependence between variables can be characterized by the multiresolution features of the NSST transform. Five real-life multivariate time series were used to compare the proposed prediction method with five state-of-the-art and 28 baseline prediction methods. The comparison results demonstrate the effectiveness of the proposed method for multivariate time-series forecasting.
查看更多>>摘要：Suspended sediment is a significant threat to the Great Barrier Reef (GBR) ecosystem. This catchment pollutant stems primarily from terrestrial soil erosion. Bulk masses of sediments have potential to propagate from river plumes into the mid-shelf and outer-shelf regions. Existing sediment forecasting methods suffer from the problem of low-resolution predictions, making them unsuitable for wide area coverage. In this paper, a novel sediment distribution prediction model is proposed to augment existing water quality management programs for the GBR. This model is based on the state-of-theart Transformer network in conjunction with the well-known finite element analysis. For model training, the emerging physics-informed neural network is employed to incorporate both simulated and measured sediment data. Our proposed Finite Element Transformer (FE-Transformer) model offers accurate predictions of sediment across the entire GBR. It provides unblurred outputs, which cannot be achieved with previous next-frame prediction models. This paves a way for accurate forecasting of sediment, which in turn may lead to improved water quality management for the GBR. (C) 2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：Learning efficient graph representation is the key to favorably addressing downstream tasks on graphs, such as node or graph property prediction. Given the non-Euclidean structural property of graphs, preserving the original graph data's similarity relationship in the embedded space needs specific tools and a similarity metric. This paper develops a new graph representation learning scheme, namely EGG, which embeds approximated second-order graph characteristics into a Grassmann manifold. The proposed strategy leverages graph convolutions to learn hidden representations of the corresponding subspace of the graph, which is then mapped to a Grassmann point of a low dimensional manifold through truncated singular value decomposition (SVD). The established graph embedding approximates denoised correlationship of node attributes, as implemented in the form of a symmetric matrix space for Euclidean calculation. The effectiveness of EGG is demonstrated using both clustering and classification tasks at the node level and graph level. It outperforms baseline models on various benchmarks. (C) 2022 Elsevier Ltd. All rights reserved.
NSTL
Elsevier