首页期刊导航|Neural Networks
期刊信息/Journal information
Neural Networks
Pergamon Press
Neural Networks

Pergamon Press

0893-6080

Neural Networks/Journal Neural NetworksSCIAHCIEIISTP
正式出版
收录年代

    Generative convolution layer for image generation

    Park, SeungShin, Yong-Goo
    10页
    查看更多>>摘要：This paper introduces a novel convolution method, called generative convolution (GConv), which is simple yet effective for improving the generative adversarial network (GAN) performance. Unlike the standard convolution, GConv first selects useful kernels compatible with the given latent vector, and then linearly combines the selected kernels to make latent-specific kernels. Using the latent-specific kernels, the proposed method produces the latent-specific features which encourage the generator to produce high-quality images. This approach is simple but surprisingly effective. First, the GAN performance is significantly improved with a little additional hardware cost. Second, GConv can be employed to the existing state-of-the-art generators without modifying the network architecture. To reveal the superiority of GConv, this paper provides extensive experiments using various standard datasets including CIFAR-10, CIFAR-100, LSUN-Church, CelebA, and tiny-ImageNet. Quantitative evaluations prove that GConv significantly boosts the performances of the unconditional and conditional GANs in terms of Frechet inception distance (FID) and Inception score (IS). For example, the proposed method improves both FID and IS scores on the tiny-ImageNet dataset from 35.13 to 29.76 and 20.23 to 22.64, respectively.
      原文链接:
    • NSTL
    • Elsevier

    Replacing pooling functions in Convolutional Neural Networks by linear combinations of increasing functions

    Rodriguez-Martinez, IosuLafuente, JulioSantiago, Regivan H. N.Bustince, Humberto...
    14页
    查看更多>>摘要：Traditionally, Convolutional Neural Networks make use of the maximum or arithmetic mean in order to reduce the features extracted by convolutional layers in a downsampling process known as pooling. However, there is no strong argument to settle upon one of the two functions and, in practice, this selection turns to be problem dependent. Further, both of these options ignore possible dependencies among the data. We believe that a combination of both of these functions, as well as of additional ones which may retain different information, can benefit the feature extraction process. In this work, we replace traditional pooling by several alternative functions. In particular, we consider linear combinations of order statistics and generalizations of the Sugeno integral, extending the latter's domain to the whole real line and setting the theoretical base for their application. We present an alternative pooling layer based on this strategy which we name "CombPool"layer. We replace the pooling layers of three different architectures of increasing complexity by CombPool layers, and empirically prove over multiple datasets that linear combinations outperform traditional pooling functions in most cases. Further, combinations with either the Sugeno integral or one of its generalizations usually yield the best results, proving a strong candidate to apply in most architectures. (c) 2022 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
      原文链接:
    • NSTL
    • Elsevier

    Knowledge-guided multi-task attention network for survival risk prediction using multi-center computed tomography images

    Zhong, LianzhenLi, CongZhang, WenjuanHu, Chaoen...
    13页
    查看更多>>摘要：Accurate preoperative prediction of overall survival (OS) risk of human cancers based on CT images is greatly significant for personalized treatment. Deep learning methods have been widely explored to improve automated prediction of OS risk. However, the accuracy of OS risk prediction has been limited by prior existing methods. To facilitate capturing survival-related information, we proposed a novel knowledge-guided multi-task network with tailored attention modules for OS risk prediction and prediction of clinical stages simultaneously. The network exploits useful information contained in multiple learning tasks to improve prediction of OS risk. Three multi-center datasets, including two gastric cancer datasets with 459 patients, and a public American lung cancer dataset with 422 patients, are used to evaluate our proposed network. The results show that our proposed network can boost its performance by capturing and sharing information from other predictions of clinical stages. Our method outperforms the state-of-the-art methods with the highest geometrical metric. Furthermore, our method shows better prognostic value with the highest hazard ratio for stratifying patients into high-and low-risk groups. Therefore, our proposed method may be exploited as a potential tool for the improvement of personalized treatment. (C) 2022 Elsevier Ltd. All rights reserved.
      原文链接:
    • NSTL
    • Elsevier

    LAP: Latency-aware automated pruning with dynamic-based filter selection

    Yang, WangdongLi, KenliLi, KeqinLiu, Chubo...
    12页
    查看更多>>摘要：Model pruning is widely used to compress and accelerate convolutional neural networks (CNNs). Conventional pruning techniques only focus on how to remove more parameters while ensuring model accuracy. This work not only covers the optimization of model accuracy, but also optimizes the model latency during pruning. When there are multiple optimization objectives, the difficulty of algorithm design increases exponentially. So latency sensitivity is proposed to effectively guide the determination of layer sparsity in this paper. We present the latency-aware automated pruning (LAP) framework which leverages the reinforcement learning to automatically determine the layer sparsity. Latency sensitivity is used as a prior knowledge and involved into the exploration loop. Rather than relying on a single reward signal such as validation accuracy or floating-point operations (FLOPs), our agent receives the feedback on the accuracy error and latency sensitivity. We also provide a novel filter selection algorithm to accurately distinguish important filters within a layer based on their dynamic changes. Compared to the state-of-the-art compression policies, our framework demonstrated superior performances for VGGNet, ResNet, and MobileNet on CIFAR-10, ImageNet, and Food-101. Our LAP allowed the inference latency of MobileNet-V1 to achieve approximately 1.64 times speedup on the Titan RTX GPU, with no loss of ImageNet Top-1 accuracy. It significantly improved the pareto optimal curve on the accuracy and latency trade-off. (C) 2022 Elsevier Ltd. All rights reserved.
      原文链接:
    • NSTL
    • Elsevier

    Sampled-data synchronization of complex network based on periodic self-triggered intermittent control and its application to image encryption

    Zhou, HuiLiu, ZijiangChu, DianhuiLi, Wenxue...
    15页
    查看更多>>摘要：The aim of this paper is to investigate exponential synchronization issue of time-varying multi-weights network with time delays (TMNTD) via periodic self-triggered intermittent sampled-data control. In particular, it is the first time to combine periodic self-triggered control and intermittent control with sampled-data, which has broader application prospects. Therein, self-triggered scheme is periodic judgment and aimed at intermittent control. And during control intervals in intermittent control, there is periodic sampled-data control. In addition, by applying tools of sampled-data control, intermittent control, event-driven control theory and stability analysis, some sufficient conditions are derived to guarantee exponential synchronization of TMNTD. After that, the theoretical results are utilized to research exponential synchronization issue of time-varying multi-weights Chua's circuits with time delays. Meantime, numerical simulations are provided to demonstrate the validity of the theoretical results. Finally, an image encryption algorithm is designed as a practical application of the developed results.
      原文链接:
    • NSTL
    • Elsevier

    Visual context learning based on textual knowledge for image-text retrieval

    Qin, YuzhuoGu, XiaodongTan, Zhenshan
    16页
    查看更多>>摘要：Image-text bidirectional retrieval is a significant task within cross-modal learning field. The main issue lies on the jointly embedding learning and accurately measuring image-text matching score. Most prior works make use of either intra-modality methods performing within two separate modalities or inter-modality ones combining two modalities tightly. However, intra-modality methods remain ambiguous when learning visual context due to the existence of redundant messages. And inter-modality methods increase the complexity of retrieval because of unifying two modalities closely when learning modal features. In this research, we propose an eclectic Visual Context Learning based on Textual knowledge Network (VCLTN), which transfers textual knowledge to visual modality for context learning and decreases the discrepancy of information capacity between two modalities. Specifically, VCLTN merges label semantics into corresponding regional features and employs those labels as intermediaries between images and texts for better modal alignment. Contextual knowledge of those labels learned within textual modality is utilized to guide the visual context learning. Besides, considering the homogeneity within each modality, global features are merged into regional features for assisting in the context learning. In order to alleviate the imbalance of information capacity between images and texts, entities together with relations inside the given caption are extracted and an auxiliary caption is sampled for attaching supplementary messages to textual modality. Experiments performed on Flickr30K and MS-COCO reveal that our model VCLTN achieves best results compared with the state-of-the-art methods. (C) 2022 Elsevier Ltd. All rights reserved.
      原文链接:
    • NSTL
    • Elsevier

    Branching time active inference: Empirical study and complexity class analysis

    Champion, TheophileBowman, HowardGrzes, Marek
    17页
    查看更多>>摘要：Active inference is a state-of-the-art framework for modelling the brain that explains a wide range of mechanisms such as habit formation, dopaminergic discharge and curiosity. However, recent implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time horizon. Fountas et al. (2020) used Monte Carlo tree search to address this problem, leading to very good results in two different tasks. Additionally, Champion et al. (2021a) proposed a tree search approach based on (temporal) structure learning. This was enabled by the development of a variational message passing approach to active inference (Champion, Bowman, Grzes, 2021), which enables compositional construction of Bayesian networks for active inference. However, this message passing tree search approach, which we call branching-time active inference (BTAI), has never been tested empirically. In this paper, we present an experimental study of the approach (Champion, Grzes, Bowman, 2021) in the context of a maze solving agent. In this context, we show that both improved prior preferences and deeper search help mitigate the vulnerability to local minima. Then, we compare BTAI to standard active inference (AcI) on a graph navigation task. We show that for small graphs, both BTAI and AcI successfully solve the task. For larger graphs, AcI exhibits an exponential (space) complexity class, making the approach intractable. However, BTAI explores the space of policies more efficiently, successfully scaling to larger graphs. Then, BTAI was compared to the POMCP algorithm (Silver and Veness, 2010) on the frozen lake environment. The experiments suggest that BTAI and the POMCP algorithm accumulate a similar amount of reward. Also, we describe when BTAI receives more rewards than the POMCP agent, and when the opposite is true. Finally, we compared BTAI to the approach of Fountas et al. (2020) on the dSprites dataset, and we discussed the pros and cons of each approach. (C) 2022 The Author(s). Published by Elsevier Ltd.
      原文链接:
    • NSTL
    • Elsevier

    Semantic consistency learning on manifold for source data-free unsupervised domain adaptation

    Zou, YanSong, ZihaoLyu, JianzhiChen, Lijuan...
    12页
    查看更多>>摘要：Recently, source data-free unsupervised domain adaptation (SFUDA) attracts increasing attention. Current work shows that the geometry of the target data is helpful to solving this challenging problem. However, these methods define the geometric structures in Euclidean space. The geometry cannot completely draw the semantic relationship between the target data distributed on a manifold. This article proposed a new SFUDA method, semantic consistency learning on manifold (SCLM), to address this problem. Firstly, we generated pseudo-labels for the target data using a new clustering method, EntMomClustering, that enhanced k-means clustering by fusing the entropy momentum. Secondly, we constructed semantic neighbor topology (SNT) to capture complete geometric information on the manifold. Specifically, in SNT, the global neighbor was detected by a developed collaborative representation-based manifold projection, while the local neighbors were obtained by similarity comparison. Thirdly, we performed a semantic consistency learning on SNT to drive a new kind of deep clustering where SNT was taken as the basic clustering unit. To ensure SNT move as entirety, in the developed objective, the entropy regulator was constructed based on a semantic mixture fused on SNT, while the self-supervised regulator encouraged similar classification on SNT. Experiments on three benchmark datasets show that our method achieves state-of-the-art results. The code is available on https://github.com/tntek/SCLM.
      原文链接:
    • NSTL
    • Elsevier

    Set-membership filtering for complex networks with constraint communication channels

    Liu, ChangYang, LixinTao, JieXu, Yong...
    8页
    查看更多>>摘要：The set-membership filtering is studied for a class of multi-rate sampling complex networks with communication capacity constraint. For reducing communication load, the weighted try-once-discard scheduling protocol is utilized to transmit the most needed measurement. To improve the filtering performance, a novel mixed compensation method is proposed to obtain a compensatory measurement that is closer to the actual value. Accordingly, a mixed compensation dependent filter is designed, and a filtering error system is obtained. Sufficient conditions are established to ensure that the filtering error system satisfies PTk-dependent constraint. Then, a new algorithm is designed to obtain the optimized ellipsoid by minimizing the constraint matrix. Finally, an illustrative example is given to demonstrate the validity of the developed filter. (C) 2022 Elsevier Ltd. All rights reserved.
      原文链接:
    • NSTL
    • Elsevier

    GIU-GANs: Global Information Utilization for Generative Adversarial Networks

    Tian, YongqiGong, XueyuanTang, JialinSu, Binghua...
    12页
    查看更多>>摘要：Recently, with the rapid development of artificial intelligence, image generation based on deep learning has advanced significantly. Image generation based on Generative Adversarial Networks (GANs) is a promising study. However, because convolutions are limited by spatial-agnostic and channel-specific, features extracted by conventional GANs based on convolution are constrained. Therefore, GANs cannot capture in-depth details per image. Moreover, straightforwardly stacking of convolutions causes too many parameters and layers in GANs, yielding a high overfitting risk. To overcome the abovementioned limitations, in this study, we propose a GANs called GIU-GANs (where Global Information Utilization: GIU). GIU-GANs leverages a new module called the GIU module, which integrates the squeeze-andexcitation module and involution to focus on global information via the channel attention mechanism, enhancing the generated image quality. Moreover, Batch Normalization (BN) inevitably ignores the representation differences among noise sampled by the generator and thus degrades the generated image quality. Thus, we introduce the representative BN to the GANs' architecture. The CIFAR-10 and CelebA datasets are employed to demonstrate the effectiveness of the proposed model. Numerous experiments indicate that the proposed model achieves state-of-the-art performance. (c) 2022 Elsevier Ltd. All rights reserved.
      原文链接:
    • NSTL
    • Elsevier