查看更多>>摘要：Adversarial examples are usually generated by adding adversarial perturbations on clean samples, designed to deceive the model to make wrong classifications. Adversarial robustness refers to the ability of a model to resist adversarial attacks. And currently, a mainstream method to enhance adversarial robustness is the Projected Gradient Descent (PGD). However, PGD is often criticized for being time-consuming during constructing adversarial examples. Fast adversarial training can improve the adversarial robustness in shorter time, but it only can train for a limited number of epochs, leading to sub-optimal performance. This paper demonstrates that the multi-exit network can reduce the impact of adversarial perturbations by outputting easily identified samples at early exits. Therefore, we can improve the adversarial robustness. Further, we find that the multi-exit network can prevent catastrophic overfitting existing in single-step adversarial training. Specifically, we find that, in the multi-exit network, (1) the norm of weights at a fully connected layer in a non-overfitted exit is much smaller than that in an overfitted exit; and (2) catastrophic overfitting occurs when the late exits have weight norms larger than the early exits. Based on these findings, we propose an approach to alleviating the catastrophic overfitting of the multi-exit network. Compared to PGD adversarial training, our approach can train a model with decreased time complexity and increased empirical robustness. Extensive experiments have been conducted to evaluate our approach against various adversarial attacks, and the experimental results demonstrate superior robustness accuracies on CIFAR-10, CIFAR-100 and SVHN. (C)& nbsp;2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：Collaborative representation-based classification (CRC), as a typical kind of linear representation-based classification, has attracted more attention due to the effective and efficient pattern classification performance. However, the existing class-specific representations are not competitively learned from collaborative representation for achieving more informative pattern discrimination among all the classes. With the purpose of enhancing the power of competitive and discriminant representations among all the classes for favorable classification, we propose a novel CRC method called the class-specific mean vector-based weighted competitive and collaborative representation (CMWCCR). The CMWCCR mainly contains three discriminative constraints including the competitive, mean vector and weighted constraints that fully employ the discrimination information in different ways. In the competitive constraint, the representations from any one class and the other classes are adapted for learning competitive representations among all the classes. In the newly designed mean vector constraint, the mean vectors of all the class-specific training samples with the corresponding class-specific representations are taken into account to further enhance the competitive representations. In the devised weighted constraint, the class-specific weights are constrained on the representation coefficients to make the similar classes have more representation contributions to strengthening the discrimination among all the class-specific representations. Thus, these three constraints in the unified CMWCCR model can complement each other for competitively learning the discriminative class-specific representations. To verify the CMWCCR classification performance, the extensive experiments are conducted on twenty-eight data sets in comparisons with the state-of-the-art representation-based classification methods. The experimental results show that the proposed CMWCCR is an effective and robust CRC method with satisfactory performance. (C)& nbsp;2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：A keyword spotting (KWS) system running on smart devices should accurately detect the appearances and predict the locations of predefined keywords from audio streams, with small footprint and high efficiency. To this end, this paper proposes a new two-stage KWS method which combines a novel multi-scale depthwise temporal convolution (MDTC) feature extractor and a two-stage keyword detection and localization module. The MDTC feature extractor learns multi-scale feature representation efficiently with dilated depthwise temporal convolution, modeling both the temporal context and the speech rate variation. We use a region proposal network (RPN) as the first-stage KWS. At each frame, we design multiple time regions, which all take the current frame as the end position but have different start positions. These time regions (or formally anchors) are used to indicate rough location candidates of keyword. With frame level features from the MDTC feature extractor as inputs, RPN learns to propose keyword region proposals based on the designed anchors. To alleviate the keyword/non-keyword class imbalance problem, we specifically introduce a hard example mining algorithm to select effective negative anchors in RPN training. The keyword region proposals from the first-stage RPN contain keyword location information which is subsequently used to explicitly extract keyword related sequential features to train the second-stage KWS. The second-stage system learns to classify and transform region proposal to keyword IDs and ground-truth keyword region respectively. Experiments on the Google Speech Command dataset show that the proposed MDTC feature extractor surpasses several competitive feature extractors with a new state-of-the-art command classification error rate of 1.74%. With the MDTC feature extractor, we further conduct wake-up word (WuW) detection and localization experiments on a commercial WuW dataset. Compared to a strong baseline, our proposed two-stage method achieves relatively 27-32% better false rejection rate at one false alarm per hour, while for keyword localization, the two-stage approach achieves more than 0.95 mean intersection-over-union ratio, which is clearly better than the one-stage RPN method.(c) 2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：The paper is devoted to investigating the quasi-synchronization issue of fractional-order multi-layer networks with mismatched parameters under delay-dependent impulsive feedback control. It is worth highlighting that fractional-order multi-layer networks with mismatched parameters, as the extension model for single-layer or two-layer ones, are constructed in this paper. Simultaneously, the intra-layer and inter-layer couplings are taken into consideration, which is more general and rarely considered in discussions of network synchronization. An extended fractional differential inequality with impulsive effects is given to establish the grounded framework and theory on the quasi-synchronization problem under delay-dependent impulsive feedback control. Moreover, in the light of the Lyapunov method and graph theory, two criteria for achieving the quasi-synchronization of fractional-order multi-layer networks with mismatched parameters are derived. Furthermore, exponential convergence rates as well as the bounds of quasi-synchronization errors are successfully deduced. Ultimately, the theoretical results are applied in a practical power system, and some illustrative examples are proposed to show the effectiveness of theoretical analysis. (C)& nbsp;2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：Deep neural networks (DNNs) are vulnerable to adversarial examples, which are crafted by imposing mild perturbation on clean ones. An intriguing property of adversarial examples is that they are efficient among different DNNs. Thus transfer-based attacks against DNNs become an increasing concern. In this scenario, attackers devise adversarial instances based on the local model without feedback information from the target one. Unfortunately, most existing transfer-based attack methods only employ a single local model to generate adversarial examples. It results in poor transferability because of overfitting to the local model. Although several ensemble attacks have been proposed, the transferability of adversarial examples merely have a slight increase. Meanwhile, these methods need high memory cost during the training process. To this end, we propose a novel attack strategy called stochastic serial attack (SSA). It adopts a serial strategy to attack local models, which reduces memory consumption compared to parallel attacks. Moreover, since local models are stochastically selected from a large model set, SSA can ensure that the adversarial examples do not overfit specific weaknesses of local source models. Extensive experiments on the ImageNet dataset and NeurIPS 2017 adversarial competition dataset show the effectiveness of SSA in improving the transferability of adversarial examples and reducing the memory consumption of the training process. (c) 2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：Multichannel signal data analysis has been crucial in various industrial applications, such as human activity recognition, vehicle failure predictions, and manufacturing equipment monitoring. Recently, deep neural networks have come into use for multichannel signal data because of their ability to automatically extract useful features from complex multichannel signals. However, deep neural networks are black-box models whose internal working mechanisms cannot be put in a form readily understood by humans. To address this issue, we have proposed an uncertainty-aware hierarchical segment-channel attention model that consists of a time segment and channel level attentions. The hierarchical attention mechanism enables a neural network to identify important time segments and channels critical for prediction, making the model explainable. In addition, the model uses variational inferences to provide uncertainty information that yields a confidence interval that can be easily explained. We conducted experiments on simulated and real-world datasets to demonstrate the usefulness and applicability of our method. The results confirm that our method can attend to important time segments and sensors while achieving better classification performance. (C)& nbsp;2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：Unsupervised feature selection is one of the efficient approaches to reduce the dimension of unlabeled high-dimensional data. We present a novel adaptive autoencoder with redundancy control (AARC) as an unsupervised feature selector. By adding two Group Lasso penalties to the objective function, AARC integrates unsupervised feature selection and determination of a compact network structure into a single framework. Besides, a penalty based on a measure of dependency between features (such as Pearson correlation, mutual information) is added to the objective function for controlling the level of redundancy in the selected features. To realize the desired effects of different regularizers in different phases of the training, we introduce adaptive parameters which change with iterations. In addition, a smoothing function is utilized to approximate the three penalties since they are not differentiable at the origin. An ablation study is carried out to validate the capabilities of redundancy control and structure optimization of AARC. Subsequently, comparisons with nine state-of-the-art methods illustrate the efficiency of AARC for unsupervised feature selection. (c) 2022 Elsevier Ltd. All rights reserved.
查看更多>>摘要：The main purpose of multi-view subspace clustering is to reveal the intrinsic low-dimensional architecture of data points according to their multi-view characteristics. Exploring the potential relationship from views is one of the most essential research focuses of the multi-view task. To better utilize the complementary and consistency information from distinct views, we propose a novel robust subspace clustering approach based on consensus representation and orthogonal diversity (RMSCCO). A novel defined orthogonality term is adopted to improve the diversity and decrease the redundance of learning subspace representation. The consensus representation and subspace learning are integrated into one unified framework to characterize the consistency from views. The groupingenhanced representation is utilized to maintain the local geometric architecture in the original data space. The l2,1-norm regularizer constraint to the noise is applied to improve the robustness. Finally, an optimization algorithm is exploited to solve RMSCCO with the Alternating Direction Method of Multipliers (ADMM). Extensive experimental results on six challenging datasets demonstrate that our approach has accomplished highly qualified performance.
查看更多>>摘要：In the absence of unseen training data, zero-shot learning algorithms utilize the semantic knowledge shared by the seen and unseen classes to establish the connection between the visual space and the semantic space, so as to realize the recognition of the unseen classes. However, in real applications, the original semantic representation cannot well characterize both the class-specificity structure and discriminative information in dimension space, which leads to unseen classes being easily misclassified into seen classes. To tackle this problem, we propose a Salient Attributes Learning Network (SALN) to generate discriminative and expressive semantic representation under the supervision of the visual features. Meanwhile, l(1,2)-norm constraint is employed to make the learned semantic representation well characterize the class-specificity structure and discriminative information in dimension space. Then feature alignment network projects the learned semantic representation into visual space and a relation network is adopted for classification. The performance of the proposed approach has made progress on the five benchmark datasets in generalized zero-shot learning task, and in-depth experiments indicate the effectiveness and excellence of our method. (C) 2022 Elsevier Ltd. All rights reserved.
Almalioglu, YasinTuran, MehmetSaputra, Muhamad Risqi U.de Gusmao, Pedro P. B....
18页
查看更多>>摘要：In the last decade, numerous supervised deep learning approaches have been proposed for visual- inertial odometry (VIO) and depth map estimation, which require large amounts of labelled data. To overcome the data limitation, self-supervised learning has emerged as a promising alternative that exploits constraints such as geometric and photometric consistency in the scene. In this study, we present a novel self-supervised deep learning-based VIO and depth map recovery approach (SelfVIO) using adversarial training and self-adaptive visual-inertial sensor fusion. SelfVIO learns the joint estimation of 6 degrees-of-freedom (6-DoF) ego-motion and a depth map of the scene from unlabelled monocular RGB image sequences and inertial measurement unit (IMU) readings. The proposed approach is able to perform VIO without requiring IMU intrinsic parameters and/or extrinsic calibration between IMU and the camera. We provide comprehensive quantitative and qualitative evaluations of the proposed framework and compare its performance with state-of-the-art VIO, VO, and visual simultaneous localization and mapping (VSLAM) approaches on the KITTI, EuRoC and Cityscapes datasets. Detailed comparisons prove that SelfVIO outperforms state-of-the-art VIO approaches in terms of pose estimation and depth recovery, making it a promising approach among existing methods in the literature.(c) 2022 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
NSTL
Elsevier