查看更多>>摘要：Integral transforms are invaluable mathematical tools to map functions into spaces where they are easier to characterize. We introduce the hyperdimensional transform as a new kind of integral transform. It converts square-integrable functions into noise-robust, holographic, high-dimensional representations called hyperdimensional vectors. The central idea is to approximate a function by a linear combination of random functions. We formally introduce a set of stochastic, orthogonal basis functions and define the hyperdimensional transform and its inverse. We discuss general transform-related properties such as its uniqueness, approximation properties of the inverse transform, and the representation of integrals and derivatives. The hyperdimensional transform offers a powerful, flexible framework that connects closely with other integral transforms, such as the Fourier, Laplace, and fuzzy transforms. Moreover, it provides theoretical foundations and new insights for the field of hyperdimensional computing, a computing paradigm that is rapidly gaining attention for efficient and explainable machine learning algorithms, with potential applications in statistical modelling and machine learning. In addition, we provide straightforward and easily understandable code, which can function as a tutorial and allows for the reproduction of the demonstrated examples, from computing the transform to solving differential equations.
查看更多>>摘要：In this paper, we visualize and quantify the predictive uncertainty of gradient-based post hoc visual explanations for neural networks. Predictive uncertainty refers to the variability in the network predictions under perturbations to the input. Visual post hoc explainability techniques highlight features within an image to justify a network's prediction. We theoretically show that existing evaluation strategies of visual explanatory techniques partially reduce the predictive uncertainty of neural networks. This analysis allows us to construct a plug in approach to visualize and quantify the remaining predictive uncertainty of any gradient-based explanatory technique. We show that every image, network, prediction, and explanatory technique has a unique uncertainty. The proposed uncertainty visualization and quantification yields two key observations. Firstly, oftentimes under incorrect predictions, explanatory techniques are uncertain about the same features that they are attributing the predictions to, thereby reducing the trustworthiness of the explanation. Secondly, objective metrics of an explanation's uncertainty, empirically behave similarly to epistemic uncertainty. We support these observations on two datasets, four explanatory techniques, and six neural network architectures.
查看更多>>摘要：Knowledge-enhanced text generation aims to enhance the quality of generated text by utilizing internal or external knowledge sources. While language models have demonstrated impressive capabilities in generating coherent and fluent text, the lack of interpretability presents a substantial obstacle. The limited interpretability of generated text significantly impacts its practical usability, particularly in knowledge-enhanced text generation tasks that necessitate reliability and explainability. Existing methods often employ domain-specific knowledge retrievers that are tailored to specific data characteristics, limiting their generalizability to diverse data types and tasks. To overcome this limitation, we directly leverage the two-tier architecture of structured knowledge, consisting of high-level entities and lowlevel knowledge triples, to design our task-agnostic structured knowledge hunter. Specifically, we employ a local-global interaction scheme for structured knowledge representation learning and a hierarchical transformer-based pointer network as the backbone for selecting relevant knowledge triples and entities. By combining the strong generative ability of language models with the high faithfulness of the knowledge hunter, our model achieves high interpretability, enabling users to comprehend the model's output generation process. Furthermore, we empirically demonstrate the effectiveness of our model in both internal knowledge-enhanced table-to-text generation on the RotoWireFG dataset and external knowledge-enhanced dialogue response generation on the KdConv dataset. Our task-agnostic model outperforms state-of-the-art methods and corresponding language models, setting new standards on the benchmark.
查看更多>>摘要：Black-box zero-th order optimizationis a central primitive for applications in fields as diverse as finance, physics, and engineering. In a common formulation of this problem, a designer sequentially attempts candidate solutions, receiving noisy feedback on the value of each attempt from the system. In this paper, we study scenarios in which feedback is also provided on the safety of the attempted solution, and the optimizer is constrained to limit the number of unsafe solutions that are tried throughout the optimization process. Focusing on methods based on Bayesian optimization (BO), prior art has introduced an optimization scheme – referred to as SafeOpt – that is guaranteed not to select any unsafe solution with a controllable probability over feedback noise as long as strict assumptions on the safety constraint function are met. In this paper, a novel BO-based approach is introduced that satisfies safety requirements irrespective of properties of the constraint function. This strong theoretical guarantee is obtained at the cost of allowing for an arbitrary, controllable but non-zero, rate of violation of the safety constraint. The proposed method, referred to as Safe-Bocp, builds on online conformal prediction (CP) and is specialized to the cases in which feedback on the safety constraint is either noiseless or noisy. Experimental results on synthetic and real-world data validate the advantages and flexibility of the proposed Safe-Bocp.
查看更多>>摘要：In federated learning systems, a server coordinates the training of machine learning models on data distributed across a number of participating client devices. In each round of training, the server selects a subset of devices to perform model updates and, in turn, aggregates those updates before proceeding to the next round of training. Most state-of-the-art federated learning algorithms assume that the clients are always available to perform training – an assumption readily violated in many practical settings where client availability is intermittent or even transient; moreover, in systems where the server samples from an exceedingly large number of clients, a client will likely participate in at most one round of training. This can lead to biasing the learned global model towards client groups endowed with more resources. In this paper, we consider systems where the clients are naturally grouped based on their data distributions, and the groups exhibit variations in the number of available clients. We present Flics-opt, an algorithm for large-scale federated learning over heterogeneous data distributions, time-varying client availability and further constraints on client participation reflecting, e.g., overall energy efficiency objectives that should be met to achieve sustainable deployment. In particular, Flics-opt dynamically learns a selection policy that adapts to client availability patterns and communication constraints, ensuring per-group long-term participation which minimizes the variance inevitably introduced into the learning process by client sampling. We show that for non-convex smooth functions Flics-opt coupled with SGD converges at $O(1/\sqrt{T})$ rate, matching the state-of-the-art convergence results which require clients to be always available. We test Flics-opt on three realistic federated datasets and show that, in terms of maximum accuracy, Flics-Avg and Flics-Adam outperform traditional FedAvg by up to 280% and 60%, respectively, while exhibiting robustness in face of heterogeneous data distributions.
查看更多>>摘要：In safety- and precision-critical control scenarios for permanent magnet synchronous motors (PMSMs), the external spontaneous disturbance causes unexpected speed drop. The disturbance occurs without routine, so it cannot be modeled specifically. The large speed drop and slow response speed cause a reduced life of the machines driven by PMSMs. Therefore, it is crucial to implement a method that can lead the controller to learn the effects caused by disturbances. To this end, this paper proposes a novel approach based on the basic structure of a backpropagation neural network (BP) for adaptive real-time adjustment in motor control. Regarding the lack of explainability of BP in existing methods, the electric motor physics is embedded into the BP (BP-PHY) gradient update part to enlarge the range of stability. To overcome the shortage of a potentially unstable output of neural network (NN), the learning parameter of NN is tailored based on the stability theory and motor physics. Finally, the proposed methods are implemented into simulations and experiments. The recovery time after disturbance decreases to 51.3% and the speed drop decreases to 50.3% compared to the basic controller of the PMSM, while the control stability of the NN is ensured.
NETL
NSTL
IEEE