Toward Deep Adaptive Hinging Hyperplanes

The adaptive hinging hyperplane (AHH) model is a popular piecewise linear representation with a generalized tree structure and has been successfully applied in dynamic system identification. In this article, we aim to construct the deep AHH (DAHH) model to extend and generalize the networking of AHH model for high-dimensional problems. The network structure of DAHH is determined through a forward growth, in which the activity ratio is introduced to select effective neurons and no connecting weights are involved between the layers. Then, all neurons in the DAHH network can be flexibly connected to the output in a skip-layer format, and only the corresponding weights are the parameters to optimize. With such a network framework, the backpropagation algorithm can be implemented in DAHH to efficiently tackle large-scale problems and the gradient vanishing problem is not encountered in the training of DAHH. In fact, the optimization problem of DAHH can maintain convexity with convex loss in the output layer, which brings natural advantages in optimization. Different from the existing neural networks, DAHH is easier to interpret, where neurons are connected sparsely and analysis of variance (ANOVA) decomposition can be applied, facilitating to revealing the interactions between variables. A theoretical analysis toward universal approximation ability and explicit domain partitions are also derived. Numerical experiments verify the effectiveness of the proposed DAHH.

Multiple-Target Homotopic Quasi-Complete Path Planning Method for Mobile Robot Using a Piecewise Linear Approach

The ability to plan a multiple-target path that goes through places considered important is desirable for autonomous mobile robots that perform tasks in industrial environments. This characteristic is necessary for inspection robots that monitor the critical conditions of sectors in thermal, nuclear, and hydropower plants. This ability is also useful for applications such as service at home, victim rescue, museum guidance, land mine detection, and so forth. Multiple-target collision-free path planning is a topic that has not been very studied because of the complexity that it implies. Usually, this issue is left in second place because, commonly, it is solved by segmentation using the point-to-point strategy. Nevertheless, this approach exhibits a poor performance, in terms of path length, due to unnecessary turnings and redundant segments present in the found path. In this paper, a multiple-target method based on homotopy continuation capable to calculate a collision-free path in a single execution for complex environments is presented. This method exhibits a better performance, both in speed and efficiency, and robustness compared to the original Homotopic Path Planning Method (HPPM). Among the new schemes that improve their performance are the Double Spherical Tracking (DST), the dummy obstacle scheme, and a systematic criterion to a selection of repulsion parameter. The case studies show its effectiveness to find a solution path for office-like environments in just a few milliseconds, even if they have narrow corridors and hundreds of obstacles. Additionally, a comparison between the proposed method and sampling-based planning algorithms (SBP) with the best performance is presented. Furthermore, the results of case studies show that the proposed method exhibits a better performance than SBP algorithms for execution time, memory, and in some cases path length metrics. Finally, to validate the feasibility of the paths calculated by the proposed planner; two simulations using the pure-pursuit controlled and differential drive robot model contained in the Robotics System Toolbox of MATLAB are presented.

Incremental Design of Simplex Basis Function Model for Dynamic System Identification

In this paper, we propose a novel adaptive piecewise linear model for dynamic system identification. It has four unique features. First, the model designs a new kind of basis function for function approximation. It maintains the uniform shape for each basis function, so as to achieve a satisfactory tradeoff between generalization ability and model complexity. Second, the model takes the structure of basis functions as decision variables to optimize the formulated identification problems instead of taking expansion coefficients as decision variables as proposed by many existing approaches. Third, we establish an incremental design strategy to solve the system identification problems. In each step of the identification, the selection of optimal basis function is a Lipschitz continuous optimization problem that is likely to be easily handled with some mature toolboxes. This incremental design strategy greatly reduces the estimation cost. Fourth, we introduce a smoothing mechanism to avoid overfitting, when the output of dynamic systems is disturbed by noise. Tests on several benchmark dynamic systems demonstrate the potential of the proposed model.

Identification of hinging hyperplane autoregressive exogenous model using efficient mixed-integer programming

A computationally efficient algorithm for hinging hyperplane autoregressive exogenous (HHARX) model identification via mixed-integer programming technique is proposed in this paper. The HHARX model is attractive since it accurately approximates a general nonlinear process as a sum of hinge functions and preserves the continuity even in a piecewise affine form. Traditional mixed-integer programming-based method for HHARX model identification can only be applied on small-scale input/output datasets due to its significant computational demands. The contribution of this paper is to develop a sequential optimization approach to build accurate HHARX model more efficiently on a relatively large number of experimental data. Moreover, the proposed framework can handle more difficult and practical cases in piecewise model identification, such as: limited submodel switching, missing output data and specified steady state. Finally, the efficiency and accuracy of the proposed computational scheme are demonstrated through modeling of two simulated examples and a pilot-scale heat exchanger.

Optimal Piecewise Linear Function Approximation for GPU-Based Applications

Many computer vision and human-computer interaction applications developed in recent years need evaluating complex and continuous mathematical functions as an essential step toward proper operation. However, rigorous evaluation of these kind of functions often implies a very high computational cost, unacceptable in real-time applications. To alleviate this problem, functions are commonly approximated by simpler piecewise-polynomial representations. Following this idea, we propose a novel, efficient, and practical technique to evaluate complex and continuous functions using a nearly optimal design of two types of piecewise linear approximations in the case of a large budget of evaluation subintervals. To this end, we develop a thorough error analysis that yields asymptotically tight bounds to accurately quantify the approximation performance of both representations. It provides an improvement upon previous error estimates and allows the user to control the tradeoff between the approximation error and the number of evaluation subintervals. To guarantee real-time operation, the method is suitable for, but not limited to, an efficient implementation in modern graphics processing units, where it outperforms previous alternative approaches by exploiting the fixed-function interpolation routines present in their texture units. The proposed technique is a perfect match for any application requiring the evaluation of continuous functions; we have measured in detail its quality and efficiency on several functions, and, in particular, the Gaussian function because it is extensively used in many areas of computer vision and cybernetics, and it is expensive to evaluate.

Transforming the canonical piecewise-linear model into a smooth-piecewise representation

A smoothed representation (based on natural exponential and logarithmic functions) for the canonical piecewise-linear model, is presented. The result is a completely differentiable formulation that exhibits interesting properties, like preserving the parameters of the original piecewise-linear model in such a way that they can be directly inherited to the smooth model in order to determine their parameters, the capability of controlling not only the smoothness grade, but also the approximation accuracy at specific breakpoint locations, a lower or equal overshooting for high order derivatives in comparison with other approaches, and the additional advantage of being expressed in a reduced mathematical form with only two types of inverse functions (logarithmic and exponential). By numerical simulation examples, this proposal is verified and well-illustrated.

A novel high level canonical piecewise linear model based on the simplicial partition and its application

The piecewise linear (PWL) model has attracted more and more attention in recent research because it can handle complex nonlinearity while maintaining linearity in local regions. A large number of compact representations for PWL modeling have been introduced, such as hinging hyperplanes and its generalized version. However, the existing methods usually give rise to many and complex subregions, which is an issue known as "curse of partitions", and hampered practical applications of PWL models. In this paper, a novel high level canonical PWL model is presented to tackle the curse of partitions. In more detail, an improved simplicial partition strategy with alterable intervals is proposed to improve the model representation capability. The proposed PWL model guarantees an unchangeable topology during training and thus a limited number of subregions after training. Several numerical experiments, and a simulated chemical process, are used to demonstrate the effectiveness of the proposed model.

Hinging hyperplanes for time-series segmentation

Division of a time series into segments is a common technique for time-series processing, and is known as segmentation. Segmentation is traditionally done by linear interpolation in order to guarantee the continuity of the reconstructed time series. The interpolation-based segmentation methods may perform poorly for data with a level of noise because interpolation is noise sensitive. To handle the problem, this paper establishes an explicit expression for segmentation from a compact representation for piecewise linear functions using hinging hyperplanes. This expression enables the use of regression to obtain a continuous reconstructed signal and, as a consequence, application of advanced techniques in segmentation. In this paper, a least squares support vector machine with lasso using a hinging feature map is given and analyzed, based on which a segmentation algorithm and its online version are established. Numerical experiments conducted on synthetic and real-world datasets demonstrate the advantages of our methods compared to existing segmentation algorithms.

The Hindmarsh-Rose neuron model: bifurcation analysis and piecewise-linear approximations

This paper provides a global picture of the bifurcation scenario of the Hindmarsh-Rose model. A combination between simulations and numerical continuations is used to unfold the complex bifurcation structure. The bifurcation analysis is carried out by varying two bifurcation parameters and evidence is given that the structure that is found is universal and appears for all combinations of bifurcation parameters. The information about the organizing principles and bifurcation diagrams are then used to compare the dynamics of the model with that of a piecewise-linear approximation, customized for circuit implementation. A good match between the dynamical behaviors of the models is found. These results can be used both to design a circuit implementation of the Hindmarsh-Rose model mimicking the diversity of neural response and as guidelines to predict the behavior of the model as well as its circuit implementation as a function of parameters.

Configuration of continuous piecewise-linear neural networks

The problem of constructing a general continuous piecewise-linear neural network is considered in this paper. It is shown that every projection domain of an arbitrary continuous piecewise-linear function can be partitioned into convex polyhedra by using difference functions of its local linear functions. Based on these convex polyhedra, a group of continuous piecewise-linear basis functions are formulated. It is proven that a linear combination of these basis functions plus a constant, which we call a standard continuous piecewise-linear neural network, can represent all continuous piecewise-linear functions. In addition, the proposed standard continuous piecewise-linear neural network is applied to solve some function approximation problems. A number of numerical experiments are presented to illustrate that the standard continuous piecewise-linear neural network can be a promising tool for function approximation.

The simplicial neural cell and its mixed-signal circuit implementation: an efficient neural-network architecture for intelligent signal processing in portable multimedia applications

This paper introduces a novel neural architecture which is capable of similar performance to any of the "classic" neural paradigms while having a very simple and efficient mixed-signal implementation which makes it a valuable candidate for intelligent signal processing in portable multimedia applications. The architecture and its realization circuit are described and the functional capabilities of the novel neural architecture called a simplicial neural cell are demonstrated for both regression and classification problems including nonlinear image filtering.