A two-stage safe screening method for non-convex support vector machine with ramp loss

Safe and accelerated screening framework for support tensor machines

Support Tensor Machines (STMs) constitute an effective supervised learning method for classifying high-dimensional tensor data. However, traditional iterative solving methods are often time-consuming. To effectively address the issue of lengthy training times, inspired by the safe screening strategies employed in support vector machines, we generalize the safe screening rule to the tensor domain and propose a novel safe screening rule for STM, which includes the dual static screening rule (DSSR), the dynamic screening rule (DGSR), and a subsequent checking verification. The screening rule initially employs variational inequalities to screen out a portion of redundant samples before training, reducing the problem scale. During the training process, the rule further accelerates training by iteratively screening redundant samples using the duality gap. We also design a subsequent checking technique based on optimality conditions to guarantee the safety of the screening rule. Building on this, we also develop a flexible safe screening framework, referred to as DS-DGSR, which incorporates the DSSR and the DGSR. It not only tackles the challenges of combining various tensor decomposition methods and the diverse scenarios of the decomposed coefficient parameter and decomposed samples in STMs, but also offers flexible adaptation and application according to the characteristics of different STMs. Numerical experiments on multiple real-world high-dimensional tensor datasets confirm the effectiveness and feasibility of DS-DGSR.

Sequential safe static and dynamic screening rule for accelerating support tensor machine

Support tensor machine (STM), as a higher-order extension of support vector machine, is adept at effectively addressing tensorial data classification problems, which maintains the inherent structure in tensors and mitigates the curse of dimensionality. However, it needs to resort to the alternating projection iterative technique, which is very time-consuming. To overcome this shortcoming, we propose an efficient sequential safe static and dynamic screening rule (SS-SDSR) for accelerating STM in this paper. Its main idea is to reduce every projection iterative sub-model by identifying and deleting the redundant variables before and during the training process without sacrificing accuracy. Its construction mainly consists of two parts: (1) The static screening rule and dynamic screening rule are first built based on the variational inequality and duality gap, respectively. (2) The sequential screening process is achieved by using the static screening rule with the different adjacent parameters and applying the dynamic screening rule under the same parameter. In the experiment, on the one hand, to verify the influence of different parameter intervals, screening frequencies, and forms of data on the effectiveness of our method, three experiments on artificial datasets are conducted, which indicate that our method is effective for any forms of data when the parameter interval is small and the screening frequency is appropriate. On the other hand, to demonstrate the feasibility and validity of our SS-SDSR, numerical experiments on eleven vector-based datasets, and six tensor-based datasets are conducted and compared with the other five algorithms. Experimental results illustrate the effectiveness and safety of our SS-SDSR.