RISE-Editing: Rotation-invariant neural point fields with interactive segmentation for fine-grained and efficient editing

Neural Radiance Fields (NeRF) have shown great potential for synthesizing novel views. Currently, despite the existence of some initial controllable and editable NeRF methods, they remain limited in terms of efficient and fine-grained editing capabilities, hinders the creative editing abilities and potential applications for NeRF. In this paper, we present the rotation-invariant neural point fields with interactive segmentation for fine-grained and efficient editing. Editing the implicit field presents a significant challenge, as varying the orientation of the corresponding explicit scaffold-whether point, mesh, volume, or other representations-may lead to a notable decline in rendering quality. By leveraging the complementary strengths of implicit NeRF-based representations and explicit point-based representations, we introduce a novel rotation-invariant neural point field representation. This representation enables the learning of local contents using Cartesian coordinates, leading to significant improvements in scene rendering quality after fine-grained editing. To achieve this rotation-invariant representation, we carefully design a Rotation-Invariant Neural Inverse Distance Weighting Interpolation (RNIDWI) module to aggregate the neural points. To enable more efficient and flexible cross-scene compositing, we disentangle the traditional NeRF representation into two components: a scene-agnostic rendering module and the scene-specific neural point fields. Furthermore, we present a multi-view ensemble learning strategy to lift the 2D inconsistent zero-shot segmentation results to 3D neural points field in real-time without post retraining. With simple click-based prompts on 2D images, user can efficiently segment the 3D neural point field and manipulate the corresponding neural points, enabling fine-grained editing of the implicit fields. Extensive experimental results demonstrate that our method offers enhanced editing capabilities and simplified editing process for users, delivers photorealistic rendering quality for novel views, and surpasses related methods in terms of the space-time efficiency and the types of editing functions they can achieve. The code is available at https://github.com/yuzewang1998/RISE-Editing.

End-to-End Ensemble Learning by Exploiting the Correlation Between Individuals and Weights

Ensemble learning performs better than a single classifier in most tasks due to the diversity among multiple classifiers. However, the enhancement of the diversity is at the expense of reducing the accuracies of individual classifiers in general and, thus, how to balance the diversity and accuracies is crucial for improving the ensemble performance. In this paper, we propose a new ensemble method which exploits the correlation between individual classifiers and their corresponding weights by constructing a joint optimization model to achieve the tradeoff between the diversity and the accuracy. Specifically, the proposed framework can be modeled as a shallow network and efficiently trained by the end-to-end manner. In the proposed ensemble method, not only can a high total classification performance be achieved by the weighted classifiers but also the individual classifier can be updated based on the error of the optimized weighted classifiers ensemble. Furthermore, the sparsity constraint is imposed on the weight to enforce that partial individual classifiers are selected for final classification. Finally, the experimental results on the UCI datasets demonstrate that the proposed method effectively improves the performance of classification compared with relevant existing ensemble methods.

Hybrid Incremental Ensemble Learning for Noisy Real-World Data Classification

Traditional ensemble learning approaches explore the feature space and the sample space, respectively, which will prevent them to construct more powerful learning models for noisy real-world dataset classification. The random subspace method only search for the selection of features. Meanwhile, the bagging approach only search for the selection of samples. To overcome these limitations, we propose the hybrid incremental ensemble learning (HIEL) approach which takes into consideration the feature space and the sample space simultaneously to handle noisy dataset. Specifically, HIEL first adopts the bagging technique and linear discriminant analysis to remove noisy attributes, and generates a set of bootstraps and the corresponding ensemble members in the subspaces. Then, the classifiers are selected incrementally based on a classifier-specific criterion function and an ensemble criterion function. The corresponding weights for the classifiers are assigned during the same process. Finally, the final label is summarized by a weighted voting scheme, which serves as the final result of the classification. We also explore various classifier-specific criterion functions based on different newly proposed similarity measures, which will alleviate the effect of noisy samples on the distance functions. In addition, the computational cost of HIEL is analyzed theoretically. A set of nonparametric tests are adopted to compare HIEL and other algorithms over several datasets. The experiment results show that HIEL performs well on the noisy datasets. HIEL outperforms most of the compared classifier ensemble methods on 14 out of 24 noisy real-world UCI and KEEL datasets.

Hybrid adaptive classifier ensemble

Traditional random subspace-based classifier ensemble approaches (RSCE) have several limitations, such as viewing the same importance for the base classifiers trained in different subspaces, not considering how to find the optimal random subspace set. In this paper, we design a general hybrid adaptive ensemble learning framework (HAEL), and apply it to address the limitations of RSCE. As compared with RSCE, HAEL consists of two adaptive processes, i.e., base classifier competition and classifier ensemble interaction, so as to adjust the weights of the base classifiers in each ensemble and to explore the optimal random subspace set simultaneously. The experiments on the real-world datasets from the KEEL dataset repository for the classification task and the cancer gene expression profiles show that: 1) HAEL works well on both the real-world KEEL datasets and the cancer gene expression profiles and 2) it outperforms most of the state-of-the-art classifier ensemble approaches on 28 out of 36 KEEL datasets and 6 out of 6 cancer datasets.

Selective Ensemble Based on Transformation of Classifiers Used SPCA

The diversity and the accuracy are two important ingredients for ensemble generalization error in an ensemble classifiers system. Nevertheless enhancing the diversity is at the expense of decreasing the accuracy of classifiers, thus balancing the diversity and the accuracy is crucial for constructing a good ensemble method. In the paper, a new ensemble method is proposed that selecting classifiers to ensemble via the transformation of individual classifiers based on diversity and accuracy. In the proposed method, the transformation of classifiers is made to produce new individual classifiers based on original classifiers and the true labels, in order to enhance diversity of an ensemble. The transformation approach is similar to principal component analysis (PCA), but it is essentially different between them that the proposed method employs the true labels to construct the covariance matrix rather than the mean of samples in PCA. Then a selecting rule is constructed based on two rules of measuring the classification performance. By the selecting rule, some available new classifiers are selected to ensemble in order to ensure the accuracy of the ensemble with selected classifiers. In other words, some individuals with poor or same performance are eliminated. Particularly, a new classifier produced by the transformation is equivalent to a linear combination of original classifiers, which indicates that the proposed method enhances the diversity by different transformations instead of constructing different training subsets. The experimental results illustrate that the proposed method obtains the better performance than other methods, and the kappa-error diagrams also illustrate that the proposed method enhances the diversity compared against other methods.

A scalable stagewise approach to large-margin multiclass loss-based boosting

We present a scalable and effective classification model to train multiclass boosting for multiclass classification problems. A direct formulation of multiclass boosting had been introduced in the past in the sense that it directly maximized the multiclass margin. The major problem of that approach is its high computational complexity during training, which hampers its application to real-world problems. In this paper, we propose a scalable and simple stagewise multiclass boosting method which also directly maximizes the multiclass margin. Our approach offers the following advantages: 1) it is simple and computationally efficient to train. The approach can speed up the training time by more than two orders of magnitude without sacrificing the classification accuracy and 2) like traditional AdaBoost, it is less sensitive to the choice of parameters and empirically demonstrates excellent generalization performance. Experimental results on challenging multiclass machine learning and vision tasks demonstrate that the proposed approach substantially improves the convergence rate and accuracy of the final visual detector at no additional computational cost compared to existing multiclass boosting.

RandomBoost: simplified multiclass boosting through randomization

We propose a novel boosting approach to multiclass classification problems, in which multiple classes are distinguished by a set of random projection matrices in essence. The approach uses random projections to alleviate the proliferation of binary classifiers typically required to perform multiclass classification. The result is a multiclass classifier with a single vector-valued parameter, irrespective of the number of classes involved. Two variants of this approach are proposed. The first method randomly projects the original data into new spaces, while the second method randomly projects the outputs of learned weak classifiers. These methods are not only conceptually simple but also effective and easy to implement. A series of experiments on synthetic, machine learning, and visual recognition data sets demonstrate that our proposed methods could be compared favorably with existing multiclass boosting algorithms in terms of both the convergence rate and classification accuracy.

RELATIONSHIP BETWEEN DATA SIZE, ACCURACY, DIVERSITY AND CLUSTERS IN NEURAL NETWORK ENSEMBLES

This paper presents an approach for analyzing relationships between data size, cluster, accuracy and diversity in neural network ensembles. The main objective of this research is to find out the influence of data size such as number of patterns, number of inputs and number of classes on various parameters such as clusters, accuracy and diversity of a neural network ensemble. The proposed approach is based on splitting data sets into different groups using the data size, clustering data and conducting training and testing of neural network ensembles. The test data is same for all groups and used to test all trained ensembles. The experiments have been conducted on 15 UCI machine learning benchmark datasets and results are presented in this paper.

Fully corrective boosting with arbitrary loss and regularization

We propose a general framework for analyzing and developing fully corrective boosting-based classifiers. The framework accepts any convex objective function, and allows any convex (for example, ℓp-norm, p ≥ 1) regularization term. By placing the wide variety of existing fully corrective boosting-based classifiers on a common footing, and considering the primal and dual problems together, the framework allows a direct comparison between apparently disparate methods. By solving the primal rather than the dual the framework is capable of generating efficient fully-corrective boosting algorithms without recourse to sophisticated convex optimization processes. We show that a range of additional boosting-based algorithms can be incorporated into the framework despite not being fully corrective. Finally, we provide an empirical analysis of the performance of a variety of the most significant boosting-based classifiers on a few machine learning benchmark datasets.

A survey of commonly used ensemble-based classification techniques

The combination of multiple classifiers, commonly referred to as a classifier ensemble, has previously demonstrated the ability to improve classification accuracy in many application domains. As a result this area has attracted significant amount of research in recent years. The aim of this paper has therefore been to provide a state of the art review of the most well-known ensemble techniques with the main focus on bagging, boosting and stacking and to trace the recent attempts, which have been made to improve their performance. Within this paper, we present and compare an updated view on the different modifications of these techniques, which have specifically aimed to address some of the drawbacks of these methods namely the low diversity problem in bagging or the over-fitting problem in boosting. In addition, we provide a review of different ensemble selection methods based on both static and dynamic approaches. We present some new directions which have been adopted in the area of classifier ensembles from a range of recently published studies. In order to provide a deeper insight into the ensembles themselves a range of existing theoretical studies have been reviewed in the paper.

OligoIS: Scalable Instance Selection for Class-Imbalanced Data Sets

In current research, an enormous amount of information is constantly being produced, which poses a challenge for data mining algorithms. Many of the problems in extremely active research areas, such as bioinformatics, security and intrusion detection, or text mining, share the following two features: large data sets and class-imbalanced distribution of samples. Although many methods have been proposed for dealing with class-imbalanced data sets, most of these methods are not scalable to the very large data sets common to those research fields. In this paper, we propose a new approach to dealing with the class-imbalance problem that is scalable to data sets with many millions of instances and hundreds of features. This proposal is based on the divide-and-conquer principle combined with application of the selection process to balanced subsets of the whole data set. This divide-and-conquer principle allows the execution of the algorithm in linear time. Furthermore, the proposed method is easy to implement using a parallel environment and can work without loading the whole data set into memory. Using 40 class-imbalanced medium-sized data sets, we will demonstrate our method's ability to improve the results of state-of-the-art instance selection methods for class-imbalanced data sets. Using three very large data sets, we will show the scalability of our proposal to millions of instances and hundreds of features.

SSC: a classifier combination method based on signal strength

We propose a new classifier combination method, the signal strength-based combining (SSC) approach, to combine the outputs of multiple classifiers to support the decision-making process in classification tasks. As ensemble learning methods have attracted growing attention from both academia and industry recently, it is critical to understand the fundamental issues of the combining rule. Motivated by the signal strength concept, our proposed SSC algorithm can effectively integrate the individual vote from different classifiers in an ensemble learning system. Comparative studies of our method with nine major existing combining rules, namely, geometric average rule, arithmetic average rule, median value rule, majority voting rule, Borda count, max and min rule, weighted average, and weighted majority voting rules, is presented. Furthermore, we also discuss the relationship of the proposed method with respect to margin-based classifiers, including the boosting method (AdaBoost.M1 and AdaBoost.M2) and support vector machines by margin analysis. Detailed analyses of margin distribution graphs are presented to discuss the characteristics of the proposed method. Simulation results for various real-world datasets illustrate the effectiveness of the proposed method.

Prototype selection for nearest neighbor classification: taxonomy and empirical study

The nearest neighbor classifier is one of the most used and well-known techniques for performing recognition tasks. It has also demonstrated itself to be one of the most useful algorithms in data mining in spite of its simplicity. However, the nearest neighbor classifier suffers from several drawbacks such as high storage requirements, low efficiency in classification response, and low noise tolerance. These weaknesses have been the subject of study for many researchers and many solutions have been proposed. Among them, one of the most promising solutions consists of reducing the data used for establishing a classification rule (training data) by means of selecting relevant prototypes. Many prototype selection methods exist in the literature and the research in this area is still advancing. Different properties could be observed in the definition of them, but no formal categorization has been established yet. This paper provides a survey of the prototype selection methods proposed in the literature from a theoretical and empirical point of view. Considering a theoretical point of view, we propose a taxonomy based on the main characteristics presented in prototype selection and we analyze their advantages and drawbacks. Empirically, we conduct an experimental study involving different sizes of data sets for measuring their performance in terms of accuracy, reduction capabilities, and runtime. The results obtained by all the methods studied have been verified by nonparametric statistical tests. Several remarks, guidelines, and recommendations are made for the use of prototype selection for nearest neighbor classification.

Learning active fusion of multiple experts' decisions: an attention-based approach

In this letter, we propose a learning system, active decision fusion learning (ADFL), for active fusion of decisions. Each decision maker, referred to as a local decision maker, provides its suggestion in the form of a probability distribution over all possible decisions. The goal of the system is to learn the active sequential selection of the local decision makers in order to consult with and thus learn the final decision based on the consultations. These two learning tasks are formulated as learning a single sequential decision-making problem in the form of a Markov decision process (MDP), and a continuous reinforcement learning method is employed to solve it. The states of this MDP are decisions of the attended local decision makers, and the actions are either attending to a local decision maker or declaring final decisions. The learning system is punished for each consultation and wrong final decision and rewarded for correct final decisions. This results in minimizing the consultation and decision-making costs through learning a sequential consultation policy where the most informative local decision makers are consulted and the least informative, misleading, and redundant ones are left unattended. An important property of this policy is that it acts locally. This means that the system handles any nonuniformity in the local decision maker's expertise over the state space. This property has been exploited in the design of local experts. ADFL is tested on a set of classification tasks, where it outperforms two well-known classification methods, Adaboost and bagging, as well as three benchmark fusion algorithms: OWA, Borda count, and majority voting. In addition, the effect of local experts design strategy on the performance of ADFL is studied, and some guidelines for the design of local experts are provided. Moreover, evaluating ADFL in some special cases proves that it is able to derive the maximum benefit from the informative local decision makers and to minimize attending to redundant ones.

RAMOBoost: Ranked Minority Oversampling in Boosting

In recent years, learning from imbalanced data has attracted growing attention from both academia and industry due to the explosive growth of applications that use and produce imbalanced data. However, because of the complex characteristics of imbalanced data, many real-world solutions struggle to provide robust efficiency in learning-based applications. In an effort to address this problem, this paper presents Ranked Minority Oversampling in Boosting (RAMOBoost), which is a RAMO technique based on the idea of adaptive synthetic data generation in an ensemble learning system. Briefly, RAMOBoost adaptively ranks minority class instances at each learning iteration according to a sampling probability distribution that is based on the underlying data distribution, and can adaptively shift the decision boundary toward difficult-to-learn minority and majority class instances by using a hypothesis assessment procedure. Simulation analysis on 19 real-world datasets assessed over various metrics-including overall accuracy, precision, recall, F-measure, G-mean, and receiver operation characteristic analysis-is used to illustrate the effectiveness of this method.