1
|
Chen Y, Yu M. Randomized phase II selection design with order constrained strata. Biometrics 2024; 80:ujad013. [PMID: 38364810 PMCID: PMC10873566 DOI: 10.1093/biomtc/ujad013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 09/06/2023] [Accepted: 11/20/2023] [Indexed: 02/18/2024]
Abstract
The exploratory nature of phase II trials makes it quite common to include heterogeneous patient subgroups with different prognoses in the same trial. Incorporating such patient heterogeneity or stratification into statistical calculation for sample size can improve efficiency and reduce sample sizes in single-arm phase II trials with binary outcomes. However, such consideration is lacking in randomized phase II trials. In this paper, we propose methods that can utilize some natural order constraints that may exist in stratified population to gain statistical efficiency for randomized phase II designs. For thoroughness and simplicity, we focus on the randomized phase II selection designs in this paper, although our method can be easily generalized to the randomized phase II screening designs. We consider both binary and time-to-event outcomes in our development. Compared with methods that do not use order constraints, our method is shown to improve the probabilities of correct selection or reduce sample size in our simulation and real examples.
Collapse
Affiliation(s)
- Yi Chen
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI 53726, United States
| | - Menggang Yu
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI 53726, United States
| |
Collapse
|
2
|
Wüthrich MV. Model selection with Gini indices under auto-calibration. EUROPEAN ACTUARIAL JOURNAL 2023; 13:469-477. [DOI: 10.1007/s13385-022-00339-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/28/2022] [Revised: 10/07/2022] [Accepted: 12/08/2022] [Indexed: 09/02/2023]
|
3
|
Heng Q, Zhou H, Chi EC. Bayesian Trend Filtering via Proximal Markov Chain Monte Carlo. J Comput Graph Stat 2023; 32:938-949. [PMID: 37822489 PMCID: PMC10564381 DOI: 10.1080/10618600.2023.2170089] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Accepted: 01/09/2023] [Indexed: 01/21/2023]
Abstract
Proximal Markov Chain Monte Carlo is a novel construct that lies at the intersection of Bayesian computation and convex optimization, which helped popularize the use of nondifferentiable priors in Bayesian statistics. Existing formulations of proximal MCMC, however, require hyperparameters and regularization parameters to be prespecified. In this work, we extend the paradigm of proximal MCMC through introducing a novel new class of nondifferentiable priors called epigraph priors. As a proof of concept, we place trend filtering, which was originally a nonparametric regression problem, in a parametric setting to provide a posterior median fit along with credible intervals as measures of uncertainty. The key idea is to replace the nonsmooth term in the posterior density with its Moreau-Yosida envelope, which enables the application of the gradient-based MCMC sampler Hamiltonian Monte Carlo. The proposed method identifies the appropriate amount of smoothing in a data-driven way, thereby automating regularization parameter selection. Compared with conventional proximal MCMC methods, our method is mostly tuning free, achieving simultaneous calibration of the mean, scale and regularization parameters in a fully Bayesian framework.
Collapse
Affiliation(s)
- Qiang Heng
- Department of Statistics, North Carolina State University
| | - Hua Zhou
- Departments of Biostatistics and Computational Medicine, UCLA
| | | |
Collapse
|
4
|
Haider C, de Franca F, Burlacu B, Kronberger G. Shape-constrained multi-objective genetic programming for symbolic regression. Appl Soft Comput 2022. [DOI: 10.1016/j.asoc.2022.109855] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
5
|
Kronberger G, de Franca FO, Burlacu B, Haider C, Kommenda M. Shape-Constrained Symbolic Regression-Improving Extrapolation with Prior Knowledge. EVOLUTIONARY COMPUTATION 2022; 30:75-98. [PMID: 34623432 DOI: 10.1162/evco_a_00294] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/16/2019] [Accepted: 03/26/2021] [Indexed: 06/13/2023]
Abstract
We investigate the addition of constraints on the function image and its derivatives for the incorporation of prior knowledge in symbolic regression. The approach is called shape-constrained symbolic regression and allows us to enforce, for example, monotonicity of the function over selected inputs. The aim is to find models which conform to expected behavior and which have improved extrapolation capabilities. We demonstrate the feasibility of the idea and propose and compare two evolutionary algorithms for shape-constrained symbolic regression: (i) an extension of tree-based genetic programming which discards infeasible solutions in the selection step, and (ii) a two-population evolutionary algorithm that separates the feasible from the infeasible solutions. In both algorithms we use interval arithmetic to approximate bounds for models and their partial derivatives. The algorithms are tested on a set of 19 synthetic and four real-world regression problems. Both algorithms are able to identify models which conform to shape constraints which is not the case for the unmodified symbolic regression algorithms. However, the predictive accuracy of models with constraints is worse on the training set and the test set. Shape-constrained polynomial regression produces the best results for the test set but also significantly larger models.
Collapse
Affiliation(s)
- G Kronberger
- Josef Ressel Center for Symbolic Regression, University of Applied Sciences Upper Austria, Softwarepark 11, 4232 Hagenberg, Austria
| | - F O de Franca
- Center for Mathematics, Computation and Cognition (CMCC), Heuristics, Analysis and Learning Laboratory (HAL), Federal University of ABC, Santo Andre, Brazil
| | - B Burlacu
- Josef Ressel Center for Symbolic Regression, University of Applied Sciences Upper Austria, Softwarepark 11, 4232 Hagenberg, Austria
| | - C Haider
- Josef Ressel Center for Symbolic Regression, University of Applied Sciences Upper Austria, Softwarepark 11, 4232 Hagenberg, Austria
| | - M Kommenda
- Josef Ressel Center for Symbolic Regression, University of Applied Sciences Upper Austria, Softwarepark 11, 4232 Hagenberg, Austria
| |
Collapse
|
6
|
Average
OLS‐Centered
Penalized Regression: A More Efficient Way to Address Multicollinearity Than Ridge Regression. STAT NEERL 2022. [DOI: 10.1111/stan.12263] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
7
|
Pastukhov V. Stacked Grenander and rearrangement estimators of a discrete distribution. Electron J Stat 2022. [DOI: 10.1214/22-ejs2045] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
Affiliation(s)
- Vladimir Pastukhov
- Department of Computer Science and Engineering, Chalmers University of Technology
| |
Collapse
|
8
|
Cui Z, Lee C, Zhu L, Zhu Y. Non-convex isotonic regression via the Myersonian approach. Stat Probab Lett 2021. [DOI: 10.1016/j.spl.2021.109210] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
9
|
Genç M. A new double-regularized regression using Liu and lasso regularization. Comput Stat 2021. [DOI: 10.1007/s00180-021-01120-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
10
|
Affiliation(s)
- Kevin Guo
- Stanford University, Statistics, Stanford, 94305-6104 United States
| | - Guillaume Basse
- Stanford University, Statistics, Stanford, 94305-6104 United States
| |
Collapse
|
11
|
Wu X, Liang R, Yang H. Penalized and constrained LAD estimation in fixed and high dimension. Stat Pap (Berl) 2021; 63:53-95. [PMID: 33814727 PMCID: PMC8009762 DOI: 10.1007/s00362-021-01229-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2020] [Revised: 02/27/2021] [Indexed: 11/26/2022]
Abstract
Recently, many literatures have proved that prior information and structure in many application fields can be formulated as constraints on regression coefficients. Following these work, we propose a L 1 penalized LAD estimation with some linear constraints in this paper. Different from constrained lasso, our estimation performs well when heavy-tailed errors or outliers are found in the response. In theory, we show that the proposed estimation enjoys the Oracle property with adjusted normal variance when the dimension of the estimated coefficients p is fixed. And when p is much greater than the sample size n, the error bound of proposed estimation is sharper thank log ( p ) / n . It is worth noting the result is true for a wide range of noise distribution, even for the Cauchy distribution. In algorithm, we not only consider an typical linear programming to solve proposed estimation in fixed dimension , but also present an nested alternating direction method of multipliers (ADMM) in high dimension. Simulation and application to real data also confirm that proposed estimation is an effective alternative when constrained lasso is unreliable.
Collapse
Affiliation(s)
- Xiaofei Wu
- College of Mathematics and Statistics, Chongqing University, Chongqing, 401331 People’s Republic of China
| | - Rongmei Liang
- College of Mathematics and Statistics, Chongqing University, Chongqing, 401331 People’s Republic of China
| | - Hu Yang
- College of Mathematics and Statistics, Chongqing University, Chongqing, 401331 People’s Republic of China
| |
Collapse
|
12
|
Minami K. Degrees of freedom in submodular regularization: A computational perspective of Stein’s unbiased risk estimate. J MULTIVARIATE ANAL 2020. [DOI: 10.1016/j.jmva.2019.104546] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
13
|
|
14
|
Gaines BR, Kim J, Zhou H. Algorithms for Fitting the Constrained Lasso. J Comput Graph Stat 2018; 27:861-871. [PMID: 30618485 PMCID: PMC6320228 DOI: 10.1080/10618600.2018.1473777] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2016] [Revised: 03/01/2018] [Indexed: 01/22/2023]
Abstract
We compare alternative computing strategies for solving the constrained lasso problem. As its name suggests, the constrained lasso extends the widely-used lasso to handle linear constraints, which allow the user to incorporate prior information into the model. In addition to quadratic programming, we employ the alternating direction method of multipliers (ADMM) and also derive an efficient solution path algorithm. Through both simulations and benchmark data examples, we compare the different algorithms and provide practical recommendations in terms of efficiency and accuracy for various sizes of data. We also show that, for an arbitrary penalty matrix, the generalized lasso can be transformed to a constrained lasso, while the converse is not true. Thus, our methods can also be used for estimating a generalized lasso, which has wide-ranging applications. Code for implementing the algorithms is freely available in both the Matlab toolbox SparseReg and the Julia package ConstrainedLasso. Supplementary materials for this article are available online.
Collapse
Affiliation(s)
- Brian R Gaines
- Department of Statistics, North Carolina State University
| | - Juhyun Kim
- Department of Biostatistics, University of California, Los Angeles (UCLA)
| | - Hua Zhou
- Department of Biostatistics, University of California, Los Angeles (UCLA)
| |
Collapse
|
15
|
Fiori S, Fioranelli N. Smooth statistical modeling of bivariate non-monotonic data by a three-stage LUT neural system. Neural Comput Appl 2018. [DOI: 10.1007/s00521-017-3215-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
16
|
|
17
|
Abstract
In this paper we present a new non-parametric calibration method called ensemble of near isotonic regression (ENIR). The method can be considered as an extension of BBQ (Pakdaman Naeini, Cooper and Hauskrecht, 2015b), a recently proposed calibration method, as well as the commonly used calibration method based on isotonic regression (IsoRegC) (Zadrozny and Elkan, 2002). ENIR is designed to address the key limitation of IsoRegC which is the monotonicity assumption of the predictions. Similar to BBQ, the method post-processes the output of a binary classifier to obtain calibrated probabilities. Thus it can be used with many existing classification models to generate accurate probabilistic predictions. We demonstrate the performance of ENIR on synthetic and real datasets for commonly applied binary classification models. Experimental results show that the method outperforms several common binary classifier calibration methods. In particular, on the real data we evaluated, ENIR commonly performs statistically significantly better than the other methods, and never worse. It is able to improve the calibration power of classifiers, while retaining their discrimination power. The method is also computationally tractable for large scale datasets, as it is O(N logN) time, where N is the number of samples.
Collapse
|
18
|
Xing EP. Semantic Pooling for Complex Event Analysis in Untrimmed Videos. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2017; 39:1617-1632. [PMID: 28113653 PMCID: PMC5570670 DOI: 10.1109/tpami.2016.2608901] [Citation(s) in RCA: 71] [Impact Index Per Article: 10.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/15/2023]
Abstract
Pooling plays an important role in generating a discriminative video representation. In this paper, we propose a new semantic pooling approach for challenging event analysis tasks (e.g., event detection, recognition, and recounting) in long untrimmed Internet videos, especially when only a few shots/segments are relevant to the event of interest while many other shots are irrelevant or even misleading. The commonly adopted pooling strategies aggregate the shots indifferently in one way or another, resulting in a great loss of information. Instead, in this work we first define a novel notion of semantic saliency that assesses the relevance of each shot with the event of interest. We then prioritize the shots according to their saliency scores since shots that are semantically more salient are expected to contribute more to the final event analysis. Next, we propose a new isotonic regularizer that is able to exploit the constructed semantic ordering information. The resulting nearly-isotonic support vector machine classifier exhibits higher discriminative power in event analysis tasks. Computationally, we develop an efficient implementation using the proximal gradient algorithm, and we prove new and closed-form proximal steps. We conduct extensive experiments on three real-world video datasets and achieve promising improvements.
Collapse
|
19
|
Naeini MP, Cooper GF. Binary Classifier Calibration using an Ensemble of Near Isotonic Regression Models. PROCEEDINGS. IEEE INTERNATIONAL CONFERENCE ON DATA MINING 2017; 2016:360-369. [PMID: 28316511 PMCID: PMC5351887 DOI: 10.1109/icdm.2016.0047] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Learning accurate probabilistic models from data is crucial in many practical tasks in data mining. In this paper we present a new non-parametric calibration method called ensemble of near isotonic regression (ENIR). The method can be considered as an extension of BBQ [20], a recently proposed calibration method, as well as the commonly used calibration method based on isotonic regression (IsoRegC) [27]. ENIR is designed to address the key limitation of IsoRegC which is the monotonicity assumption of the predictions. Similar to BBQ, the method post-processes the output of a binary classifier to obtain calibrated probabilities. Thus it can be used with many existing classification models to generate accurate probabilistic predictions. We demonstrate the performance of ENIR on synthetic and real datasets for commonly applied binary classification models. Experimental results show that the method outperforms several common binary classifier calibration methods. In particular on the real data, ENIR commonly performs statistically significantly better than the other methods, and never worse. It is able to improve the calibration power of classifiers, while retaining their discrimination power. The method is also computationally tractable for large scale datasets, as it is O(N log N) time, where N is the number of samples.
Collapse
Affiliation(s)
| | - Gregory F Cooper
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, USA
| |
Collapse
|
20
|
|
21
|
|
22
|
Abstract
We consider regression scenarios where it is natural to impose an order constraint on the coefficients. We propose an order-constrained version of ℓ 1-regularized regression (Lasso) for this problem, and show how to solve it efficiently using the well-known Pool Adjacent Violators Algorithm as its proximal operator. The main application of this idea is to time-lagged regression, where we predict an outcome at time t from features at the previous K time points. In this setting it is natural to assume that the coefficients decay as we move farther away from t, and hence the order constraint is reasonable. Potential application areas include financial time series and prediction of dynamic patient outcomes based on clinical measurements. We illustrate this idea on real and simulated data.
Collapse
Affiliation(s)
- Robert Tibshirani
- Department of Health Research, & Policy, and Statistics, Stanford University, Stanford, CA 94305
| | - Xiaotong Suo
- Institute for Computational & Mathematical Engineering, Stanford University, Huang Engineering Center, 475 Via Ortega, Suite 060, Stanford, CA 94305
| |
Collapse
|
23
|
|
24
|
Abstract
Many least-square problems involve affine equality and inequality constraints. Although there are a variety of methods for solving such problems, most statisticians find constrained estimation challenging. The current article proposes a new path-following algorithm for quadratic programming that replaces hard constraints by what are called exact penalties. Similar penalties arise in l1 regularization in model selection. In the regularization setting, penalties encapsulate prior knowledge, and penalized parameter estimates represent a trade-off between the observed data and the prior knowledge. Classical penalty methods of optimization, such as the quadratic penalty method, solve a sequence of unconstrained problems that put greater and greater stress on meeting the constraints. In the limit as the penalty constant tends to ∞, one recovers the constrained solution. In the exact penalty method, squared penalties!are replaced by absolute value penalties, and the solution is recovered for a finite value of the penalty constant. The exact path-following method starts at the unconstrained solution and follows the solution path as the penalty constant increases. In the process, the solution path hits, slides along, and exits from the various constraints. Path following in Lasso penalized regression, in contrast, starts with a large value of the penalty constant and works its way downward. In both settings, inspection of the entire solution path is revealing. Just as with the Lasso and generalized Lasso, it is possible to plot the effective degrees of freedom along the solution path. For a strictly convex quadratic program, the exact penalty algorithm can be framed entirely in terms of the sweep operator of regression analysis. A few well-chosen examples illustrate the mechanics and potential of path following. This article has supplementary materials available online.
Collapse
Affiliation(s)
- Hua Zhou
- Department of Statistics, North Carolina State University, Raleigh, NC 27695-8203 ( )
| | | |
Collapse
|
25
|
Fang Z, Meinshausen N. LASSO Isotone for High-Dimensional Additive Isotonic Regression. J Comput Graph Stat 2012. [DOI: 10.1198/jcgs.2011.10095] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
26
|
Tibshirani R. Regression shrinkage and selection via the lasso: a retrospective. J R Stat Soc Series B Stat Methodol 2011. [DOI: 10.1111/j.1467-9868.2011.00771.x] [Citation(s) in RCA: 1675] [Impact Index Per Article: 128.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
|