Research Group "Stochastic Algorithms and Nonparametric Statistics"

Jelena Bradic (Princeton)

Regularization for Cox's Proportional Hazards Models with NP Dimensionality

Abstract: High throughput genetic sequencing arrays with thousands of measurements per sample and a great amount of related censored clinical data have increased demanding need for better measurement specic model selection. In this paper we establish strong oracle properties of non-concave penalized methods for non-polynomial (NP) dimensional data with censoring in the framework of Cox's proportional hazards model. A class of folded-concave penalties are employed and both LASSO and SCAD are discussed specically. We unveil the question under which dimensionality and correlation restrictions can an oracle estimator be constructed and grasped. It is demonstrated that non-concave penalties lead to signicant reduction of the rrepresentable conditionneeded for LASSO model selection consistency. The large deviation result for martingales, bearing interests of its own, is developed for characterizing the strong oracle property. Moreover, the non-concave regularized estimator, is shown to achieve asymptotically the information bound of the oracle estimator. A coordinate-wise algorithm is developed for nding the grid of solution paths for penalized hazard regression problems, and its performance is evaluated on simulated and gene association study examples

Vladimir Koltchinskii (Georgia Institute of Technology, Atlanta, USA)

Von Neumann Entropy Penalization and Low Rank Matrix Estimation

Abstract:Let be a Hermitian nonnegatively denite m m matrix of unit trace (a density matrix of a quantum system) and let Xj be Hermitian m m matrices (observables). Suppose that, for the system prepared n times in the same state , the measurements of X1; :::;Xn have been performed and let Y1; :::; Yn be the outcomes of these measurements. A basic problem in quantum state tomography is to estimate the unknown density matrix based on the measurements (X1; Y1); :::; (Xn; Yn). Since E(Xj) = tr(Xj); j = 1; :::; n; this is a matrix regression problem that is especially interesting when the target density matrix is large, but it either has a small rank, or can be well approximated by low rank matrices. We will discuss an approach to this problem based on a penalized least squares method with complexity penalty dened in terms of von Neumann entropy. Under the assumption that the matrices X1; :::;Xn have been sampled at random from a probability distribution in the space of Hermitian matrices, we derived oracle inequalities showing the dependence of the estimation error on the accuracy of approximation of by low rank matrices.

Richard Song (UC Berkeley)

Large Vector Auto Regressions

Abstract:A challenging issue for nonstructural macroeconomic forecasting is to determine which varia- bles are relevant. One approach is to include a large number of economic and nancial variables, which has been shown to lead to signicant improvements for forecasting using the dynamic factor modeling method. The challenges in practice come from a mixture of serial correlation (time dynamics), high dimensional (space) dependence structure and moderate sample size (relative to dimensionality). To this end, an integrated solution that can address these three challenges simultaneously is appealing. A more direct approach \large vector auto regressions" as we study here, combines the two-step estimation procedure of dynamic factor modeling into one step, is also more suitable for variable-to-variable relationship studies (impulse response and interpretation), allows \individualized" endogeneity or exogeneity, and is able to select the variables and lags simultaneously. Three types of estimators are carefully studied. The tuning pa- rameters are chose via a data driven \rolling scheme" method to optimize the forecasting performance. We show that this method can produce the estimator as ecient as the oracle. A macroeconomic forecasting problem is considered to illustrate this method, which demonstrates its superior to the existing estimators.

Peter X. K. Song (University of Michigan)

Composite likelihood Bayesian information criterion for model selection in high dimensional correlated data

Abstract:Composite likelihood methodology has received an increasing attention recently in the literature as an effective way of handling high-dimensional correlated variables. Such data are often encountered in biomedical and social sciences studies; for example, spatio-temporal data, social networks and gene regulatory networks. Due to complicated dependency structures, the full likelihood approach often renders to intractable computational complexity. This imposes difficulty on model selection as most of the traditionally used information criteria require the evaluation of the full likelihood over a finite model space. We propose a composite likelihood version of the Bayesian information criterion (BIC) and establish its large sample property for the selection of the true underlying model. Under some mild regularity conditions, the proposed BIC is shown to be selection consistent, where the number of potential model parameters is allowed to increase to infinity at a certain rate of the sample size. Simulation studies demonstrate the empirical performance of this new BIC criterion, especially for the scenario that the number of parameters increases with the sample size. Also, a data analysis example is illustrated. Reference: Gao and Song (2010). Composite likelihood Bayesian information criteria for model selection in high dimensional data. JASA 105, 1531-1540.

Shota Gugushvili (Niederlande)

$\sqrt{n}$-consistent parameter estimation for systems of ordinary differential equations: bypassing numerical integration via smoothing

Abstract:We consider the problem of parameter estimation for a system of ordinary differential equations from noisy observations on a solution of the system. In case the system is nonlinear, as it typically is in practical applications, an analytic solution to it usually does not exist. Straightforward estimation methods like the ordinary least squares method consequently depend on repetitive use of numerical integration of the system in order to determine its solution for different parameter values, and to find the parameter estimate that minimises the objective function. Use of repetitive numerical integration induces huge computational load on such estimation methods. We study consistency of an alternative estimator that is defined as a minimiser of an appropriate distance between a nonparametrically estimated derivative of the solution and the right-hand side of the system applied to a nonparametrically estimated solution. This smooth and match estimator (SME) bypasses numerical integration of the system altogether and reduces the amount of computational time drastically compared to ordinary least squares. Moreover, we show that under suitable regularity conditions this smooth and match estimation procedure leads to a $\sqrt{n}$-consistent estimator of the parameter of interest. The talk is based on a joint work with Chris Klaassen.

tba

Abstract:

Catalina Stefanescu-Cuntze (ESMT)

Modeling the Loss Distribution

Abstract:In this paper we focus on modeling and predicting the loss distribution for credit risky assets such as bonds and loans. We model the probability of default and the recovery rate given default based on shared covariates. We develop a new class of default models that explicitly account for sector specic and regime dependent unobservable heterogeneity in rm characteristics. Based on the analysis of a large default and recovery data set over the horizon 1980{2008, we document that the specication of the default model has a major impact on the predicted loss distribution, while the specication of the recovery model is less important. In particular, we nd evidence that industry factors and regime dynamics aect the performance of default models, implying that the appropriate choice of default models for loss prediction will depend on the credit cycle and on portfolio characteristics. Finally, we show that default probabilities and recovery rates predicted out-of-sample are negatively correlated, and that the magnitude of the correlation varies with seniority class, industry, and credit cycle.

Valentin Patilea (Rennes)

Adaptive Estimation of Var with Time-Varying Variance : Application to Testing Linear Causality in Mean and Var Order

Abstract:Linear Vector AutoRegressive (VAR) models where the innovations could be unconditionally heteroscedastic and serially dependent are considered. The volatility structure is deterministic and quite general, including breaks or trending variances as special cases. In this framework we propose Ordinary Least Squares (OLS), Generalized Least Squares (GLS) and Adaptive Least Squares (ALS) procedures. The GLS estimator requires the knowledge of the time-varying variance structure while in the ALS approach the unknown variance is estimated by kernel smoothing with the outer product of the OLS residuals vectors. Different bandwidths for the different cells of the time-varying variance matrix are also allowed. We derive the asymptotic distribution of the proposed estimators for the VAR model coefficients and compare their properties. In particular we show that the ALS estimator is asymptotically equivalent to the infeasible GLS estimator. This asymptotic equivalence is obtained uniformly with respect to the bandwidth(s) in a given range and hence justifies data-driven bandwidth rules. Using these results two applications are proposed. First, we build Wald tests for the linear Granger causality in mean which are adapted to VAR processes driven by errors with a non stationary volatility. It is also shown that the commonly used standard Wald test for the linear Granger causality in mean is potentially unreliable in our framework. Second, we investigate the properties of the classical portmanteau tests when the innovations are unconditionally heteroscedastic. We find that the asymptotic distributions of the OLS residual autocorrelations can be quite different from the standard chi-square asymptotic distribution. Corrected portmanteau tests which take into account changes in the volatility using data driven adjustments for the critical values are proposed. Monte Carlo experiments and real data examples illustrate the theoretical results. The talk is based on joint work with Hamdi Raïssi.

Xiaohong Chen (Yale)

On Inference of PSMD Estimators of Semi/Nonparametric Conditional Moment Models

Abstract:In this paper, we consider estimation and inference of functionals of unknown parameters satisfying (nonlinear) semi/nonparametric conditional moment restrictions models. Such models belong to the difficult (nonlinear) ill-posed inverse problems with unknown operators, and include nonparametric quantile instrumental variables (IV), single index IV regressions and nonparametric endogenous default model as special cases. We first establish the asymptotic normality of the penalized sieve minimum distance (PSMD) estimators of any functionals that may or may not be root-n estimable. We show that the profiled optimally weighted PSMD criterion is asymptotically chi-square distributed, which provides one simple way to construct confidence sets. We also establish the validity of bootstrap based confidence sets for possibly non-optimally weighted PSMD estimators. We illustrate the general theory by constructing confidence bands for a nonparametric quantile instrumental variables Engel curve in a Monte Carlo study and a real data application (by Xiaohong Chen and Demian Pouzo).

Abstract:tba

Prof. Natalie Neumeyer (Universität Hamburg)

Some specification tests in nonparametric quantile regression

Abstract: Due to their great flexibility as well as robustness and equivariance under monotone transformations quantile regression models have become increasingly popular in many fields of statistical applications. We consider hypotheses tests for model assumptions in nonparametric quantile regression models that are based on estimated empirical processes. In particular we consider a test for location-scale models, a test for monotonicity of the conditional quantile curve and a test for significance of covariables.

Victor Chernozukov (MIT)

Intersection Bounds: Estimation and Inference

Abstract: We develop a practical and novel method for inference on intersection bounds, namely bounds defined by either the infimum or supremum of a parametric or nonparametric function, or equivalently, the value of a linear programming problem with a potentially infinite constraint set. Our approach is especially convenient in models comprised of a continuum of inequalities that are separable in parameters, and also applies to models with inequalities that are non-separable in parameters. Since analog estimators for intersection bounds can be severely biased in finite samples, routinely underestimating the length of the identified set, we also offer a (downward/upward) median unbiased estimator of these (upper/lower) bounds as a natural by-product of our inferential procedure. Furthermore, our method appears to be the first and currently only method for inference in nonparametric models with a continuum of inequalities. We develop asymptotic theory for our method based on the strong approximation of a sequence of studentized empirical processes by a sequence of Gaussian or other pivotal processes. We provide conditions for the use of nonparametric kernel and series estimators, including a novel result that establishes strong approximation for general series estimators, which may be of independent interest. We illustrate the usefulness of our method with Monte Carlo experiments and an empirical example.

Patricia Reynaud-Bouret (Université de Nice)

Hawkes process as models for some genomic data

Abstract: It seems that some of the genomic data, such as positions of words on the DNA or positions of transcription regulatory elements may hint for synergies between them. One of the statistical possible model to catch those interactions is the Hawkes process, which has been rst introduced to model earthquakes. Gusto and Schbath have introduced this model for genome analysis. However if maximum likelihood methods exist and if AIC criterion is usually used to select a correct number of parameters, this combination has been proved to be not accurate when the complexity of the family of parametric models is high. After discussing the Hawkes models (multivariate or not) and explaining what has been done from a parametric point of view (eventually combined with AIC), I will explain what adaptive model selection can and cannot do and also what thresholding in certain cases and Lasso methods may improve.