Skip to content

Nonparametric Estimation from Incomplete Observations

Why this mattered

Kaplan and Meier made it possible to estimate a survival curve directly from incomplete time-to-event data without imposing a parametric lifetime distribution. The key move was to treat censoring not as a nuisance that forced discarding cases or grouping observations, but as information about risk sets: at each observed event time, the estimator updates survival using the number still under observation just before that time. Under the paper’s explicit independence assumption between lifetime and loss time, this product-limit estimator preserved the temporal ordering of deaths and losses and yielded a nonparametric maximum-likelihood estimate of the survivor function.

The paradigm shift was practical as much as mathematical. Clinical follow-up studies, reliability tests, and life-testing experiments could now use all available partial observations rather than reducing them to crude actuarial intervals or complete-case summaries. A patient lost to follow-up after two years, for example, still contributed evidence of survival up to two years; a machine withdrawn before failure still informed the risk set until withdrawal. This changed survival analysis into a general statistical language for censored data, making rigorous comparison and estimation possible in settings where complete observation was structurally unrealistic.

The Kaplan-Meier estimator became the foundation on which much of modern event-history analysis was built. Later methods such as the log-rank test, Cox proportional hazards regression, competing-risk extensions, and contemporary clinical-trial survival endpoints all depend on the same conceptual infrastructure: risk sets, censoring assumptions, and nonparametric survival estimation. Its enduring influence comes from separating the estimand, the survival function, from strong distributional assumptions while making the limits of inference explicit.

Abstract

Abstract In lifetesting, medical follow-up, and other fields the observation of the time of occurrence of the event of interest (called a death) may be prevented for some of the items of the sample by the previous occurrence of some other event (called a loss). Losses may be either accidental or controlled, the latter resulting from a decision to terminate certain observations. In either case it is usually assumed in this paper that the lifetime (age at death) is independent of the potential loss time; in practice this assumption deserves careful scrutiny. Despite the resulting incompleteness of the data, it is desired to estimate the proportion P(t) of items in the population whose lifetimes would exceed t (in the absence of such losses), without making any assumption about the form of the function P(t). The observation for each item of a suitable initial event, marking the beginning of its lifetime, is presupposed. For random samples of size N the product-limit (PL) estimate can be defined as follows: List and label the N observed lifetimes (whether to death or loss) in order of increasing magnitude, so that one has 0≤t 1ǐ≤t 2ǐ≤ … ≤t N ǐ. Then P(t)= II. [(N – r)/(N – r + 1)], where r assumes those values for which tr ≤t and for which tr ǐ measures the time to death. This estimate is the distribution, unrestricted as to form, which maximizes the likelihood of the observations. Other estimates that are discussed are the actuarial estimates (which are also products, but with the number of factors usually reduced by grouping); and reduced-sample (RS) estimates, which require that losses not be accidental, so that the limits of observation (potential loss times) are known even for those items whose deaths are observed. When no losses occur at ages less than t, the estimate of P(t) in all cases reduces to the usual binomial estimate, namely, the observed proportion of survivors.

Sources