lineage.compare_emissions

Cross-validated comparison of the Gamma/Bernoulli and competing-risks emissions.

The two emissions are densities over different observation spaces. The competing-risks form in lineage.states.StateDistributionCR puts a density on when a cell died, which the Gamma/Bernoulli form has no way to express -- it scores a death as a bare Bernoulli outcome and discards the time. Comparing their raw likelihoods would charge the competing-risks model for predicting strictly more.

The headline metric therefore coarsens death timing away, leaving a space both models describe:

outcome

competing risks

Gamma/Bernoulli

transition t

f_D(t) S_X(t)

p f_D(t)

death

1 - P(divide)

1 - p

censored, c

S_D(c) S_X(c)

S_D(c)

The transition branches carry identical total mass, so that comparison is like for like. The censored branch is where the Gamma/Bernoulli form ignores that a censored cell also did not die, and so claims more probability than it is entitled to -- see outcome_mass(), which shows its total exceeding one by up to ~10% for the short-horizon cells that make up much of this data. That bias runs in the Gamma model's favour, which makes the comparison conservative.

Run as python -m lineage.compare_emissions <population> <states> <reps>, e.g. python -m lineage.compare_emissions AllLapatinib 2,3,4 5.

Attributes

MODELS

PHASE_COLS

TIME_FLOOR

Functions

outcome_mass(a, scale, p_div, horizon)

Total probability each emission assigns across the outcomes of one phase.

coarse_logpdf(dist, x)

Log likelihood of a two-phase observation with death timing coarsened away.

build(pops, cls, num_states[, mask_seed, frac])

Rebuild populations under emission class cls, masking frac of cells.

_state_weights(gammas, ci, li, idxs)

Normalized posterior over states for the given held-out cells.

heldout_LL(objs, gammas, held[, scorer])

Held-out log likelihood, marginalized over the fitted state posterior.

fate_logloss(objs, gammas, held)

Held-out log likelihood of the binary fate alone.

death_time_LL(objs, gammas, held)

Density the competing-risks model puts on held-out death times, given a death.

run(pop_name, k_list, reps[, seed0])

Fit both emissions on the same masked data and emit one JSON record per rep.

Module Contents

lineage.compare_emissions.MODELS
lineage.compare_emissions.PHASE_COLS
lineage.compare_emissions.TIME_FLOOR = 1e-10
lineage.compare_emissions.outcome_mass(a, scale, p_div, horizon)

Total probability each emission assigns across the outcomes of one phase.

For a cell watched until horizon the outcomes are: transition at some t <= horizon, death at some t <= horizon, or still going at the horizon. A well-formed emission spreads exactly probability one over those.

Returns:

(Gamma/Bernoulli mass, competing-risks mass)

Parameters:

  • a (float)

  • scale (float)

  • p_div (float)

  • horizon (float)

Return type:

tuple[float, float]

lineage.compare_emissions.coarse_logpdf(dist, x)

Log likelihood of a two-phase observation with death timing coarsened away.

Parameters:

x (numpy.ndarray)

Return type:

numpy.ndarray

lineage.compare_emissions.build(pops, cls, num_states, mask_seed=None, frac=0.25)

Rebuild populations under emission class cls, masking frac of cells.

Masking is driven by an rng over the tree shapes alone, so a given mask_seed hides exactly the same cells whichever emission class is used, and the two models are scored on identical held-out sets.

Returns:

(populations, held-out records of (lineage index, cell indices, obs))

Parameters:

  • pops (list)

  • num_states (int)

  • frac (float)

lineage.compare_emissions._state_weights(gammas, ci, li, idxs)

Normalized posterior over states for the given held-out cells.

Parameters:

  • ci (int)

  • li (int)

  • idxs (numpy.ndarray)

Return type:

numpy.ndarray

lineage.compare_emissions.heldout_LL(objs, gammas, held, scorer=coarse_logpdf)

Held-out log likelihood, marginalized over the fitted state posterior.

Parameters:

  • objs (list)

  • gammas (list)

  • held (list)

Return type:

tuple[float, int]

lineage.compare_emissions.fate_logloss(objs, gammas, held)

Held-out log likelihood of the binary fate alone.

Both models emit a division probability per phase, so this compares them over an identical observation space with no duration density involved at all.

Parameters:

  • objs (list)

  • gammas (list)

  • held (list)

Return type:

tuple[float, int]

lineage.compare_emissions.death_time_LL(objs, gammas, held)

Density the competing-risks model puts on held-out death times, given a death.

This is the piece the Gamma/Bernoulli emission cannot score at all, so it is reported on its own rather than folded into the comparison. A positive value means the fitted death clock is sharper than a one-per-hour reference.

Parameters:

  • objs (list)

  • gammas (list)

  • held (list)

Return type:

tuple[float, int]

lineage.compare_emissions.run(pop_name, k_list, reps, seed0=0)

Fit both emissions on the same masked data and emit one JSON record per rep.

Parameters:

  • pop_name (str)

  • k_list (list[int])

  • reps (int)

  • seed0 (int)