Module mixture :: Class BayesMixtureModel
[hide private]
[frames] | no frames]

Class BayesMixtureModel

source code


Bayesian mixture models

Instance Methods [hide private]
 
__init__(self, G, pi, components, prior, compFix=None, struct=0, identifiable=1)
Constructor
source code
 
__str__(self)
String representation of the DataSet
source code
 
__eq__(self, other)
Interface for the '==' operation
source code
 
__copy__(self)
Interface for the copy.copy function
source code
 
modelInitialization(self, data, rtype=1)
Perform model initialization given a random assigment of the data to the models.
source code
 
mapEM(self, data, max_iter, delta, silent=False, mix_pi=None, mix_posterior=None, tilt=0)
Reestimation of maximum a posteriori (MAP) mixture parameters using the EM algorithm.
source code
 
removeComponent(self, ind)
Deletes a component from the model.
source code
 
updateStructureBayesian(self, data, objFunction='MAP', silent=1)
Updating structure by chosing optimal local merge with respect to the posterior.
source code
 
updateStructureBayesianFullEnumerationFixedOrder(self, data, objFunction='MAP', silent=1)
Updating structure by chosing optimal local merge with respect to the posterior.
source code
 
updateStructureBayesianBottomUp(self, data, objFunction='MAP', silent=1)
Updating structure by chosing optimal local merge with respect to the posterior.
source code
 
KLFeatureRanks(self, data, comps, silent=False)
Ranks the features by the symmetric relative entropy between the parameters induced in a structure where all components in 'comps' are merged and the parameters induced by all components (i.e.
source code
 
randMaxTraining(self, data, nr_runs, nr_steps, delta, tilt=0, objFunction='MAP', rtype=1, silent=False)
Performs `nr_runs` MAP EM runs with random initial parameters and returns the model which yields the maximum likelihood.
source code
 
shortInitMAP(self, data, nr_runs, nr_init, nr_steps, delta, tilt=0, nr_seed_steps=5, silent=False)
EM strategy:
source code
 
bayesStructureEM(self, data, nr_repeats, nr_runs, nr_steps, delta, tilt=0, objFunction='MAP', silent=False, min_struct=1, rtype=1)
EM training for models with CSI structure.
source code
 
minimalStructure(self)
Finds redundant components in the model structure and collapses the structure to a minimal representation.
source code

Inherited from MixtureModel: EM, EStep, EStep_old, MStep, classify, flatStr, formatData, identifiable, initStructure, isValid, merge, pdf, posteriorTraceback, printClusterEntropy, printStructure, printTraceback, randMaxEM, reorderComponents, sample, sampleDataSet, sampleDataSetLabels, sampleSet, sampleSetLabels, structureEM, sufficientStatistics, updateFreeParams, updateStructureGlobal, update_suff_p, validStructure

Method Details [hide private]

__init__(self, G, pi, components, prior, compFix=None, struct=0, identifiable=1)
(Constructor)

source code 

Constructor

Parameters:
  • G - number of components
  • pi - mixture weights
  • components - list of ProductDistribution objects, each entry is one component
  • prior - MixtureModelPrior object
  • compFix - list of optional flags for fixing components in the reestimation the following values are supported: 1 distribution parameters are fixed, 2 distribution parameters and mixture coefficients are fixed
  • struct - Flag for CSI structure, 0 = no CSI structure 1 = CSI structure
Overrides: ProbDistribution.__init__

__str__(self)
(Informal representation operator)

source code 

String representation of the DataSet

Returns:
string representation
Overrides: ProbDistribution.__str__
(inherited documentation)

__eq__(self, other)
(Equality operator)

source code 

Interface for the '==' operation

Parameters:
  • other - object to be compared
Overrides: ProbDistribution.__eq__
(inherited documentation)

__copy__(self)

source code 

Interface for the copy.copy function

Overrides: ProbDistribution.__copy__
(inherited documentation)

modelInitialization(self, data, rtype=1)

source code 

Perform model initialization given a random assigment of the data to the models.

Parameters:
  • data - DataSet object
  • rtype - type of random assignments. 0 = fuzzy assignment 1 = hard assignment
Returns:
posterior assigments
Overrides: MixtureModel.modelInitialization

mapEM(self, data, max_iter, delta, silent=False, mix_pi=None, mix_posterior=None, tilt=0)

source code 

Reestimation of maximum a posteriori (MAP) mixture parameters using the EM algorithm.

Parameters:
  • data - DataSet object
  • max_iter - maximum number of iterations
  • prior - an appropriate MixtureModelPrior object
  • delta - minimal difference in likelihood between two iterations before convergence is assumed.
  • silent - 0/1 flag, toggles verbose output
  • mix_pi - [internal use only] necessary for the reestimation of mixtures as components
  • mix_posterior - [internal use only] necessary for the reestimation of mixtures as components
  • tilt - 0/1 flag, toggles the use of a deterministic annealing in the training
Returns:
tuple of posterior matrix and log-likelihood from the last iteration
Overrides: MixtureModel.mapEM
(inherited documentation)

removeComponent(self, ind)

source code 

Deletes a component from the model.

Parameters:
  • ind - index of component to be removed
Overrides: MixtureModel.removeComponent

updateStructureBayesian(self, data, objFunction='MAP', silent=1)

source code 

Updating structure by chosing optimal local merge with respect to the posterior.

Features: - store merges in a learning history to prevent recomputation of merge parameters

  • compute parameters of candidate structures from parameters of groups to be merged
Parameters:
  • data - DataSet object
  • silent - verbosity flag
Returns:
number of structure changes

updateStructureBayesianFullEnumerationFixedOrder(self, data, objFunction='MAP', silent=1)

source code 

Updating structure by chosing optimal local merge with respect to the posterior.

Enumerates all possible structures for each feature seperately, i.e. returns the optimal structure for the given ordering of features. For a complete enumeration of the structure space, including permutation of feature order use the functions in fullEnumerationExhaustive.py.

Parameters:
  • data - DataSet object
  • silent - verbosity flag
Returns:
number of structure changes

updateStructureBayesianBottomUp(self, data, objFunction='MAP', silent=1)

source code 

Updating structure by chosing optimal local merge with respect to the posterior. The procedure starts with the minimally complex structure, i.e. every feature has a single distribution.

Parameters:
  • data - DataSet object
  • silent - verbosity flag
Returns:
number of structure changes

KLFeatureRanks(self, data, comps, silent=False)

source code 

Ranks the features by the symmetric relative entropy between the parameters induced in a structure where all components in 'comps' are merged and the parameters induced by all components (i.e. the uninformative structure).

This gives a ranking of the relevance,i.e. discriminatory information of features for distinguishing a subset of components.

Parameters:
  • data - DataSet object
  • comps - list of component indices
Returns:
list of tuples of (feature index, score) pairs in descending score order

randMaxTraining(self, data, nr_runs, nr_steps, delta, tilt=0, objFunction='MAP', rtype=1, silent=False)

source code 

Performs `nr_runs` MAP EM runs with random initial parameters and returns the model which yields the maximum likelihood.

Parameters:
  • data - DataSet object
  • nr_runs - number of repeated random initializations
  • nr_steps - maximum number of steps in each run
  • delta - minimim difference in log-likelihood before convergence
  • tilt - 0/1 flag, toggles the use of a deterministic annealing in the training
  • silent - 0/1 flag, toggles verbose output
Returns:
log-likelihood of winning model

shortInitMAP(self, data, nr_runs, nr_init, nr_steps, delta, tilt=0, nr_seed_steps=5, silent=False)

source code 

EM strategy:

  • 'nr_init' random initial models
  • short EM runs with each start model
  • take model with best likelihood for long EM run.

The short runs are set to 5 iterations by default

Parameters:
  • data - DataSet object
  • nr_runs - number of repeated random initializations
  • nr_init - number of random models for each run
  • nr_steps - maximum number of steps for the long training run
  • delta - minimim difference in log-likelihood before convergence
  • tilt - 0/1 flag, toggles the use of a deterministic annealing in the training
  • nr_seed_steps - number of EM steps for each seed model, default is 5
  • silent - 0/1 flag, toggles verbose output
Returns:
log-likelihood of winning model

bayesStructureEM(self, data, nr_repeats, nr_runs, nr_steps, delta, tilt=0, objFunction='MAP', silent=False, min_struct=1, rtype=1)

source code 

EM training for models with CSI structure. First a candidate model is generated by using the randMaxMAP procedure, then the structure is trained.

Parameters:
  • data - DataSet object
  • nr_repeats - number of candidate models to be generated
  • nr_runs - number of repeated random initializations
  • nr_steps - maximum number of steps for the long training run
  • delta - minimim difference in log-likelihood before convergence
  • tilt - 0/1 flag, toggles the use of a deterministic annealing in the training
  • silent - 0/1 flag, toggles verbose output
  • min_struct - 0/1 flag, toggles merging of components with identical paramters
Returns:
log-likelihood of winning model

minimalStructure(self)

source code 

Finds redundant components in the model structure and collapses the structure to a minimal representation.

Overrides: MixtureModel.minimalStructure
(inherited documentation)