spr_decorr.c File Reference

Calculate a good decorrelation matrix given the full covariance gaussians for an HMM-system with diagonal covariance matrices. More...

Detailed Description

Calculate a good decorrelation matrix given the full covariance gaussians for an HMM-system with diagonal covariance matrices.

spr_decorr [-method decorr method](1) [-SLDA scale axes (LDA)](0.0) [-niter max #iter](8)
    [-eps convergence threshold](1e-3) <-cov 2nd order stats>
    [-wgt new weight vector for the gaussians] [-ncov transformed stats]
    <-mat output decorr matrix> [-imat initial decorr matrix] [-xmat existing transf]
    [-tmat temp. results] [-mvg new mvg] [-from start param](0) [-inc add #params](-1)
    [-PVT](flag: pivoting) [-gc GLDA-criterium](MCE) [-gf GLDA-function](EXP7)
    [-VC variance compensation](1) [-greedy fast&greedy](Ggf) [-Fopt Full/Inc-search](/)
    [-SVD GLDA-search](Ggf) [-NDcost Non-diag cost](1.0) [-Reps Rank eps](1e-7)
    [-class class info] [-msk mask] [-VCW file/wgt_vec]

Parameters

-method<em>decorr	method Decorrelation/optimization method: (0) nop, (1) approx (fast) LS decorr, (2) LS decorr, (3) weighted LS decorr, (4) ML decorr, (-1) generalize LDA (MIDA), (-2) joint GLDA&decorr, (-3) LDA
-SLDA<em>scale	axes (LDA) Use the LDA-transformation as initial decorrelation matrix instead of the default PCA-transformation. The axes are scaled acording to their importance to the power <SLDA>.
-niter<em>max	#iter Maximum number of iterations.
-eps<em>convergence	threshold Stop if the relative improvement is less than <eps> for more than two consequetive iterations.
-cov<em>2nd	order stats The full covariance matrices (statistics) for the gaussians.
-wgt<em>new	weight vector for the gaussians Overrule the weights stored in the full covariance matrices.
-ncov<em>transformed	stats Write the transformed (+ truncated) 2nd order statistics to disk.
-mat<em>output	decorr matrix The resulting decorrelation matrix.
-imat<em>initial	decorr matrix An initial decorrelation matrix (starting point for the iterative optimization algorithm). If not specified, the PCA or LDA transformation is used as initial value for the decorrelation matrix. A special value is EYE, which will use the identity matrix as initial transformation.
-xmat<em>existing	transf Existing transformation matrix used to create the statistics. The transformation matrices written to disk are the combined transformation.
-tmat<em>temp.	results Write intermediate results to disk (GLDA-only). The file-name should contain a 'i' where the index should come.
-mvg<em>new	mvg The new gaussians set to be used if the decorrelation matrix is used in the preprocessing.
-from<em>start	param Start optimizing from parameter <from> (method -1 & -2).
-inc<em>add	#params Add <inc> optimal parameters (method -1 & -2).
-PVTflag	pivoting Do optimal pivoting, i.e. increased stability, at the cost of a slower evaluation (method -1 & -2).
-gc<em>GLDA-criterium</em><a	name="spr_decorr.gc" class="el"> Criterium to optimize (method -1 and -2): MCE: Minimize Classification Error (default) MMI: Minimal Mutual Information (loss) MI2: Least squares approximation of MMI OVL: Minimize overlap SOV: Minimize the scaled overlap.
-gf<em>GLDA-function</em><a	name="spr_decorr.gf" class="el"> Warp-function applied to the basic overlap measure (method -1 and -2): LIN: Linear (no warping) EXP6: Modified exponential EXP7: Modified exponential ERF: Correct warp function for MCE (1-dim/equal covar). MIC: Approx. warp function for MMI (1-dimensional).
-VC<em>variance	compensation Compensation for differences in variance (method -1 and -2): (0) no compensation (1) symmetric compensation (normalized overlap measure) (2) same as 1 + extra factor based on the sigma ratio (-1) non symmetric compensation for MMI (-2) non symmetric compensation for MCE
-greedy<em>fast&amp;greedy</em><a	name="spr_decorr.greedy" class="el"> Select the greedy level (method -1 & -2), must be either '/' or combination of the following letters: (G) add new parameters independent from existing set. (g) add new parameters dependent on the already present params. (f) optimize the global set (also changes the first parameters) at the end. (F) do a global optimisation for every additional parameter (and not only at the end).
-Fopt<em>Full/Inc-search</em><a	name="spr_decorr.Fopt" class="el"> Do a more profound search, i.e. less chance to get stuck in a local minimum (method -1 & -2). See the -greedy option for more details.
-SVD<em>GLDA-search</em><a	name="spr_decorr.SVD" class="el"> Use SVD to optimize the search (method -1 & -2). See the -greedy option for more details.
-NDcost<em>Non-diag	cost Cost of having non-diagonal covariance matrices in the GLDA-solution (method -2).
-Reps<em>Rank	eps Epsilon value to determine the rank of the within class covariance matrix. Should be a small non negative value.
-class<em>class	info Define multiple gaussians per class, and eventually, define multiple independent sets of classes. The first field in a line gives the class a certain gaussian belongs to, the second field gives the set. The gaussians cannot be shared, and all gaussians in on class must be listed consequetive (the same holds for the sets).
-msk<em>mask</em><a	name="spr_decorr.msk" class="el"> Define a (symmetric) masking matrix. Allow non-zero elements in the transformation matrix at certain positions only. The masking elements must be 8-bit integers. Only works in combination with an initial matrix (use 'EYE' for the identity matrix) – see the '-imat' option.
-VCW<em>file/wgt_vec</em><a	name="spr_decorr.VCW" class="el"> Vector Component Weighting: in the LDA method, assign different weights (priorities) to each axis in the input space (implementation: left&right multiply the withing class covariance matrix with diag(1/sqrt(VCW))). Either a file name is specified or an expression (starting with '[').

Calculate a good decorrelation matrix given the full covariance gaussians for an HMM-system with diagonal covariance matrices Calculate a good decorrelation/discriminant matrix given the full covariance gaussians for an HMM-system with diagonal covariance matrices. Four decorrelation and two discriminant methods are provided:

0

Don't do anything (can be used to derive the new gaussian set).

1

Find the orthonormal transformation that minimizes the sum of the square of the off diagonal elements.

2

Find the orthonormal transformation that minimizes the sum of the square of the off diagonal elements with normalization for the size of the gaussian (diagonal elements).

3

Find the linear transformation that minimizes the sum of the square of the off diagonal elements with normalization for the size of the gaussian. The current implementation does not always converge!

4

Find the linear transformation that maximizes the log likelihood of the data if the classes are to be modelled with a diagonal covariance matrix (see ICASSP98, M. Gales, Cambridge University for more details).

-1

Calculate the generalized LDA transformation of the parameters. Note that the transformation matrix is always square, although only the first p rows (number of paramters requested) are of use. The rows are ordered according to their importance (most important linear combinations first), except when the greedy flag is set to 0. Two basic criteria are available: Minimal Classification Error and Minimal Entropy (or Minimal Mutual Information loss). Note that all methods are approximative.

-2

The same as method (-1), except for the extra cost assigned to non-diagonal covariance matrices in the resulting sub-space. This optimization is extremely slow.

All methods use an iterative optimization algorithm. A good initial value for the decorrelation matrix is the PCA-tranformation that maps the average of all covariance matrices to the unity matrix. If no external matrix is given, this value is taken as initial value (or a scaled version of the LDA-transformation id the SLDA-flag is used). For the generalized LDA, the default initial transformation matrix is the LDA-transformation.

Good flags for the generalized LDA (empirically determined) are:

for MCE

-gc SOV -gf EXP7

for MMI

-gc MMI -gf LIN

See Also

spr_fv_gauss.c, and spr_calc_lda.c

Author

Kris Demuynck

Date

12/05/98

Revision History:

12/05/98 - KD

creation

30/07/98 - KD

added the fourth decorrelation method (ML)

05/08/98 - KD

added the generalized LDA

30/01/99 - KD

improved generalized LDA (MCE/MMI/OVL)

19/06/12 - KD

code cleanup+speedup & bug fix in the multi-set glda condition