BSS_WEIBULL_ICA_IJEST10-02-11-106

8/7/2019 BSS_WEIBULL_ICA_IJEST10-02-11-106

1/8

M. El-Sayed Wahed / International Journal of Engineering Science and Technology

Vol. 2(11), 2010, 6739-6746

BLIND SOURCE SEPARATUION BASED

ON THE WEIBULL MIXTURE MODELM. El-Sayed Wahed

Department of Mathematics, Faculty of Science

Zagazig University, Zagazig, (Egypt).

E-mail: [email protected]

Abstract.

Modeling the distribution of the observed data with a parametric approach is an important tool of statistical

signal processing. In applications where the data have many clusters, like image segmentation, a multi componentprobabilistic model representation such as mixture modeling is required. Actually, in many real applications in the

field of image processing, Audio, blind equalization, there are a need to better approximate the observed data and

the use of mixture approach. This paper aims to provide a realistic distribution based on Mixture of weilbull

distribution, which has the advantage to characterize parameters estimation in each component in the mixture. Wepropose a formulation of the Nelder-Mead under weilbull distribution. For this, the method Likelihood function is

used to update the mixture parameters. An application of this technique is considered for modeling signals. In which

we used it as score function for blind signal separation (BSS), we resort to the Fast Independent Component

Analysis (FICA) approach whilst to adaptively estimate the parameters of such score function.

Keywords:

Mixture Weilbull Distribution (MWD), Nelder-Mead (NM), Maximum Likelihood (ML), Independent

Component Analysis (ICA), Blind Signal Separation (BSS).

1. Introduction

The problem of independent component analysis (ICA) has received wide attention in various fields such as

biomedical signal analysis and processing (EEG, MEG, ECG), geophysical data processing, data mining, speech

recognition, image recognition and wireless communications (Amari and Cichocki, 1998; Amari et al, 1997; Belland Sejnowski, 1995; Cichock et al., 1997; Gardner, 1991; Slock, 1996). In many applications, the sensorysignals(Observations obtained from multiple sensors) are generated by a linear generative model which is unknownto us . In other words, the observations are linear instantaneous mixtures of unknown source signals and the

objective is to process the observations in such a way that the outputs correspond to the separate primary source

signals.

The mixture modeling technique has been widely used for the estimation of the probability density functionand has found significant applications in various domains (see for example, Refs. [10, 2,11]). We propose here the

use of the weilbull distribution as the basic distribution in the mixture [9]. The weilbull distribution has been

employed to model and detect Gaussian and non-Gaussian signals recently in speech modeling and image and video

coding. We focus in this paper on the parameters estimation of the mixture weilbull model. To these end NM

method used to obtain an ML (Maximum Likelihood) has been developed. In this method the shape parameters areupdated by numerical optimization of the Likelihood function. Applications of this technique in BSS are considered.

The rest of this paper is organized as follows: In section 2 we introduce the Mixture of Weibull (MW)

approach. In Section 3, we present our approaches for the parameters estimation. In section 4, we introduce BSS. Insection 5, we discuss the Application of the MW model in (BSS).

2. The Mixture of Weibull Distribution

A mixture of Weibull is a parametric statistical model which assumes that the data originates from aweighted sum of several Weibull sources. More specifically, a MW is defined as[12]:

k

i

iiiii cxpwxp1

),,|()|( (1)

ISSN: 0975-5462 6739


2/8


Vol. 2(11), 2010, 6739-6746

kicw iiii ,...,2,1,),,,( .

k is the number of mixture density components.

iw is the ith mixture weight and satisfies

k

i

ii ww1

1,0 .

),,|( iiii cxp is an individual density of the Weibull density which is characterized by the

following probability density function [9].

iiiiii cxp

|c-x|-exp|cx|

2

1),,|( i

1

iiii (2)

where ic is the location parameter, 0i is the scale parameter, 0i is the shape parameter .By varying theparameters of (Weibull), its possible to characterize a large class of distributions such as gaussian, subgaussian

(more peaked, than gaussian, heavier tail) and supergaussian (flatter, more uniform).Fig 1 show some examples of

pdfs for MW for k=2 ( k=1 is Weibull that introduce in[9]) .Thanks to the shape parameters the MW is more

flexible and can approximate a large class of statistical distributions, this distribution requires to estimate k*4

parameters, kicm iiii ,...,2,1),,,(

. Particular we discuss the estimation of these parameters in detail inthe following sections.

-4 -2 0 2 4 6 8 10 120

0.1

0.2

0.3

0.4

0.5

g

f

h

w

Fig 1:the probability density functions of MW

for 9,7,2,4,3,4,5,2,3,7,2,4,7,2,2 21221 cck with different weight

.3,0.4,0.60.7,01 m , 12 1 mm and 8,1,3,21 are ),,,( whfg respectively.

3. The log-likelihood function to estimate MWD parameters

The log-likelihood function of (2), given by [2].

k

1i

1-N

1j

iiiiiji, ]),,c|(xpw[log)|L( jx (3)

whereN the sample size and kiip j ,...,1()x|(ji, , )]1,0[ Nj represents the conditional

expectation of ip given the observation jx ,means the posterior probability that jx belongs to the ith

component.

ISSN: 0975-5462 6740


3/8


Vol. 2(11), 2010, 6739-6746

In the case of Generalized Gamma distribution, if we substitute (2) into (3) and after some manipulation we obtain

the following form of )|L( x :

k

i

N

j

k

i

N

j

ijiijiiijii ccw1 1 1 1

,ji, x||x|log)1()2log()(log)(log)(log)|L( x

(4)

Accordingly, we obtain for ki ,...,2,1 the following nonlinear equation related to the estimated parameters byderivatives of the log-likelihood function with respect to iiic ,, and setting these derivatives to zeros we obtain

the following equations.

0)(

,,0)(

,0)(

iii

LL

c

L

|x|x|x(5)

N

j

iiiiiii

i

ij ccci

1

0|x|log|x||x|log1

(6)

N

jiii

iij

i

c1 0|x|

1

(7)

N

j

iiiiiiii

ii

i

ij csignccsignc

i

1

10)x(|x|)x(

|x|

1

(8)

After a little mathematical manipulation, the ML estimate of i is obtained.

0

1|x|

|x|

||log

1

1

1

1

N

j ijN

k

ikki

N

k

ki

ijji

N

j

ji

i

i

i

c

c

cu

(9)

Where i is the estimator of i then the estimator of i and ic are given by the following equations:

N

k

ikki

N

k

ki

i

ic1

1

|x|

(10)

0|x|)x(|x|

1

1

N

j

ijiiij

ij

i

jiiccsign

c

(11)

4. Blind Signal Separation (BSS).

By definition, independent component analysis (ICA) is the statistical method that searches for a lineartransformation, which can effectively minimize the statistical dependence between its components [10]. Under the

physically plausible assumption of mutual statistical independence between these components, the most application

of ICA is blind signal separation (BSS). In its simplest form, BSS aims to recover a set of unknown signals, the so-

called original sourcesT

n21 (t)]s(t),....,s(t),[s(t)s n

R , by relying exclusively on information that can be

extracted from their linear and instantaneous mixturesT

m21 (t)]x(t),....,x(t),[x(t) xm

R , given by

ISSN: 0975-5462 6741


4/8


Vol. 2(11), 2010, 6739-6746

m,2,1,,(t)(t) tsAx (12)

where Amxn

R is an unknown mixing matrix of full rank and m n. In doing so, BSS remains truly (blind) inthe sense that very little to almost nothing be known a priori for the mixing matrix or the original source signals.

Often sources are assumed to be zero-mean and unit-variance signals with at most one having a Gaussian

distribution. The problem of source estimation then boils down to determining the un-mixing matrix W

nxm

R

such that the linear transformation of the sensor observation is

(t)(t) xWu , t = 1,2,,n , (13)

where u(t)=nT

n21 (t)]u(t),....,u(t),[u R yield an estimate of vectors(t) corresponding to the original or truesources. In general, the majority of BSS approaches perform ICA, by essentially optimizing the negative log-likelihood (objective) function with respect to the un-mixing matrix W such that

n

l 1

lu |)det(|log)](upE[log),L( l WWu , (14)

where E[.] represents the expectation operator and )(up lu l is the model for the marginal probability density

function (pdf) of lu , for all nl ,...,2,1 . Normally, matrix W is regarded as the parameter of interest and the pdfs of the sources are considered to be nuisance parameters. In effect, when correctly hypothesizing upon the

distribution of the sources, the maximum likelihood (ML) principle leads to estimating functions, which in fact are

the score functions of the sources [7].

)(log)( lul

ll updu

du

l (15)

In principle, the separation criterion in (14) can be optimized by any suitable ICA algorithm where contrasts are

utilized (see e.g., [7]). A popular choice of such a contrast-based algorithm is the so-called fast (cubic) converging

Newton-type (fixed-point) algorithm, normally referred to as FastICA [1], based on

])((E[T

k1k uuDWW klll uuEdiag W])))([( , (16)

where, as defined in [8],

)]))(uE[])u(udiag(1/(E[ ' lllll D , (17)

withT

nn2211 )](u),....,(u),(u[(t) being valid for all nl ,...,2,1 . In the ICA framework, accuratelyestimating the statistical model of the sources at hand is still an open and challenging problem [7]. Practical BSS

scenarios employ difficult source distributions and even situations where many sources with very different pdfs are

mixed together. Towards this direction, a large number of parametric density models have been made available in

recent literature. Examples of such models include the generalized Gaussian density (GGD)[4], the generalizedlambda density(GLD) and the generalized beta distribution (GBD) or even combinations and generalizations such as

super and generalized Gaussian mixture model (GMM) [5], the generalized gamma density (GGD) [3]. In the

following section we show example Mixture Weibull Density (MWD) for signal modeling in blind signal

separation.5. Application of MWD in blind signal separation

Novel flexible score function is obtained, by substituting (1) into (15) for the source estimates

nll ,...,2,1,u , it quickly become obvious that our proposed score function inherits a generalized parametricstructure ,which in turn can be attributed to the highly flexible MWD parent model . In this case, a simple calculus

yield the flexible BSS score function

ISSN: 0975-5462 6742


5/8


Vol. 2(11), 2010, 6739-6746

k

i

ii

il

i

iliililiii

lu

li

ii

lcu

cucusigncuwup 1

22

l||

1||exp)(||

2

11)|(u

(18)

In principle )|( ll u .Is capable of modeling a large number of signals, such as speech or communication signals.

This is due to the fact that its characterization depends explicitly on all parameters kicw iiii ,...,2,1,,,,

Other commonly used score functions can be obtained by substituting appropriate values for iiii cm ,,, in

(18), for instance, when 1k we have score function

1||||

)()|(u 1111

1

1

l1

cu

cu

cusignl

l

l

l (19)

When 11 and 11 we have a scaled form of the GGD-based score function constitutes such a special case of(17).

1

111||)()|(

cucusignu llll (20)

The function)|(

ll ucould become singular, in some special cases, essentially those corresponding to heavy-

tailed (or sparse) distribution defined for 0|| il cu . In practice, to circumvent such deficiency, the denominatorin (18) can be modified slightly to read function

k

i

ii

il

i

iliililiii

lu

li

ii

lcu

cucusigncuwup 1

22

l||

1||exp)(||

2

11)|(u

(21)

where is a small positive parameter (typically around 104) which, when put to use, can almost always guarantee

that the discontinuity of (18) or values in or approaching the region 0|| il cu is completely avoided.

6. Numerical experiments

To investigate the separation performance of the proposed MWD-based FastICA BSS method, a set of

numerical experiments preformed in which we consider two example one artificial at 2k and anther real (speechsignal), we illustrate this in the following two examples

6.1. Example 1

In this example 2k and the data set used consists different realizations of independent signals, thenumber of data samples has also been designed to be relatively small, for example, N = 250.The source signals are

mixed (noiselessly) with randomly generated full rank mixing matrices A. The FastICA method is implemented in

the so-called simultaneous separation mode, whereas the stopping criterion is set to410 . FastICA is executed

using the flexible MWD model is used to model the distribution of the unknown sources, while (9)(11) are

employed to adaptively calculate the necessary parameters of the MWD-based score function defined in (20).

Now to show the performance in this case we consider three source signals (source) where this signals generatedrandom from MWD. Let the mixing matrix A and un-mixing matrix W are defined as follow

,

.48.32.17

.86.65.75

.37.79.56

A

1.86871.8798-1.9485

0.16460.1924-0.2184

0.25660.2177-0.2128

W

By using the equation sAx we obtain mixed signals as show in Fig (2) where mixing signals in left and sourcesignals in right

ISSN: 0975-5462 6743


6/8


Vol. 2(11), 2010, 6739-6746

0 100 200 3000

500

1000

fisrt mixing

mixing signals

0 100 200 3000

500

1000

second mixing

0 100 200 3000

200

400

third mixing

0 100 200 3000

500

1000Source signals

fisrt source

0 100 200 3000

5

10

second source

0 100 200 3000

200

400

third source

Fig(2)

After using FastICA we recover the sources and we show the estimated signals in left and original signals in right inFig (3) with different in scale only.

0 100 200 3000

5

first estimated

estimated Signals

0 100 200 3000

5

10

second estimated

0 100 200 3000

2

4

third estimated

0 100 200 3000

500

1000Source Signals

first source

0 100 200 3000

5

10

second source

0 100 200 3000

200

400

third source

Fig (3)

6.2. Example.1.

Consider three speech signals as sources, mixing matrix A and demixing matrix W are given as follow

.50.78.95

.96.56.22

.32.8977.

A and

.0589.0874.0429

.0982.0467.0159

.8760.0340.0987

W

ISSN: 0975-5462 6744


7/8


Vol. 2(11), 2010, 6739-6746

By using the equation sAx we obtain mixed signals as show in figure (4) where mixing signals in left andoriginal signals in right. We recover the source by using FastICA and we show the estimated signals in left withscales, permutation and original signals in right in figure (5).

0 2000 4000 6000 8000-1

0

1

fisrt mixing

mixing signals

0 2000 4000 6000 8000-1

0

1

second mixing

0 2000 4000 6000 8000-0.5

0

0.5

third mixing

0 2000 4000 6000 8000-1

0

1Source signals

fisrt source

0 2000 4000 6000 8000-0.5

0

0.5

second source

0 2000 4000 6000 8000-1

0

1

third source

Fig(4)

6. Algorithm performance

The separation performance for ICA algorithm is evaluated with the cross-talk error measure

n

j

n

i jlnl

jin

i

n

j linl

ji

p

p

p

pPI

1 1

2

1

2

1 1

2

1

2

1||max

||1

||max

||

(20)

Note that here jip represents the elements of the permutation matrix P = WA, which after assuming that all

sources have been successfully separated should ideally reduce to a permuted and scaled version of the identity

matrix. The separation performance for first example is dBPI 72.18 and for second example

is dBPI 01.26

0 2000 4000 6000 8000-5

0

5

fisrt estimated

estimated signals

0 2000 4000 6000 8000-10

0

10

second estimated

0 2000 4000 6000 8000-5

0

5

third estimated

0 2000 4000 6000 8000-1

0

1Source signals

fisrt source

0 2000 4000 6000 8000-0.5

0

0.5

second source

0 2000 4000 6000 8000-1

0

1

third source

ISSN: 0975-5462 6745


8/8


Vol. 2(11), 2010, 6739-6746

Fig(5)

5.Conclusion

The main motivation of this study is to provide estimators for the MWD parameters. For this, a new

formulation of the NM algorithm is conducted to include the parameters estimation. We have derived a novel

parametric family of flexible score functions, based exclusively on the MWD model. To calculate the parameters of

these functions in an adaptive BSS setup, we have chosen to maximize the ML equation with the NM optimization

method. Simulation results show that the proposed approach is capable of separating mixtures of signals.

Reference:

[1] A.Hyvarinen, E.Oja, A fast fixed-point algorithm for independent component analysis, Neural computation, vol 9, no 7, October 1997, pp.1483-1492.

[2] C. Tzagkarakis, A. Mouchtaris, P. Tsakalides; .Musical Genre Classification VIA Generalized Gaussian and Alpha-Stable Modeling,ICASSP 2006 Proceedings. Vol.5, 14-19 May 2006 pp 217.220.

[3] E.W. Stacy, A generalization of the gamma distribution, Ann. Math. Stat. vol 33, no 3, September 1962, pp. 1187 -1192.[4] K. Kokkinakis, A.K. Nandi, Exponent parameter estimation for generalized Gaussian probability density functions with application to

speech modeling, Signal Process. vol 85, no 9, September 2005,pp. 1852-1858.[5] J.A. Palmer, K. Kreutz-Delgado, S. Makeig, Super-Gaussian mixture source model for ICA, in: Proceedings of the International

Conference on Independent Component Analysis and Blind Signal Separation, Charleston, SC, USA, March 5-8 ,2006, pp. 854-861.

[6] J. Eriksson, J. Karvanen, V. Koivunen, Source distribution adaptive maximum likelihood estimation of ICA model, in: Proceedings of theSecond International Conference on ICA and BSS, Helsinki, Finland, June 19-22, 2000, pp. 227- 232.

[7] J.-F. Cardoso, Blind signal separation: statistical principles, Proc. IEEE, vol 86, no 10, October 1998, pp. 2009- 2025.

[8] J. Karvanen, V. Koivunen, Blind separation methods based on Pearson system and its extensions, Signal Process. vol 82, no 4, April 2002,pp. 663-673.

[9] M. El-Sayed Wahed, (2009):Blind signal separation using an adaptive Weibull distribution. International Journal of Physical SciencesVol. 4(5), pp. 265-270.

[10] Pekka Paalanen, Joni-Kristian Kamarainen, Jarmo Ilonen, Heikki Klviinen; .Feature representation and discrimination based on Gaussianmixture model probability densities Practices and algorithms. Pattern Recognition 39 (2006) pp. 1346-1358.

[11] Vimal Bhatia, Bernard Mulgrew, .Non-parametric likelihood based channel estimator for Gaussian mixture noise, Signal Processing Vol.87 (2007) pp 2569-2586.

[12] Wendy L. Martinezand Angel R. Martinez computational statistics with matlab,Boca Raton London New York Washington, D.C.

ISSN: 0975-5462 6746

BSS_WEIBULL_ICA_IJEST10-02-11-106

Documents

Transcript of BSS_WEIBULL_ICA_IJEST10-02-11-106