An IR architecture for BSS in strong nonlinearconvolutive environments

Daniele Vigliano, Raffaele Parisi, Aurelio UnciniINFOCOM Dpt.

Universit'a degli Studi di Roma "La Sapienza", Rome, ItalyDaniele.vig1ianogZposte.it

Abstract- This paper introduces an IIR flexible ICA approach that CPNL-C is currently the most general nonlinearto the problem of blind source separation in convolutive convolutive model and it comprises all other mixingnonlinear environments. The proposed algorithm performs environments as particular cases.separation in the presence of convolutive mixing of postnonlinear convolutive mixtures (CPNL-C), it is based on 1* SEPARATION IN NONLINEAR CONVOLUTIVEredundancy reduction and realizes source separation by ENVIRONMENTSminimizing the output mutual information. Experimentalresults are described to show the effectiveness of the described This section describes the BSS problem in nonlineartechnique. environment. Let x[n] (n=l,... ,N) be the vector of mixed

accessible signals and s[n] the vector of hidden independentI. INTRODUCTION sources. The nonlinear convolutive mixture can be expressed

as: x[n]=FiQs[n],..,s[n-L]}, where F1.} is a convolutiveFirst studies about Independent Component Analysis aimedat resolving the well-known cocktail party problem in nonlinear distorting function. The solution y[n] of the BSSinstantaneous and convolutive environments. As a matter of problem can be expressed as y [n] = H s [n]}= G a F s [n]}.fact it came out that linear mixing models are unrealistic and In instantaneous environments ICA recovers the originalpoor in many practical situations. This observation motivated a sources up to some trivial acceptable non-uniqueness, so thatgrowing interest in separation of signals in the presence ofnon in general outputs can be scaled and delayed versions oflinear convolutive mixing environments. Recent advances in flipped inputs. In a more general convolutive nonlinear case,BSS ofnonlinear mixing models were reviewed in [1]. the issue of separating mixtures with the only constraint of

Several papers explored Post Nonlinear mixing problems output independence and no other a priori assumption is(PNL) in instantaneous [2] and convolutive environments affected by a strong non uniqueness [l][l 1]. Given[7], but only few works (e.g. [3][4][11]) dealt with the issue independent inputs, several well-known examples show theof existence and uniqueness of the solution. ICA algorithms existence ofmaps that can produce independent outputs evenfor BSS are strongly influenced by the matching with the with non diagonal Jacobian matrix. The independencestatistical properties of the signals involved in separation. In constraint alone is not strong enough to recover originalorder to properly deal with this problem, the Flexible ICA sources from generic nonlinear mixing environments [2].was introduced to obtain a better matching of the algorithm The most important issue for generic nonlinear problemsparameters with the pdf of signals. is to ensure the presence of conditions granting the

Recent studies aimed at increasing the complexity of possibility to achieve the desired solution (e.g. [4] and [11] ).mixing models, from single-block nonlinear structures to In this paper the same general idea was applied: addingmulti-block convolutive architectures. In [5] the mixing "soft" constraints (a priori assumptions) to the problem canenvironment was a PNL mixing followed by an produce the uniqueness of the solution. A priori knowledgeinstantaneous mixing (PNL-L), while in [11] a PNL mixing about the mixing model is exploited to design the recoveryblock followed by a convolutive one (PNL-C) was network: the so called "mirror" demixing model is used.considered. Finally in [10] the mixing environment was theconvolutive mixing of a convolutive post-nonlinear mixture III. MIXING-DEMIXING ARCHITECTURES(CPNL-C) and FIR networks were employed. This section describes the mixing (CPNL-C) and

This paper explores the performance of a fully recurrent demixing model. Based on the choice of the mixing CPNL-CIIR architecture in separating sources in the presence of a model (Fig. 1), the mirror model has been chosen asCPNL-C mixing environment. It is important to underline demixing model.

0-7803-9390-2/06/$20.00 ©)2006 IEEE 4563 ISCAS 2006

Ark] F[.] B[k] N N LQs[n] a [k] c[n b [k] x[nJ Yj [n] = ZJhvh [n]+ ,qjh[n]yh[n-m]t1[C [01]] h=1 h=1 m=1h.j

[k]0:^::i:-.'t 1 1''l :;l 5- bis[k]Bvh[n]=gh [rh [n]] (2)

rh [n] , Whxh L[n] + , Y,Pjh [n]rh[n-m]h=1 h=1 m=1

a[k] b, [k] h.j

Figure 1. Block diagram of the convolutive nonlinear mixing model In (2) G[.] is the Nxl vector of nonlinear compensating(CPNL-C model). functions, one for each channel, W, Z, Q[k] and P[k] are

NxN FIR matrices respectively of Lw, Lz, Lp and Lq taps.In Fig. 1 A[k] and B[k] are NxN FIR matrices and Q[k] and P[k] are such that qii[k]=O and p,1[k]=O. TheF[c[n]]=[fH[c,[n]], fA[c, [n]]]T is the Nx 1 vector of transfer matrix of the convolutive block of Figure 2. can benonlinear distorting functions (invertible and strictly expressed as:monotone functions). The CPNL-C mixing environment isexpressed by [ w1i + w21p12[z] W12 + W22P12[z]l

r [z] 1 -p12[Z]p21[Z] 1 - P12[z]P21 [z]I (3)x[n]=F[s]=B[n]*F[A[n]*s[n]] (1) x[z] - w21 +wHp21[z] W22+W21P21[]Z|

1i-P12[Z]p21[z] 1-P12 [z]P21 [z]and represents the convolutive mixing of a convolutive post-nonlinear mixture. Most of mixing environments described Knowledge about the particular kind of mixing model is thein literature can be viewed as particular cases ofthe CPNL-C key to avoid the strict non uniqueness of the solution. Suchmodel. assumption limits the weakness of the output independence

condition, thus reducing the cardinality of all possibleA[k] G[.] B[k] independent output solutions. With this constraint the problem

[n] ............. is.

g,Ai.:Dv *r,[n]7] [i]2 - is empirically confirmed by the attractive results ofX/2 q2 [k]] S[n] / \lXexperimental tests.

W1 P1 = Z21 / W IV. BLIND DEMIXING ALGORITHM AND NETWORK MODEL2

where vi(yi)=Py (yi )/py (yi) are the so called Score The learning rules for each parameter of the setFunctions (SF). In this paper, Spline Neurons are used to () = zu[a],wts [b], Pml [k], qnr [h], Qg,Q } are:perform on-line estimation of both Score FunctionsyV (y, [n]) and nonlinear compensating functions g, (r1 [n]) (for -T Tmore detail see [5]-[7], [11]). The closed form expression of k= - 'Vy [n]spline neuron which represents the ii score function is: 7 +a_3/aQj =-L Tu + Ty (z)j T M

y (ui,Q) =1/2 T1(u1)MQv (5) =/_W-WT _TX[]+

where T (u.p[u.3 Uj2 ui 1] and (O

The "Separation Index" Sj (dB), introduced in [9], gives a results with white signals; the reason of this behavior lie inmeasure of how much channel j-th is separated from the the form of the cost function (7).others; it can be expressed as:

VI. CONCLUSION

F 2 2 ~~~~~~~~~~Thispaper describes a recurrent ICA approach to the2I 2 BSS problem in the CPNL-C mixing environment. TheKi=Io l ,(Y(j),j E jj Yc(j),k (9j proposed algorithm is based on an IIR architecture. On thek#j ~~~~~demixing side, the use of mirror model confirms theexistence and the uniqueness of the solution to the CPNL-C

where y,jis the i-th output signal when only the j-th input separation problem. Results show good separation and goodsi na is pesen and (j)is th outut chnnel

nonlinear compensation. The use of such architecture allowssina is peet adu()i h upt canl for a more compact representation of the recovering modelcorresponding to the j-input. The trend of this index (Figure and grants a more accurate demixing. The recovering4. a-b) confirms the improving of separation during the network performs the on line estimation of both nonlineartraining for both tests. For both tests, Fig. 3 and Fig. 4 show compensating functions and Score Functions by Splinehow the algorithm is successful in performing the separation Neurons leading to a better matching and producing a moreof the output signals. accurate learning.

fima) b)REFERENCES[1] Jutten, C., Karhunen, J., (2003), "Advances in Nonlinear Blind

Sources Separation", 4th International Symposium on ICA and BSS(ICA2003), April 2003, Nara, Japan.

[2] Taleb, A., (2002), "A Generic Framework for Blind SourcesSeparation in Structured Nonlinear Models", In IEEE Trans. on signal

Als l'l -I, " . 1~~~~~~~~~~~prcesig vol. 50. no 8 August 2002.3]Taleb,, A.,, Jutten,, C.,, (1999),, "Sources Separation in post nonlinear

mixtures",, In IEEE Trans. on signal processing,, vol. 47. no 10 August 1999.[4] Hyvarinen,, A.,, Pajunen,, P.,, (1999),, "Nonlinear Independent

Component Analysis: Existence and Uniqueness Results",, NeuralNetworks 12(2): 429-439!,1999.

[5] Solazzi,, M.,, Uncini,, A.,, (2004),, "Spline Neural Networks for BlindSeparation of Post-Nonlinear-Linear Mixtures",, In IEEE Trans. onCircuits and Systems I Fundamental Theory and Applications,, Vol.

Figure 3. a-b-c) voices: joint pdf of original sources,, input mixture and1 N.4 p87-82,Ari204

joint pdf of independent output; d-e-f) white signals: joint pdf of original [6] Uncini,, A.,, Vecci,, L.,, Piazza,, F.,, (1998),, "Learning and approximationsources,, joint pdf of input mixture and joint pdf independent output. capabilities of adaptive Spline activation function neural network", In NN,

Vol. I11, no. 2,, pag. 259-270 March 1998.[7] Milani,, F.,, Solazzi,, M.,, Uncini,, A.,, (2002),, "Blind Source Separation

Separation Index S"arahcm Indexof convolutive nonlinear mixtures by flexible spline nonlinear

I SId~14 functions",, Proc. ofIEEE ICASSP'02,, Orlando,, USA,, May,, 2002.[8] Zade,, M. B.,, Jutten,, C.,, Najeby,, K.,, (2001),, "Blind Separating,,

W ~~~~~~~~~~~~~~~~~~~ConvolutivePost nonlinear Mixture",, ICA 2001 In Proc. of the 3rd12 6' ~~~~~~~~~~~~~~~~Workshopon Independent Component Analysis and Signal Separation

4'~~~~~~~~ I~~~(CA2001I),, San Diego (California,, USA),, 2001I, pp. 138-143.[9] Shobben,, D.,, Torkkola,, K.,, Smaragdis,, P.,, (1999),, "Evaluation of

blind signal separation methods",, In Proc. of ICA and BSS,, Aussois,,France,,January 11-15,, 1999.

w~ iso 2W 2$03m360 4DO w 2oo 2W m [10] Vigliano D.,, Parisi R. and Uncini A.,, (2004)!, "A flexible approach toEpwhs ~ a) Emsb) a novel BSS convolutive nonlinear problem: preliminary result",,

Proc. of "Italian Workshop on Neural Networks (WIRN04)",, Perugia,,Springer-Verlag Ed.,, Sept 2004.

Figure 4. Separation index during the training; a) separation of white [11] Vigliano,, D.,, Parisi R.,, Uncini,, A.,, (2005),, An Information Theoreticsignals,, b) separation of vocal signals. Approach to a Novel Nonlinear Independent Component Analysis

Paradigm,, In Press On Elsevier Signal Processing Special Issue onIn [10] a similar mixing environment was approached by Information Theoretic (2005).

a FIR-based architecture. The separation performances [12] Choi,, S.,, Cichocki A.,, (1997),, "Adaptive Blind Separation of speechobtained with the recurrent algorithm are clearly superior. In signals: Cocktail party problem",, ICSP97,, Seoul,, Korea,, 26-28,, pp.

addition, whiletheFIR matrix requires 15-tap filters, the 6~~~~~.- 17-622, Augs 1997.AfN