Implementation of Implementation of Infomax ICA Algorithm Infomax ICA Algorithm with Low-Power Analog with Low-Power Analog

CMOS CircuitsCMOS Circuits

Ki-Seok Cho and Soo-Young Lee

Brain Science Research Center and Department of Electrical Engineering and Computer Science

Korea Advanced Institute of Science and Technology

2

Why Analog Infomax-ICA Chip?

Advantages– Massive Parallel Processing– Low power Consumption – Simple Architecture

Related Researches– ICA chip using Herault-Jutten algorithm

• 1992, Cohen & Andreas• Current mode sub-threshold MOS implementation• Convergence is sensitive to the mixing condition

– Infomax ICA algorithm is less sensitive to the mixing condition than Herault-Jutten one

3

Used Algorithm

Instantaneous Mixture– Linear & no delayed mixture

Maximize Entropy with Natural Gradient

–

](U)SUM[WW

WUSUM

WXUT

g

ttt WWW 1

W(u)uIWWW

yW Tφ

)(

TH

4

11W1X

)(2 1u

41W4X

14W

)(2 4u

44W

1U 4U

1sum

4sum

Architecture multiplier

current summation

r learning rate

4 x 4 ICA network in one Chip

5

Test Results in Waveforms Source Signals (s1, s2)

two different male’s voice

16 kHz sampled

Mixed Signals (x1, x2)

Instantaneous mixture

Mixing Mtx A is

Separated Signals (o1, o2)

Recovered original sources

8.07.0

6.08.0A

6

Test Results in SNRsX1 X2 O1 O2

S1 0.8 2.8 -3.63 7.5

S2 3.9 1.5 10.1 -1.5

n

n

nresultnsource

nsourcedBSNR 2

2

][][

][log10)(

source[n] a sampled source signal

result[n] a sampled mixed signal or separated signal

n iteration number

SNRs in dB

7

Fabricated Chip

2.8mm x 2.8mm

AMS CMOS 0.6um

2 poly-3 metal

Analog Digital Hybrid Process

Die Photo of a Fabricated ICA Chip

8

Advantages of Proposed ChipModular Structure: Can be extended to any size network

All-in-One chip: Does not need any DSP nor AD/DA converter

High Speed: Massive parallel processing is possible

1X

)(2 1u

NX

)(2 Nu

1U NU

1sum

Nsum

Chip1 Chip2

Chip N. . .

9

Discussion Some Problems

Did not consider the density of circuits Only for Super-Gaussian source case We could test only 2x2 case because of offsets

Further Works Sub-threshold circuits Can be applied to Sub-Gaussian case Simple network using 1-Q multiplier Need noise-robust circuits

10

Adding Bias & Shifting

(u)w

H(y)w

W(u)uIWWW

H(y)W

wWu

00

TT

o

2

1

2

1

2221

1211

2

1

o

o

w

w

z

z

ww

ww

u

u

z

)tanh()(ˆ

)tanh()(

uu

uu

Adding Bias Wo

For Positive Input z

Shifting of non-linear

Function For Positive u

11

Positive Weight Network

z1

z2

z3

(x1+x2+x3)

u1

u2

u3

(u)

wp11wp21

-

-

wo1

wo3

wo2

)1(

)()()(

211122111

1122111

11221111

zzwwzwz

wwzwz

wwzwzu

popp

popp

o

12

Nonholonomic ICA

gradient norminal with ,wuwuwwuwuw

wuwuwwuwuwtW

ICA icnonholonom with ,wuwu

wuwutW

uf

f

tFttt

yydiagt

tytttt

tWttt

2 2

1 1

2 2

1 1

ji ij

ii

n n

T

)()(

)()()(

)(

0

))(()()()1(

)(

)()())(()()()(

)()()()1(

2221212221211121

2221211221211111

121111

222212

,,11

uWW

WyyW

WW

In the 2 x 2 network Using Nonholonomic ICA

13

o

d4

3

daccbba4

3

d5o

da4

c3

cb

ba1

kI

V

VVVVVVVI

III

VI

VVI

VI

VVI

VVI

exp

)()()(exp

exp

)(exp

exp

)(exp

)(exp

21

21

4

321

4

3

22

1

I1

I2

M1

M2

M3I3

Io

I4

I3 M4

M5

GS

DSG

DSGDD

V

VnVnV

VkT

qV

kT

qV

nkT

qI

L

WI

exp

expexpexp

expexpexp0

Triple Input 1-Q multiplier