1 © 2005 nokia v1-filename.ppt / yyyy-mm-dd / initials dspgateway architecture in detail yoshiya...

1 © 2005 Nokia V1-Filename.ppt / yyyy-mm-dd / Initials

DSPGateway Architecture in Detail

Yoshiya Hirase

Nokia Research Center

Nokia Japan Co. Ltd.

2 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Agenda

• DSP Gateway Features

• Demonstration Using Nokia 770


DSP Gateway Features

• Open Source Software‐ Integrated in Nokia 770 Internet Tablet

• Simple interface for DSP• Efficient inter-processor communication (ARM <-> DSP)• Block data transfer via shared memory

• DSP Task management‐ Dynamic Loader

• Fault-tolerant mechanism

• Power management


History

• 2003/6: Released the DSP Gateway version 1.0 for OMAP1510 at SourceForge

• 2004/8: Supported OMAP1610/5912

• 2004/9: Merged into the Linux OMAP tree

• 2005/2H: Nokia 770 released with DSP Gateway

The latest DSP Gateway ver 3.3 supports OMAP 15XX (including 5910) and 16XX (including 1710 and 5912).


DSP Interface: Device Files

• DSP task devicesinterface to DSP task

• DSP control devicemiscellaneous DSP control

(e.g. DSP configuration)

• DSP memory deviceDSP memory access

• DSP task watch devicefor dynamic loader

• DSP error detection deviceerror info delivery

DSP ARM(Linux)

Mailbox

“task1” “task2” “task3”

/de

v/dspta

sk/ ta

sk1

/de

v/dspta

sk/ ta

sk2

/de

v/dspta

sk/ ta

sk3

user app.

dspctl utility

/de

v/dspctl/

ctl

/de

v/dspctl/

me

m

INT handler

DSP Gateway library

DSP memory

device I/F

DSP control

device I/F

DSP task device I/F

DSP Gateway driver

INT handler

DSP/ BIOS TASK

DSP/ BIOS TASK

DSP/ BIOS TASK

task API

DSP task

watch device I/F

/de

v/dspctl/

twch

Dynamic Loader Daemon

/de

v/dspctl/

err

DSP error

detection device I/F


Inter-Processor Communication

• Communications via OMAP mailbox mechanism

(16bit + 16bit)

Mailbox registersARM

OMAP

(16bit + 16bit)

memory

IPBUF

interrupt

DSP

interrupt

IPBUFIPBUF

internal memory


Inter-Processor Communication: IPBUF

Global IPBUF:‐ Used to transfer block data to/from DSP

task

– Prepared by the system and managed with alloc & free functions

– Fixed block size, restricted block count

Private IPBUF:‐ Used to transfer block data to/from DSP

task

– A task can define an exclusive buffer, with arbitrary size

System IPBUF:‐ Used to transfer extra data between

systems of ARM and DSP

Global IPBUF

Private IPBUF

System IPBUF

1 2 3 4

ARM

(alloc)

DSP

1 task A

ARM DSP

task B Privat

e IPBUF

task B

task B

(free)

Global IPBUF

task A

ARM DSP

system systemSystem IPBUF


Inter-Processor Communication:

Mailbox commands


Memory Mapping

• DSP internal memory‐ DARAM, SARAM

• External memory‐ SDRAM

SDRAM

00_0000 DARAM SARAM

PDROM DSP Memory Space

Linux Virtual Address Space

0000_0000

C000_0000

Physical Memory

E000_0000

SARAM DARAM

E100_0000 FF_FFFF

DSP space shadow area

Kernel memory area MPU MMU

DSP MMU


Task Management: Dynamic Loader


Task Management: DSP Application Life Cycle

DSP Gateway

process 1

open()

write()

read()

close()

Linux application Device driver DSP kernel

open() return

open()

DSP application

activate DSP task

complete

write()input data / control command

output dataread()

close()

deactivate DSP task

complete

close() return

ARM DSP

DSP task dynamically

created

DSP task dynamically

deleted

write() return

read() return

task running


int fd_mp3file, fd_dsp;

DATA buf[BUFSZ];

int Decode_MP3(void)

{

// Open MP3 decoder device

fd_dsp = open(“/dev/dsptask/mp3dec”, O_RDWR);

// Initialize MP3 decoder

ioctl(fd, MP3_DECODE_INIT);

// Decode MP3 audio file

while (ReadMp3File(fd_mp3file, buf, BUFSZ) != EOF) {

// Send encoded data to MP3 decoder in DSP

write(fd_dsp, buf, BUFSZ);

}

// Close MP3 decoder device

close(fd_dsp);

}

MP3 Pseudo program (ARM side)

Issue TCTL command to

DSP

Issue BKSND command to

DSP


MP3 Pseudo program (DSP side)Uns rcv_tctl(struct dsptask *task, Uns ctlcmd, Uns *ret, Uns arg)

{

switch (ctlcmd) {

case MP3_DECODE_INIT:

InitIMDCT();

InitFilters();

break;

...

}

Uns rcv_bksnd(struct dsptask *task, Uns bid, Uns cnt)

{

/* Decode received MP3 data */

Dequantize();

ReduceAlias();

IMDCT();

...

/* Send decoded PCM data to the Audio Device Driver */

...

}

Handle MP3_DECODE_I

NIT of TCTL command

Handle BKSND command


Nokia 770 Internet Tablet Features

• Display‐ High-resolution (800x480) touch screen with up to 65,536 colors

• Memory‐ Engine: DDR RAM 64MB‐ Memory: Flash 128MB (>64MB for user)‐ Memory card: 64MB RS-MMC (Reduced Size - MultiMediaCard)

• Supported file formats‐ Audio: MP3, Real Audio, AAC, WAV, AMR‐ Image: JPEG, GIF, BMP, TIFF, PNG, Animated GIF format, SVG-tiny, ICO‐ Video: MPEG1, MPEG4, Real Video, H.263, 3GP

(RED items are decoded on DSP, BLUE items are routed through DSP)• Connectivity

‐ WLAN: 802.11b/g‐ Bluetooth specification: 1.2‐ USB device mode for PC connectivity‐ RS-MMC‐ 3.5 mm stereo audio out


Multimedia Software Architecture in Nokia 770

ARM DSP

Linux DSP/BIOS

GStreamerPlugin #1

(Codec)

Plugin #2

Plugin #3

Codec #2

Codec #3

DSP Gateway

Multimedia Player


Future Work

• OMAP2420 support (including IVA support)

• Driver Abstraction, and expansion for generic AMC (Asymmetric Multi-Core) processors


Demo: Play Media Files using DSP Gateway

•System log message

•Device status

•System log message

•Device status

Play sound or movie

Play sound or movie


References:

• Project page:‐ http://sourceforge.net/projects/dspgateway

• Articles:‐ 森　英悟 , 小林　俊裕 , 高橋　清隆 . “オープン・ソース・ソフトウェア

DSPGatewayを用いた Linuxで使う OMAP DSP部のソフトウェア開発環境 .” Interface: CQ Publishing, May 2004, 156-164

http://www.cqpub.co.jp/interface/contents/2004/200405.htm


Thank you


Appendix: File manager Image


Appendix: Audio Player Image


Appendix: BKSND command in log


Appendix: TDEL command in log


Appendix: Task Status Transition


Appendix: Log message while playing MP3


Appendix: Performance Evaluation Example

• Memory mapping improves data transfer well‐ Data send latency (to DSP) by 44%-68%‐ Data receive latency (from DSP) by 27-55%

MPU bksnd := MPU sends a data blocks to DSP

MPUbkreq_DSPbksnd := MPU sends a block request to DSP -> DSP receives it & then sends the block back -> MPU receives the data block from DSP

Memory write/read := MPU writes/reads data on the mapped memory

1 © 2005 nokia v1-filename.ppt / yyyy-mm-dd / initials dspgateway architecture in detail yoshiya...

Documents

created dsp task

nrctokyo dsp interface

arm alloc dsp

dsp gateway version

block data tofrom dsp

dsp data bufbufsz int

latest dsp gateway ver

return task