Introduction toCharm++ Machine Layer

Gengbin ZhengParallel Programming Lab

4/3/2002

Machine Layer OverviewMachine Layer Overview

Startup and initialization; Low level communication

CmiFreeSendFn, etc CmiSyncBroadcastAllFn(), etc

SMP support Multiple worker threads and one

comm thread; Thread creation and state

initialization;

Supported PlatformsSupported PlatformsCharm version OS Communication Compiler------------------- --------- ------------------- -------------------- net-linux PC Linux UDP/Myrinet GNU compiler net-linux-ia64 IA64 Linux UDP/Myrinet GNU compiler net-axp Alpha UDP GNU compiler net-cygwin Win32/cygwin UDP GNU compiler net-win32 Win32 UDP MS Visual C++ net-sol Solaris UDP GNU compiler net-irix IRIX UDP GNU compiler net-hp HP-UX UDP GNU compiler net-ppc-darwin MacOS X UDP GNU C++

compiler mpi-linux PC Linux MPI GNU compiler mpi-linux-ia64 IA64 Linux MPI GNU compiler mpi-axp Alpha MPI GNU compiler mpi-linux-axp Alpha Linux MPI GNU compiler mpi-origin Origin2000 MPI C++ compiler mpi-sp AIX MPI AIX xlC Compiler origin2000 Origin2000 shared-mem SGI C++ compiler t3e Cray T3E shared-mem Cray C++ compiler

Communication Communication ArchitectureArchitecture

Communication API (Send/Recv)

Net MPI Shmem

UDP(machine-eth.c)

TCP(machine-tcp.c)

Myrinet(machine-gm.c)

Machine Layer Files LayoutMachine Layer Files Layout

Configuration files for each main platform: conv-mach.h conv-common.h conv-mach-opt.h from:

smp, scyld, bluegene, gm, vmi, tcp … cc, cxx, kcc, cc64, acc, ecc …

Implementations: common.net, common.mpi, common.shmem machine.c

How ‘build’ worksHow ‘build’ works

build charm++ net-linux gm smp kcc bluegenebuild charm++ net-linux gm smp kcc bluegene Sort gm, smp and bluegene Mkdir net-linux-bluegene-gm-smp-kcc Cat conv-mach-[kcc|bluegene|gm|smp].h to

conv-mach-opt.h Cat conv-mach-[kcc|bluegene|gm|smp].sh to

conv-mach-opt.sh Gather files from common.net, etc (Makefile) Make charm++ under

net-linux-bluegene-gm-smp-kcc/tmp

Parallel Program StartupParallel Program Startup Net version - nodelist

Charmrun node compute node

Rsh/ssh(IP, port)

my node

(IP, port)

Broadcast all nodes

(IP, port)

compute node

Rsh/ssh(IP, port)

Converse lifeConverse life

ConverseInit Global variables initialization; Start worker threads;

ConverseRunPE for each Charm PE Per thread initialization; Loop into scheduler: CsdScheduler()

Message formatsMessage formats

Net version#define CMK_MSG_HEADER_BASIC { CmiUInt2 d0,d1,d2,d3,d4,d5,hdl,d7; }

Dgram Header length handlerxhandler

MPI version#define CMK_MSG_HEADER_BASIC { CmiUInt2 rank, root, hdl,xhdl,info,d3;

}

infohandlerxhandlerrank root d3

SMP support

MPI-smp as an example Create threads: CmiStartThreads Worker threads work cycle

See code in mahcine-smp.c Communication thread work cycle

See code in machine-smp.c