Helper Threads via Virtual Multithreading on an experimental

Itanium 2 processor platform.

Perry H Wang et. Al.

Outline

Helper threads VMT ideas Implementation details

• Hardware

• Firmware

• Compiler Results Conclusion.

Helper threads

Used in Multi-threaded architectures to prefetch hard-to-predict delinquent data or compute hard-to-predict branches.

Threads share resources as fetch bandwith and functional units.

Hyper Threading (Intel - P4)

Each hardware thread context is exposed as logical processor to the OS.

OS finds threads for execution and binds them to the logical processor.

User has to use OS-visible thread API to create and manage threads.

Helper Threads - Issues

Resource contention among multiple helper threads

Adaptable invocation for different program phases.• Threads have to be self-throttling.

OS based thread synchronization is unpredictable and has long latency ( ~micro secs)

Virtual Multithreading

Single processor supports multiple thread contexts. Monitors long latency micro-architectural events. Switches to different Instruction in same program

in 100 cycles.

OS transparent Uses firmware support in Itanium 2 processor to

reduce context switch time.

Context switch requires

Advantages with VMT on Itanium

Ability to track micro-architectural events without involvement of the OS. Eg: Last level cache misses.

Large register set partitioned by compiler for helper threads• Register communication is easier

• Value Synchronization - no memory comm.

OS context switches allow threads to be resumed on any processor.

New Instructions

YieldSynchronous transfer to

VMT thread, similar to branch misprediction

Yield conditionalTransfer only when

pipeline stalls at some later instruction.

Execution proceed, instructions retire

No pipeline stall instruction behaves as nop

Key Characteristics

Self throttling – main and helper threads keep counters to track progress (iteration counter)• Helper thread falls behind -> reload value

• Helper thread runs too far ahead -> relinquish ctrl.

Main thread begins execution at instruction that triggered helper thread invocation.

VMT preserves thread continuation of helper threads -> helper thread can restart where it stopped.

Key Characteristics

VMT has to maintain• Initial instruction address.

• Continuation instruction address

Compiler preserves 2 registers for the purpose.

Support for multiple helper threads can be done by reprogramming these registers.

Itanium Firmware

Programmable debugging hardware support for PAL• To enable silicon debugging

& validation. PAL can program PMU to

monitor and count events of interest - opcode monitoring, instruction addr., Data addr.

Debugging hardware can trigger a PAL handler when the monitored even occurs.

Firmware

VMT mechanism emulated by firmware infrastructure. Opcode monitoring to simulate yield and yield conditional. PAL programs PMU to track

• Last level cache misses

• Pipeline stalls

• Instructions with special opcodes. Thread switch latency = pipeline flush + overhead

for manipulating registers. (~140 cycles giving 60 cycles of computation time when memory miss ~200 cycles)

Experimental machine.

4 way 1.5 Ghz Itanium 2 processor based MP system with 16 GB of RAM.

Separate 16 KB 4-way set associate L1 I- and D-cache

Shared 256 KB 8-way set associative L2 cache. 6 MB 24-way L3 cache that can be configured as 1

MB 4-way set associative cache.

workloads

MCF – combinatorial optimization.

VPR – FPGA Circuit Placement and Routing

DOT – graph layout optimization tool

DSS system running on 100 GB IBM DB2 database

6 queries with long run time and span large portions of database.

95% cpu utilization, 40 concurrent threads

Compiler and optimizations

Electon –O3, IPA, Profile guided opt. , Itanium2 specific opt.

Recompiled to obtain threads and linked with original binaries.• Register partitioning to minimize VMT context switch

• Aggressive software prefetching with profile feedback.

• Ld.s , chk, predication, branch prediction hints.

SpeedUp A few helper threads give good

speedup Significant fraction of L3

misses are removed from main thread (avg. 48%). Capacity misses in L3 are due to pointer chasing.

Helper thread size is small. Helper thread can contain

control flow dependencies also. Throughput improved by

reducing latency of individual threads

Conclusion

Fly-weight context switching 5.8 – 38.5% increase for SPEC2000 INT 5-12% speedup on DSS workload. VMT threads are to be invoked based on

program behavior depending on number of cache misses.

My view on limitations

Requires large register files and firmware support. Too Itanium (not adaptable to other architectures).

Scalability of helper threads.(# helper threads running at one time…to complex)