an initiative towards open network-on-chip benchmarks c. grecu, a. ivanov (univ. of british...

An Initiative towards Open An Initiative towards Open Network-on-Chip Network-on-Chip

BenchmarksBenchmarks

C. Grecu, A. Ivanov (Univ. of British Columbia), C. Grecu, A. Ivanov (Univ. of British Columbia), P. Pande (Washington State P. Pande (Washington State Univ., US), A. Jantsch (Royal Institute of Technology, SE), E. Salminen (Tampere Univ., US), A. Jantsch (Royal Institute of Technology, SE), E. Salminen (Tampere Univ. of Technology, FIN), U. Ogras, R. Marculescu (Carnegie Mellon University, Univ. of Technology, FIN), U. Ogras, R. Marculescu (Carnegie Mellon University,

US)US)

VTS 2007

OutlineOutline

Who: NoC WG Who: NoC WG Why: Objectives and BenefitsWhy: Objectives and Benefits What: Overview of NoC BenchmarksWhat: Overview of NoC Benchmarks

Performance BenchmarksPerformance Benchmarks Test and DFT Benchmarks Test and DFT Benchmarks Fault-Tolerance/Reliability Benchmarks Fault-Tolerance/Reliability Benchmarks

When and Where: When and Where: Agenda (2007)Agenda (2007)

SummarySummary

NoC Challenges and OpportunitiesNoC Challenges and Opportunities

Engineering Knowledge and Practice Engineering Knowledge and Practice – Worldwide, industrial and academicWorldwide, industrial and academic

Quality Quality – Increased product quality and performance at lower costIncreased product quality and performance at lower cost

EvaluationEvaluation– Comparison and tradeoff Comparison and tradeoff

AutomationAutomation– Standards, CAD, support and maintenanceStandards, CAD, support and maintenance

Adoption Adoption – New products, new applicationsNew products, new applications

Deployment Deployment – Manufacturing and salesManufacturing and sales

WhoWho

Academic participants Academic participants (to date)(to date)C. Grecu, A. Ivanov (UBC - Canada) C. Grecu, A. Ivanov (UBC - Canada)

R. Marculescu, U. Ogras (CMU - USA)R. Marculescu, U. Ogras (CMU - USA)

A. Jantsch (KTH - Sweden)A. Jantsch (KTH - Sweden)

P. Pande (WSU - USA)P. Pande (WSU - USA)

E. Salminen, A. Kulmala (TUT - Finland)E. Salminen, A. Kulmala (TUT - Finland)

Industrial Participation Industrial Participation (to date) (to date) SonicsSonics

NokiaNokia

ToshibaToshiba

Administrative/LogisticalAdministrative/LogisticalOCP-IPOCP-IP

Where and When Where and When

Globally Globally

Bi-weekly teleconferenceBi-weekly teleconference

Occasional face-to-face meetingsOccasional face-to-face meetings– E.g., DATEE.g., DATE

NoC Benchmarks – Benefits NoC Benchmarks – Benefits Goals of NoC WG are specifically to Goals of NoC WG are specifically to increaseincrease and and accelerateaccelerate

– Worldwide Worldwide understandingunderstanding of NoC paradigm in industry and academe of NoC paradigm in industry and academeEngineering and scientific development disseminated through technical Engineering and scientific development disseminated through technical literature literature General advancement of knowledge and state of the art and practiceGeneral advancement of knowledge and state of the art and practice

– Quality Quality of NoC solutionsof NoC solutionsEnabling higher performance/low-cost products Enabling higher performance/low-cost products

– Evaluation & comparisonEvaluation & comparison of NoC solutions of NoC solutionsEnabling fair and objective competition (academic and industrial) Enabling fair and objective competition (academic and industrial)

– Methodology development & automationMethodology development & automation (CAD) for NoC solutions (CAD) for NoC solutionsEnabling reproducibility and porting from platform to platform, company to Enabling reproducibility and porting from platform to platform, company to company, company, Open standards developmentOpen standards development

– AdoptionAdoptionFostering management confidence, increase know-how and availability of Fostering management confidence, increase know-how and availability of highly qualified personnel highly qualified personnel

– DeploymentDeploymentAssessing and designing for manufacturability, quality and reliability of NoC-Assessing and designing for manufacturability, quality and reliability of NoC-based productsbased products

NoC Benchmarks – General ObjectivesNoC Benchmarks – General Objectives

Repository of “standard” NoC Information Repository of “standard” NoC Information Representation Format & Design Data Representation Format & Design Data

– Representation FormatRepresentation FormatDescription “rules” and “requirements" Description “rules” and “requirements" Simulation Platforms/Parameters Simulation Platforms/Parameters Metrics and Measurement Methodology Metrics and Measurement Methodology

– Design DataDesign DataDesign specifications (Design specifications (black boxblack box))

– IP cores (nature, number)IP cores (nature, number)

Design specifications (Design specifications (white boxwhite box))– Topology (NoC communication fabric)Topology (NoC communication fabric)

Data Traffic Models/DataData Traffic Models/Data– Synthetic, application specificSynthetic, application specific

Interface informationInterface information

– Above to be captured through “Specifications/Deliverables Document” (in Above to be captured through “Specifications/Deliverables Document” (in progress)progress)

Challenge(s)Challenge(s)

Achieving goals under constraintsAchieving goals under constraints– IP protection/sensitivitiesIP protection/sensitivities

HardwareHardware

ProtocolsProtocols

TrafficTraffic

Application-specificApplication-specific

– RelevanceRelevanceAbstractionAbstraction

Required complexityRequired complexity

Performance BenchmarksPerformance Benchmarks

Benchmark ProgramsBenchmark Programs– Programs or models mimicking real applications Programs or models mimicking real applications

jointly exercising communication architecture (NoC) jointly exercising communication architecture (NoC) interfaced to the processing (computation) interfaced to the processing (computation) platform/elements as well as the system design platform/elements as well as the system design methodologymethodology

– Useful for assessing effectiveness of particular NoC Useful for assessing effectiveness of particular NoC for given application or application domainfor given application or application domain

Micro-BenchmarksMicro-Benchmarks– Abstracted model aimed at exercising only a specific Abstracted model aimed at exercising only a specific

aspect of an NoCaspect of an NoCE.g., routing algorithmsE.g., routing algorithms

Benchmark ProgramsBenchmark Programs

Programs written in C, SystemC, VHDL ….Programs written in C, SystemC, VHDL ….Information providedInformation provided– FunctionalityFunctionality

Application modelApplication modelMapping and scheduling of application tasks to PEsMapping and scheduling of application tasks to PEsSet of models for target PEsSet of models for target PEs

– UsageUsageDirections to connect PEs to NoCDirections to connect PEs to NoCInstructions for configuration and compilation, and executionInstructions for configuration and compilation, and execution

– Topology and Mapping (Optional)Topology and Mapping (Optional)Size and topology of NoCSize and topology of NoCStructure and number of routers Structure and number of routers Resource (PE) bindingResource (PE) binding

Benchmark programs aim at aggregate performance of NoCBenchmark programs aim at aggregate performance of NoC

Performance Benchmarks – Performance Benchmarks – HowHow

Propose to use Propose to use Communication-Centric Application Communication-Centric Application ModelingModeling– Based on Communication Task Graphs (CTGs)Based on Communication Task Graphs (CTGs)

Finite State Machines (FSMs) used to model PEs in real Finite State Machines (FSMs) used to model PEs in real applications applications

– Communication tasks Communication tasks

– Computation tasksComputation tasks

– Allows for effectively hiding proprietary/sensitive Allows for effectively hiding proprietary/sensitive information of specific (real) applications information of specific (real) applications

Communication-Centric Communication-Centric Application Model – Application Model – A pictureA picture

1

4

2

5

3

6

7 8

9

10

11

(25,48) (46,13)

(16,25)(22,36)

(16,25)

(16,25)

(18,40)

(40,14)

(40,14)

(25,22)

(25,22)

(bandwidth, latency)

PEnPE3PE2PE1

NoC communication medium

• application model

• annotated CTG

• application mapping

• tasks onto PEs

• computation architecture

• computation parameters

• communication architecture

• communication parameters and models

Micro-Benchmarks (Micro-Benchmarks (Bs)Bs)

Bs focus on single aspect/parameter of NoC-based Bs focus on single aspect/parameter of NoC-based architecturearchitecture

– PacketsPacketsDelay, latency, bandwidth, jitter, power consumptionDelay, latency, bandwidth, jitter, power consumptionRouting, switching, buffering, flow controlRouting, switching, buffering, flow control

– TransactionsTransactionsPacketization, end to end flow control, streamingPacketization, end to end flow control, streamingProtocol and interface block evaluationProtocol and interface block evaluation

– Traffic EnvironmentTraffic EnvironmentTemporal and spatial traffic distributions Temporal and spatial traffic distributions Congestion, arbitration, buffering, flow control mechanismsCongestion, arbitration, buffering, flow control mechanisms

– QoSQoS““best effort” trafficbest effort” trafficGuaranteed servicesGuaranteed services

– Scalability (network size & topology)Scalability (network size & topology)

Performance Benchmarks Performance Benchmarks

Additional FeaturesAdditional Features– Measurement point and methodology Measurement point and methodology

specificationspecification– Interfaces and Sockets Interfaces and Sockets

Flexibility (built-in through specification format)Flexibility (built-in through specification format)– Core-centric and interconnect agnosticCore-centric and interconnect agnostic– Allow Network Interfaces to deliver standard signals to Allow Network Interfaces to deliver standard signals to

NoC fabricNoC fabric– E.g., OCP, AXI , …E.g., OCP, AXI , …

NoC Test BenchmarksNoC Test Benchmarks

General Objective (“Why”)General Objective (“Why”) To measure efficiency of test methods and DFT To measure efficiency of test methods and DFT

architectures on NoCs-based systemsarchitectures on NoCs-based systems e.g.,e.g.,

TAM designTAM design Test wrapper designTest wrapper design Test schedulingTest scheduling

Test Benchmarks - Test Benchmarks - RequirementsRequirements

Coverage Metrics DefinitionCoverage Metrics Definition– Fault models and setFault models and set

Low-level (stuck-at, open/shorts, crosstalk)Low-level (stuck-at, open/shorts, crosstalk)High-level (mis-routing, data corruption, packet loss)High-level (mis-routing, data corruption, packet loss)

– Test modes supported (off-line, on-line)Test modes supported (off-line, on-line) Test Type DefinitionTest Type Definition

StructuralStructural Fabrication correctnessFabrication correctness

Functional & ParametricFunctional & Parametric Interaction between cores (computation) and data transport infrastructure Interaction between cores (computation) and data transport infrastructure

(communication)(communication)

Captured as integral part of NoC Test BenchmarksCaptured as integral part of NoC Test Benchmarks

Enables Test Methodology Efficiency EvaluationEnables Test Methodology Efficiency Evaluation How: Apply to set of exemplary NoC-based systemsHow: Apply to set of exemplary NoC-based systems What: figure of merit parameters, e.g., test time, power, coverage, silicon area, What: figure of merit parameters, e.g., test time, power, coverage, silicon area,

etc.etc.

NoC Test BenchmarksNoC Test Benchmarks

Input: Standard, reproducible NoC Test CircuitsInput: Standard, reproducible NoC Test Circuits Superset of ITC’02 SoC Test Benchmarks Superset of ITC’02 SoC Test Benchmarks

Format and circuits Format and circuits Augmented/alteredAugmented/altered

NoC fabric(s)NoC fabric(s)Size, etc.Size, etc.

Necessary information of NoC fabric(s)Necessary information of NoC fabric(s) ConnectivityConnectivity

TopologyTopology Components – switches, routers, buffersComponents – switches, routers, buffers

Test-related dataTest-related data Number of scan-chains per routerNumber of scan-chains per router Number and size of buffersNumber and size of buffers Number of tests sessions (per router, channel)Number of tests sessions (per router, channel)

Per test session: number of test patterns, power dissipation, etc.Per test session: number of test patterns, power dissipation, etc.

Test Benchmarks – NoC Fabric DescriptionTest Benchmarks – NoC Fabric Description

XML-basedStructuredPortableFlexible

Test info for Routers Channels (links) Inter-component connectivity: routers & cores

TAM Versatile scenarios

NoC reuse Dedicated Combination of above

NoC Benchmarks for On-line Test & NoC Benchmarks for On-line Test & Fault-ToleranceFault-Tolerance

ObjectivesObjectives On-Line test efficiencyOn-Line test efficiency Fault-tolerance qualities/capabilitiesFault-tolerance qualities/capabilities

Evaluated under the “mission mode” operating conditions Evaluated under the “mission mode” operating conditions of NoC of NoC Direct connection with Performance Benchmarks*Direct connection with Performance Benchmarks*

Required specificationsRequired specifications NoC fabric (hardware) descriptionNoC fabric (hardware) description Operating Conditions (traffic models, application data) Operating Conditions (traffic models, application data) Fault injection Fault injection

* www.ocpip.org/socket/whitepapers/ NoC-Benchmarks-WhitePaper-15.pdf

Benchmarks for On-Line Test & Benchmarks for On-Line Test & Fault-ToleranceFault-Tolerance

Fault Injection Methodology

• Permanent and transient faults

• Fault models

• low-level (stuck-at, open, cross-talk, bit-flip)

• high-level (mis-routing, data corruption, packet loss)

• Assign

• permanent fault probabilities to NoC components

• transient fault probabilities to NoC data

• Account for defect clustering and burst errors

Pf (channel)

Pf (buffer) Pf (logic)

Pf (data)- message, packet, flit, bit -

Benchmarks for Benchmarks for On-Line TestOn-Line Test & & Fault-ToleranceFault-Tolerance

On-line/Fault-Tolerance Quality MetricsOn-line/Fault-Tolerance Quality Metrics PerformancePerformance

Detection (Error/Fault) coverageDetection (Error/Fault) coverage Error/Failure recoveryError/Failure recovery Detection/Recovery timeDetection/Recovery time

Cost/Performance ImpactCost/Performance Impact Si area Si area Complexity Complexity QoS degradationQoS degradation Power dissipationPower dissipation

Agenda -- 2007Agenda -- 2007Q1 Q1 – Public release of White PaperPublic release of White Paper– Recruitment of active industrial participants on WGRecruitment of active industrial participants on WG– Development of Specifications DocumentDevelopment of Specifications Document

Q2Q2– Public disclosures at DATE NoC Workshop, VTS, NoC Symposium …Public disclosures at DATE NoC Workshop, VTS, NoC Symposium …– Completion of Specifications DocumentCompletion of Specifications Document– Initial development and implementation of benchmarksInitial development and implementation of benchmarks

PerformancePerformanceTest and Fault ToleranceTest and Fault Tolerance

Q3Q3– Public release of Specifications DocumentPublic release of Specifications Document– Final development and implementation of Benchmarks (Release #1)Final development and implementation of Benchmarks (Release #1)

Q4 Q4 – 11stst release (beta) of benchmarks release (beta) of benchmarks– Initial evaluation and reports of benchmarksInitial evaluation and reports of benchmarks

SummarySummary

Help WANTED & Welcome!Help WANTED & Welcome!– Opinions/ContributionsOpinions/Contributions

NoC CircuitsNoC Circuits

ProgramsPrograms

Traffic Data/ModelsTraffic Data/Models

ApplicationsApplications

Special acknowledgements to OCP-IPSpecial acknowledgements to OCP-IP

THANK YOU!THANK YOU!