ultra-large-scale (uls) systems & their impact on technology & society friday, september 11,...

Ultra-Large-Scale (ULS) Systems Ultra-Large-Scale (ULS) Systems & Their Impact on & Their Impact on

Technology & SocietyTechnology & Society

Friday, April 21, 2023Friday, April 21, 2023

Dr. Douglas C. Schmidt [email protected]

www.dre.vanderbilt.edu/~schmidt

Vanderbilt University Nashville, Tennessee

Institute for Software Integrated

Systems

CPU & network performance has increased by orders of magnitude in past decades

1,200 bits/sec to 10+ Gigabits/sec

The Road Ahead

Extrapolating these trends another decade or so yields• ~10 Gigahertz desktops• ~100 Gigabits/sec LANs• ~100 Megabits/sec wireless• ~10 Terabits/sec Internet backbone

10 Megahertz to 4+ Gigahertz

Unfortunately, software quality & productivity hasn’t improved as

rapidly or predictably as hardware – especially for mission- & safety-

critical systems

Example Mission- & Safety-Critical Systems

Why Hardware Improves So Consistently

Advances in hardware & networks stem largely from R&D on standardized & reusable APIs & protocols

x86 & Power PC chipsets TCP/IP

ApplicationSoftware

ApplicationSoftware

ApplicationSoftware

ApplicationSoftware

Why Software Fails to Improve as ConsistentlyIn general, software has not been as standardized or reusable as hardware, especially for DRE systems

Standard/COTS Hardware & Networks

Proprietary & Stovepiped Application & Infrastructure Software

What Makes Software Hard?

• Organizational impediments

• Economic impediments

• Administrative impediments

• Political impediments

• Psychological impediments

• etc. …

“Human Nature” Technical Complexities

Accidental Complexities

•Low-level APIs & debug tools

•Algorithmic decomposition

Inherent Complexities

•Function & quality property assurance

•Scheduling & synchronization

•Deadlock

•etc. …

www.dre.vanderbilt.edu/~schmidt/reuse-lessons.html

Key Challenges for Software Developers

LogicalLogicalViewView

PhysicalPhysicalViewView

DevelopmentDevelopmentViewView

ProcessProcessViewView

Developers & users of software face challenges in multiple dimensions


Determining units of abstraction for system (de)composition,

reuse, & validation

• Popular technologies & tools provide inadequate support for

• Checking pre-/post-conditions & invariants

• Specifying, analyzing, & optimizing dependencies

• Expressing design intent more clearly using domain concepts

Key Challenge for Software Developers


Our Solution ApproachDevising composable

abstractions whose interfaces & QoS properties can be

specified/analyzed/optimized via metadata

Deployment-Time Fusion

Component

Required Interface Provided Interface Event SinkEvent Source

Physical Assembly

Collocation Group Application AssemblyDeployment Plan

www.dre.vanderbilt.edu/~schmidt/IEEE-RTAS-08.pdf

Total Footprint

18% reduction

45% reduction


Integrating/deploying diverse new & reusable application components in a networked environment to ensure

end-to-end QoS requirements


• Configuring & customizing components for application requirements & run-time environments

• Automated optimized mapping of components onto nodes in target environments


Our Solution Approach


Model-driven development & analysis techniques for optimizing,

verifying, & automating the deployment & configuration process

Gigabit Ethernet

www.dre.vanderbilt.edu/~schmidt/JSS-06.pdf


Devising execution architectures, concurrency models, & communication styles

that ensure multi-dimensional QoS & correctness of new/reusable components


• Identifying & reducing performance & robustness risks earlier in system lifecycle

• Satisfying multiple (often conflicting) QoS demands

• e.g., secure, real-time, reliable

• Satisfying QoS demands in face of fluctuating/insufficient resources

• e.g., mobile ad hoc networks (MANETs)


register

clie

nt

kit

sub

task

req.

sub

tas

k

sub

task

res.

Skoll Server(s)

Configuration Model

Enable Feature{1 = Yes, 0 = NO}CALLBACK

Enable Feature{1 = Yes, 0 = NO}POLLER

run(Multiple/run_test) = 1 → (Compiler = SUNCC5_1)

AMI = 1 → CORBA_MSG = 1

Constraints

Test T runnable{1 = Yes, 0 = NO}run(T)

Enable Feature{1 = Yes, 0 = NO}CORBA_MSG

Enable Feature{1 = Yes, 0 = NO}AMI

Compiler{gcc2.96, SUNCC5_1}Compiler

InterpretationSettingsOption

Enable Feature{1 = Yes, 0 = NO}CALLBACK

Enable Feature{1 = Yes, 0 = NO}POLLER

run(Multiple/run_test) = 1 → (Compiler = SUNCC5_1)

AMI = 1 → CORBA_MSG = 1

Constraints

Test T runnable{1 = Yes, 0 = NO}run(T)

Enable Feature{1 = Yes, 0 = NO}CORBA_MSG

Enable Feature{1 = Yes, 0 = NO}AMI

Compiler{gcc2.96, SUNCC5_1}Compiler

InterpretationSettingsOption

Intelligent Steering Agent(ISA)

Intelligent SteeringAgent

AdaptationStrategies

…

SubtaskResults

ConfigurationModel

ClientCharacteristic

Output: Subtask

Subtask Code

<sub-task><download>

<cvs>cvs.doc.wustl.edu</cvs><module>ACE+TAO</module> <version>v5.2.3</version>

</download>…

</subtask>

Adaptation strategiese.g., nearest neighbor

Automatic characterizatione.g., classification trees

www.dre.vanderbilt.edu/~schmidt/IEEE-TSE-07{a,b}.pdf



Distributed continuous quality assurance techniques run on virtual computing grid

Skoll clients(s)

Visualizatione.g., scoreboardwww.dre.vanderbilt.edu/scoreboard


(De)composing systems into reusable modules (e.g., packages, subsystems,

libraries) that achieve/preserve required QoS properties

• Popular technologies & tools provide inadequate support for avoiding

• Cyclic dependencies, which make unit testing & reuse hard

• Excessive link-time dependencies, which bloat the size of executables

• Excessive compile-time dependencies, where small changes trigger massive recompiles




Feature-oriented specialization optimization tool, e.g., layer folding, deployment platform, memoization,

constant propagation, etc.

Standards-based, General-purpose, Layered Middleware Architecture

Container

ClientOBJREF

in argsoperation()out args +

return

IDLSTUBS

ORBINTERFACE

IDLSKEL

Object Adapter

ORB CORE GIOP/IIOP/ESIOPS

Component(Servant)

Services

ProtocolInterface

ComponentInterface

ServicesInterface

DII

DSI

Average end-to-end measures

improved by ~43%

Customized Middleware Stack

Feature-Oriented CUStomizer (FOCUS)

www.dre.vanderbilt.edu/~schmidt/ACM-EuroSys-06.pdf

New Challenges: Ultra-Large-Scale (ULS) SystemsA ULS System has unprecedented scale in some of these dimensions:

• Lines of code

• Amount of data stored, accessed, manipulated, & refined

• Number of connections & interdependencies

• Number of hardware elements

• Number of computational elements

• Number of system purposes & user perception of these purposes

• Number of routine processes, interactions, & “emergent behaviors”

• Number of (overlapping) policy domains & enforceable mechanisms

• Number of people involved in some way

• …

ULS systems are socio-economic ecosystems comprised of software-intensive systems, people, policies, cultures, & economics

ULS systems report is available from www.sei.cmu.edu/uls

Example Emerging ULS SystemsULS systems are systems of systems at Internet scale

ULS system examples from Linda Northrop’s OOPSLA 2006 keynote talk

Characteristics of ULS systems that arise because of their scale include

• Decentralization

• Inherently conflicting, unknowable, & diverse requirements

• Continuous evolution & deployment

• Heterogeneous, inconsistent, & changing elements

• Erosion of the people/system boundary

• “Normal” failures

• New paradigms for acquisition & policy

Scale Changes Everything

• These characteristics appear in some of today’s systems, but in ULS systems they dominate

• These characteristics undermine the assumptions that underlie today’s technology approaches, rendering incremental solutions inadequate

Serialized Phasing is Common in ULS Systems

Application components developed after

infrastructure is sufficiently mature

Software Lifecycle Timeline

Level of

Ab

stra

ctio

n

System infrastructure components developed

first


Level of

Ab

stra

ctio

n

System integration & testing is performed after application development

is finished

Serialized Phasing is Common in ULS Systems

Integration Surprises!!!

Complexities of Serialized Phasing

Development Timeline

Level of

Ab

stra

ctio

n

Still in development

Ready for testingComplexities

• System infrastructure cannot be tested adequately until applications are done

Complexities of Serialized Phasing

Development Timeline

Level of

Ab

stra

ctio

n

End-to-end performance of

critical path?

System bottleneck?

Complexities

• System infrastructure cannot be tested adequately until applications are done

• Entire system must be deployed & configured (D&C) properly to meet end-to-end QoS requirements

• Existing tools & platforms have poor support for realistic “what if” evaluation

QoS needs of components in ULS systems often unknown until late in lifecycle

Promising Approach for ULS System Challenges:

System Execution Modeling (SEM) Tools

EnsureDesign

Conformance

Express &ValidateDesignRules

Conduct “What If”Analysis

Tools to express & validate design rules

• Help applications & developers adhere to system specifications at design-time

Tools to ensure design rule conformance

• Help properly deploy & configure applications to enforce design rules throughout system lifecycle

Tools to conduct “what if” analysis

• Help analyze QoS concerns prior to completing the entire system, i.e., before system integration phase

SEM tools should be applied continuously when developing software elements

SEM Tool Approach for Planning AspectApproach

• Develop Component Workload Emulator (CoWorkEr) Utilization Test Suite (CUTS) so architects & systems engineers can conduct “what if” analysis on evolving systems by

CUTS integrates nicely with continuous integration servers & test-driven development

3. Tools synthesize emulation code & configuration metadata to deploy system in target environment

4. Metrics are collected & analyzed to explore how design alternatives from multiple computational & valuation perspectives affect system QoS

1. Software architects, developers, & systems engineers specify structure of an enterprise DRE system

2. They associate necessary QoS characteristics with individual components (e.g., CPU utilization) or system as a whole

• Application components are represented as Component Workload Emulators (CoWorkErs)

• CoWorkErs can be interconnected by the PICML tool to form operational strings

• CoWorkErs are programmed using the Component Behavior Modeling Language (CBML)


Level of

Ab

stra

ctio

n

www.cs.wustl.edu/~schmidt/PDF/CUTS.pdfwww.dre.vanderbilt.edu/~schmidt/IEEE-RTCSA-06.pdf

Emulating Computational Components in CUTS

• Workload Modeling Language (WML) is used to parameterize CBML actions with realistic & executable workload



Level of

Ab

stra

ctio

n

High-level Overview of WML’s

Modeling Structure

High-level Overview of WML’s

Modeling Structure

• Workload Modeling Language (WML) is used to parameterize CBML actions with realistic & executable workload

• Fully parameterized models are used to generate emulation code for the target architecture

• i.e, spec. compliant code



Level of

Ab

stra

ctio

n

www.cs.wustl.edu/~schmidt/PDF/QoSPML-WML.pdfwww.dre.vanderbilt.edu/~schmidt/WML-book-chapter.pdf

Software Lifecycel Timeline

Level of

Ab

stra

ctio

n

• Benchmark Manager Web (BMW) Utility provides analytical capabilities for CUTS

• e.g., showing performance of actions in each CoWorkEr, or evaluating non-functional concerns of mission critical operational strings

Analyzing Systemic Performance in CUTS

www.dre.vanderbilt.edu/~schmidt/IEEE-ECBS-2008.pdf

Concluding Remarks• The emergence of ULS systems

requires significant innovations & advances in tools & platforms

• Not all technologies provide the precision we’re accustomed to in legacy small-scale systems

• Advances in computing technology & related disciplines needed to address ULS systems challenges

• Significant groundwork layed in various R&D programs

• Much more research needed for ULS systems

• Currently working with SEI, OSD, & NSF on ULS systems R&D

ultra-large-scale (uls) systems & their impact on technology & society friday, september 11,...

Documents