generic support for policy-based self-adaptive systems (introducing agile) richard john anthony...

Generic Support for Policy-Based Self-Adaptive Systems

(Introducing AGILE)

Richard John AnthonyUniversity of [email protected]

www.PolicyAutonomics.net

Introduction

This work is towards a generic policy toolkit to support the implementation of autonomic computing systems in which the policies themselves can be adapted dynamically and automatically.

A generic solution must:

• Be flexible to enable a diverse set of behaviours to be defined;

• Be sufficiently expressive to capture simple and complex business logic;

• Support re-use of behaviour to reduce policy development time;

• Be accessible to non autonomics-expert software developers working in a diverse range of application fields;

• Facilitate integration with legacy applications.

Richard Anthony University of Greenwich SAACS 2006 1

Current State-of-Practice in Policy-based Autonomics

Three distinct levels of sophistication are found in current approaches:

1. Policy rules are statically embedded. The template configuration is exposed and can be modified between executions.

2. Policy updates are open-loop. An external entity (usually a human) identifies, or is alerted to, potential configuration optimisations, which can be applied between executions.

3. Closed-loop adaptation, where the policy dynamically and automatically adapts its own rule-base or template settings during execution. (Few schemes target or achieve this level at present).


(a) P (b) T P

T P(c)

TTT P(d)

M

PPP P(e)

M

T P(f)

P

MT

TTT

PPP

Manual configuration Policy (rule based) Library of templates

Template configuration Meta policy Library of policies

Direct configuration / control Controlled system Persisted state

Feedback / monitoring / update State propagation

Pictographic representation of policy life-cycles found in current practice


Extending the state of the art

The new policy language and library support self-adaptive policy behaviour:

The policy is able to adapt its own behaviour as well as adapting thecontrolled system.

This gives rise to a number of envisioned policy-system configurationsbeyond current practice.


T P(g)

T P(h)

PPP P(i)

M

TTTPPP P(j)

MT

P

MT

TTT

PPP

Manual configuration Policy (rule based) Library of templates

Template configuration Meta policy Library of policies

Direct configuration / control Controlled system Persisted state

Feedback / monitoring / update State propagation

Advanced policy configurations beyond the current state of practice


Differential between existing and proposed approaches (V. generalised)

• Some governance delegated to a meta-policy.• Dynamic reconfiguration at the level of policies, and their components such as rules.

Controlledcomponent

Manager

Policy

Controlledcomponent

Manager

Policy

Meta-Policy

State ofPractice ?

Proposed /Targeted PolicyPolicy


The Policy Expression Language

Extends the state of the art in policy languages:

Explicit support for dynamic self-adaptation of policies over both shortand long term through persisted configuration changes.

Policy suites, in which a particular type of policy can have manydifferently-geared instances (e.g. cautious and aggressive versions).

Suites of templates for a given policy-type allow different initial configurations to be used, depending on circumstances.

Meta-policies can be used to select amongst many policy-instances and templates at run-time.

Features to support bounded behaviour and enhance stability.

Facilitates policy-object reuse.


Overview of the policy expression language features (1)

OO approach facilitates re-use of behaviour, through reusing tested objects.

A polymorphic Variable-type (PolicyObject) facilitates referencing of policy objects within the policy rules and actions themselves.

Hierarchical support for three levels of ‘policy’:

1. Templates provide configuration parameters which are used to initialise the other types of policy.

2. Normal policies contain the low-level autonomics logic.

3. Meta-policies provide higher-level adaptation.

A single policy script may contain all three types of ‘policy’.

Meta-policies support ‘hot-swapping’ between normal policy instances.


The Policy Library

Easy integration with legacy code:Provides internal support for many powerful language-support features. Clear demarcation between internally managed and externally maintained aspects of policy behaviour.

Supports use by non-autonomics-experts: Simplifies mapping of business logic into policy specifications, through features such as:

• Powerful library-internal validation and semantic checking,• Distinction between internal and external variables,• Automatic restriction of low-level adaptation within higher-level expressed bounds,• Silent conflict / violation handling.


Integrating AGILE with applications (New-build or Legacy Code)


Application / Component

AGILERun-timeLibrary

Load Policy Script

Load Template(s)

Persist State

Save run-time trace

Pass in environmental values

Read return values

Case Example: A multi-policy stock trading scheme (1)

Conforms to configuration (i) in the policy taxonomy (beyond current state of practice).

Illustrates the use of a meta-policy and the advantage of dynamic switching between instances of business policies.

The stock system is representative of many real-world problems that have:Highly complex behaviour,Sensitivity to several sources of environmental volatility,Non-linear sensitivity to some environmental parameters, Many dimensions of freedom.

For a system with any of these characteristics, it is not realistic to closely track the system over its entire behaviour range with a single policy.



Trading decisions are subject to influences such as price behaviour trends, the rate of change in traded volumes and the rate of change in price, etc.

A single, generalised (and thus coarse) policy could lead to significant sub-optimality across a wide spectrum of behaviours.

One solution is to divide the application behaviour space into several zones.

Each zone requires a stock-price-based trading strategy, contextualised by a given range of rate of change in traded volumes and in price.

Zone # 2

Zone # 3Zone # 1

Zone # 4

Rate of Volume Traded change

Rate of Price

change

The policies have different configuration but are of the same ‘type’ since theyaddress the same business decision (but arrive at their decisions differently).


PolicymaintainedthresholdsPolicy

Time

Price PolicymaintainedthresholdsPolicy

Time

Price


Each policy is configured such that its operational envelope (that over which it guarantees safe and desirable behaviour) is tuned to map closely onto its ‘zone’.

A self-adaptive policy may adjust its own thresholds over time, or these might be supplied externally.



Individual policies need not be aware of the delimitation of their zones.

A MetaPolicy is employed to select between the different business policies.

The MetaPolicy monitors the behaviour of the target system and determines which policy should be employed at any given moment.

Policymaintainedthresholds

Policy

Time

Price

MetaPolicy

Policymaintainedthresholds

Policy

Time

Price

MetaPolicy



There is a risk of instability if simple cut-off points are used to determine the policy selection.

If the behaviour of the monitored system is close to one of the cut-offs there isa possibility that the policies will be switched to and fro, causing extra work forthe system.

The ToleranceRangeCheck language object facilitates easy configuration of dead-zones, enhancing stability (localised oscillatory behaviour in the target system is not mirrored in the behaviour of the MetaPolicy).

Policy # 2

Policy # 3Policy # 1

Policy # 4

Rate of ‘Volume Traded’ change

Rate of ‘Price’ change

Price rate-changedead-zone

Volume rate-change dead-zone

Policy # 2

Policy # 3Policy # 1

Policy # 4

Rate of ‘Volume Traded’ change

Rate of ‘Price’ change

Price rate-changedead-zone

Volume rate-change dead-zone


Stock-trading multi-policy exampleScript part 1

<PolicySuite PolicyType="StockTradingPolicySuite">

<EnvironmentVariables><EVariable Name="E_P_RateChange" Type="long"/><EVariable Name="E_V_RateChange" Type="long"/><EVariable Name="E_ExpectedFuturePrice" Type="long"/><EVariable Name="E_CurrentPrice" Type="long"/>

</EnvironmentVariables><InternalVariables>

<IVariable Name="Previous_P_ZoneLOW" Type="bool"/><IVariable Name="Previous_V_ZoneLOW" Type="bool"/><IVariable Name="Current_P_ZoneLOW" Type="bool"/><IVariable Name="Current_V_ZoneLOW" Type="bool"/><IVariable Name="P_DZ" Type="long"/><IVariable Name="V_DZ" Type="long"/><IVariable Name="DZ_Tolerance" Type="long"/><IVariable Name="Margin" Type="long"/><IVariable Name="Cautious" Type="bool"/><IVariable Name="I_CautiousAction" Type="PolicyObject"/><IVariable Name="I_AggressiveAction" Type="PolicyObject"/>

</InternalVariables><Templates>

<Template Name="T_MetaTemplate"><Assign Variable="Previous_P_ZoneLOW" Value="true"/><Assign Variable="Previous_V_ZoneLOW" Value="true"/><Assign Variable="P_DZ" Value="60"/><Assign Variable="V_DZ" Value="50"/><Assign Variable="DZ_Tolerance" Value="10"/>

</Template></Templates> Richard Anthony University of Greenwich SAACS 2006 16

<ReturnValues><ReturnValue Name="Hold" Value="1"/><ReturnValue Name="Buy_Cautious" Value="2"/><ReturnValue Name="Buy_Aggressive" Value="3"/><ReturnValue Name="Sell_Cautious" Value="4"/><ReturnValue Name="Sell_Aggressive" Value="5"/>

</ReturnValues><Actions>

<Action Name="A_M1"><EvaluateTRC TRC="C_P_DZ"/><EvaluateTRC TRC="C_V_DZ"/><EvaluateRule Rule="R_M1"/><EvaluateRule Rule="R_M2"/><EvaluateRule Rule="R_M3"/><Yield Policy="Policy4"/>

</Action><Action Name="A_Set_P_Previous">

<Assign LHS="Current_P_ZoneLOW" RHS="Previous_P_ZoneLOW"/></Action><Action Name="A_Set_V_Previous">

<Assign LHS="Current_V_ZoneLOW" RHS="Previous_V_ZoneLOW"/></Action><Action Name="A_Set_P_LOW">

<Assign LHS="Current_P_ZoneLOW" RHS="true"/> <Assign LHS="Previous_P_ZoneLOW" RHS="Current_P_ZoneLOW"/></Action><Action Name="A_Set_V_LOW">

<Assign LHS="Current_V_ZoneLOW" RHS="true"/><Assign LHS="Previous_V_ZoneLOW" RHS="Current_V_ZoneLOW"/>

</Action>Richard Anthony University of Greenwich SAACS 2006 17

Script part 2

<Action Name="A_Set_P_HIGH"><Assign LHS="Current_P_ZoneLOW" RHS="false"/><Assign LHS="Previous_P_ZoneLOW"

RHS="Current_P_ZoneLOW"/></Action><Action Name="A_Set_V_HIGH">

<Assign LHS="Current_V_ZoneLOW" RHS="false"/><Assign LHS="Previous_V_ZoneLOW"

RHS="Current_V_ZoneLOW"/></Action><Action Name="A_Policy1">

<Yield Policy="Policy1"/></Action><Action Name="A_Policy2">

<Yield Policy="Policy2"/></Action><Action Name="A_Policy3">

<Yield Policy="Policy3"/></Action><Action Name="A_Trade1">

<Assign LHS="Margin" RHS="40"/><Assign LHS="Cautious" RHS="true"/><EvaluateTRC TRC="C_Trade"/>

</Action><Action Name="A_Trade2">

<Assign LHS="Margin" RHS="15"/><Assign LHS="Cautious" RHS="true"/><EvaluateTRC TRC="C_Trade"/>

</Action><Action Name="A_Trade3">

<Assign LHS="Margin" RHS="50"/><Assign LHS="Cautious" RHS="false"/><EvaluateTRC TRC="C_Trade"/>

</Action>


Script part 3

<Action Name="A_Trade4"><Assign LHS="Margin" RHS="20"/><Assign LHS="Cautious" RHS="false"/><EvaluateTRC TRC="C_Trade"/>

</Action><Action Name="A_Hold">

<Return ReturnValue="Hold"/></Action><Action Name="A_Buy">

<Assign LHS="I_CautiousAction" RHS="A_BuyCautious"/><Assign LHS="I_AggressiveAction"

RHS="A_BuyAggressive"/><EvaluateRule Rule="R_Mode"/>

</Action><Action Name="A_Sell">

<Assign LHS="I_CautiousAction" RHS="A_SellCautious"/><Assign LHS="I_AggressiveAction"

RHS="A_SellAggressive"/><EvaluateRule Rule="R_Mode"/>

</Action><Action Name="A_BuyCautious">

<Return ReturnValue="Buy_Cautious"/></Action><Action Name="A_SellCautious">

<Return ReturnValue="Sell_Cautious"/></Action><Action Name="A_BuyAggressive">

<Return ReturnValue="Buy_Aggressive"/></Action><Action Name="A_SellAggressive">

<Return ReturnValue="Sell_Aggressive"/></Action>

</Actions>Richard Anthony University of Greenwich SAACS 2006 19

Script part 4

<Rules><Rule Name="R_M1" LHS="Current_P_ZoneLOW" Op="EQ" RHS="true" LOp="AND"

LH2="Current_V_ZoneLOW" Op2="EQ" RH2="true" ActionIfTrue="A_Policy1"ElseAction="Null"/>

<Rule Name="R_M2" LHS="Current_P_ZoneLOW" Op="EQ" RHS="false"LOp="AND" LH2="Current_V_ZoneLOW" Op2="EQ" RH2="true"ActionIfTrue="A_Policy2" ElseAction="Null"/>

<Rule Name="R_M3" LHS="Current_P_ZoneLOW" Op="EQ" RHS="true" LOp="AND"LH2="Current_V_ZoneLOW" Op2="EQ" RH2="false" ActionIfTrue="A_Policy3"ElseAction="Null"/>

<Rule Name="R_Mode" LHS="Cautious" Op="EQ" RHS="true"ActionIfTrue="I_CautiousAction" ElseAction="I_AggressiveAction"/>

</Rules>

<ToleranceRangeChecks><TRC Name="C_P_DZ" Check="E_P_RateChange" Compare="P_DZ"

Tolerance="DZ_Tolerance" ActionInZone="A_Set_P_Previous"ActionLower="A_Set_P_LOW" ActionHigher="A_Set_P_HIGH"/>

<TRC Name="C_V_DZ" Check="E_V_RateChange" Compare="V_DZ"Tolerance="DZ_Tolerance" ActionInZone="A_Set_V_Previous"ActionLower="A_Set_V_LOW" ActionHigher="A_Set_V_HIGH"/>

<TRC Name="C_Trade" Check="E_CurrentPrice" Compare="E_ExpectedFuturePrice"

Tolerance="Margin" ActionInZone="A_Hold" ActionLower="A_Buy"ActionHigher="A_Sell"/>

</ToleranceRangeChecks> Richard Anthony University of Greenwich SAACS 2006 20

Script part 5

<Policies><Policy Name="HighLevelPolicy"

PolicyType="MetaPolicy"><Load Template="T_MetaTemplate"/><Execute Action="A_M1"/>

</Policy><Policy Name="Policy1" PolicyType="NormalPolicy">

<Execute Action="A_Trade1"/></Policy><Policy Name="Policy2" PolicyType="NormalPolicy">



<Execute Action="A_Trade4"/></Policy>

</Policies></PolicySuite>


Script part 6

AGILE V1.1 Run-Time Log

Policy: HighLevelPolicy Type: MetaPolicy Load Template - Skipped (already loaded)Action: A_M1ToleranceRangeCheck: C_P_DZ Guard was True: TRC evaluated

Variable: E_P_RateChangeValue: 57Variable: P_DZ Value: 60Variable: DZ_Tolerance Value: 10

TRC evaluated In-Zone: Action=A_Set_P_PreviousAction: A_Set_P_Previous Assign: LHS=Current_P_ZoneLOW RHS=Previous_P_ZoneLOWToleranceRangeCheck: C_V_DZ Guard was True: TRC evaluated

Variable: E_V_RateChangeValue: 40Variable: V_DZ Value: 50Variable: DZ_Tolerance Value: 10

TRC evaluated Outside-Lower-Zone: Action=A_Set_V_LOWAction: A_Set_V_LOW Assign: LHS=Current_V_ZoneLOW RHS=true Assign: LHS=Previous_V_ZoneLOW RHS=Current_V_ZoneLOWRule: R_M1 Guard was True: Rule evaluated

Variable: Current_P_ZoneLOW Value: false Variable: true Expression 'Current_P_ZoneLOW EQ true' was False

Variable: Current_V_ZoneLOW Value: true Variable: true Expression 'Current_V_ZoneLOW EQ true' was True

Rule evaluated False: Action=Null


Stock-trading multi-policy exampleRun-time log - part 1

Rule: R_M2 Guard was True: Rule evaluatedVariable: Current_P_ZoneLOW Value: false Variable: false

Expression 'Current_P_ZoneLOW EQ false' was TrueVariable: Current_V_ZoneLOW Value: true Variable: true

Expression 'Current_V_ZoneLOW EQ true' was True Rule evaluated True: Action=A_Policy2

Action: A_Policy2 Yield: Policy: Policy2 Type: NormalPolicy

Action: A_Trade2 Assign: LHS=Margin RHS=15 Assign: LHS=Cautious RHS=trueToleranceRangeCheck: C_Trade Guard was True: TRC evaluated

Variable: E_CurrentPrice Value: 78Variable: E_ExpectedFuturePrice Value: 95Variable: Margin Value: 15

TRC evaluated Outside-Lower-Zone: Action=A_BuyAction: A_Buy Assign: LHS=I_CautiousAction RHS=A_BuyCautious Assign: LHS=I_AggressiveAction RHS=A_BuyAggressiveRule: R_Mode Guard was True: Rule evaluated

Variable: Cautious Value: true Variable: true Expression 'Cautious EQ true' was True Rule evaluated True: Action=A_BuyCautiousAction: A_BuyCautious

ReturnValue: Buy_Cautious Richard Anthony University of Greenwich SAACS 2006 23

Run-time log - part 2

Conclusion

The policy language supports several dimensions of adaptability:

- Low-level changes enforced by a particular policy;

- Dynamic adjustments to internal state variables (e.g. including the parameters that define the position and width of dead-zones);

- Automatic switching of policies to ensure that the current policy always remains within its optimal operational envelope;

- Longer-term adaptation by persisting modified state in the form of dynamically created templates.

Support for dead-zones enables stability despite potentially high adaptability.

The library mechanisms automatically enforce adherence to constraints such as maximum value for a given variable (these attributes are themselves dynamically re-configurable).

The library facilitates integration into legacy code - many applications in current use need self-management, without complete re-development.


Further work …

Further development of language specification• Support for utility functions and fuzzy comparisons are planned,• Richer attribute sets on objects.

Further development of library and interface• Flexible persistence support,• An instrumentation interface for external monitoring.

Development of a suite of demonstrator / validator applications

AGILE Library and documentation available at: www.PolicyAutonomics.net


generic support for policy-based self-adaptive systems (introducing agile) richard john anthony...

Documents

p p p p e

t t t p p p p j

t p f p

t p g t p h p p p p

p b t p t p c t t t

policy rules

policy languages

policy updates