impact 2012 1640 - bpm design considerations when optimizing business processes to automated
TRANSCRIPT
1
Design Considerations When Optimizing Business Processes to Automated Kim Clark, Senior IT Specialist, IBM Brian Petrini, Senior IT Architect, IBM
1640
2
Please Note
IBM's statements regarding its plans, directions, and intent are subject to change or withdrawal at IBM's sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.
IBM Software Group
Agenda § What do we mean by Progressive Process Optimization?
§ What to we mean by Automation?
§ Deep dive: Progressive Task Optimization
§ Managing change over the longer term
IBM Software Group
What do we mean by Process Optimization?
Create Account
Capture Details
Send Documents
Sign Documents
Activate Account
Customer
Call centre
Back office
Notify Customer
It is critical to drive optimization based on the business goals, not technical optimization capabilities.
Why? – Business Goals – Reduce headcount
– Maximise high value staff
– Increase volumes
– Meet performance targets
– Improve user experience
– Make better informed decisions
– Improve business agility
How? – Optimization Techniques – Path Re-engineering
– Resource Utilisation Re-engineering
– Interaction Re-engineering
– Decision Re-engineering
– Automation
IBM Software Group What do we mean by progressive process optimization?
“The journey is the goal”
§ Change takes time – Cultural, and infrastructural changes
can’t be made overnight.
§ Phased delivery – Change must be delivered iteratively
both within and across projects.
§ Design for change – Flexible solutions and agile methods
don’t just happen
Kaizen (改善?), Japanese for "improvement", or "change for the better" refers to philosophy or practices that focus upon continuous improvement of processes in manufacturing, engineering, game development, and business management. http://en.wikipedia.org/wiki/Kaizen
IBM Software Group Process optimization techniques § Path Re-engineering
– Refine the routes through the process to reduce steps, or time – Parallel flows, sequence improvements, wait time optimization, simulation, fast paths, task prioritisation
§ Resource Utilisation Re-engineering – Route the work to the appropriately skilled and/or most available people.
– Participant grouping, substitution, skills based routing. –
§ Human Interaction Re-engineering – Improve the efficiency of the user interface
– Guided activity, reduced navigation, improved situational context, reduced sign-on, simplified actions, data-pre-fetch, asynchronous completion
§ Decision Re-engineering – Extract logic from the process to simplify tasks, improve routing and enable dynamic
change – Complex rule automation, path optimization, participant routing refinement, adaptive optimization, prioritisation
§ Re-engineering typically includes Automation – Move work between people automatically, and perform tasks without user involvement
– Process modelling, process implementation, basic user interfaces, integration, aggregation, composition, orchestration
IBM Software Group An unmodelled process can’t be optimised There are benefits to process modelling alone
§ Documents a single end-to-end view of the process – Shows the participants, the tasks involved, and the flow between them. – Process is still completely manual, but better understood. – Re-engineering of the process flow is easier due to the improved visibility. – Simulations of current and to-be process can be performed.
§ But – Participants have to know who does the next step – They also need to know how to reliably transfer the work/ownership – Participants require significant knowledge. – Process re-engineering will result in significant retraining of staff
Create Account
Capture Details
Send Documents
Sign Documents
Activate Account
Customer
Call centre
Back office
Notify Customer
IBM Software Group
Lean / Six Sigma
§ LEAN – Focuses on removing waste, reducing end-to-end lead times, and making
work flow.
– Root causes are typically known.
§ SIX SIGMA – Focuses on reducing variation and defect rates.
– Root causes are not known.
IBM Software Group Path Re-engineering
Create and Activate Account
Request Account
Customer
Call centre
Back office
Notify Customer
Create Account
Capture Details
Send Documents
Sign Documents
Activate Account
Customer
Call centre
Back office
Notify Customer
IBM Software Group
Agenda § What do we mean by Progressive Process Optimization?
§ What to we mean by Automation?
§ Deep dive: Progressive Task Optimization
§ Managing change over the longer term
IBM Software Group Business Process Management (BPM) provides a single, executable view of the process
After BPM
Current State Future State
ExecutiveManagement
Call Centre
Regional RiskManagement
Local Finance and Ops
OffshoreOperation
1
2
3
4
5
6
Process
ExecutiveManagement
Call Centre
Regional RiskManagement
LocalFinanceand Ops
OffshoreOperation
Process Definition à Management Orchestration à Defined Work Segments
à Measurable Results
Hidden work à Process Variation à Complexity à Missing Information à
Chaos
IBM Software Group
Shared Process Model
Reports / Analytics
Managers / Analysts
Business Modeler
Business Modelers
Graphical Implementation
Process Developers
User Portal / Forms
Users
“… the picture IS the process …”
IBM Software Group
What are the types of automation?
Automation
Create Account
Capture Details
Send Documents
Sign Documents
Activate Account
Customer
Call centre
Back office
Notify Customer
Create Account
Capture Details
Send Documents
Sign Documents
Activate Account
Customer
Call centre
System
Notify Customer
Review Documents
IBM Software Group
Types of Automation: Flow Automation
§ Definition – Having the end to end process flow controlled by
a system rather than by humans deciding what to do next.
– All the individual tasks/activities in the process could still be manual, it is only the process flow that has been automated.
§ Issues – Might we now just be giving tasks to people
faster than they can complete them?
§ Benefits – More efficient throughput – Process is known, and made consistent – Process state is visible – Process is measurable. Process
transitions are captured and can be reported on.
– Work is distributed, prioritised
§ Skills Involved – BPMN, Lean Systems Thinking, Six
Sigma, simulation, business activity monitoring
Create Account
Capture Details
Send Documents
Sign Documents
Activate Account
Customer
Call centre
Back office
Notify Customer
IBM Software Group Flow automation brings runtime statistics
Create Account
Capture Details
Send Documents
Sign Documents
Activate Account
Customer
Call centre
Back office
Notify Customer
End-to-end
Customer visible delays
Queuing time
Customer response time
10 mins
15 mins
20 mins 2 mins
2 mins
Business Goals + Runtime Statistics = Optimization Recommendations
IBM Software Group Types of Automation: Task Automation
§ Definition – Taking an individual task that is currently
done by a human, and enabling a system to perform the action instead.
– This does not necessarily mean it is also part of an automated process. It could be called by the user via their user interface, and the process flow still belong to the users.
§ Issues – Is the task worth automating? How often
does it happen? How much of the current end to end time does it take?
§ Benefits – Faster activity completion
– Improved data integrity and consistency – Higher availability
§ Skills involved – User interface design, integration, SOA
Create Account
Capture Details
Send Documents
Sign Documents
Activate Account
Customer
Call centre
System Notify Customer
Review Documents
IBM Software Group
Agenda § What do we mean by Progressive Process Optimization?
§ What to we mean by Automation?
§ Deep dive: Progressive Task Optimization
§ Managing change over the longer term
Progressive Task Optimization
1. Assisted Swivel Chair Tell me when and how to do the task, and let the next person in line know when I’m done.
2. Linked external window
with request context
Take me to the place where I can do the task, and fill in the details for me.
3. Linked external window
with request context and callback
Take me to the place where I can do the task, fill in the details. When I complete the work, send the results back to the process and move the process onwards.
4. Integrated linked external window
Combine all the places I normally go to together on one integrated page so it feels like one system.
5. Decision supported sub-process
Ensure I only do the task when it’s really necessary, and that I only the part of the task that I’m most suited to.
6. Custom UI calling services Create a screen specifically for my task, that makes me as effective as possible.
7. External UI calling BPM API
Bring the tasks seamlessly into my existing system’s user interface.
8. Fully automated task Do the task for me.
There are many options between a manual task, and an automated task.
Manual A
utomated
IBM Software Group 1. Assisted “Swivel Chair”
§ Process model is implemented in the simplest form in a BPMS. – Work is transferred between the participants automatically, and prioritised
automatically. Workload distribution is managed automatically including for example substitutes during vacations, and balancing work across multiple teams. Managers can focus on assistance rather than distribution of work.
§ Tasks are screens explaining what to do with context specific data. – Only the relevant data and guidance for the task is presented to the user.
– A new participant would need less training.
– Lower skilled workers could be used.
§ No integration with other systems. – This level of automation is very simple to implement since the BPMS is a
completely self contained solution.
§ Issues – User must return to the task to say they have completed it.
– If we require data from the other system in the process, the user must re-key it.
– We have no idea whether the user really did the work, or whether they did it correctly.
Capture Details
Create Account
Send Documents
Sign Documents
Activate Account
Customer
Call centre
Back office
Notify Customer
1. Perform a credit check on the customer using the standard procedure to ensure that their credit rating is above the threshold set in the guidelines.
2. If the credit score is acceptable, initialise an account using the account setup guidelines.
Instructions
Smith Surname: John Firstname:
15/07/1985 DOB: 1, High St., London Address:
800 Score: AC3957495 New Account Num:
Complete Close
“Tell me when and how to do the task, and let the next person in line know when I’m done”
IBM Software Group 2a. Link with request context – via pop-up
§ Instructions provide links to external systems’ screens
– User does not have to navigate other system
– Screen is already written
– Users may already be familiar with the popup screen
– Changes to the screen don’t directly affect the process.
§ Request context passed via link – Context can be coded into the URL (or via HTTP POST
parameters).
– May allow navigation to correct screen, and even passing of data to avoid re-keying of request data.
§ Issues – User must understand enough about the other system to
complete the action.
– No communication is possible between the two windows due to “same origin policy”* so:
– If we require data from the external system in the process, the user must re-key it.
– User must return to the task window to say they have completed it.
– Requires that other systems allow URL based navigation and/or data passed in this way. If it does not, it would require changes to that system.
– What is the security model? How will the user be identified and logged onto the external system?
1. Perform a credit check on the customer using the credit check system to ensure that their credit rating is above the threshold set in the guidelines.
2. If the credit score is acceptable, create an account via the CRM system.
Instructions
Smith Surname: John Firstname:
15/07/1985 DOB: 1, High St., London Address:
800 Score: AC3957495 New Account Num:
Complete Close
Surname:
Firstname:
DOB:
Address:
Create New Account
CRM System – Create New Account
15/07/1985
1, High St., London
Smith
John
http://.../crm/createaccount?surname=Smith... re-k
ey
AC3957495 Account Num:
*See later slides for options
“Take me to the place where I can do the task, and fill in the details for me”
IBM Software Group 2b. Link with request context – embedded via iFrame
§ Embedded frame – An alternative to the pop-up is to embed the
external system’s screen using an iFrame.
– Again, if the external system can be navigated, and it’s page populated using the URL and HTTP parameters, then the iFrame will “appear” to be integrated into the page.
§ Issues – The iFrame is essentially a still seen by the
browser as a completely separate browser window. All the same issues as before therefore still exist.
– Note that for security reasons it is impossible to navigate in java script into the iFrame if it comes from a different domain to the main page. This is most noticeable in the fact that the user still has to re-key any data that needs to be drawn back into the process from the external system.
1. Perform a credit check on the customer using the credit check system to ensure that their credit rating is above the threshold set in the guidelines.
2. If the credit score is acceptable, create an account using the CRM panel below.
Instructions
Smith Surname: John Firstname:
15/07/1985 DOB: 1, High St., London Address:
800 Score: AC3957495 New Account Num:
Complete Close
Create New Account
CRM – Create New Account
re-k
ey
AC3957495 Account Num:
Surname:
Firstname:
DOB:
Address:
15/07/1985
1, High St., London
Smith
John
“Take me to the place where I can do the task, and fill in the details for me”
IBM Software Group Cross-domain communication and the “same origin policy”
Main page
iFrame
Proxy Server www.proxydomain.com
Web Server www.domain2.com
Web Server www.domain1.com
Main page
iFrame Web Server www.domain2.com
Web Server www.domain1.com
Javascript cannot navigate between main page and iFrame due to “same origin policy”.
With a proxy, the main page and iFrame both originate from same domain from the browser’s perspective, so navigation is now possible. Only works for relatively simple pages. Security complexities soon become apparent.
For more information on postMessage and ways around the “same origin policy”: https://www.ibm.com/developerworks/web/library/wa-crossdomaincomm
IBM Software Group 3. Link with call back
§ Completion of work in the external system results in a call back to BPMS triggering an intermediate event. e.g.
– Web service call to BPMS
– Queue topic to which BPMS subscribes
– BPMS polling an external system
§ User does not need to: – Wait for the external system to complete its work
– Re-key any data from the external system into the task
– Return to the task window and complete the task
§ Issues – Reliant on changes to external system, or access to it at low level.
– The integration for the call back may be point to point – not very re-usable.
– Note anti-pattern relating to “non-tasks”, and introduction of event gateway.
Surname:
Firstname:
DOB:
Address:
Create New Account
CRM GUI – Create New Account
15/07/1985
1, High St., London
Smith
John
AC3957495 Account Num:
Capture Details
Create Account
Send Documents
Sign Documents
Activate Account
Customer
Call centre
Back office
Notify Customer
CRM System
“Take me to the place where I can do the task, fill in the details. When I complete the work, send the results back to the process and move the process onwards.”
http://.../crm/createaccount?surname=Smith...
IBM Software Group 4. Integrated linked external window
§ Integrated frame/window – Communication enabled
between task and embedded window
§ Issues – postMessage required
changes to the external application to send events to task.
– postMessage only supported on HTML 5 enabled browsers
1. Perform a credit check on the customer using the credit check system to ensure that their credit rating is above the threshold set in the guidelines.
2. If the credit score is acceptable, create an account using the CRM panel below.
Instructions
Smith Surname: John Firstname:
15/07/1985 DOB: 1, High St., London Address:
800 Score: AC3957495 New Account Num:
Complete Close
Create New Account
CRM System – Create New Account
post
Mes
sage
AC3957495 Account Num:
Surname:
Firstname:
DOB:
Address:
15/07/1985
1, High St., London
Smith
John
For more information on postMessage and ways around the “same origin policy”: https://www.ibm.com/developerworks/web/library/wa-crossdomaincomm
“Combine all the places I normally go to together on one integrated page so it feels like one system”
IBM Software Group
Cross-domain communication – new options in HTML 5
§ “postMessage” facility – Windows can explicitly listen for
messages from other windows that originate from a different domain.
– Standardised via HTML 5
§ Issues – Still relatively immature
– Not supported by older browsers
– Requires new code in source of both main page and iFrame page.
Main page
iFrame
parent.window.postMessage( accountNumber, “http://domain”);
window.addEventListener( function(event) { $(“newaccountnumber”).innerHTML = event.data; } );
For more information on postMessage and ways around the “same origin policy”: https://www.ibm.com/developerworks/web/library/wa-crossdomaincomm
IBM Software Group
Create Account
5. Decision Supported
§ Reduce how often the task is included in the process – Average time of the task is reduced.
– Better distribution of work between junior and senior resource.
– Really a “flow optimization”, but could be seen as “task optimization” if we consider that the sub-process as still representing the original task
§ Externalise the decision criteria to a business rule – Analysis of process statistics may turn up further opportunities to bypass the task, and these can then be introduced at runtime.
– Passing more process data to the rule allows for most options at runtime. However, it also couples the rule to the process data model.
– Rules can be derived from historical data from the current process. Look for trends in the monitored data.
Check Credit
Capture Details
Initialise Account
Call centre
Back Office (Juniors)
Credit Check?
Yes (1%)
No (99%)
Business Rule Management System
Credit Check Rules Rule 1: Customer is staff member Rule 2: Customer has balance > $1000 Rule 3: …
Back Office (Seniors)
“Ensure I only do the task when it’s really necessary, and that I only the part of the task that I’m most suited to”
IBM Software Group 6. Custom UI calling services
§ Single user interface, with service calls to underlying systems
– UI calls services via ESB, or directly via API
– Users need not have to be familiar with user interfaces of underlying systems.
– No re-keying of data – Resulting data easily drawn back into the
process. – If errors occur, user is still has visibility of
where the error occurred in order to take remedial action.
§ Issues – Required systems must be available via
an exposed API. – Response from services must be real-
time. 1-3 seconds typically. – User is still co-ordinating the requests to
the systems. Could be further automated. – User interface is coupled to the APIs of
other systems. ESB pattern should be considered.
1. Retrieve a credit check on the customer using button below to ensure that their credit rating is above the threshold set in the guidelines.
2. If the credit score is acceptable, create an account using the button below.
Instructions
Smith Surname: John Firstname:
15/07/1985 DOB: 1, High St., London Address:
Score:
Account:
Complete Close
Credit Check
Create Account
CRM System
Credit Vetting Agency
Validate Address
Address Validation Service
“Create a screen specifically for my task, that makes me as effective as possible.”
IBM Software Group
Interface Characteristics Functional Definition
Principal data objects Operation/function Read or change Request/response objects
Technical Interface Transport Protocol Data format
Interaction type Request-response or fire-forget Thread-blocking or asynchronous Batch or individual
Performance Response times Throughput Volumes Concurrency Message size
Integrity Validation Transactionality Statefullness Event Sequence Idempotence
Security Identity/Authentication Authorisation Data Ownership Privacy
Reliability Availability Delivery assurance
Error Handling Error Management capabilities Known exception conditions
“Interface Characteristics: Capturing Integration Complexity for BPM and SOA” http://www.ibm.com/developerworks/websphere/techjournal/1112_clark/1112_clark.html
IBM Software Group 7. External UI calling task API
§ External system’s UI augmented to include data from BPM
– Users remain within the system they are already familiar with.
– UI can change largely independently of the process.
– UI can be completely custom, so can really hone the design based on user centred design, minimising key counts, etc.
§ Issues – Users unaware they are interacting with a
business process – CRM system UI programmers need to
familiarise with the BPM API. – CRM system is coupled to BPM. May be
better to surface the BPM API as services on an ESB.
– What about transactionality between the calls to “Create Account” and “Update Task”
Create New Account
CRM System – Create New Account
AC3957495 Account Num:
Surname:
Firstname:
DOB:
Address:
15/07/1985
1, High St., London
Smith
John
BPM API is documented: http://publib.boulder.ibm.com/infocenter/dmndhelp/v7r5mx/index.jsp?topic=%2Fcom.ibm.wbpm.bpc.doc%2Ftopics%2Fcdev_restapis.html Summary of usage: http://www.ibm.com/developerworks/websphere/library/techarticles/1108_thaker/1108_thaker.html?ca=drs-
BPM System
Jane Jones 23/09/1990
Fred Bloggs 13/10/2002
John Smith 15/07/1985
Account creation requests
Get
Tas
k Li
st
Update
Task
CRM System
Create
Account
“Bring the tasks seamlessly into my existing system’s user interface.”
IBM Software Group 8a. Partially automated task § Minimise the user interaction
– Data for the task is pre-fetched. Interaction need not be real-time – could be asynchronous enabling better workload balancing on back end systems.
– Actions performed as a result of the task are done asynchronously following the task.
– Data is presented in a context specific format
– Human resource is used only for what it is best at – e.g. complex decision making
– Good for “mobile” users. Review task is much simpler, and can be viewed and acted upon on a smaller device. E.g. “inline tasks completion”
§ Issues – User loses awareness of how to interact with back
end systems should they need to revert to manual processing.
– Changes to the back end systems’ APIs break the process. (ESB pattern would help with decoupling here).
– User may need results from the follow on actions (e.g. may need the account number created to pass back to the customer).
– Services/APIs need to be appropriately exposed for use by automated consumers
Check Credit
Capture Details
Initialise Account
Call centre
Back office
Credit Check?
No
Yes System
CRM System
Credit Vetting Agency
Business Rules
Review
Create Account
“Do the task for me, and I’ll just review it.”
IBM Software Group
SRP
GUI
LRP
Presentation Layer
Process and Composite Service Layer
Atomic Service Layer
How different are UIs and Processes as requesters?
GUI Graphical User Interface HT Human Task SRP Short-running Process LRP Long-running Process
Synchronous call
waiting HT
IBM Software Group Differences between UI consumer and BPM consumer Characteristic User Interface
Consumer Automated Consumer
Functional Full data typically required Response may have to be acknowledgement rather than completion
Opportunity for key data only. Can wait for completion
Data Integrity Rarely transactional all the way from the UI itself. Optimistic locking generally required for scalability Idempotence may be required to avoid duplicate submissions Event sequencing less likely to be an issue.
Can participate or even control a global transaction Pessimistic locking often preferable for simplified error handling Idempotence less of an issue if transactionality present. Event sequencing and race conditions become more common concern
Error Handling User often part of error handling strategy Data validation ideally handled by user screen
User not online, so most error paths must be codified. Late validation errors very challenging. Validation must be pushed back to capture point.
Security Requestor’s identity can be used for actions. Adjustments to the data can be handled by the requestor immediately.
Requestor no longer present when doing actions. Whose identity should be used and how. How are asynchronous errors are resolved, who can see/change the data?
Availability Services must be available when users are present.
Brief outages can be tolerated.
Performance Response times must be user acceptable, even at peak usage times. Batch processing generally unacceptable.
Response times can be longer. Throughput is the priority. Can process in batch if necessary.
For further detail see “Transitioning from Integration and SOA to BPM” at: http://www.websphereusergroup.org.uk/wug/agenda/33
IBM Software Group
Create Account
8b. Fully automated task
§ Remove user interaction – End to end time vastly reduced
– Throughput vastly increased (orders of magnitude)
– Headcount reduced/released for more important work
§ Issues – Many exception handling now pushed to
the domain of IT operations.
– Exceptions harder to diagnose.
– Business less familiar with the process and the back end systems, so less able to assist with exceptions.
Check Credit
Capture Details
Create Account
Call centre
Credit Check?
No
Yes
System
CRM System
Credit Vetting Agency
Credit Check Rules
Review
Review Rules
“Just do the task for me.”
IBM Software Group What is STP? § STP = “Straight Through Processing”
– The primary goal of STP is to increase throughput, typically by removing humans from the process.
– This requires both flow and task automation and also significant re-engineering.
– If the end goal is STP, you must design for this from the start.
§ STP = “Fully automated?” – Rarely – STPs automate the most commonly traversed paths.
– Most have rarely occurring edge cases that are still handled manually.
Create Account
Notify Customer
System Activate Account
IBM Software Group
Is full automation the goal?
§ “Should we automate a person pushing 2 buttons once a year?”
§ “It costs $100K to automate the work and $200K to make the team redundant. We will then save $20K each year as a result.”
§ “Press 1 for your account balance, 2 for new credit card or 9 to listen to another 100 choices"
§ “What do you mean, I need to land the plane? I normally just push a button. I haven’t had to actually land a plane in 20 years!”
IBM Software Group Business Goals for Process Optimization
Reduce headcount Handle the same amount of work with less people in order to cut costs, or free up resources for alternate work.
Maximise high value staff
Free up core knowledge workers from mundane activity, to focus on higher value activities, in order to gain greater value from the investment in high cost staff.
Increase volumes Scale up the volume of process instances that can be handled in a given time in order to increase retain or increase profit margins.
Meet performance targets
Focus on performance indicators such as “end to end” process time, to avoid being penalised for failing to meet service level agreements (SLAs).
Improve user experience
Improve user traction with the process, in order to employee retention, and/or retain customers.
Make better informed decisions
Expose the aggregate process statistics to enable more insightful decisions due to the runtime statistics naturally available from the automation of process flows.
Improve business agility
Enable the business to react more quickly to changes in the market due to the adoption of agile methods and tools
IBM Software Group
Agenda § What do we mean by Progressive Process Optimization?
§ What to we mean by Automation?
§ Deep dive: Progressive Task Optimization
§ Managing change over the longer term
IBM Software Group
Design time “green field” optimizations
When does optimization occur?
Project 1 Project 2 Project 3
Production
Production time “brown field” optimizations
Initial Development
Changes via runtime “points of variability”
“green field” optimizations
IBM Software Group
How is change management different
§ Green field process change (Initial implementation time) – Changes can be deployed immediately by replacing the process
application. – Existing test process instances can typically be re-created.
§ Brown field process change (Production time changes) – Process instances are present, and may be for some time. – Changes to the following require a new process application
– Process flow (BPMN/BPEL) – Process data model – User interfaces (coaches/task clients) – Inline rules – Interfaces/services
– To take full advantage of the changes, we must migrate process instances – We need to consider how intermediate events will correlate with process
instances. – Points of variability need to be designed in, and externalised if changes
are going to be possible without re-deployment.
IBM Software Group
Iteration cycles within the SOA Layers
§ Complexity of implementation – The level of complexity to implement a given service operation becomes higher as it moves down through the layers. An apparently simple
operation for a consumer may result in aggregation, or multiple transactions, or even a trigger a whole business process in the layers below, and where the service exposure may be a straightforward web service, the actual integration may be complex data formats over unusual protocols. As such, small changes in the upper layers typically balloon into larger changes lower down.
§ Dependencies between layers – Components lower in the stack typically have more things dependant on them for re-use, so are very quickly locked into providing
backward compatibility. This results in more complex versioning mechanisms, and greater re-factoring considerations.
§ Persistence of in flight data – Artefacts lower in the stack are more likely to have persistent data associated with them (such as business process instances, and human
tasks) resulting in data migration challenges between iterations.
§ Visibility to the business – Presentation layers (graphical user interfaces), by their nature, are the main point of contact between the business and the system. As
such they receive significantly more attention in terms of incoming changes. Fortunately also, sitting at the top of the stack, they can undergo significant change without enforcing any changes to the lower layers. Therefore the pace of change specifically in a presentational component can be extremely high compared to the other layers.
Shorter
Iteration Cycle
Longer
Some form of iterative design can be done for implementation at all levels within the reference architecture. However, it is important to recognise that there are a number of reasons that iteration cycles often need to be longer as we move deeper in the stack.
IBM Software Group
Optimization and Agility – orthogonal concerns
§ Optimization: Implement changes that improve KPIs – Whether those changes are implemented in an agile way or not, if the
performance increases, it is still optimization.
§ Agility: Implement changes rapidly and iteratively – Some changes contribute to optimization, but some do not. They may for
example be adding additional functionality to the process to meet new needs. Adding changes for example for regulatory compliance, might even reduce performance.
– Agility is about optimising the implementation process.
IBM Software Group Runtime Optimization: Enabling points of variability in production
§ Decision logic and values – Externalise to database tables (values) or rules (logic).
§ Process flow – Publish new versions of flow, and migrate process instances as necessary. (Be aware of process migration
constraints) § User interface
– Make key aspects of the UI dynamically configurable. (Consider testability) – Ensure user interface is separately deployable from process application.
§ Participant routing – Use externalised groups (e.g. from LDAP) – Externalise routing service (e.g. use rules/tables)
§ Reporting – Externalise reporting framework – Ensure suitably rich data set is available exposed up front (Consider performance implications)
§ Integration points – Decouple process from endpoint using persistence layer – Expose integration points as re-usable services using the ESB pattern.
§ Data model – Reduce touch points on the parts of the data model that may change. – Externalise and/or make dynamic the parts of the process that do touch data. (Consider testability) – Hold only high level metadata/keys in the process. (Consider UI response times)
Anything could change. Focus on what will change. The more flexible you make it, the harder it is to test.
A runtime process is very deliberately constrained, or testing it would be impossible. However, it is recognised that some things are more likely to change over time.
IBM Software Group
Improving dynamicity in very long lived or complex processes
§ Pluggable Sub-processes § Chained Processes
Very Complex Process
Controlling Long-Lived Process
Short-lived
process
Short-lived
process
Long-lived
process
Short-lived
process
Very Long Process
Long-lived
process
Short-lived
process
Long-lived
process
Long-lived
process
New or existing system or data store
IBM Software Group
Facades for service versioning Logical Component Boundary
Module Module Module
Update Customer
Delete Customer
v1
Retrieve Customer
Create Customer
v1 Module
Delete Customer
v2
Customer Object v2
Customer Object v1
Customer Object v1
Customer Object v1
Customer Object v1
Customer Object v1
Module
Create Customer
v2
Customer Object v1
Customer Service Façade v1
Module
Create Retrieve Update Delete
Module
Customer Service Façade v2
Customer Object v2
Create Retrieve Update Delete
Maps Routing
Module
Maps
IBM Software Group
Versioning Strategy for Interfaces – Compatibility
§ Compatible changes Consumer not affected – Additional operations
– Additional independent data structures – New optional structures/attributes on the request
object
– New runtime (unchecked/undeclared) faults
§ Incompatible changes Consumer affected unless provided as a
new endpoint
– Changes to operation parameters* – Changes to existing data structures
– New mandatory structures/attributes on the request object
– New parameters (optional or mandatory) on the response object**
– Changes to existing operation names
– Changes to namespaces – New business (checked/declared) faults
* and ** A blanket rule on there being no changes to the structure of response objects can be overly restrictive. It can make it extremely difficult for service implementations to evolve. An alternative approach where additional data within response objects must tolerated by consumers can reduce implementation and refactoring costs, but at the loss of some interface contract precision, added complexity around testing and some introduced runtime risk.
IBM Software Group Types of response data structure change: Additional attributes
<response> <order> <reference>123<reference> <quantity>2</quantity> <productId>KI987</productId> </order> </response>
<response> <order> <reference>123</reference> <quantity>2</quantity> <productId>KI987</productId> <status>SHIPPED</status> </order> </response>
Formally not backward compatible as consumer is passed unexpected data. However…
– This scenario is very common, due to general incremental evolution of interfaces over time.
– Treating this as non-backwardly compatible change is very expensive both in terms of service implementation refactoring and also for existing consumers what may want the change but would have to move to a new version of the interface to use it.
– Many (but not all) consumers can accommodate it without any refactoring of consumer code.
– Primarily depends on whether consumers validate the overall data structure (e.g. against the XML Schema) or simply extract the parts they need (e.g. via XSLT/xpath).
– Decision can be made within a domain of governance to consider this backward compatible.
IBM Software Group
Cross Domain Model
Domain 2 Specific Model
Domain 3 Specific Model
Domain 1 Specific Model
Domain 2 Specific Model
Domain 3 Specific Model
Domain 1 Specific Model
Handling Cross Domain Dependencies
E
F A B
C D C’
E F A
B C
D
A Entity Definition (i.e. a schema for a BO)
Exposed Service showing the data definitions it uses
F’
We often find that we can’t create perfect dividing lines between domains, and there is still some need for services to use some objects from other domains. This could suggest that we have the domain boundaries wrong, but it may be a genuine case. How can we handle this interdependency between domains? We don’t want consumers of the services of one domain to have to include all the object definitions from other domains.
There are a number of different mechanisms here depending on the situation: Definition E has been moved into a cross components library that it can be assumed all domains will probably need to import. Objects in this domain must be in very common use, and must be extremely stable in their definition as changes to them will cause a significant refactor/retest. Common examples are often not business data, and more likely to be technical objects used by frameworks such as a logging object. . Definition F’ is the same as E, but where it has been found that when the definition is used in other domains, it typically doesn’t need the full complexity it needs in its own domain, so the shared copy is simplified to make it more usable. Again, they must be in common use and extremely stable. An example would be the small, but very specific amount of customer data often required contain just title, first name, surname, customer number etc, but not the full customer object graph with it’s related child objects. Definition C’ is a simplified object, like F’, but it is not commonly used across domains or has a format specific to the use in Domain 1, or is insufficiently stable to be shared in the cross domain model. An example would be the customer data required on a quotation form specific to a type of insurance product. A health insurance might need to capture a small amount of health relevant information such as whether someone is a smoker.
IBM Software Group
Data Object A - Aggregated Attribute 1 - Aggregated Attribute 2
Data encapsulation in process variables
Data Object type “Any”
Type A Type B
Short-running component
that understands
Child B Short-running component
that understands
Child A
Long Running Business Process
invokes invokes
reads aggregates but not child objects
aggregated data e.g. sum, count
The long running process need only understand the wrapper object The more replaceable short running components understand the child objects
“router”
IBM Software Group
Dynamic SOA and BPM Best Practices for Business Process Management and SOA Agility
Author: Marc Fiammante Publication Date: July 24, 2009 ISBN: 978-0137018918
IBM Software Group
Common pitfalls in optimization
§ Don’t assume you must optimize all tasks to the same level – Use simulation or historical statistics to optimize the critical path
§ Don’t do it all in one go – Recognise the benefits of Progressive Task Optimization
§ Optimization must be relevant – Optimize to a quantifiable, business goal such as a service level agreement
§ Don’t over-optimise – It might not be advantageous to be able to pay invoices more quickly!
Agenda – Summary
§ What do we mean by Progressive Process Optimization?
§ What to we mean by Automation?
§ Deep dive: Progressive Task Optimization
§ Managing change over the longer term
IBM Software Group
This presentation uses content from the following: § Information on postMessage and other ways around the “same origin policy”:
– https://www.ibm.com/developerworks/web/library/wa-crossdomaincomm
§ Interface Characteristics: Capturing Integration Complexity for BPM and SOA
– http://www.ibm.com/developerworks/websphere/techjournal/1112_clark/1112_clark.html
§ SOA maturity in relation to solution design – http://www.ibm.com/developerworks/websphere/library/techarticles/0904_clark/0904_clark.html
§ “Transitioning from Integration and SOA to BPM” – http://www.websphereusergroup.org.uk/wug/agenda/33
§ Process implementation types: Patterns based design for process-based solutions
– http://www.ibm.com/developerworks/websphere/library/techarticles/1004_clark/1004_clark.html
§ “Process-oriented modeling for SOA” – http://www.ibm.com/developerworks/architecture/library/ar-procmod1
IBM Software Group
Other useful references § Dynamic SOA and BPM: Best Practices for Business Process Management
and SOA Agility – Marc Fiammante, 2009. ISBN: 978-0137018918
§ BPM API – http://publib.boulder.ibm.com/infocenter/dmndhelp/v7r5mx/index.jsp?topic=%2Fcom.ibm.wbpm.bpc.doc%2Ftopics%2Fcdev_restapis.html
– and a summary of its usage: – http://www.ibm.com/developerworks/websphere/library/techarticles/1108_thaker/1108_thaker.html?ca=drs-
§ Scaling BPM Adoption from Project to Program with IBM BPM – http://www.redbooks.ibm.com/abstracts/sg247973.html
§ Five Guidelines to Better Process Modeling – http://wiki.lombardi.com/display/commwiki/Five+Guidelines+to+Better+Process+Modeling
§ The Enterprise Service Bus, re-examined Updating concepts and terminology for an evolved technology
– http://www.ibm.com/developerworks/websphere/techjournal/1105_flurry/1105_flurry.html
§ “Design service interfaces with industry models” – http://www.ibm.com/developerworks/industry/library/ind-models/index.html
© 2010 IBM Corporation 57
IBM Software Services Zone for WebSphere ibm.com/websphere/serviceszone/
§ Emerging technology resources including proven, prescribed, and repeatable assets & offerings to accelerate Mobile, Cloud, and BPM adoption.
§ Visibility across worldwide skills, capabilities, and education that only IBM Software Services for WebSphere can bring to your project.
§ Access to WebSphere practitioners’ insight on project trends, best practices and emerging technologies through personal videos, blogs, articles & more.
§ Discover defined and proven offerings to get your project started quickly.
What’s New
The destination for WebSphere services-related resources, offerings & technical skills to help you achieve innovative growth.
Visit us in the Solution Center!
• Ped SE-2: IBM Software Services for WebSphere • Ped SE-3: IBM WebSphere Education
• Process Innovation Zone
58
We love your Feedback! § Don’t forget to submit your Impact session and speaker
feedback! Your feedback is very important to us, we use it to improve our conference for you next year.
§ Go to impactsmartsite.com from your mobile device § From the Impact 2012 Online Conference Guide:
– Select Agenda
– Navigate to the session you want to give feedback on
– Select the session or speaker feedback links – Submit your feedback
59
© IBM Corporation 2012. All Rights Reserved.
IBM, the IBM logo, ibm.com are trademarks or registered trademarks of International Business Machines Corp.,
registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies.
A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/
copytrade.shtml.
Copyright and Trademarks