[ieee 2006 13th asia pacific software engineering conference (apsec'06) - bangalore, india...

Introducing MDA in a large IT consultancy organization

Vinay Kulkarni, Tata Research Development and Design

Centre, Pune, INDIA vinay.vkulkarni@ tcs.com

Sreedhar Reddy, Tata Research Development and Design

Centre, Pune, INDIA sreedhar.reddy@ tcs.com

AbstractWe discuss our experience of introducing MDA

approach, supported by our MDA toolset, in an IT consultancy and software development services organization. Initially, our MDA toolset provided a set of modeling notations for specifying different layers of an enterprise application and a set of code generators that transform these models into platform-specific implementations. In spite of considerable amount of code being generated, it didn’t translate into commensurate productivity gain over the entire development lifecycle of large projects. What was missing was support for collaborative development. Providing suitable abstractions and tool support for a collaboration-based development method improved the situation considerably. Enterprise product lines and large development projects benefited the most resulting in improved productivity, better code quality, platform independence and easier change management. However, the steep learning curve and inadequate debugging support for models were significant hurdles. Small to medium sized projects found this MDA approach too heavy and restrictive, and preferred a flexible and lightweight meta-data driven aspect-oriented approach. Our experience suggests that MDA will find greater acceptance if complimented with well-defined and flexible methodologies that support collaborative development.

1. Introduction

Faced with the problem of developing large and complex applications, industry practice uses a combination of non-formal notations and methods. Different notations are used to specify the properties of different aspects of an application and these specifications are transformed into their corresponding implementations through the steps of a development process. The development process relies heavily on manual verification to ensure the different pieces integrate into a consistent whole. This is an expensive and error prone process demanding large teams with

broad-ranging expertise in business domain, architecture and technology platforms. To address this problem, Object Management Group (OMG) an industry consortium working has proposed a Model Driven Architecture (MDA) [11] that provides modeling notations for specifying different aspects of an application, and means to transform these platform-independent specifications into a platform-specific implementation. We discuss our experience of introducing MDA approach, supported by our MDA toolset, in an organization that provides a wide variety of IT consultancy and software development services. The approach has been tried out on enterprise product lines, several large (i.e. effort in hundreds of person-years) development projects and a few small (i.e. effort in few person-months) to medium sized (i.e. effort in few person-years) projects over the last ten years.

We begin with a description of our model-driven development approach. We then describe our model-based code generators for developing client-server applications. We argue for the need to shift the developer-centric focus to a development-centric focus and describe our solution for the same. We discuss our experience of using model-based techniques for developing product-lines, several large enterprise applications and a few small-to-medium sized projects. We argue for the need of a lightweight and flexible approach to develop small-to-medium sized applications and present a metadata-driven aspect-oriented approach for the same. We conclude with an analysis of our experience.

2. Our model-driven development approach

The development of an application starts with an abstract specification that is to be transformed into a concrete implementation on a given target architecture [5]. The target architecture is usually layered with each layer representing one view of the system e.g. Graphical User Interface (GUI) layer, application logic layer and database layer. The modeling approach

0-7695-2685-3/06 $20.00 © 2006 IEEE 419

XIII ASIA PACIFIC SOFTWARE ENGINEERING CONFERENCE (APSEC'06)0-7695-2685-3/06 $20.00 © 2006

constructs the Application specification using different abstract views - GUI layer model, App layer model and Db layer model each defining a set of properties corresponding to the layer it models as shown in Fig. 1. Corresponding to these specifications are the three meta models - GUI layer meta model, App layer meta model and Db layer meta model which are views of a single Unified meta model. Having a single meta model allows us to specify integrity constraints to be satisfied by the instances of related model elements within and across different layers. This enables independent transformation of GUI layer model, App layer model and DB layer model into their corresponding implementations namely GUI layer code, App layer code and Db layer code with assurance of integration of these code fragments into a consistent whole. These transformations can be performed either manually or using code generators. The transformations are specified at meta model level and hence are applicable for all model instances. If each individual transformation implements the corresponding specification and its relationships with other specifications correctly then the resulting implementations will glue together giving a consistent implementation of the specification as depicted in Fig. 1. Models can be kept independent of implementation technology and the application specifications can be targeted to multiple technology platforms through code generation. Construction of application specification in terms of independent models helps divide and conquer. Modeling helps in early detection of errors in application development cycle. Associated with every model are a set of rules and constraints that define validity of its instances. These rules and constraints

could include rules for type checking and for consistency between specifications of different layers. Automated code generation results in higher productivity and uniformly high code quality.

3. The use of modeling for developing client-server applications

3.1 Architecture of client-server applications

A typical client-server application is implemented across three architecture layers – user interface, application functionality and database. Each layer is implemented on a different platform supporting different primitives. For example, User interface platforms like Visual Basic provide windows and controls as implementation primitives. Application logic is implemented in a programming language like C++ or Java with classes and methods as the primitives while the database layer is implemented using tables and columns in a relational database system. These three layers are implemented independently and combined later to get an integrated implementation [5]. A user interacts with an application through its user interface. The user feeds in information using forms and browses over available information using queries and reports. Forms, queries and reports are implemented in a target platform using standard graphical user interface primitives such as windows, controls and buttons. A window is a unit of interaction between the user and the system and is composed of controls and buttons. A control accepts or presents data in a specific format. The user can perform a specific task by clicking on a button.

Application layer implements the business functionality in terms of business logic, business rules and business process. The functionality is modeled using classes, attributes, methods and associations between classes. A business process models the application as a set of tasks and defines the order in which these tasks are executed. Each task is implemented by a method of a class. Business logic specifies the computations to be performed by the application in terms of methods. The language for specifying business logic frees the application developer from low-level implementation concerns such as memory management, pointers, resource management etc and is retargettable to programming languages of choice such as Java, C++, C# etc.

The database layer provides persistence for application objects using RDBMS tables, primary key

Fig. 1. Model based development approach

View ofUnified meta model

GUI layer meta model App layer meta model Db layer meta model

Decomposes into

Application specification

GUI layer model App layer model Db layer model

Instance of

Composed of Application implementation

GUI layer code App layer code

Model based code generation

Db layer code

0-7695-2685-3/06 $20.00 © 2006 IEEE 420


and query based access to these tables, and an object oriented view of these accesses to the application layer.

3.2 Model-based generation of client-server applications

We defined a necessary and sufficient unified meta model to capture the specification requirements of all the architectural layers. Figure 2 shows the unified meta model and meta models specific to each architectural layer as views of the unified meta model with the associations spanning across the architectural layer models highlighted.

The toolset comprised of a modeling tool, a model repository and a set of model-based code generators one each for an architectural layer. The modeling tool was in fact a meta modeling tool which enabled users to define a purpose-specific meta model and create instances of the same. The models are stored in a common shared model repository. A model-based code generator transformed the platform independent model, pertaining to the architectural layer specific view, to the desired platform-specific implementation by hard-coding the choice of the various non-functional concerns, namely, design strategies, technology platform and architecture. For example, data manager layer generator transforms the platform independent instance of db layer meta model of fig. 2 to generate

database schema definition and primary-key based Create, Read, Update and Delete (CRUD) methods for a persistent class. In addition to the specificities of the chosen RDBMS, the data manager layer has to address the problem of mapping the inherently hierarchical world of objects to inherently flat world of relational database tables. Different strategies for this object-relational mapping exist, namely, one class to one table, entire class hierarchy to one table etc. Similarly, different strategies exist for flattening a class hierarchy into tables, namely, replication of all the attributes of inherited class into the table corresponding to the inheriting class, replication of just the primary key attributes etc. Also, database access can be implemented in multiple ways, namely, ProC, ODBC, JDBC etc. We hard-coded these choices in the implementation of the purpose-specific data manager layer generator. The same approach was followed while implementing generators for other architectural layers as well. Figure 3 shows the first implementation of our MDA toolset that essentially was a set of model-based code generators for developing client-server applications.

Model validation helped early detection of errors in application development cycle. Model-based code generation resulted in application-wide consistent implementation of design strategies, guidelines and best practices in a person-independent manner. This turned out to be a significant advantage especially

Fig. 2. Unified meta model for code generation

0..1

Class

Method

Association

has

source

DataType

ofType

destinatio

Attribute

has

Task

Process

has

realizes

precedes

1..* 1..*

1..*

*

1..* 1..*

*

*1..*

* *

**

* *

Table

Column Key

has has1..* 1..*

**

*

UIAttribute*

Window

UIClass Button

has has

has

opens

1..*

*

*

*

*

1..*

* mapsTo* *

maps0..10..1

composed1..* *

*cal0..1

Attribute

mapsTo 0..1

0..1

implemen0..1

0..1

GUI layer meta model

App layer meta model Db layer meta model

0-7695-2685-3/06 $20.00 © 2006 IEEE 421


when teams comprised of few experts and a large number of inexperienced developers. In spite of significant amount of code being generated, getting a large team to work in an independent and yet coordinated manner turned out to be difficult. As a result, there was no appreciable gain in productivity across the entire development cycle.

3.3 Shift to a development-centric focus

A typical business application can be divided into a set of interacting functional units e.g. Foreign Exchange, Business Partner, Retail Banking of a banking system. A functional unit has high internal cohesion and interacts with other functional units in a well-defined manner. This functional decomposition can be used as the basis for arriving at a development process wherein separate teams can be assigned to work on separate functional units. Interactions between functional units manifest in a supplier-consumer relationship consistency of which needs to be ensured during application development. Ensuring this consistency is typically a manual, effort intensive and error prone process.

We introduced component as an abstraction to model the functional decomposition. Component is a unit of development and enables an application to be modeled as a set of inter-dependent components. A component specifies its interface in terms of model elements such as classes, operations etc. The supplier-consumer relationship between components is explicitly modeled through depends association between two components. A component can only use

the model elements specified in the interface of the component it depends upon. Explicit modeling of component interfaces enables automated consistency checking of supplier-consumer relationships between components. A component specification consists of model and code, and supplier-consumer relationship is honored both for model and code. A component has two associated workspaces, namely, model workspace and code workspace. A workspace provides change isolation. As a result, a consumer component is not immediately impacted by changes introduced in its supplier components. A special workspace is provided for exchanging models and code between components. A (supplier) component keeps its model and code in this special workspace for sharing from where its consumer components pick them up. Automated validation of models, later in their respective workspaces, ensures consistency of component specifications. The component abstraction with explicitly modeled dependency enables an application to be modeled as a set of inter-related components. The workspace concept enables independent development of a component. A special workspace and a protocol for sharing artifacts between workspaces facilitate a large team to develop an application in a coordinated manner. The meta model shown in Fig. 4 specifies the core infrastructure that enables a tool-assisted component-based development process. The process is realized through a set of roles each responsible for performing a set of well-defined tasks on a component in a workspace.

The component abstraction, supporting toolset and associated development method enabled divide-and-conquer of application development. We could assign separate teams to develop components independently

Fig. 3. A set of model-based code generators

hrdgen

GUI layer meta model

App layer meta model

Db layer meta model

Unified meta model

PowerBuilderC++

Oracle

guigen dmgen

GUI layer model App layer model Db layer model

instanceOf instanceOf instanceOf

viewOf

Fig. 4. Meta model to support coordinated development

Tool

automatesTaskIn

*

Process

Task

belongsTo

precedes

1..*

**

*

Roleplays

1..* *

Interface

Component

has depends

1

** 1

Workspace

CodeWorkspace

Model Workspace

1 *has

User

playsRoleOn

1..*

*

1

*

*

automates

performs* *

0-7695-2685-3/06 $20.00 © 2006 IEEE 422


with assurances of their integration into a consistent whole. Explicit modeling of interface and dependency of components provided greater structure to application specification. This enhanced structure was used to compute the exact impact of a change and led to optimal testing effort. A role essentially identifies a set of logically coherent process steps. For instance all modeling related tasks are grouped into the modelerrole, all coding related tasks are grouped in the programmer role, all workspace synchronization related tasks are grouped in the manager role, all setup related tasks are grouped in the administrator role. Clear separation of responsibilities and assigning individuals with requisite skills to play suitable roles led to a disciplined development process. This ensured that the advantages of model-based code generation were not lost due to process bottlenecks or inefficiencies. Certain development process related errors were detected earlier or eliminated altogether. We partitioned an application into components on the basis of functional rather than architectural separation, i.e. each component had complete specifications of a functional area comprising all the architectural layers, thus facilitating system testing. Independent workspaces coupled with support for process traceability enabled separate teams to be assigned for

development and testing leading to effective use of human and computing resources.

4. Experience of introducing MDA approach

The approach presented in this paper has been realized in our model driven development environment [10]. Over the last ten years, the approach has been to deliver enterprise product lines, several large development projects and a few small to medium sized projects [8]. Over the last 10 years, more than 60 large enterprise applications (several millions of lines of code) belonging to various industry verticals have been delivered on a variety of technology platforms using this approach. Table 1 describes a representative usage of the toolset for developing large applications. As shown in the table, application was specified in terms of models for the various architectural layers namely, application layer, database layer and GUI layer, and a complete implementation generated therefrom. As can be seen, this resulted in a significant productivity in terms of the lines of code. Moreover, encoding of design strategies, guidelines and best practices into the code generators resulted in uniformly high code quality. Generation of interface code between the

Specifications Generated code ProjectDomainmodel(no of classes / screens)

Size (kloc)

Kind Size (kloc)

KindTechnology Platforms

Straight Through Processing system

334 / 0 183 Business logic,Business rules,Queries

3271 Application layer, Database layer, Architectural glue

IBM S/390, Sun Solaris, Win NT, C++, Java, ICS, MQ Series, WebSphere, DB2

Negotiated dealing system

303 / 0 46 Business logic,Queries


IBM S/390, Win NT, C++, CICS, MQ Series, COM+, DB2

Distributormanagementsystem

250 / 213 380 Business logic,Business rules,Queries, GUI

2670 Application layer, Database layer, GUI layer, Architectural glue

HP-UX, Java, JSP, Weblogic, Oracle, EJB

Insurance system 105 / 0 357 Business logic,Business rules,Queries


IBM S/390, Sun Solaris, C++, Java, CICS, DB2, CORBA

Table 1. Representative usage summary

0-7695-2685-3/06 $20.00 © 2006 IEEE 423


various architectural layers ensured smooth integration of independently generated code artifacts.

All the project teams initially found it hard to switch over from the conventional code-centric development approach to model-centric approach. The difficulties were essentially due to the steep learning curve and perceived loss of control over development artifacts. Some of the problems encountered were due to the state of the tool support available. In model-driven development approach, part of the specification is in model form and part in code form. However, debugging support was available only at the code level leading to difficulties in debugging. Also, the cycle-time required to introduce a small change and verify its correctness was found to be significantly greater for the model-driven approach than traditional code-centric approach. However, the fact that a model-level change gets automatically reflected at multiple places in a consistent manner was appreciated.

Several projects had a product-family nature wherein a product-variant needed to be quickly put together and customized to meet the specific requirements of a customer. Model-driven development approach helped in quickly retargeting the application functionality on multiple technology platforms with purpose-specific choice of architecture and design strategies. This was achieved using a relatively unskilled workforce as the technology and architecture concerns were largely taken care of by the tools [6]. The tool-assisted component-based development process helped in early detection of errors that would otherwise have led to late-stage integration problems. Also, all the projects reported significant improvements in productivity and quality. Large development projects found the approach attractive due to the assurances of consistent application of chosen design strategies, guidelines and best practices across the application. Promise of productivity enhancements even with a relatively unskilled workforce made it worthwhile for product-lines and large projects to invest in learning/training and to cope with the tooling irritants.

However, small-to-medium sized projects found the model-driven approach too heavyweight and restrictive. Shorter project durations didn’t allow for

investments in learning/training. Need to deliver quickly made the teams intolerant to tooling irritants. The perceived loss of control of the development process and artifacts was the principal reason for these projects not taking to model-driven approach.

Many architectural and design strategies cut across the layers. In the earlier realization of the approach, this required each tool to be aware of these cross cutting aspects. As a result, customizing for such cross cutting aspects required consistent modifications to several tools leading to maintenance problems. We have developed a new building block abstraction to specify these aspects as modular concerns that can be suitably composed [7]. A purpose-specific variant of model-based code generator can now be generated from its specifications [9]. We have developed the core infrastructure to realize a software factory [2] for supporting variants of model-based code generators as a product line [4]. We believe these improvements will make model-driven approach more attractive. Our experience of diffusion of these tools from a technology management perspective is discussed in [15].

5. Metadata-driven aspect-oriented software development

Figure 5 describes our metadata-driven aspect-oriented approach that treats code itself as the model thus bringing the advantages of model-driven development to small and medium sized projects, and removing model-code dichotomy and associated problems. The approach has been illustrated in the context of J2EE development and can be easily extended to support .Net development also. Implementation code of an enterprise application can be broadly classified into business logic and code for solution architecture that addresses non-functional concerns such as design strategies, architecture and technology platform. The central idea is to annotate business logic, specified using a popular programming language, with tagswhere a tag encodes a choice among a non-functional concern so as to generate the code for solution architecture from its declarative specification.

0-7695-2685-3/06 $20.00 © 2006 IEEE 424


We fall back on model-to-model transformation [13] and model-to-text transformation [12] techniques for code generation. The basic UML [14] class model can be easily extracted from the business logic specified using a popular programming language like Java. The code generator for a required solution architecture can be specified as a composition hierarchy of tags. We build on the ideas of separation of concerns [16] and aspect weaving [3] to develop the building block abstraction that specifies contribution of a tag towards code generation. Building block is the unifying abstraction to specify a tag in terms of i) model corresponding to the specific concern, ii) transformation of the concern-specific model into UML class model, and iii) transformation of the resultant model to the desired code fragments. Ability to compose tags into a hierarchy leads to two types of building blocks, namely, leaf building block and composite building block. A leaf building block specifies the code fragment to be generated for a tag and a composite building block specifies how to compose the code fragments its child building blocks have generated.

The approach was supported through an open extensible Eclipse-based [1] toolset resulting in a development process that is flexible and lightweight as compared to model-driven development process. Separation of architect role (to specify tags) from developer role (to specify business logic and use tags) led to effective utilization of resources. The reusable nature of building blocks enabled reuse even across projects. Template based code generation strategy enabled easy adherence to customer-specific standards, guidelines, best practices etc. The composable nature of building blocks, the ability to define purpose-

specific meta models and plug-in architecture due to Eclipse resulted in easy customizability, extensibility and improved maintainability of the toolset. The approach facilitated creation of a repository of reusable artifacts. Meta tools driven by a library of reusable, easy to adapt solution accelerators enabled even inexperienced teams to deliver high code on schedule. Low or no learning curve, adherence to industry standards and dependence on freeware further accelerated acceptance of the approach by the developer community. Within 5 months of introduction, with minimum push from our side, 20 small to medium sized projects have been developed using this approach.

6. Summary Modern business systems need to cater to rapidly evolving business requirements in an ever-shrinking window of opportunity while keeping pace with rapid advances in technology. To address this problem, MDA provides modeling notations for specifying different aspects of an application, and means to transform these platform-independent specifications into a platform-specific implementation. We discussed our experience of introducing MDA approach, supported by our MDA toolset, in an organization that provides a wide variety of IT consultancy and software development services. The approach has been tried out on enterprise product lines, several large development projects and a few small to medium sized projects. Initially, our MDA toolset provided a set of modeling notations for specifying different layers of an enterprise application and a set of code generators that transform these models into platform-specific implementations. Though this resulted in considerable amount of code being generated, it didn’t translate into commensurate productivity gain over the entire development lifecycle. Providing a set of abstractions for better organization of application specifications, role-based interactions through an integrated development environment and a tool-assisted development method improved the situation considerably. In our experience, enterprise product lines benefited the most from MDA resulting in improved productivity, better code quality, platform independence and easier change management. Large development projects essentially benefited due to uniformly high code quality and smoother integration. However, both found the steep learning curve a significant initial hurdle. Inadequate debugging support for models was another significant deterrent. Small to medium sized projects found the MDA approach too heavy and restrictive. Instead, a

Fig. 5. Metadata-driven aspect-oriented development

Business logic

Tag processor

Class model Tag definitions

Code fragments

Weaver

extract

0-7695-2685-3/06 $20.00 © 2006 IEEE 425


metadata-driven aspect-oriented approach was found more suitable in these projects.

The focus of MDA community so far has been on standardizing modeling notations and transformations in order to facilitate tool interoperability. From our experience, we feel, the focus now needs to shift towards evolving suitable development methods. The development method should also address the needs of small to medium sized projects that find the present MDA approach too heavy and restrictive for their needs.

7. References 1. Eclipse – a universal tool platform.

http://www.eclipse.org/ 2. Jack Greenfield and Keith Short, Software factories:

Assembling applications with patterns, models, frameworks and tools, Wiley, 2004.

3. Gregor Kiczales, John Lamping, Anurag Mendhekar, Chris Maeda, Cristina Videira Lopes, Jean-Marc Longtier and John Irwin. Aspect oriented programming. ECOOP’97 LNCS 1241, pp 220-242. Springer-Verlag. June 1997.

4. Vinay Kulkarni and Sreedhar Reddy. Enterprise business application productline as a model-driven software factory. International Workshop on Software Factories at OOPSLA 2005 (http://softwarefactories.com/workshops/OOPSLA-2005/Papers/Kulkarni.pdf)

5. Vinay Kulkarni, R. Venkatesh and Sreedhar Reddy. Generating enterprise applications from models. OOIS’02, LNCS 2426, pp 270-279. 2002.

6. Vinay Kulkarni and Sreedhar Reddy. Generating enterprise applications from models – experience and best practices. Workshop on Best Practices of Model Driven Software Development at OOPSLA 2005 (http://www.softmetaware.com/oopsla2005/kulkarni.pdf)

7. Vinay Kulkarni and Sreedhar Reddy. Integrating Aspects with Model Driven Software Development. Software Engineering Research and Practice 2003: 186-197

8. Vinay Kulkarni and Sreedhar Reddy. Model-driven development of enterprise applications. UML Satellite Activities 2004: 118-128

9. Vinay Kulkarni and Sreedhar Reddy. Separation of Concerns in Model-Driven Development. IEEE Software 20(5): 64-69 (2003)

10. MasterCraft – Component-based Development Environment. Technical Documents. Tata Research Development and Design Centre. http://www.tata-mastercraft.com

11. MDA – Model Driven Architecture http://www.omg.org/mda/

12. MOF Models to Text Transformations http://www.omg.org/cgi-bin/doc?ad/05-05-15

13. MOF Query / View / Transformations http://www.omg.org/cgi-bin/doc?ad/05-09-01

14. OMG, "UML Infrastructure 2.0 Draft Adopted Specification", 2003, http://www.omg.org/uml/

15. V C S Prasad, Vinay Kulkarni and Uday Ramteerthkar. Diffusion Experiences of Developer Productivity Tools in an IT Services Company in India, accepted at 3rd IEEE International Conference on Management of Innovation and Technology 2006

16. Peri Tarr, Harold Ossher, William Harrison and Stanley M. Suttom Jr. N Degrees of separation: Multi-dimensional separation of concerns. Proceedings of the International Conference on Software Engineering (ICSE'99) pp 107-119.

0-7695-2685-3/06 $20.00 © 2006 IEEE 426
