A DISCIPLINE FOR IMPROVINGSOFTWARE PRODUCTIVITYRobert M. Factor and William B. Smith _

Robert M. Factor isdirector of theAnalysis SystemsLaboratory at AT&TBell Laboratories,Middletown, NewJersey. Mr. Factorjoined AT&T in 1971,and is responsible forthe development ofoperations supportsystems. He has aB.S. in electricalengineering from theUniversity of Denver,and both an M.S. inelectrical engineeringand Ph.D. in systemsengineering from CaseWestern Reserve Uni­versity. William B.Smith is executivedirector of the U.S.and InternationalOperations SystemsDivision of Bell Labor­atories at Middletown.His division developsoperations systems fortelecommunicationscompanies here andoverseas. Mr. Smithjoined AT&T in 1962after receiving a B.S.in electrical engineer­ing from the Universityof Maryland. He sub­sequently received anM.S. in electrical(continued on page 9)

AT&T TECHNICAL JOURNAL •JULY/AUGUST 1988

Software is the key to the functionality needed bytoday's telecommunications products and services. Tra­ditionally, the creation ofsoftware has beena labor­intensive endeavor. Because of the tremendous growthin software needs during the past two decades, contin­uing emphasis has been given to finding ways toimprove software development productivity. AT&T BellLaboratories established a program in 1986 with thegoal of tripling software development productivitywithin three years, i.e., improving it by a factor ofthree. This paper describes the program's frameworkand the technology being pursued to respond to thischallenge.Introduction

Since the opening ofthe modern telecommunications age in1964 with the introduction ofNo. lESS™ switch and associated opera­tions systems, software has played anexpanding role in theachievement ofnew telecommunications functions both tomeet cus­tomer needs through products and services and to operate and maintainthe telecommunications network ina cost-effective manner. Today, sig­nificant AT&T revenues come from software; and, within a few years,software sales will begin to exceed hardware sales. Similar trends areevident with otherproducts and services provided bythe telecommuni­cations industry.

Since itsinception, software production hasbeenalabor­intensive endeavor. AtAT&T, about 40percent ofthetotal R&D commu­nity devotes most ofitstime todesigning, coding, and testing software.An additional 20percent supports theinitial and final phases ofthesoft­ware product's life cycle: specifications orrequirements and delivery.Combined, theseefforts annually produce about 50million lines ofcode innew products released tocustomers. Moreover, three-fourths ofthepeo­ple involved with software work inteams onlarge projects thatrequireanywhere from 150,000 tofour million lines ofcode.

With such tremendous growth insoftware needs during the

past two decades, continuing attention hasbeenfocusedonways to make the software development process moreproductive. In the telecommunications business, softwaredevelopment costs are sucha large percentage ofsalesthat efficiency ofproduct development will dominate acompany's profitability andeven its survivability. In thepast, much ofthe effective academic work done to supportsoftware development productivity centeredaround "pro­gramming-in-the-small,"! tasks suchas program editing,compiling, and debugging. Forlarge software projects,however, productivity depends primarily onhow "program­ming-in-the-large" tasks are performed andmanaged. 1

Thesetasks include specification, design, andtest coordi­nation andthe supporting activities ofsoftwareconfiguration management andversion control. Building onwhat had previously been done at AT&T to improve pro­ductivity inall those software development tasks, thecompany in 1986 focused on a goal of improving productiv­itybya factor of three within three years. The factor wasestablished not so much to measure software and produc­tivity precisely as to posea challenge to the R&Dcommunity. ThisissueoftheAT&T Technical Journaldescribes several activities developed specifically inresponse to this challenge.

Obviously, introduction ofa new technology can­notbe applied uniformly to evolving andnew projects. Thedegree to which a project is bound to existing technologiesdepends on its stage ofdevelopment. Nonetheless, manyof the new technologies canbe applied to evolving projectsandshould produce significant productivity gains.

Specific projects are used as targets to validatenew technologies and to obtain "industrial strength" ­that is, high quality androbustness - in the tools that pro­vide the technologies. Improved productivity will not onlypermit development ofmore new systems, butwill alsoencourage replacing mature systems withones that havelower software maintenance costs. The overall result willbe better customer satisfaction, more revenues, andimproved profitability.

Framework to Achieve Software Productivity GoalsThe challenge to triple development productivity

in three years demanded a program formulated to demon-

strate results. Early on, wedecided to adopt a rigoroussystem-engineering approach. Bell Laboratories had devel­oped the discipline ofsystems engineering in the 1950s toconceive, implement, anddeploy large products and com­plex services. The discipline has servedthe company well.Today, it represents about 15percent ofthe effort withinBell Laboratories to define requirements for software­based projects within focused system-engineering groups.We wanted to use the samesystem-engineering focus toimprove productivity in the software process itself.

SoftwareSystem Engineering Process.The first step inthe process was to model the effort needed to develop soft­ware for large projects (see Figure 1). Although softwaredevelopment within the AT&T R&D community includes allthe traditional life-cycle stages," activities in the differentstagesoverlap in time, project, andpersonnel. Effort spentonany activity varies widely across projects. Thesevaria­tions from the traditional life cycle have evolved toaccommodate the high degreeofinnovation andintegration(with hardware and existing software) needed to build newtelecommunication products. Despite the variations, wewere able to map detailed models of the software develop­ment cycle into four stages: requirements, design,programming/unit test, andsystem test.

We did notdirectly address the maintenance stageofa product. Ourgoal was to improve productivity for theinitial release. We reasoned that, ifwereached ourgoalsuccessfully, the same productivity gains would be carriedinto the product's maintenance phase.

In addition, weexpected productivity increases tobe accompanied byan increase in quality. Because soft­ware development is a labor-intensive endeavor, ourexpectation was based on the belief that technologiesresulting inproductivity improvements would reduce com­plexity inall stages ofthe development process, and thatthis reduction ofprocess complexity would also improvequality.

Ourearly work required a definition ofsoftwaremetrics. We investigated alternative definitions ofsoftwaremetrics forproductivity;" including function points. Weconcluded that the straightforward measure of"noncom­mentary lines ofcode per staff year," including reusedcode, was best forour initial purpose. (One outgrowth of

AT&T TECHNICAL JOURNAL •JULY/AUGUST 1988

3

4

Figure 1. Softwaredevelopmentenvironment.

this early work was a more rigorous program ofmeasure­ment to develop a better baseforcomparison andestimation. The program is described inthe paper by Leh­der, Yu, and Smith inthis issue.) This metric is easytodefine and measure; noalternative metrics have demon­strateda clearsuperiority. In addition, because Cprogramming language is a standard inthe AT&T technicalcommunity, we have a veryconsistent measure.

Our work then turned to identifying the variouscomponents and important factors involved indevelopinglarge software projects. Surveys were taken within theAT&T R&D community to estimate how available humanresources were distributed among the four stages ofthedevelopment process. Although there was some spread byproject, the mean was determined to be about 10percentofeffort inrequirements, 20percent indesign, 50percent

AT&T TECHNICAL JOURNAL •JULY/AUGUST 1988

inprogramming/unit test, and 20percent insystem test­ing. Experienced software managers and lead engineersthenestimated how application ofvarious technologiesaffects each step ofthe software development process.The value ofthese improvements wasweighted by thepercentage ofeffort spent oneach stageto determine themost promising areasinwhich to apply eitherexisting butnotyet applied technology, or new technology.

Our results, for example, showed that productivityshould double with increased component (building block)reuse. Early results from model projects have confirmedthis estimate. Improvement by a factor ofabout 1.7 is pos­sible with identifiable technology advances insoftwaresupport tools (e.g., administration tools, automated testand analysis tools); a factor of1.4 can be achieved byimproving languages and code generators (also known as

pattern or generation reuse). The combined application ofthese technologies, even discounting theirmutual interac­tions, is expected to result inthe desired productivity gain.

Technology ThrustsHaving concluded through engineering analysis

that the desired productivity gains could be realized, wenext analyzed areas ofhigh technology (Figure 1)fromwhich specific, high-priority technologies could beidentified.

In selecting specific, high technologies, it isimportant to review technologies currently inuse. Themajority ofsoftware inAT&T is developed onUNIX® sys­temsand usesC programming language. One advantage ofthe UNIX system environment is the availability ofopeninterfaces that permit the introduction ofmany tools tosupport the software development process. For example,there are tools that profile a routine's real-time use, testits complexity and style, and check syntax and semanticsagainst preset standards. The Clanguage' itself, with itshigh level ofportability and its extendibility via C++,5 is apowerful system programming language used throughoutthe world. For programming-in-the-large tasks, MAKE,SCCS, and otherchange-management systems haveexisted onUNIX systems for a number ofyears. 6,7 MAKEis a UNIX system build program for software development;SCCS stands for "source code control system."

The challenge is to improve continually all thesecurrent facilities, both conceptually and at the performancelevel, and to come upwith new facilities to integrate andautomate many othertasksin the process. In evaluatingnew technology to answer thatchallenge, we consideredpotential payoff and the likelihood ofsuccess. Asa result,we concluded that the following thrusts should be givenhighest priority in the nearterm:- Product administration and configuration management- Software reuse- Languages- Project management.

Product Administration and Configuration Management.One area of high-priority attention was improved soft­ware administration capability so that the state of thesoftware could be put under stringent discipline through-

out a product's life cycle.Asoftware product consists notonly ofthe source

code but also ofassociated source documents such asproduct installation scripts; makefiles; test scripts; modifi­cation requests (MRs); requirement, design, and userdocuments; and business and quality plans. Although eachsource component can have different versions andreleases, all are related functionally (e.g., anMR mayresult inmodifications to several files) and structurally (aparticular product feature may have beenimplemented as acollection ofsub-features). More than one person may beresponsible for any component. In addition, as noted ear­lier, the entire product goes through several stages duringits lifetime. These stages overlap intime, project and per­sonneL Disciplined administration ofthesecomponents iscruciaL Their interrelationships with other components,personnel, and life-cycle stages are similarly criticaL

Concurrently, this administration must be auto­mated as much as possible without intruding onthe naturalwork habits ofthe people involved in the development.This required careful human engineering ofthe alreadyevolving administration discipline and proper automation ofits tasks. Two systems, SABLE and NMAKE, were cen­tral to this thrust. These systems, described by Cichinskiand Fowler in this issue, were based onearlier UNIX sys­tem-based tools SCCS and MAKE.

SoftwareReuse. Thebiggest opportunity for pro­ductivity improvement was judged to be inthe field ofreuse. Reuse can be exercised at many levels ofapplica­tion. It has, infact, beena mainstay ofproductivityimprovement work for many years at AT&T.

Reuse can be obtained eitherby using existingcomponents with little or nomodification or by reusingtemplates. In the latter sense, generation is a form ofreuse. If the domain ofapplication is sufficiently under­stood and can be modeled accurately, thentechniques existto create systems with program generators. Newapproaches to producing thesegenerators from the applica­tion descriptions have beenformulated and applied incertain projects. These approaches are described inthepaper by Cleaveland and Kintala in this issue.

For telecommunications applications, however, thegreatest productivity payoffs result when large existing

AT&T TECHNICAL JOURNAL •JULY/AUGUST 1988

5

6

Figure 2. Process model forasset reuse.

components are reused. One level ofcomponent reuse isto build successive software generic releases onpreviousreleases ofthe same application. (We apply the term"generic" to a series ofsoftware products thatserve abroad functional areato recognize this level ofreuse.) Asecond level ofreuse is to apply standard system compo­nents and services available with the operating systemembedded in the product; one ofthe strengths oftheUNIX operating system is the set ofcomponents and ser­vices available with it. The reuse ofapplicationcomponents borrowed from a close, cooperative associa­tion with otherprojects represents a third level ofreuse.

Thereusearea ofgreatest importance to ourcur­rent software productivity effort is the utilization ofcomponents designed for reuse as partofanengineeringplan. These components, truly envisioned as assets, maybe relatively small (such as string manipulation routines),ofmoderate size (such as errorhandling packages), orrather large (such as recovery or system reconfigurationsubsystems),

Figure 2 shows a version ofanasset-reuse modelthatdescribes the essence ofa design process based onengineered reuse. Thefirst step in the process is to utilizereusable assets from a pool to obtain a desired solution. Ifall assetsalready exist in the pool, thena system develop-

AT&T TECHNICAL JOURNAL •JULY/AUGUST 1988

ment effort would consist of:• Pulling the assets together to provide anintegrated

software product that includes user and administrativedocumentation

• Testing the product to assure thatit has metrequirements

• Delivering the product to the customer.Currently, reusable assets thatmeet the needs ofall thecomplex telecommunications products we build are rare.However, certain classes oftelecommunications productscan benefit substantially from existing software assets.

Inaddition, a process has been created to obtainengineered assets both bycatalogingexisting assetsandchoosing those thatfit the "engineered" environment(including system performance requirements) and by con­tracting internally for developments to provide new assetsconsistent with theirdevelopments and the engineeringspecification for reusable assets. These assetsare thenplaced into the asset pool for use onfuture developments.Thepaper byAnderson, Beck, and Buonanno in this issuedescribes the basis leading to large-scale component reusethrough modular components.

Languages. Akey tool inany software developmentactivity is, ofcourse, the programming language used towrite the software's programs. In the past, AT&T hadmoved from using assembly languages to using high-levellanguages, such as C, to build its telecommunications prod-

ucts. The use ofhigh-level languages reduces thecomplexity associated withthe programming process. Itdoes so bymoving away from the needto baseprogramson the intrinsic nature ofcomputers andtoward program­ming basedonthe nature ofthe problem.

Systems currently being built, and telecommunica­tions products envisioned for the future, could be 10timesmore complex thanearliersystems in control and data-flowstructures, interprocedure interactions, andvolume andefficiency requirements. Yet, to improve productivity lev­els, weneedto reduce the complexity ofthe programmingprocess. Todoso, wewantto use more expressive powerin the general-purpose languages, and more languagesdesigned for special-purpose applications.

C++, an extension of C, provides support fordata abstraction andobject-oriented programming para­digms. The ability ofC++ to encapsulate relatedapplication concepts intoa single structure, called a class,opens the way to designing systems that promote reuse inthe programming process andcomplements the componentreuse described in the preceding section. Its inheritancemechanism reduces the effort ofhandling the complexitymentioned above. At the sametime, C++ is compatiblewith existing C software andexpertise, bothimportantAT&T assets. Thiscompatibility isessentialforits rapidandgraceful introduction intoa wide variety ofAT&T proj­ects. The paperbyCoplien, Dewhurst, andKoenig in thisissuedescribes C++ in this context.

Application languages provide a second way toimprove productivity. It is more natural and, therefore,more productive for the application builder to specify andprogram ina language that has the application conceptsbuilt intoit. AWKB is an excellent example ofanapplicationlanguage for writing programs to manipulate datastoredonsequential files on the UNIX system. Languages withfinite-state-machine concepts built into themwill be natu­ralfor the call-processing applications in switchingsystems. These languages canalso help developers specifyandbuild prototypes rapidly. The paperbyBrown et al. inthis issueexplains the concepts in greater detail.

Project Management. Providing a methodology (pro­cedures anddocumentation) to manage the developmentprocess is a key part of the productivity improvement pro-

gram. An important step is the definition anddocumentation ofa methodology that could be applied totargeted projects. In AT&T, the software developmentmethodology is featured in the Software Project Manage­mentWorkshop, a training course described inanearlierissue oftheAT&T Technical Journal. 9 During this one-weekworkshop, four principles ofeffective management-plan­ning, organizing, monitoring, andcontrolling-are taughtthrough exercises andlectures. The end result of theworkshop is an actual project plan generated bypartici­pants. The workshop's curriculum was built onAT&T'sexperiences with both successful andunsuccessful proj­ects, as well as from published work. 10,11Methodologiesmust, ofcourse, be backed byassociated tools to permitease ofimplementation. Product administration and confi­guration management (described byCichinski andFowler'spaper in this issue) andthe software estimation technology(see the paper byLehder, Yu, and Smith, also in this issue)are examples of tools developed directly to support themethodology's basic discipline.

Future Directions. Much ofthe technology usedtoimprove software development productivity was initiated inresearch programs during the past decade. Thus, as partofourongoing productivity program, wewant to supportcontinuing research efforts andfind ways to transfer theresulting technology rapidly into development projects.Considerable research is now being done onmethods suchas increasing computer aids in the frontend ofthe designprocess and generation ofsoftware; program-flow­structure visualization; integrated version, build, andtestenvironments; andspecification languages. Anumber ofdistributed workstation trials are under way that experi­mentwith approaches for improved implementation andexecution environments. These andotherareas ofinvesti­gation that show promise offuture gains are described inthe paperbyBelanger, Bergland, andWish in this issue.

Relationship of Technologies to Productivity OpportunitiesThe state-of-the-art technologies and theirappli­

cation, reported in this issue, were chosen for theiranticipated payoff in improving oursoftware developmentprocess. Software projects throughout AT&T abound withevidence ofimmediate productivity improvements that

AT&T TECHNICAL JOURNAL •JULY/AUGUST 1988

7

8

result directly from application ofthese technologies. Forexample, SABLE is usedinmore than 180 projects. WhenSABLE is used, new projects come onboard within a daybecause the system automatically sets up the entire mech­anism to administer the project and manage changes to it.Similarly, NMAKE routinely decreases the time interval byfactors offive or more when compared to technology usedinMAKE, NMAKE's predecessor. Reusable componentshave also helped reduce the time intervals between newproducts.

It is obviously difficult to measure precisely pro­ductivity improvements attributable to thesetechnologies.This is because ofthe interrelationships among aspects ofsoftware development and the idiosyncrasies indifferentproject environments. Moreover, as noted earlier, newtechnology cannot be introduced uniformly inboth evolvingand new projects because the degree ofcommitment toexisting technologies is a function ofa project's stage ofdevelopment.

Another approach to determining payoff is similarto the one advocated recently by Boehm. 12 His conclusionsare statedas a set ofsixopportunities to drive down soft­ware development costs:- Getthe best from people- Make steps more efficient- Eliminate steps- Eliminate rework- Build simpler products- Reuse components.Our technology thrustsmap into theseopportunities.Some ofthesemappings are straightforward and obvious;others are not.

For example, we believe thateffective estimationtechniques will improve productivity in two ways. First,we get the best from people if we can accurately estimatethe effort and schedules for a product. If schedules are tootight, inefficiencies occur and quality drops. Second, realis­ticestimates have a tendency to drive projects to buildsimpler products, or to increase component reuse becauseoftherecognition that complex customized developmentsinvolve higher costs and risks.

Thefocus is onthe most important issues involvedinproviding the functionality required ofthe development.

AT&T TECHNICAL JOURNAL •JULY/AUGUST 1988

It is significant to note thatwe use a process discipline thatforms the framework for development activities, notamethodology thatgives step-by-step instructions. Weencourage variances in the discipline to allow for local dif­ferences and for the discipline's growth.

Productivity andQuality. No discussion about produc­tivity would be complete without considering theinteraction between quality and productivity. Although thepapers inthis issue focus ontechnologies that improveproductivity directly, quality hasanimportant and directrelationship with productivity. Improvements inqualityreduce rework, and reduced rework means increased pro­ductivity. AT&T's aggressive quality program"complements the productivity work described here.

Quality insoftware products is often presented inthe context ofthe system's error-free behavior inuserenvironments.s-" For AT&T to be a leader inthe produc­tion ofhigh-quality software, the software must notonlybe free of"bugs," butmust also satisfy many othercus­tomer expectations. It must, as well, include attributessuch as elegance indesign, reduced complexity, transpar­ency ofthe design in the implementation through use ofhigh-level languages, and smooth integration ofthe infor­mation generated during the various stages ofthe systemthatmap onto the ability to evolve rapidly to meetuserneeds. The tools thatrepresent the technologies describedinthis issue help intheseadditional aspects ofquality. Forexample, C++ reduces the complexity ofthe design para­digm byletting a developer work with high-level conceptualobjects thatencapsulate into a single structure both thedata and the operations ondata. Reusable modules simplifysoftware development byproviding plug-compatible com­ponents, thereby allowing the system developer toconcentrate onthe essential elements ofthe new function­ality thatis to be added. Reduced complexity directlyincreases both productivity and quality.

Technology Transfer. Finally, to obtain desiredincreases inproductivity and quality, the technologies mustevolve. They must be embedded in the industrial-strengthtools used both to build the products we provide to ourcustomers and inthe products themselves. Obtainingindustrial strength in the tools requires a transfer ofthetechnologies from research to general use through a con-

trolled process. 14 AtAT&T, the organization whose charteris to improve the R&D community's productivity throughtechnology transferrecognized the increasing importanceofits task. The organization has increased its efforts tomake protoypes ofthese technologies, then test them inamanner identical to all otherAT&T products. It recognizesthat technologies that improve productivity mustalsoimprove quality. The tools andbuilding blocks that embodythe technology must, therefore, also be ofextremely highquality.

ConclusionLeadership in telecommunications products and

services depends notonly onworld-class hardware tech­nology, butalso increasingly on customer satisfactionbased onfunctionality, quality, timeliness, and costofsoft­ware products. During the past 20years, much progresshasbeenmade to convert the art ofsoftware developmentinto anengineering discipline that better servescustomerneeds. The papers in this issue represent some ofthesteps taken within AT&T. The reported improvements,however, are just part ofa journey that will notendas longas customers continue to look for even more effectiveways to meet their telecommunications needs.

References1. E DeRemer and H. H. Kron, "Programming-in-the-Large Versus

Programming-in-the-Small," IEEE Transactiems em SoftwareEngineering, Vol. SE-2, No.2, June1976, pp. 80-86.

2. B. Boehm, Software Engineering Economics, Prentice-Hall,Englewood Cliffs, New Jersey, 1981.

3. C. Jones, ProgrammingProductivity, McGraw-Hili, New York,1986.

4. B. W. Kernighan and D. M. Ritchie, The C Programming Lan­guage, Prentice-Hall, Englewood Cliffs, New Jersey, 1988.

5. B. Stroustrup, The C++ Programming Language, Addison­Wesley, Reading, Massachusetts, 1986.

6. UNIX Time-Sharing System, Bell System Technicaljournal, July/August 1978, Vol. 57, No.6, Part 2.

7. UNIXSystem V User Reference Manual, Release 3.2, AT&T, June1988.

8. A. V. Aho, B. W. Kernighan, and P.]. Weinberger, The AWKProgramming Language, Addison-Wesley, Reading, Massachu­setts, 1988.

9. 1. B. Robertson andG. A. Secor, "Effective Management ofSoftware Development," AT&T Technicaljournal, Vol. 65, No.2,MarchiApril1986, pp. 94-101.

10. P. B. Crosby, Quality isFree, McGraw-Hili, New York, 1979.11 .T. Demarco, Cemtrolling Software Projects, Yourdin Press, New

York,1982.12. B. W. Boehm, "Improving Software Productivity," Computer, Vol.

20, No.9, September1987, pp. 43-57.13. Quality: TheoryandPractice, AT&T Technicaljournal, Vol. 65,

No.2, MarchiApril 1986.14. D. G. Belanger, "Technology Transfer: ASupplier View," IEEE

Second Workshop em Software Technology Transfer, June1987.

Biographies (continued)engineering from Princeton University and a Ph.D. in com­puter science from the University of Pennsylvania.

(Manuscript received May 24, 1988)

AT&T TECHNICAL JOURNAL' JULY/AUGUST 1988

9