Download - October 8, 2015CLIMEPage 1 Constraining Software Evolution Steven P. Reiss Brown University
April 21, 2023 CLIME Page 1
Constraining Software Evolution
Steven P. ReissBrown University
April 21, 2023 CLIME Page 2
Software is Multidimensional
Architecture
SpecificationsDesign
Test Cases
User InterfaceConventions
HistoryDocumentation
CodeModule Structure
Component Structure
DBMS SchemaPatterns
April 21, 2023 CLIME Page 3
Software Artifacts
• Dimensions are reflected in artifacts– Typically files, sometimes databases– Some artifacts are “paper only”
• Generally one-to-one– But some artifacts have multiple dims– And some dims are spread over multiple
artifacts
• But it is the artifacts that are edited
April 21, 2023 CLIME Page 4
Evolution is One-dimensional
• Artifacts evolve separately– Thus dimensions evolve independently– Different views become inconsistent– Design, documentation, etc. ignored– Software itself becomes inconsistent
• Some dimensions don’t evolve– Some dimensions don’t have artifacts– These are totally ignored
• Result is poor, unmaintainable software
April 21, 2023 CLIME Page 5
What is Needed
• CONSISTENT EVOLUTION – OF ALL DIMENSIONS OF SOFTWARE– OVER THE LIFE OF THE SOFTWARE
• This is our goal
April 21, 2023 CLIME Page 6
What We Propose
• A tool to ensure artifacts consistency– Initially as the software is developed– During maintenance as the software evolves
• The CLIME system
April 21, 2023 CLIME Page 7
Outline of Talk
• Requirements for consistent evolution• Constraints as a mechanism• Implementing constraints in CLIME• On-going and future work
April 21, 2023 CLIME Page 8
Requirements for the Tool• Work with existing tools & artifacts• Handle a wide range of dimensions• Be extensible• Be bidirectional• Support partial & incomplete checking• Be able to locate points of inconsistency• Have low overhead and be unintrusive• Handle static and dynamic properties• Work incrementally
April 21, 2023 CLIME Page 9
Insight Yields Methodology• Artifacts are constraints on artifacts• Design
– Constraints on the code– Constraints on the code’s behavior
• Specifications– Constraints on the solution
• In general– Software artifacts can be kept consistent by
viewing each as constraints on other artifacts and checking these constraints.
April 21, 2023 CLIME Page 10
Basic Approach
SoftwareArtifacts
Constraints
ConsistencyChecker
InconsistencyReport
April 21, 2023 CLIME Page 11
Problems with this Approach
• Determining what artifacts are involved• How to extract the information needed
for constraints from the artifacts• How to define constraints from this
information• How to check constraints • What to report to the developer• Making it incremental and complete
April 21, 2023 CLIME Page 12
Related Work: XLinkIt• Constraining XML documents
– Constraints are XLink/XPath queries– Documents can be dynamically generated
• Has been applied to software engineering– Small number of dimensions– By dynamically generating from java code– Comparing to XML representation of UML
• Problems– NOT incremental, complete, automatic, compatible,
powerful enough, practical at large scales
April 21, 2023 CLIME Page 13
What are the Artifacts
• Need to define the domain– Essentially the project being worked on– This concept already exists in environments
• Need to identify artifacts– These are typically stored as files now– Identify artifacts using file types
• We handle this using a project manager– Or from existing programming environment
April 21, 2023 CLIME Page 14
Project Manager: CLIP
• Project determines the set of artifacts• Provides set of files to rest of system
– Based on directory list (include + exclude)– Based on patterns– Based on artifact file types
• Also specifies environment for files• Can be automatically generated
– From existing environments (Eclipse, …)
• Front end also available
April 21, 2023 CLIME Page 15
Project Interface
April 21, 2023 CLIME Page 16
Information From Artifacts• First we need to understand the artifacts
– Artifacts alone are too varied and complex• We need specific information
– To define the constraints– Different constraints need different info
• Solution: abstract artifact information– Simpler than considering whole artifact– Tools to extract the needed information
• This must be done incrementally• Store the result in a database
April 21, 2023 CLIME Page 17
Artifact AbstractionSoftwareArtifacts
Abstractor Abstractor Abstractor Abstractor
Update Information
Database Manager
Database
Update Manager
April 21, 2023 CLIME Page 18
Artifact Abstraction
• Update Manager– Detects what has changed– Runs appropriate abstractor(s) for each
artifact
• Update Information– Commands to delete sets of tuples (patterns)– Commands to add tuples
• Database Manager– Relational DBMS (JDBC/Postgresql)
April 21, 2023 CLIME Page 19
Current Abstractors
• Source– Symbol table information (Modified Jikes)
• Refs, defs, types, hierarchy, scopes, …– Semantic information (Using Soot)– Documentation information (CLAD: Doclet)
• UML (Rose or any XMI-based tool)– Class & message interaction diagrams
• CVS History– File version and revision history
April 21, 2023 CLIME Page 20
Test Case Abstraction• Desired information
– What are the test cases– Test coverage of each test case
• CITE: Test coverage tool– Extension of junit testing program– Use jikesbt to instrument code
• Modified junit that uses cite– Produces coverage information when tests
are run as well as the set of test cases– Line, branch, routine, call coverage
April 21, 2023 CLIME Page 21
Cite Coverage<TESTRUN DATE='1034344045851' PROJECT='onsets' USER='spr' NAME='spr.onsets.TestBitSet.testFirst'
SIGNATURE='()V' STATUS='PASS' RUNID='17921'> <METHOD NAME='spr.onsets.OnsetBitSet.check' CALLED='35'> <BLOCK INDEX='18429' START='54' END='54' COUNT='35'> <BRANCH TOBLOCK='18431' COUNT='33' /> <BRANCH TOBLOCK='18430' COUNT='2' /> </BLOCK> <BLOCK INDEX='18430' START='54' END='54' COUNT='2'> <BRANCH TOBLOCK='18432' COUNT='2' /> </BLOCK> <BLOCK INDEX='18431' START='54' END='54' COUNT='33'> <BRANCH TOBLOCK='18432' COUNT='33' /> </BLOCK> <BLOCK INDEX='18432' START='54' END='54' COUNT='35' /> </METHOD> <METHOD NAME='spr.onsets.OnsetBitSet.firstBit' CALLED='3'> <CALLS NAME='spr.onsets.OnsetBitSet.check' COUNT='35' /> <BLOCK INDEX='18500' START='134' END='134' COUNT='3'> <BRANCH TOBLOCK='18504' COUNT='3' /> </BLOCK> <BLOCK INDEX='18501' START='135' END='135' COUNT='35'> <BRANCH TOBLOCK='18503' COUNT='33' /> <BRANCH TOBLOCK='18502' COUNT='2' /> </BLOCK> <BLOCK INDEX='18502' START='135' END='135' COUNT='2' /> <BLOCK INDEX='18503' START='134' END='134' COUNT='33'> <BRANCH TOBLOCK='18504' COUNT='33' /> </BLOCK> <BLOCK INDEX='18504' START='134' END='134' COUNT='36'> <BRANCH TOBLOCK='18501' COUNT='35' /> <BRANCH TOBLOCK='18505' COUNT='1' /> </BLOCK> <BLOCK INDEX='18505' START='138' END='138' COUNT='1' /> </METHOD></TESTRUN>
April 21, 2023 CLIME Page 22
Database Manager: CLIDE• Maintains the database based on
commands– First it finds all tuples that match remove
patterns using queries– Matches new tuples to those to remove– Adds any really new tuples to the database – Updates any modified tuples– Removes tuples that are really deleted
• Generates an update file– What was added, deleted, modified– Based on UIDs
April 21, 2023 CLIME Page 23
CLIDE: UIDs
• Unique Ids are used to represent tuples– Each relevant tuple has one
• Database manager maintains these– If tuple is only modified, it keeps its UID– Based on set of key fields
• UIDs from abstractor => global UIDs– Provide pointers in the tuples
• These provide low-level handles to the actual data
April 21, 2023 CLIME Page 24
CLIDE: Table Definition?xml version="1.0"?><!DOCTYPE CLIME:TABLES SYSTEM "file:/pro/clime/xml/schema/climetable.dtd" >
<CLIME:TABLES xmlns:CLIME='file:/pro/clime/xml/schema/clime'>
<CLIME:ENUM NAME="HistoryType"> <CLIME:CONST NAME="NONE" VALUE="0" /> <CLIME:CONST NAME="CVS" VALUE="1" /> </CLIME:ENUM>
<CLIME:TABLE NAME="HistoryFile" > <CLIME:FIELD NAME="Id" TYPE="Id" KEY="1" /> <CLIME:FIELD NAME="FileName" TYPE="String" /> <CLIME:FIELD NAME="HistoryType" TYPE="HistoryType" /> <CLIME:FIELD NAME="RawFile" TYPE="String" /> <CLIME:FIELD NAME="CurrentVersion" TYPE="String" /> <CLIME:MATCH IDFIELD="Id" FIELDS="FileName RawFile" /> </CLIME:TABLE>
<CLIME:TABLE NAME="HistoryVersion"> <CLIME:FIELD NAME="Id" TYPE="Id" KEY="1" /> <CLIME:FIELD NAME="ForFile" TYPE="Id" SOURCE="HistoryFile" /> <CLIME:FIELD NAME="FileName" TYPE="String" /> <CLIME:FIELD NAME="Version" TYPE="String" /> <CLIME:FIELD NAME="Author" TYPE="String" /> <CLIME:FIELD NAME="HasDesc" TYPE="Boolean" /> <CLIME:MATCH IDFIELD="Id" FIELDS="ForFile Version" /> </CLIME:TABLE>
</CLIME:TABLES>
April 21, 2023 CLIME Page 25
CLIDE: Transitive Closure
• Transitive closure is needed in software– Symbols available in scope– Subclass relationship– Control/data flow; type inference; …– TC is not a relational operation
• CLIDE provides transitive relations– Automatically built and maintained– Update is done incrementally– Can be queried against directly
April 21, 2023 CLIME Page 26
Defining Constraints
• Defined in terms of artifact abstractions• Multiple types of constraints are possible• Most constraints can be defined as
relationships among the abstractions– Examples– This allows us to define metaconstraints
• Issue: maintaining traceability– Source of the constraint; Point of violation– This leads to constraint instances
April 21, 2023 CLIME Page 27
Using Metaconstraints
Metaconstraints
ArtifactDatabase
ConstraintFinder
Constraint Instances
April 21, 2023 CLIME Page 28
Metaconstraints
• Predicate equations (first order logic)
• S is the base relation determines when x is applicable specifies what must be true
– For this constraint to hold– Typically starts with for all or exists
• These are general logic equations– With the full power of SQL available
)()()( xxSx
April 21, 2023 CLIME Page 29
Example<CLIME:CONSTRAINT TYPE="QUERY" NAME="uml_source_class_correspondence" > <CLIME:EXPR OP="FORALL" VAR="x" TABLE="UmlClass"> <CLIME:EXPR OP="AND">
<CLIME:EXPR OP="NOTNULL"> <CLIME:EXPR OP="FIELD" FIELD="Name" VAR="x" /> </CLIME:EXPR> <CLIME:EXPR OP="EQL">
<CLIME:EXPR OP="FIELD" FIELD="ClassType" VAR="x" /> <CLIME:EXPR OP="INT" VALUE="0" />
</CLIME:EXPR> </CLIME:EXPR> <CLIME:EXPR OP="EXISTS" VAR="y" TABLE="SrcDefinition">
<CLIME:EXPR OP="AND"> <CLIME:EXPR OP="EQL"> <CLIME:EXPR OP="FIELD" FIELD="SymbolType" VAR="y" /> <CLIME:EXPR OP="INT" VALUE="3" />
</CLIME:EXPR> <CLIME:EXPR OP="EQL"> <CLIME:EXPR OP="FIELD" FIELD="Name" VAR="x" /> <CLIME:EXPR OP="FIELD" FIELD="Name" VAR="y" />
</CLIME:EXPR> </CLIME:EXPR> </CLIME:EXPR> </CLIME:EXPR></CLIME:CONSTRAINT>
April 21, 2023 CLIME Page 30
Current Metaconstraints
• UML Class Diagrams Source– Every class has a corresponding source class– Interface matches a source interface– Class generalization matches hierarchy– Interface generalization matches hierarchy– UML operations correspond to methods– UML attributes correspond to fields– UML associations reflected in source
April 21, 2023 CLIME Page 31
Current Metaconstraints
• Source UML Class Diagrams– Public classes appear in UML model– Public interfaces appear in UML model– Generalizations among public classes– Generalizations using interfaces– Public methods appear in UML model– Associations using fields appear in UML
• Test Cases Source– Source covered by last test not changed
April 21, 2023 CLIME Page 32
Current Metaconstraints
• Documentation => Source– Parameters, See Also, Throws correspond to
source
• Source => Documentation– Public classes and interfaces are documented
April 21, 2023 CLIME Page 33
Generic Constraints
• Some constraints don’t have a source– Style constraints (e.g. naming conventions)– Usage constraints (e.g. language rules)
• Some constraints have no artifact– Design patterns
• Database manager allows explicit tuples– Generic (style constraints)– Project-specific (design patterns)
• Constraints defined from these tuples
April 21, 2023 CLIME Page 34
Current Defined Constraints
• Style Constraints– Class naming conventions– Method naming conventions– Field naming conventions– Local variable naming conventions– Constant naming conventions
• Pattern Constraints– Façade patterns
April 21, 2023 CLIME Page 35
Current Defined Constraints
• Language usage constraints– All parameters not starting with _ are used– All fields are read at least once– All fields are written at least once– All methods are called at least once– Fields are either private or protected– Any routine returning type X must return a
new instance• Note this is specialized by the particular project
April 21, 2023 CLIME Page 36
Maintaining Constraints• Metaconstraints
– Need to be translated into actual constraints– One for each matching item in the database
• Several tasks need to be done– Look for new constraints to be added– Check new constraints for validity– Check prior constraints for applicability– Check prior constraints for validity– Remove dead constraints
• This has to be done incrementally
April 21, 2023 CLIME Page 37
Constraint Instances
• Basis for checking, updating, reporting• An instance contains
– The meta constraint– The database object that is its source (UID)– Database objects that validate/invalidate it
• Exists : The particular item that was found• For All : Any items that did not match
– Any tables that affect the constraint– Current status of the instance (valid/invalid)
• Stored in the database
April 21, 2023 CLIME Page 38
CLIC: Generating Constraints
Metaconstraint
Constraint Manager
Test Query
{UID,true|false}
Constraint Instance
Item Query
Assoc UIDsAssoc Tables
April 21, 2023 CLIME Page 39
Test Query
• XML metaconstraint mapped to SQL• Returns a set of UIDs and status
– SELECT x.uid,true FROM … WHERE <holds> UNION SELECT x.uid,false FROM … WHERE <not-holds>
• Restricted to UIDs that change– WHERE x.uid IN ...
• Used for new constraints• Used for checking existing constraints
April 21, 2023 CLIME Page 40
Item Queries
• Find UIDs for a constraint instance– Of those items that made is succeed or fail
• XML metaconstraint mapped to SQL– Multiple queries generated for each instance
• One for each level of nesting• Using information from prior level
– Used to determine when to recheck constraint
• Table associations determined directly– For cases which are NOT-based
April 21, 2023 CLIME Page 41
Showing Constraints: CLICUI
• Constraint instances– Have enough information for traceability
• UIDs of constraint mapped to database object• Database object has descriptive information
– Have enough information for understanding• UIDs of exceptions or associated objects• Again can be mapped to database objects
• What’s left is to report the results– In a meaningful way– Integrated with an environment
April 21, 2023 CLIME Page 42
Constraint User Interface
April 21, 2023 CLIME Page 43
CLICUI
• User interface for browsing constraints– By type, by status, by source
• Constraints kept up to date automatically• Details obtained from database
– Relation-based user output– Location information from relation
• Location information to the environment– “Goto” in the editor – Tell the environment to display something
April 21, 2023 CLIME Page 44
Eclipse-based Interface
April 21, 2023 CLIME Page 45
CLIME Experiences
• CLIME used for several projects– On a game program (UML, source, tests, …)– On itself (43,500 LOC)– On SOOT (200KLOC), Eclipse (1.2MLOC)
• CLIME on itself– Generates about 2300 constraints– Unobtrusively used– Bugs found through style and usage
constraints
April 21, 2023 CLIME Page 46
CLIME: Summary
• Integrating evolving software is the next step in development environments
• This can be done by constructing and using constraints among the artifacts
• A wide variety of artifacts and constraints can be managed cheaply and effectively
April 21, 2023 CLIME Page 47
CLIME: Summary
• The constraint mechanism works with existing tools and artifacts
• It extends to a variety of dimensions• It is extensible• It supports associating inconsistencies
with artifacts• It has low overhead and is unobtrusive
April 21, 2023 CLIME Page 48
Current Research
• Additional artifacts and constraints– Documentation, language usage,
• Artifacts requiring complex constraints– UML interaction diagrams
• Behavioral specifications and constraints• User-friendly front ends
April 21, 2023 CLIME Page 49
Complex Constraints
• Queries can’t check all constraints– Language constraints (CCEL/Parasoft Jtest)
require the abstract syntax tree– Dynamic constraints require static/dynamic
analysis
• The associated analysis is best done outside of queries
April 21, 2023 CLIME Page 50
Artifact AnalysisSoftwareArtifacts
Abstractor
Update Information
Update Manager
Program Analysis
Result Files
April 21, 2023 CLIME Page 51
Program Analyzers
• Abstract syntax tree queries– CCEL-type queries against a standard AST– Evaluated dynamically inside Jikes– Query results output in XML
• Static analysis queries– Finite-state specification of desired behavior– Static check if that behavior
• Must occur, can occur, must not occur
• Dynamic checking
April 21, 2023 CLIME Page 52
UML Interaction Diagrams
April 21, 2023 CLIME Page 53
UML Interaction Constraints
• What does it mean to check these– Statically using program analysis– Dynamically using trace information
• Build an FSA from interaction diagram– Check this against control flow– Check this against trace
• Difficult to do in a SQL query– But often doable through static analysis
April 21, 2023 CLIME Page 54
Behavior Constraints
• How a class or library should be used– This is part of the specification– Should be part of evolution
• Ensuring new software meets specifications• Checking effects of changes to existing software
• Example– Always return new instance– Iterators in Java
• Initialize – { hasNext() – next() }
April 21, 2023 CLIME Page 55
Checking Behavior
• Can sometimes be done statically– Through program analysis– Can detect most simple instances– Can isolate potential violations
• Can also be done dynamically– By adding checking code (aspects)– By looking at trace data
April 21, 2023 CLIME Page 56
Defining Behavior
• Specify a set of program events– Calls, returns, variable sets, etc.– Can be parameterized (e.g. by object)
• Define extended finite state machines– Trigger event that creates machine instance– Machines have associated variables– Actions can set and test variables– Events cause transitions – Illegal transitions trigger errors
April 21, 2023 CLIME Page 57
Behavior Constraint<CHECKS> <CHECK META="iterator_check" NAME="iterator_check"> <CLASSES> <USE ID="C1" CLASS="java.util.Iterator"/> <USE ID=“C2” CLASS=“java.util.Colleciton” /> </CLASSES> <IGNORE> <CLASS NAME="java." /> <CLASS NAME="javax." /> </IGNORE> <EVENTS> <EVENT ID="E1" NAME="NewIterator" TYPE="CALLRETURN" VALUE="C1" METHOD="iterator" TRIGGER="1" CLASS="C2" /> <EVENT ID="E2" NAME="hasNext" TYPE="CALLRETURN" OBJECT="C1" METHOD="hasNext" /> <EVENT ID="E3" NAME="next" TYPE="CALLRETURN" OBJECT="C1" METHOD="next" /> <EVENT ID="E4" NAME="hasMore" TYPE="CALLRETURN" OBJECT="C1" METHOD="hasMoreElements" /> <EVENT ID="E5" NAME="nextElement" TYPE="CALLRETURN" OBJECT="C1" METHOD="nextElement" /> </EVENTS> <AUTOMATON START="S1" TRIGGER="NEW">
<STATE ID="S1" ACCEPT="1" NAME="Start" > <ON EVENT="E1" GOTO="S2" />
</STATE> <STATE ID="S2" ACCEPT="1" NAME="IteratorAllocated" > <ON EVENT="E2" GOTO="S3" /> <ON EVENT="E3" GOTO="S4" /> <ON EVENT="E4" GOTO="S3" /> <ON EVENT="E5" GOTO="S4" /> </STATE> <STATE ID="S3" ACCEPT="1" NAME="hasNextCalled" > <ON EVENT="E3" GOTO="S2" /> <ON EVENT="E5" GOTO="S2" /> </STATE> <STATE ID="S4" ERROR="1" NAME="nextCalled“/ >
</AUTOMATON> </CHECK></CHECKS>
April 21, 2023 CLIME Page 58
Behavior Checker: CLAC
• Takes FSM based definition• Does dynamic checking of application
– Currently using trace files– Easily modified to used class file patching
• Used to test iterators & tokenizers in several systems– Needed to make FSM more complex– Found one problem with our code this way
April 21, 2023 CLIME Page 59
Future Research
• Show applicability to a wide variety of artifacts on real software systems
• Handling behavior specifications efficiently and statically where possible
• Extending this to handle component and net-based software– Appropriate representations and constraints
• Validate the approach on a large scale
April 21, 2023 CLIME Page 60
Future Research
• Extending the concept from detecting dependencies to fixing them– Automatic updates are sometimes possible
• Configuration management using constraints
• New design tools based on constraints
April 21, 2023 CLIME Page 61
Acknowledgements
• NSF (ACI9982266, CCR9988141, CCR9702188)
• Manos Renieris, Shriram Krishnamurthy• Christina Kennedy, Tom Wooldridge,
Brock Pytlik, Daniel Licata, Curran Nachbar, Tudor Antoniu
April 21, 2023 CLIME Page 62
Questions / Comments