working effectively with legacy perl code
TRANSCRIPT
![Page 1: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/1.jpg)
Working Effectively with Legacy Perl Code
Erik RantapaaFrozen Perl 2010
![Page 2: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/2.jpg)
What is Legacy Code?
![Page 3: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/3.jpg)
What is Legacy Code?
Uses old/obsolete perl and modulesNot well factored / organically grownOriginal developers goneNo one fully understands itUses outdated practicesHard to modify without breaking itLong new developer ramp-up timeNo documentation (or wrong documentation)No tests
On the other hand... Doing useful work - generating revenue
![Page 4: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/4.jpg)
What is Legacy Code?
Michael Feathers - "Code without tests"
Ward Cunningham - "Accumulated technical debt"
Stuart Halloway - "Legacy is the degree to which code: - fails to capture intent - fails to communicate intent - captures irrelevant detail (ceremony)"
![Page 5: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/5.jpg)
Example Legacy App
e-commerce applicationover 10 years old"rewritten" at least a couple of timesworked on by lots of different developers> 1000 .pm files> 150 database tables, >2000 columns3 templating systems, 100's of template filesused perl 5.8.0 until recentlylots of old versions of CPAN modules (some customized)didn't compile with -cw until recentlyrm -rf not an option
![Page 6: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/6.jpg)
Overview
Unit Testing from a Perl perspectiveWorking with the Code BaseInstrumentationFuture Directions
![Page 7: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/7.jpg)
![Page 8: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/8.jpg)
The Case for Testing
"... With tests we can change our code quickly and verifiably. Without them we really don't know if our code is getting better or worse. ..." - Michael Feathers
![Page 9: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/9.jpg)
Cost of Fixing Defects
![Page 10: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/10.jpg)
The Feedback Cycle
![Page 11: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/11.jpg)
Testing vs. Debugging
Debugging:manual set-up (set breakpoints, step code, etc.)temporarily modifies code (printf debugging)manual verificationpay for it every time
Testing:no source code modificationautomated set-up and verificationpay once when you create the testreap the rewards every time you run it
![Page 12: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/12.jpg)
Your Application
![Page 13: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/13.jpg)
A Unit Test
![Page 14: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/14.jpg)
Unit Testing
isolates a small piece of functionalityeliminates dependencies on other components through mocking / substitutionideally runs quicklyprovides automated verification
Benefits:
safety net during refactoring after refactoring becomes a regression testspeeds up the Edit-Compile-Run-Debug cycle
![Page 15: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/15.jpg)
Impediments to Unit Testing
Main impediment: the way the application is glued together.
use LWP;...sub notify_user { my ($user, $message) = @_; ... my $ua = LWP::UserAgent->new; ... $ua->request(...); }
![Page 16: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/16.jpg)
Strongly Coupled Concerns
sub emit_html {
}
![Page 17: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/17.jpg)
Dependency Breaking TechniquesAdapt ParameterBreak Out Method ObjectDefinition CompletionEncapsulate Global ReferencesExtract and Override CallExtract and Override Factory MethodExtract and Override GetterExtract ImplementerExtract InterfaceIntroduce Instance DelegatorIntroduce Static SetterLink Substitution
Parameterize ConstructorParameterize MethodPrimitive ParameterPull Up FeaturePush Down DependencyReplace Function with Function PointerReplace Global Reference with GetterSubclass and Override MethodSupersede Instance VariableTemplate RedefinitionText Redefinition...
![Page 18: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/18.jpg)
Sprouting
replace a block of code with a subroutine/method call (even for new code)
Benefits:
simple and safe code transformationcode block can be run independentlypermits the code to be redefined
![Page 19: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/19.jpg)
Sprouting Example - DBI calls
Any DBI call is a good candidate for sprouting. DBI call = an action on a business conceptSprouting permits:
testing of SQL syntaxtesting of query operationremoval of interaction with the database
![Page 20: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/20.jpg)
Dependency Injection
# use LWP;...sub notify_user { my ($user, $message, $ua ) = @_; ... # my $ua = LWP::UserAgent->new; ... $ua->request(...); } "Ask for - Don't create"
![Page 21: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/21.jpg)
Preserving Signatures
use LWP;...sub notify_user { my ($user, $message, $ua ) = @_; ... $ua ||= LWP::UserAgent->new; ... $ua->request(...); }
![Page 22: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/22.jpg)
Perl-Specific Mocking/Isolation Techniques
![Page 23: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/23.jpg)
Replacing a Package
alter @INC to load alternate code
alter %INC to omit unneeded code
![Page 24: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/24.jpg)
Replacing Subroutines
Monkey patch the symbol table:
no warnings 'redefine'; *Product::saveToDatabase = sub { $_[0]->{saved} = 1 }; *CORE::GLOBAL::time = sub { ... }
Create accessors for package lexicals
![Page 25: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/25.jpg)
Modifying Object Instances
Re-bless to a subclass:
package MockTemplateEngine;our @ISA = qw(RealTemplateEngine);sub process { ... new behavior ...}...$template = RealTemplateEngine->new(...);bless $template, 'MockTemplateEngine';
![Page 26: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/26.jpg)
Use the Stack
Define behavior based on caller(). Use in conjunction with monkey patching subs.
![Page 27: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/27.jpg)
Exploiting Perl's Dynamicism
Use the dynamic capabilities of Perl to help you isolate code and create mock objects.Not the cleanest techniques, but they can greatly simplify getting dependency-laden code under test.
![Page 28: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/28.jpg)
Manual Testing
Not always bad Some times it's the best option:
automated verification is difficult / impossibleautomated test to costly to create
![Page 29: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/29.jpg)
Manual Testing Example
![Page 30: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/30.jpg)
Using an Wrapper Layer
![Page 31: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/31.jpg)
Real-World Experience
Writing the first test is very difficultIt gets easier!Biggest win: automated verificationNext biggest win: efficient testsManual testing is ok (save what you did!)
![Page 32: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/32.jpg)
More Real-World Experience
Isolating code will help you understand the dependency structure of your applicationLet your tests guide refactoringTestable code ~ clean code ~ modular code
![Page 33: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/33.jpg)
Working with the Code Base
![Page 34: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/34.jpg)
Source Control
Get everthing under source control:perl itselfall CPAN modulesshared librariesApache, mod_perl, etc.
![Page 35: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/35.jpg)
Reliable Builds
Automate building and deployment Break dependencies on absolute paths, port numbersEnable multiple deployments to co-exist on the same machine
![Page 36: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/36.jpg)
Logging
make logs easy to accessimport logs into a databaseautomate a daily synopsis
SELECT count(*), substr(message, 0, 50) as "key"FROM log_messageWHERE log_time BETWEEN ... AND ...GROUP BY keyORDER BY count(*) DESC;
![Page 37: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/37.jpg)
Wrappers
Create deployment-specific wrappers for perl and perldoc:
#!/bin/shPATH=...PERL5LIB=...LD_LIBRARY_PATH=...ORACLE_HOME=......
/path/to/app/perl "$@"
![Page 38: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/38.jpg)
Create Tools
$ compile-check Formatter.pm
Runs app-perl -cw -MApp::PaymentService::Email::Formatter
Full module name determined from current working directory.
![Page 39: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/39.jpg)
Automated Testing
Continuous IntegrationSmoke TestsRegression Tests
![Page 40: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/40.jpg)
Instrumentation
![Page 41: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/41.jpg)
Devel::NYTProf
Pros:full instrumentation of your codetiming statistics on a per-line basiscall-graphs, heat mapuseful for
Cons:big performance hit (not suitable for production)you need to drive it
Also see Aspect::Library::NYTProf for targeted profiling
![Page 42: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/42.jpg)
Devel::Leak::Object
Original purpose: find leakage of objects due to cyclic references
Modifications:
Record how (the stack) objects get createdCount the number of each type of object createdDetermine where objects of a certain class are createdRobust facility to instrument bless and DESTROY.
![Page 43: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/43.jpg)
dtrace
Pros:instrument events across the entire systemruns at nearly full speed - usable in production
Cons:only available for Mac OS and Solarisperl probes still under development
![Page 44: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/44.jpg)
Future Directions
More static analysismanually added soft type assertionsmore help in editors/IDEsmore refactoring toolsbetter compilation diagnosticscatch mistyped subroutine namescatch errors in calling methods/subscatch other type errors
![Page 45: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/45.jpg)
Future Directions
More dynamic instrumentionusable in productionlow performance hitmainly interested in:
the call chain (stack)types of variables / subroutine parameters / return values
dtrace? modified perl interpreter?
![Page 46: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/46.jpg)
Future Directions
Automated Instrumentationbuild a database of the runtime behavior of your programcollect design details not expressed in the sourcemine the database to infer internal rules:
call graphssignature of subroutines
use rules to aid static analysis
![Page 47: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/47.jpg)
A Refactoring Problem
Want to change $self->{PRICE} to $self->getPrice:
package Product;
sub foo { my ($self, ...) = @_; ... ... $self->{PRICE} ... ...}
![Page 48: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/48.jpg)
A Refactoring Problem
Within package Product → easy!
package Product;
sub getPrice { ... }
sub foo { my ($self, ...) = @_; ... ... $self->getPrice ... # know type of $self ...}
![Page 49: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/49.jpg)
A Refactoring Problem
Can we change this?
package SomeOtherClass;
sub bar { ... ... $product->{PRICE} ...}
Need to know if $product is from package Product (or a subclass)
![Page 50: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/50.jpg)
A Refactoring Problem
Look at context for help.
package SomeOtherClass;
sub bar { ... my $product = Product->new(...); ... ... $product->{PRICE} ...}
![Page 51: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/51.jpg)
A Refactoring Problem
Now what?
package SomeOtherClass;
sub bar { my ($self, $product, ...) = @_; ... ... $product->{PRICE} ...}
Need to figure out where SomeOtherClass::bar() gets called.
Instrumentation DB → just look up the answer
![Page 52: Working Effectively With Legacy Perl Code](https://reader033.vdocuments.us/reader033/viewer/2022052303/554f5c9ab4c905c8088b4751/html5/thumbnails/52.jpg)
References
"Working Effectively with Legacy Code" - Michael Feathers "Clean Code Talks" - googletechtalks channel on YouTubeWriting Testable Code - Misko Hevery http://misko.hevery.com/code_reviewers_guide"Clean Code: A Handbook of Agile Software Craftsmanship," Robert C. Martin, ed.