rt4 - the whole sordid story
DESCRIPTION
Building RT 4 took a lot more work than we'd expected. In the end, we learned some useful lessons and ended up with a great product.TRANSCRIPT
Hi, I'm Jesse Vincent
From Boston, MA in the US
Perl Hacker
Current Perl 5 “pumpking”
Android Hacker (K-9 Mail)
Kindle Hacker (Savory)
Former Perl 6 project manager
Hi I'm Jesse
Original Author of RT
Partner at Best Practical
RT project lead
(That just I don't get to code much anymore)
Best Practical
We make RT
We sell support
We sell training
We sell consulting
We sell custom development
Our offices
Boston, MA
Moscow, Russia
Hangzhou, China
Pinglin, Taiwan
(We're 11 people)
RT: Request Tracker
General purpose ticketing system
GNU GPLv2
Continuousdevelopmentsince 1996
O(100) downloadsevery day
What is a ticketing system?
It keeps track of whatneeds to get done
It keeps track of whatgot done
...along with lots of metadata
...and business logic
...and access control
...It's just a TODO list
On some very serious drugs
not
Important properties ofticketing systems
● Everything has a unique ID● Everything has a timestamp● History can't be edited or erased
What do you use a ticketing system for?
Network operations
Bug tracking
Call center
Helpdesk
Customer service
Work orders
Accounts Payable
Accounts Receivable
Vacation rentals
Youth counselling
Workflow
What we use RT for
Bug Tracking
Bronze/Silver/Gold support
Customer Development
Resumes
Inbound Sales Inquiries
Sales Leads
Accounts Payable
Accounts Receivable
Who uses RT?
Who else uses RT?
http://requesttracker.wikia.com/wiki/RTUsers
RT Scales
It scales down
(for testing or development)
Run RT on your laptop
SQLite
Standalone web server
It scales up
Largest RT I know about
40,000-70,000 tickets
...every day
(Nearly 1 ticket/second)
Multiple front-end app servers
Big database server with hot standby
Designed to be hookableand pluggable
Plugins
● rt-action-linearescalate
● rt-action-notifygroup
● rt-ajaxyreplypage
● rt-authen-bitcard
● rt-authen-openid2
● rt-bugtracker
● rt-bugtracker-public
● rt-condition-complex
● rt-crypt-smime
● rt-extension-activityreports
● rt-extension-activityreports-billing
● rt-extension-addadminccsonqueuechange
● rt-extension-attributewalker
● rt-extension-captcha
● rt-extension-cloneticket-withdata
● rt-extension-commandbyemail
● rt-extension-commandbymail
● rt-extension-commentoncreate
● rt-extension-customfield-hideemptyvalues
● rt-extension-datediscordian
● rt-extension-extractcustomfieldvalues
● rt-extension-formtools
● rt-extension-jsgantt
● rt-extension-ldapimport
●
● rt-extension-log-memoryusage
● rt-extension-menubarsearches
● rt-extension-mergeusers
● rt-extension-mergeusershistory
● rt-extension-nagios
● rt-extension-notificationmatrix
● rt-extension-priorityasstring
● rt-extension-quickcalls
● rt-extension-quickdelete
● rt-extension-quickupdate
● rt-extension-reportspam
● rt-extension-rt_cpan_org
● rt-extension-spawnlinkedticketinqueue
● rt-extension-utils
● rtfm
● rtfm-extension-articletemplate
● rtir
● rtx-calendar
● rtx-emailcompletion
● rtx-ticketlist-transactions
● rtx-workflowbuilder
(and a bunch more created by RT users)
RT 4.0
Now available?
Not quite
Christmas 2010:
4.0.0RC1
March 24, 2011:
4.0.0 RC7
Release next week?
hcchien has been askingme to do a talk on RT4
since at least 2006.
I've been promising “next year” for 5 years.
We started RT4 inSeptember 2007
I named it 3.999-DANGEROUS
In literature, they call that foreshadowing
I do public RT trainingsa few times a year
I talk about RT's history
These are the slides I use
A Brief History of RT
RT 0.9 (1996)
● Designed for use at a single company
● 2 sysadmins● 30 users
RT 1.0 (1999)
● Same as RT 0.9
+ a bit more courage● Used at hundeds of companies● Dozens of CSRs● Thouands of requests per day● Intense guilt
RT 2.0 (2001)● Total rewrite● Just after Jesse escaped Microsoft● DBIx::SearchBuilder● Abstraction● Whole new UI● No more frames● “Keywords”
RT 3.0 (2003)
● Overhauled web interface● Extension mechanisms● Internationalization● Custom fields● Cleaner internals● Tests
RT 3.2 (2004)
● New search UI● Spreadsheet / RSS output● Outgoing mail preview and logging● UI improvements● No major structural changes● More tests
RT 3.4 (2005)
● Reimplemented Custom Fields● Custom fields on users, groups
transactions● Generalized Transaction system● Faster, Faster, Faster● Prettier● Even more tests
RT 3.6 (2006)
● All-CSS layout and styling● Customizable homepage● Built in charts and reports● Ticket "reminders"● Comprehensive test coverage● Cleaner code
RT 3.8 (2008)
● More user preferences● Timezones● Theme● Ticket history order● New configuration system● Even more tests
RT 3.8 (continued)
● “Favorite” tickets● Ticket relationship graphs● Branded queues● iCal feeds● PGP support
RT 4.0 (2008?)
Never trust a vendor who makes promises about unreleased products
That's really what it said!
It was sort of a joke
...little did I know
All Taiwanese know that4 is very unlucky
You're supposedto just skip 4
Nobody warned me
...until last night!
In my culture, 6 is the unlucky number.
Along came 2006
We started thinkingabout building RT 4.0
RT is big
http://www.flickr.com/photos/swiv/4426214075/
RT is big
http://www.flickr.com/photos/daymin/4715213393/
RT is complex
http://www.flickr.com/photos/18909153@N08/5241036226/
RT is complex
RT is old
RT is old
http://www.flickr.com/photos/paulgissane/163290720
What would we change?
Modernize the API
Remove insane features
Use a framework!
Jifty
AJAX
Modern API
Lots of testing affordances
Plack
Automatic Database Schema Management
UI Helpers
So, we started refactoring
Not a from-scratch rewrite
..but pretty close
No deadline
"It'll be ready when it's ready"
No fixed deliverable
"We want it to be good"
So, we went to work.
What went wrong?
We moved files around
Git made that sort of ok
We decided to modernizeour coding style
We started renamingclasses and methods
RT's API was oldand InterCapped
The modern perl worldis prettier_looking
We built refactoring tools
We ported thefull test suite
RT 3.6/3.7 were still inactive development
We were fixing lots of bugs in 3.6
It was a constant battleto merge forward
3 years in, RT 3.999 was largely "done"
60% of the code of RT 3.8
Almost the samefeature set as RT 3.8.0
Almost the sametest suite as 3.8.0
Much cleaner
Ran on Jifty
Ran on Plack
New "Ticket Lifecycles" system
Much of the UI portedto Template::Declare
It was a lot better
It killed off lots of badold API decisions
...but mostly better forRT's developers
2627 commits
3 years of development
1484 files changed, 174558 insertions(+), 319761 deletions(-)
3.999 was different than 3.8 in some important ways
The API was recognizably the same
3.999 was better than 3.8 insome important ways
The API was 100% incompatible
There were lots and lots of reasons to make the change
There are lots and lots of RT extensions out there.
We've done at least 75
There are more on CPAN
Just about every RT instance has some local customizations
RT has been downloaded about 100 times every day
For the past 5+ years
How many of you have ever customized or extended
some software?
Guess what happens whenyou change an API?
Ever had an API changebreak your customization?
RT has many users
They rely on many, manyRT extensions
● rt-action-linearescalate
● rt-action-notifygroup
● rt-ajaxyreplypage
● rt-authen-bitcard
● rt-authen-openid2
● rt-bugtracker
● rt-bugtracker-public
● rt-condition-complex
● rt-crypt-smime
● rt-extension-activityreports
● rt-extension-activityreports-billing
● rt-extension-addadminccsonqueuechange
● rt-extension-attributewalker
● rt-extension-captcha
● rt-extension-cloneticket-withdata
● rt-extension-commandbyemail
● rt-extension-commandbymail
● rt-extension-commentoncreate
● rt-extension-customfield-hideemptyvalues
● rt-extension-datediscordian
● rt-extension-extractcustomfieldvalues
● rt-extension-formtools
● rt-extension-jsgantt
● rt-extension-ldapimport
●
● rt-extension-log-memoryusage
● rt-extension-menubarsearches
● rt-extension-mergeusers
● rt-extension-mergeusershistory
● rt-extension-nagios
● rt-extension-notificationmatrix
● rt-extension-priorityasstring
● rt-extension-quickcalls
● rt-extension-quickdelete
● rt-extension-quickupdate
● rt-extension-reportspam
● rt-extension-rt_cpan_org
● rt-extension-spawnlinkedticketinqueue
● rt-extension-utils
● rtfm
● rtfm-extension-articletemplate
● rtir
● rtx-calendar
● rtx-emailcompletion
● rtx-ticketlist-transactions
● rtx-workflowbuilder
We broke all of them
(all the extensions)
(all the users)
RT 3.999 had 40% lesscode than RT 3.8
RT 3.999 lookedjust like RT 3.8
RT 3.999 had almost thesame features as 3.8
...with one really big change
RT 3.x has a systemcalled “Scrips”
Scrips let you buildcustom business logic
They're sort of like“if...then...” statements
They can fire after any update
RT's approvals systemuses Scrips
RT's email-sending rulesuse Scrips
They're really powerful
...but not omnipotent
Scrips are unchangedsince RT 2.0
We decided to replaceScrips in RT 3.999
I wanted a simplemacro language
clkao built the backend
“I'm not building a stupid macro language. If we're doing this, it should support eval and apply”
We created lorzy
It was a lispy language
...with named, typed parameters
We ripped out Scrips and dropped in lorzy
Greenspun's Tenth Rule
Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.
...at least we did it on purpose?
So, have I sold RT 3.999 to you?
I don't like hurting users.
I don't like hurting customers.
Last summer, we threw away3 years of work.
[sad panda]
What'd we learn?
Second system syndromehurts a lot
. o O { My problem is that I actually have users
I don't want to alienate }
A working test suite is not a magic bullet
Incremental updates vs
gut renovation
Got a working system?
Make the smallest changethat could possibly work
Build for your users
When we do client work, they say that they want high-quality software last week for almost
free.
"Good, fast and cheap, pick any two"
Turns out that youmust pick two.
Last summer, we startedRT4 over again
What'd we do differentthis time?
We were workingfor a customer
The customer wantedRT 3.8...
...with a lot of extensions.
They wanted to pay us to integrate those extensions.
We had a deadline
We had a fixed set of deliverables
We had a mostly fixed set of deliverables
We got to work
The client wanted frequentbeta releases
The new RT 3.9 needed tobe deployable at any time
Lots of topic branches
We started pulling in work we'd already done as core features
RTFM became Articles
Stock answers
Really useful
Lots of folks don't install it
So, now you don't get a choice
Lifecycles
Originally built for 3.999 & backported to 3.8
Mobile Interface
Date custom fields
IP address custom fields
Full-text search for MySQL, Postgres and Oracle
Gmail-style foldingof ticket history
Some new features, we built from scratch
New auditing anddebugging tools
A brand new visual theme
A brand new theme editor
A new permissions editor UI
Dropdown and radio-button custom fields
Autocompleters for usersand email addresses
Ported to Plack
Some stuff, you'd never notice
Overhauled how wedo boilerplate code
(Fewer files change.They change less)
Cut down our use of autoconf
Made the test suite much faster
Eliminated "noise"from the test suite
Successfully deliveredto customer
Since then, we've been stabilizing, fixing bugs and
improving docs
RC 1 came out just after christmas
RC 7 came out last week
We're very, very close.
Try out RT4
http://bestpractical.com/rt/download.html
http://bestpractical.com/rt/git.html
Release Candidate means we think it's ready for production
If it breaks, email [email protected]
Thank you!
Questions?