yapc10 cdt world domination
DESCRIPTION
TRANSCRIPT
![Page 1: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/1.jpg)
Catalyst, DBIC, and TT for world domination
![Page 2: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/2.jpg)
My Background
• Lazy (Programmer)
• Often involved in sys admin (repetitive) work
• Leverage CPAN so I don’t have to figure things out others already have (Lazy)
![Page 3: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/3.jpg)
In the beginning...
• automate maintenance
• systems checks, report by email.
• check status of various processes/disk/memory
• automate user tasks
Scripts to:
![Page 4: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/4.jpg)
In the beginning...
• automate maintenance
• Job failures difficult to detect
Scripts to:
![Page 5: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/5.jpg)
In the beginning...
• systems checks, report by email
• Inbox flooded.
• Still have to read and process them
• Follow ups often require system access
Scripts to:
![Page 6: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/6.jpg)
In the beginning...
• check status of various processes/disk/memory ($> appname_health.pl)
• often requires privileged access
• output in fixed font
Scripts to:
![Page 7: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/7.jpg)
In the beginning...
• automate user tasks (mkuser, unlock account, etc)
• user at least has to have a login
Scripts to:
![Page 8: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/8.jpg)
DBI + CGI
![Page 9: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/9.jpg)
CGI - Benefits
• Accessible to anyone with a web browser
• Programs easy to deploy
• No need to login to host
![Page 10: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/10.jpg)
CGI - Bad habits
• Accessible to anyone with a web browser
• Perl compile cost every page hit
• Long running processes not easily possible
• HTML Code mixed in with perl code (HERE or qw{}) print q{<HTML>
hello world</HTML>};
![Page 11: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/11.jpg)
DBI - Benefits
• Standardized access to back end data
• Database not perl process does hard query work
![Page 12: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/12.jpg)
DBI - Bad habits• Vulnerable to SQL injection
• Perl code is hard coded to a specific DB type (MySQL, Postgres, Oracle, Sybase, MSSQL)
• Redundant prep/execute/extract code usually splashed all over the code
• Detection of errors usually not standardized
• SQL code (often many lines long) intermixed with perl code. (Hard to read)
• Badly formed SQL can reveal sensitive information to end user
![Page 13: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/13.jpg)
DBI - Bad habits
• Vulnerable to SQL injection
• select * foo where bar = “$test”
• $test = q{5; insert into admin_table...};
• $test = q{“blah}
• Difficult to handle BLOB data
• Placeholders limited
• where of select, set of update, values for insert)
• From clause inaccessible via placeholders
![Page 14: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/14.jpg)
DBI + CGI
![Page 15: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/15.jpg)
Catalyst, DBIx::Class, TT
![Page 16: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/16.jpg)
The Learning Curve• Moose!
• DBIx::Class
• TT
• Catalyst
• Model/View/Controller paradigm a difficult balance to maintain
• Perl Testing techniques in Catalyst
• Catalyst unit testing (how do I exclude the models in the test?)
• Model testing (build a test DB complicated enough to test the model)
• Selenium (minor changes in TT easily break tests)
![Page 17: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/17.jpg)
Moose
• Easy to learn for if you’ve done OO programming before
• http://search.cpan.org/perldoc?Moose::Cookbook
• No more new!?!?
• Not critical to learn but useful when thinking about how Catalyst works
![Page 18: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/18.jpg)
DBIx::Class• Best way to call SQL but read like perl
• Standard interface to returning query from submy $rs = $schema->resultset(‘CD’) ->search({ year => 2000 }, { prefetch => 'artist' });while ($rs->next) {
print $rs->artist . “\n”;}
my $dbi = DBI->connect(...) or die DBI->error;my $query = q{select artist from CD where year = 2000};my $sth = $dbi->prepare($query) or die;$sth->execute or die;while(my ($artist) = $sth->fetchrow_array) {
print “$artist\n”;}
![Page 19: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/19.jpg)
DBIx::Class table setup
• Schema Loader Magic! (unless it’s not a well built Schema)
• Make sure fkeys,pkeys setup in db first if you can.
• Not all Pg features pull from schema loader
• DBIx::Class::Schema::Loader to load schema
• Alternatively: cd ~catapp; script/create.pl model CatalystModelName...
![Page 20: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/20.jpg)
DBIx::Class - Caveats
• wrap insert/updates in eval {} or x; blocks
• txn_do($code_ref) (still inside eval block)
• left outer joins by default. (null rows removed)
• Understand when a query is a row or a result set. They behave differently.
• inflate/deflate only works on rows, not $rs
• relation columns can have same name as variables. causes you to have to do $row->get_column
![Page 21: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/21.jpg)
Learning TT
• Think Cool PHP (if it was perl)
• http://template-toolkit.org/docs/manual/
• Syntax differences can be subtle and TIMTOWTDI
• Catalyst will do all the heavy lifting here.
• Mailing list is very friendly.
![Page 22: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/22.jpg)
TT: Advice from the trenches
• “Spend a solid day or more dedicated to learning Template::Toolkit. Andy Wardley has done an amazing job at creating and maintaining this system. By spending a little time up front, it will save you a TON of time in the future and get you familiar with its capabilities. Learn how to make it sort arrays, sort hash keys, loop through keys of hashes, etc - things that are vital to the template development for web applications.”
• TT manual can be confusing to the beginner.
• “some basic concepts were buried in the middle of examples of various features, so I had to read about almost everything before I could do more than the trivial.”
• “I felt the TT language had too many syntactic variations for accomplishing the same thing, and again the examples randomly chose which version they used in order to demonstrate this breadth, but until you get to the part which explains the equivalences, you are left scratching your head trying to figure out what the semantic differences are.”
• “Aside from that, TT seems to be a powerful, stable templating system.”
![Page 23: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/23.jpg)
TT - Trenches II
• Case matters: [% FOR foo IN chairs.keys.sort %]
• Prevent line wrap: [% author -%]
• Altering the stash is local (sometimes): [% foo = ‘HELP’ %]
![Page 24: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/24.jpg)
Learning Catalyst (are we there yet?)
• CPAN Tutorial
• Catalyst Examples from SVN
• Catalyst Advent Calendar
• Johnathan Rockway’s Book “Catalyst”
• Initial setup is 90% of the battle
• catalyst.pl AppName
![Page 25: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/25.jpg)
Cost Justification for the learning Curve
• Multi-developer approach easier when code is broken into multiple modules
• Automated testing setup for you
• Stupid DBI mistakes no longer a risk (Now it’s stupid DBIC mistakes but they fail more consistently)
• DB Structure separated from code in one place.
• Annoying HTML page setup for each CGI script centralized
• Model structure allows for easy re-use outside of Catalyst
• Application portable to multiple developers with self contained web server.
![Page 26: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/26.jpg)
MVC
• Spend time thinking about and talking about the demarcation for MVC
• http://en.wikipedia.org/wiki/Model-view-controller
![Page 27: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/27.jpg)
View
• What presents the data to the user
• HTML
• JSON
• file download
• csv
![Page 28: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/28.jpg)
Controllers
• Air traffic control of the application
• Receives user input
• Authenticates and authorizes access with help of models
• Requests information from models based on user input
• Data cleansing of user input
• Formats data (input and output) to present back to view for display
![Page 29: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/29.jpg)
Models• Library of calls for controllers and outside programs to:
Query/update/insert information by:
• Invoking programs
• Accessing databases
• Messaging (email, IM)
• Maintains Data integrity where pkey/fkeys stop.
• Enforces authorization based on controllers provided credentials
• Centralizes auditing of events
![Page 30: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/30.jpg)
Lessons learned from Controller Development
• Do all authentication and authorization in root.pm via auto (Private)
• Take care not to do view or model work in the controllers. This is the easiest place to do too much
• Common Controller routines or repetitive code sections indicates you have a model routine
![Page 31: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/31.jpg)
Lessons learned from Model Development
• Pod EVERYTHING. It takes a very short time to forget what it does or why it’s there or how to use it.
• Inheritance is probably not the right solution - Use Moose Roles instead to separate the sections of your models for better management
• Models provided us a way to standardize back end interface to on-boarding.
• Create a common library to validate hash passed to model
• my $params = validate_hash(shift, qw/date_inserted id owner on_hold/) or return toss(“reason...”);
![Page 32: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/32.jpg)
Is it a model or controller job?
• Whether you agree with all of this doesn’t matter. What matters is you become aware of the coding issues and decide which way you’re going to go.
![Page 33: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/33.jpg)
Model test suite
• Can be difficult to create test databases complicated enough to validate functionality or replicate errors.
• Clear /re-build database between tests
• input/output can be complex. Build helper to make life easier (test_model_func({model=>.., func => ..., args => ...., toss =>...,}))
• Controller bugs require a test to fix in model
![Page 34: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/34.jpg)
Apache + FastCGI
• Allows app to be scalable.
• Multiple perl engines can run in the back end
• Engines can be on external hosts
• Scales down connections on idle
• Remember [% c->uri_for(‘/’) %] in all your templates
• Catalyst::View::TT->CATALYST_VAR
![Page 35: Yapc10 Cdt World Domination](https://reader034.vdocuments.us/reader034/viewer/2022042623/54b4a6b74a7959a0168b4593/html5/thumbnails/35.jpg)
In the end
• Splitting up areas of responsibility between controllers and models helps enforce:
• documentation
• interface standards
• testing