one year later: reflections on developing with extbase and fluid
DESCRIPTION
Following the 2010 TYPO3 conference in Dallas, Cast Iron Coding decided to drop development of its internal MVC framework and base all future development on ExtBase. In the past year, we have encountered challenges with ExtBase as well as opportunities, and the purpose of this talk will be to share with the audience the lessons we've learned as well as some of our successes. A large part of the talk will be devoted to showing what's possible with ExtBase in real-world settings, including discussions about1. using ExtBase with an Oracle database backend and DBAL/ADOB2. utilizing ExtBase in backend scheduler tasks3. moving to ExtBase, ExtJS, and ExtDirect for backend module development4. Replacing HTML elements in the backend with FLUID HTML elements5. Moving from TemplaVoila to fluid-based page templates; and6. Our experience using ExtBase with an AJAX-driven frontend plugin that allows users to search data stored in the client's Salesforce account.Rather than go into great technical detail, I will discuss what we learned from these various examples. However, I will provide audience members with some code examples that may be useful for future projects of their own. Finally, I will discuss how the switch to ExtBase impacted our business from a productivity/revenue standpoint, and discuss what sorts of projects we see as a good fit for ExtBase.TRANSCRIPT
A Year LaterReflections on Developing with ExtBase and Fluid
Date: June 10, 2011
Author: Zach Davis, Cast Iron Coding, Inc.
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #1
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #2
•Located in Portland, Oregon
•Founded in 2004 with two employees; now at seven employees (six
developers) plus local sub-contractors
•95% of all sites we build are built with TYPO3
•While we are a full-service agency, CIC’s core competency is
development; we have developed hundreds of TYPO3 extensions for
our clients.
•Our clients are drawn from a variety of industries but we have some
specialization in non-profit and higher education sites
About Cast Iron Coding and how we came to TYPO3
Background
Background
About CIC, about this talk
The goal of today’s talk is to share some of our experiences with ExtBase/Fluid and to discuss specific uses of ExtBase/Fluid that have been helpful on recent projects.
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #3
•Low quality extension—hard to configure, lack of Typoscript abstraction, presentation layer hard-coded into code—
extensions in the extension library meant that for most projects we ended up developing custom extensions for clients.
•Consequently, the cost associated with pretty standard features was often too high for clients, especially for smaller clients
on tight budgets
•This made it increasingly more difficult for us to compete with Wordpress , which has increasingly come to be seen as a
viable CMS, on smallish (under 15k) projects.
•Instead of hoping for better extensions in the TER, we chose to devote time to finding a way to a) reduce the time required to
develop extensions and b) increase the amount of code that was reusable between projects (or make sure we were writing
code that was focused on the domain)
In 2008, we needed a way to reduce development time and costs to remain competitive with other open source systems such as Wordpress (for smaller projects) and Drupal or Joomla (for larger projects)
There were a number of challenges that we experienced working with TYPO3 in 2008 and 2009:
CICMVC: Our first attempt at MVC in TYPO3
CICMVC
Our first attempt at MVC in TYPO3
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #4
In response to these challenges, CIC developed CICMVC to replace piBase.
CICMVC is a light-weight, high-performance MVC layer that sits on top of piBase. Some parts of it worked really well for us:
CICMVC: Our first attempt at MVC in TYPO3
CICMVC
Our first attempt at MVC in TYPO3
•Few constraints, which isn’t always a bad thing! Because we can ensure quality control in-house, we didn’t need to include
the sorts of constraints that are present in FLUID and ExtBase.
•Therefore, CICMVC uses PHP views—not so popular these days, perhaps, but not much more syntactically difficult than
FLUID, and quite a bit faster.
•Core classes of CICMVC include: controller, model, helper, view, peer, query, form, requests, request filters, exceptions, email
and partials.
•Built-in models and peers for common objects (Digital asset and digital asset peer, user models, countries and states, etc).
•Separate pi classes (pi1, pi2, etc) and page types are used for different output modes (HTML, JSON, XML, etc).
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #5
CICMVC: Our first attempt at MVC in TYPO3
CICMVC
Our first attempt at MVC in TYPO3
•Database accessed by way of “peer” classes which relied on query objects. In most cases, records could be received easily
without writing much SQL . The query object used arrays to store the query components. In more complicated cases, the
query object and the peer class offer easy, direct access to the underlying TYPO3 DB class. We _never_ ran into performance
problems in relation to database access.
•Basic security accomplished with basic request filters. Variables coming from GET or POST need to be defined as strings,
integers, etc, and are validated as such before being passed to controllers. For most extensions, this was sufficient.
•CICMVC was not an academic exercise. It was a real-world attempt to dramatically reduce development time in order to
increase company revenue while also reducing client cost
•More lightweight than ExtBase, feels significantly faster and more responsive.
•The year after we started using CICMVC widely across all projects, we saw revenue increase by about 75% while the
company stayed relatively small. Without wanting to mistake correlation for causation, our strong sense is that CICMVC had a
huge effect on CIC’s bottom line.
Some parts of it worked really well for us:
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #6
CICMVC: Our first attempt at MVC in TYPO3
CICMVC
Our first attempt at MVC in TYPO3
•Lack of an ORM meant that persisting objects from the TYPO3 front-end was never as easy as it should have been. Instead of
persisting objects automatically, we built a form class that made error handling, validation, etc easier. (Creating and persisting
objects from a front-end form was the developer’s responsibility—not the framework’s—which made sense because there
often is not a simple 1-1 relationship between form and model.)
•Because it was a custom solution without a community behind it, our clients had to place a great deal of trust in their long-
term relationship with CIC in order to use CICMVC as the foundation for a large project.
•CICMVC was not an academic exercise. If developers weren’t careful, business logic could wind up in a view or presentation
logic could wind up in a model. Lack of constraints required CIC to place a lot of trust in its developers, even with internal QA
processes.
•Limited feature set and lack of configurability
•Lack of modularity: view helpers were stored in one large “helper” class rather than as individual classes
•Magic getters and setters on models were both a blessing and a curse
Other parts weren’t so great...
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 Conference
CICMVC
A few examples
Slide #7
Some CICMVC ExamplesEmory Center for the Study of Law and Religion
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 Conference
CICMVC
A few examples
Slide #8
Some CICMVC ExamplesCarnegie Corporation of New York
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 Conference
CICMVC
A few examples
Slide #9
Some CICMVC ExamplesCarnegie Corporation of New York
Models and
Peers
Modules
PublicationViews
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #10
We make the leap
Goodbye CICMVC, hello ExtBase!
In mid-2010, we drop CICMVC and switch to ExtBase
Oracle Storage Backend Fluid Page Templates Backend Reporting Tools
Custom ExtBase Content Elements
Tides: Project Directory and SFDC
Tides: Impact Stories
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #11
In mid-2010, we drop CICMVC and switch to ExtBase
Goodbye CICMVC, hello ExtBase!
We make the leap
•We wanted clients to have a solutions that other TYPO3 developers could support. It’s already a challenge to sell US
customers on TYPO3 with only a handful of vendors here; further lock-in due to a custom extension framework makes that
pitch even more difficult.
•We wanted to give back to the TYPO3 project, which we didn’t feel was possible with CICMVC. Originally, before ExtBase,
we thought we might be able to share CICMVC eventually. However, once ExtBase took off, it was clear that another MVC
solution wouldn’t be helpful.
•We bought into the thinking behind Domain Driven Design. Staying current in terms of best practices and writing code that
can be changed without extensive refactoring keeps us competitive. DDD pushes us to think more conceptually, which
makes us better developers (and, IMO, better thinkers!)
We had a number of reasons for making the leap
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #12
In mid-2010, we drop CICMVC and switch to ExtBase
Goodbye CICMVC, hello ExtBase!
We make the leap
•Performance, performance, performance! Would our clients perceive ExtBase extensions as slow and under performing?
•We were concerned about decreased productivity. The kickstarter wasn’t quite there yet, which meant having to do more
work to setup tables. Writing getters and setters on models is a pain. Fluid syntax took some getting used to and writing view
helpers can be time consuming.
•We were concerned that ExtBase wasn’t stable enough, even in late 2010, and that APIs would change after we’d done a
project.
•We worried that the persistence layer was too restrictive, and that some of the approaches expected by DDD were not well
suited for real-world applications.
•We suspected that ExtBase would not play nice with other key TYPO3 features, especially localization and Workspaces.
And some concerns...
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #13
Our first attempt: ExtBase content elements
ExtBase Content Elements
Hard to do with piBase, easy with ExtBase
We thought we’d take something difficult—creating editor-friendly, custom content elements—and see if we could make it easier by using ExtBase
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #14
Sidebar CalloutContent Element
ExtBase Content Elements
Hard to do with piBase, easy with ExtBase
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #15
Custom content elements defined in ext_tables.phpext_tables.php:
ext_localconf.php:
ExtBase Content Elements
Hard to do with piBase, easy with ExtBase
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #16
Define actions in ContentController class
ExtBase Content Elements
Hard to do with piBase, easy with ExtBase
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #17
Setup a Fluid view for the content element
ExtBase Content Elements
Hard to do with piBase, easy with ExtBase
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #18
Our first attempt: ExtBase content elementsUsing the same approach, we created a handful of similar elements.
ExtBase Content Elements
Hard to do with piBase, easy with ExtBase
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #19
•Editors LOVE these content elements. Without much work on our part, clients get a library of custom content elements that
force them to be consistent without being overly constrained.
•After you create the first one, additional custom content elements are very cost effective. We include a placeholder
extension called “t3site” in our starter package that contains a controller for custom content elements and a sample custom
content element. The ext_tables config can be easily reproduced and new content elements can be created that use the
same controller.
•Someday, we should create a model for tt_content records that could be shared across these different content element
types.
•We save time (thereby making our work more profitable) because we don’t have to set up classes in the RTE that allow
editors to create similar element with <P> tags and whatnot, and because we don’t have to map and maintain a bunch of
FCEs.
•Editors get a better user interface because instead of flexforms, they get standard TYPO3 tt_content records with the
conventional fields that they’re used to.
Our first attempt: ExtBase content elementsWe learned a few things from these custom content elements
ExtBase Content Elements
Hard to do with piBase, easy with ExtBase
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #20
Our second attempt: a standard extBase search/listplug-inStructurally similar to the Carnegie publications plug-in, Tides project directory and impact stories seemed like a good candidate for moving away from CICMVC
•Built around quite a few simple models: Constituents, GeographicScopes, Issues, Programs, Projects, Strategies, Services,
and Stories
•Data for project directory is stored in Salesforce, pulled into TYPO3 every couple hours via Salesforce’s SOAP web service
• Story views include issue list, list of stories, search results (for AJAX requests), single story, and story teaser box
•Project directory views include list of projects, lat/long coordinates (for Google Maps request, output as JSON), search
results (for AJAX requests), and single project.
ExtBase Front-end Plugins
A typical ExtBase use case
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #21
Tides Project Directory
ExtBase Front-end Plugins
A typical ExtBase use case
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #22
Tides Issues Stories
ExtBase Front-end Plugins
A typical ExtBase use case
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #23
Our second attempt: a standard extBase search/listplug-in
One of ExtBase’s most important (and overlooked) capabilities is that it can unify back-end and front-end functionality. There’s some hackery involved, but using the same repositories in both sides of TYPO3 as well as ExtBase’s persistence layer saves considerable time.
ExtBase Front-end Plugins
A typical ExtBase use case
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #24
Our second attempt: a standard extBase search/listplug-inNonetheless, ExtBase’s persistence layer wasn’t sufficient for handling the complex queries (not all that complex, perhaps?) generated by the search interface.
ExtBase Front-end Plugins
A typical ExtBase use case
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #25
Oracle Storage Backend for ExtBase
It worked!
Our third attempt: Oracle ExtBase Storage Backend Client needed a searchable white paper/video library. We couldn’t, in good conscience, create piBase extensions for this, and felt that we had to use ExtBase. One requirement was that TYPO3 be running on an Oracle DB
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #26
•Out of the box, DBAL pretty much ignores ExtBase because ExtBase pretty much ignores the TYPO3 DB class. Instead of using
methods like exec_SELECTquery, the ExtBase persistence layer uses passes SQL directly to the sql_query method on TYPO3’s
DB class. DBAL, as I understand it, doesn’t really parse these queries.
•So, instead of trying to use these DB methods, we followed ExtBase’s example and made a OracleDBBackend Class that
generates queries that conform to Oracle’s query syntax.
•Because the objects don’t change on the front-end, we didn’t implement addRow, updateRow or other save-related
methods. Those are on our roadmap
•Setting a different storage backend for Extbase in ext_localconf isn’t hard (and while it works, this probably shouldn’t be
considered a public API):
Our approach:
Our third attempt: Oracle ExtBase Persistence Layer
Oracle Storage Backend for ExtBase
It worked!
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #27
•Avoiding DBAL as much as possible is definitely the way to go. Instead of making DBAL smarter about parsing ExtBase
queries, it’s easier to just make ExtBase generate syntactically correct Oracle queries.
•We ran into more problems with TYPO3/Oracle/DBAL in general than we did with ExtBase specifically, once we
implemented our storage backend.
•Performance, while not great, is tolerable. Because of the keyword search, the output is essentially uncached
•Work involved with this was really not all that difficult. We probably spent about 2 developer days on this.
•Not ExtBase related, per se, but worth mentioning: if TYPO3 is to truly succeed as an “enterprise” CMS, it needs better
Oracle support. Developers who have worked on DBAL have done a great job, but there’s more to do. Would be nice if
companies using TYPO3 and Oracle could pool resources.
What we learned:
Our third attempt: Oracle ExtBase Persistence Layer
Oracle Storage Backend for ExtBase
It worked!
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #28
•Uses FLUID for page templates
•Does not require mapping mechanisms like TV, automaketemplate, parts and subparts, and other approaches
•Instead, Typoscript Objects are made available to the Fluid page template, just as objects are made available to fluid
templates in ExtBase
•Editors use backend layouts for the editing interface, which is more consistent with TYPO3’s overall back-end UI than TV
•FCEs are no longer needed because we have custom content elements, which are just as easy to setup, easier to use, and
easier to track in Git
•We’re looking for a column solution (like Bernhard’s nested content elements extension or Snowflake’s columns extension in
the TER).
•Relationship between tt_content and pages is obscured in XML, which makes some tasks difficult
We’ve never been entirely satisfied with TemplaVoila. FCEs mean flexforms, and flexforms provide an inconsistent UI for editors. TV stores configuration (mapping) in the DB, so we can’t easily track it in Git. The classic page module and it’s limitations was a non-starter.
We needed a new approach, so we started building the fluid pages extension.
Our fourth attempt: Fluid Page Templates
Fluid Page Templates
Lots of possibilities for continued development
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #29
Our fourth attempt: Fluid Page Templates
Fluid Page Templates
Lots of possibilities for continued development
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #30
Our fifth attempt: Reports
Reports
A little ExtBase, a lot of ExtJs and ExtDirect
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #31
Conclusions
Our first attempt at MVC in TYPO3
•On low to medium traffic sites, with relatively flat data, ExtBase is clearly a better choice than piBase
•Even with more complex data or with larger sets of data, there’s a good chance that you can circumvent ExtBase’s storage
backend in your repositories by writing your own queries, especially if the application is read-only on the front-end.
•We get greater code reusability in ExtBase as we add models, services, and view helpers to our own ExtBase helper
extension.
•We think that Fluid, in particular, should be more widely used throughout TYPO3. It’s silly (and super-confusing for
newcomers) that we have so many templating methods in TYPO3 (TemplaVoila, parts/subparts/markers, and FLUID). TYPO3
should try to standardize on FLUID and continue to develop the FLUID TEMPLATE cObj so that it can be more useful as a tool
for page templates.
Overall, results are positive. We’ll continue using ExtBase!
Conclusions!
Friday, June 10, 2011
Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #32
Conclusions
Our first attempt at MVC in TYPO3
•Sharing models and repositories between the front-end and the back-end can be a huge time saver, especially when you
have to import data into TYPO3 from an external system for display on the front-end
•As recent discussions have shown, the persistence layer is a real pain point for ExtBase developers. Given TYPO3’s
idiosyncrasies, I have my doubts about the possibilities offered by Doctrine. Instead, I’d like it to be EASIER for developers to
get low-level database access and to create objects out of query results in the repositories (thereby moving away from the
one-to-one model / DB table relationship).
Overall, results are positive. We’ll continue using ExtBase!
Conclusions!
Friday, June 10, 2011