tales from the postgres front - and what we can learn

24
© 2013 EDB All rights reserved. 1 Tales from the Postgres Front Marc Linster | September 2014

Upload: enterprisedb

Post on 19-Nov-2014

226 views

Category:

Software


5 download

DESCRIPTION

Studying when and why PostgreSQL users – even veteran pros – call us for help can go a long way toward educating the community on emerging skill sets and rising demand for specific kinds of expertise. This presentation was delivered by Marc Linster, Senior Vice President, Products and Services at EnterpriseDB. He explains why database pros should make friends in the worlds of operating systems and storage area networks, why their knowledge of Linux should extend beyond Red Hat, and why they need to learn more than a few tuning tips. Mining the support ticket database of EnterpriseDB, a major Postgres software and services provider, gives great insight into what DBAs need to know and what awaits if they don’t heed the advice.

TRANSCRIPT

  • 1. Tales from the Postgres FrontMarc Linster | September 2014 2013 EDB All rights reserved. 1

2. Agenda EnterpriseDB Overview (quick) Migrations Postgres Support Experiences Health ChecksWhy are we doing this? Share some of our metrics with the community Learn how to resolve customer problems faster Drive our product strategy to make customer problems go away 2013 EDB All rights reserved. 2 3. Postgres PlusAdvanced Server Postgres PlusManagement Performance 2014 EnterpriseDB Corporation. All rights reserved. 3Cloud DatabaseHigh AvailabilityREMOTEDBA 24x7SUPPORTPROFESSIONALSERVICESTRAININGEDB ServesAll Your Postgres NeedsPostgreSQLSecurity 4. POSTGRESinnovationServices& training 2014 EnterpriseDB Corporation. All rights reserved. 4ENTERPRISEreliability24/7supportEnterprise-classfeatures & toolsIndemnificationProductroad-mapControlThousandsof developersFastdevelopmentcyclesLow costNo vendorlock-inAdvancedfeaturesEnabling commercialadoption of Postgres 5. Migrations 2014 EnterpriseDB Corporation. All rights reserved. 5 6. Migrations Majority of interest for migrations from Oracle, followed by MSSQL, DB2 (few) and MySQL (rare) Rapid increase in interest from MS SQL Server in 2014 Obstacles Missing language equivalents (e.g., autonomous_transaction, merge) Scalability (vertical and RAC-like) Need additional constructs in compatibility layer (EDB proprietary 10thgeneration) Greatly facilitated by Object Relational Frameworks, such asHibernate great indicator for easy migration to Postgres Experience shows that 70%+ of migrations can be executedquickly and result in comparable performance 2014 EnterpriseDB Corporation. All rights reserved. 6 7. Postgres Support Experiences 2014 EnterpriseDB Corporation. All rights reserved. 7 8. Why Analyze Support Tickets Adoption of Postgres is growing fast More and more Postgres users are not not dyed in the wool Postgresusers they come from Oracle, MS SQL, DB2, etc. Postgres is moving into more complex and more mission-criticalapplications Question patterns are changing from simple to complex Expertise level is changing when the customer calls, it is with a(more) difficult problem or urgent issue How do we make Postgres users even more successful? Targeted white papers & how-to instructions focused on non-Postgresusers Pro-active training for users as part of our customer enablementprocess Balanced product strategy of new features, new capabilities andusability 2014 EnterpriseDB Corporation. All rights reserved. 8 9. Ticket Volumes & Multi Year Growth Q1 2014: ~ 2500 total customers Analysis in this presentation is based on a representativesample of tickets raised over a 12 months period 2014 EnterpriseDB Corporation. All rights reserved. 9 10. Tickets and PlatformsObservations Postgres Plus Advanced Server: Strong emphasis on proprietary or commerciallysupported OSS PostgreSQL: Significantly less emphasis on proprietary or commercially supportedOSS even though the customer has commercially supported PostgresDistribution 2014 EnterpriseDB Corporation. All rights reserved. 1077%Linux 11. When do customers call? 2014 EnterpriseDB Corporation. All rights reserved. 11 12. What Do Customers Call About100%90%80%70%60%50%40%30%20%10%0%What are customers calling about?POC Dev Deployment Maintenance 2014 EnterpriseDB Corporation. All rights reserved. 12TuningProduct AwarenessHow toEnhancementsCorruptionBug# Tickets PhaseCategory POC Dev Deployment Maintenance Grand TotalBug 8% 22% 9% 7% 10%Corruption 0% 0% 0% 6% 2%Enhancements 3% 4% 1% 0% 1%How to 41% 33% 41% 29% 35%Product Awareness 46% 39% 44% 39% 41%Tuning 3% 2% 5% 19% 10%Grand Total 100% 100% 100% 100% 100% 13. Where Do Users Need Help# Tickets PhaseCategory POC Dev Deployment Maintenance Grand TotalConnectors 0% 2% 1% 0% 3%Bug 0% 0% 0% 0% 1%How to 0% 1% 0% 0% 1%Product Awareness 0% 1% 0% 0% 1%Database 10% 9% 17% 37% 73%Bug 1% 2% 1% 2% 6%Corruption 0% 0% 0% 2% 2%Enhancements 0% 0% 0% 0% 1%How to 4% 3% 6% 10% 23%Product Awareness 4% 3% 8% 15% 31%Tuning 0% 0% 1% 7% 9%Replication 1% 4% 4% 2% 12%Bug 0% 1% 0% 0% 2%How to 0% 0% 3% 1% 4%Product Awareness 1% 2% 1% 0% 4%Tuning 0% 0% 0% 1% 1%Utilities 1% 3% 6% 3% 12%Bug 0% 1% 1% 0% 2%Enhancements 0% 0% 0% 0% 0%How to 1% 1% 3% 1% 6%Product Awareness 0% 1% 3% 1% 5%Grand Total 12% 17% 28% 42% 100% 2014 EnterpriseDB Corporation. All rights reserved. 13 14. Select Top questions (not in rank order) Deployment/Database/Product Awareness & How To How to set up connection pooling? How to set up Hot Standby and Streaming Replica? How to add table spaces? How to partition? How to upgrade from major version? Maintenance/Database/Product Awareness & How To How to find bloat in tables & indexes? How to enable auditing selectively without performance impact? What are the effects of changes to the values of postgresql.conffile parameters wal_writer_delay, bgwriter_delay? What is the correct configuration of hba_conf? 2014 EnterpriseDB Corporation. All rights reserved. 14 15. Lessons Learned from the AHC 2014 EnterpriseDB Corporation. All rights reserved. 15 16. Architectural Health Check Systematic top-to-bottom review of the customersPostgres production installation Identifies issues and improvement opportunities in: Security Performance Backups Replication Maintenance Upgrades The following analysis is based on a set of 20 AHCsexecuted in 2013 2014 EnterpriseDB Corporation. All rights reserved. 16 17. 100%90%80%70%60%50%40%30%20%10%0%effec ve_cache_sizeTopDatabaseParameterAdjustments-OrderedbyFrequencywal_buffersmaintenance_work_memshared_buffersbgwriter_lru_maxpageswork_memrandom_page_costcheckpoint_segmentsbgwriter_lru_mul pliercpu_tuple_costcheckpoint_comple on_targetcheckpoint_ meout 2014 EnterpriseDB Corporation. All rights reserved. 17Red bars: experts expected topparameters, includingvacuum_cost_limit, log_line_prefix,log_min_duration_statement,log_checkpointscheckpoint_warningautovacuum_vacuum_cost_delayautovacuum_vacuum_thresholdprefixwaitsstatementfactorline_lock_scale_log_log_dura vacuum_on_log_autovacuum_min_autovacuum_nap melog_autovacuum_min_dura on 18. Kernel Parameters1614121086420 2014 EnterpriseDB Corporation. All rights reserved. 18 19. Postgres Platform DistributionPostgreSQL, 2014 EnterpriseDB Corporation. All rights reserved. 1917Postgres PlusAdvancedServer, 3Postgres PlusCloudDatabase, 1 20. Backup and StandbySnapshot7%Backup Approach 2014 EnterpriseDB Corporation. All rights reserved. 20Standby Approachhot47%None18%cold12%warm23%PhysicalOnly36%LogicalOnly29%FSPhysicalandLogical14%None14%43% of customers could not executePITR (prior to AHC) some had NObackup in place30% of customers would have beenslow or unable to recover from failure 21. Maintenance TasksMissingIndexesBloated Tables 2014 EnterpriseDB Corporation. All rights reserved. 21UnusedIndexes39%BloatedIndexes30%13%9%MissingPK9%Bloat management continuesto be a problem 22. Observations from the AHCs Windows was less prevalent (10%) than indicated insupport tickets may be due to a focus on Windowsfor development versus production Most DBAs do not appear to be familiar enough withthe storage system/SAN, and the DAS/SAN/Fiberchannel/iSCSI/NFS decisions OS System Adjustments (dirty_background_ratio &dirty_ratio) are also a major adjustment (70% & 45%) indicating the need for more cooperation betweenDBAs and Sys Admins Customers would benefit from targeted educationabout Postgres-specific OS parameters and storagesystem/SAN guidance 2014 EnterpriseDB Corporation. All rights reserved. 22 23. Summary The analysis helped us understand where we need tocollect more data during our support interactions tobetter pinpoint problem trends and drive knowledgebase development This will drive our product development and featureprioritization so that we can make customer problemsgo away 2014 EnterpriseDB Corporation. All rights reserved. 23 24. Thank you Many EDB Team Members contributed to this analysis Thom Brown Deepanshu Sharma Robert Haas Kevin Grittner Gabrielle Roth Contact Marc [email protected]+1 (617) 306 6059 2014 EnterpriseDB Corporation. All rights reserved. 24