the mongodb tools company - studio 3t...andrew davidson, vp, cloud products, mongodb “the biggest...

19
MongoDB Trends Report 2020 3T Software Labs The MongoDB Tools Company

Upload: others

Post on 19-Jan-2021

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

MongoDB Trends Report 2020 3T Software Labs

The MongoDB Tools Company

Page 2: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

Table of Contents

Key Findings

MongoDB and other database technologies

Polyglot data in the cloud

Data differences in the cloud

Moving MongoDB to the cloud

Concerns about cost vs. performance

MongoDB Atlas and the DBaaS wave

Methodology

Further Reading

3

4

8

9

10

11

13

14

18

With commentary from

Matthias GelbmannCo-FounderDB-Engines

Page 7

Andrew DavidsonVP, Cloud ProductsMongoDB

Page 9

Kimberly WilkinsMongoDB Technical LeadPercona

Page 11

2

Page 3: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

Overview

This report reviews MongoDB usage trends, including cloud adoption and MongoDB’s use alongside traditional relational database technologies and newer, NoSQL database technologies. The results are based upon over 16,000 responses from developers, DevOps engineers, DBAs, systems architects, C-level executives and business founders, all running MongoDB in production. They were collected through a series of online surveys conducted between June 2017 and September 2020.

Key Findings

3

• The number of developers abandoning SQL technologies continues to increase.

Page 6

• NoSQL databases are being moved to the cloud faster than their SQL counterparts.

Page 9

• Europe (EMEA), arguably because of GDPR, is significantly behind the rest of the world in moving data to the cloud.

Page 10

• Growth of MongoDB Atlas, especially the paid tier, continues at breakneck speed.

Page 12

• MongoDB's considerable growth in the last few years has been accompanied by a rising tide in NoSQL technologies across the board, often at the expense of less scalable SQL systems.

Page 6

Page 4: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

MongoDB and other database technologies

4

Page 5: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

With the growth of analytics and the rise of big data, it is unsurprising that database sizes are growing faster than ever. In just two years we’ve seen fewer and fewer MongoDB users with small databases of less than 10 gigabytes, and more and more users with large databases. In 2017, less than 3% of MongoDB users reported their largest database was over 1 terabyte. In 2020, that number has tripled to over 9%.

Databases are getting bigger, but developers are also using more of them, employing more database technologies every year. In 2017 a MongoDB user only used on average 2 different technologies, either relational or non-relational. In 2020, the average is over 3 per user.

5

20 %

40 %

60 %

2017 2018 2020

9.3 %

4.0 %2.8 %

42.7 %

53.3 %56.2 %

Less than 10GB10GB - 100 GB100GB - 1TBOver 1TB

Figure 1.01 Size of largest database over time

Figure 1.02 Average number of different database technologies per MongoDB user over time

0.0

1.0

2.0

3.0

4.0

2017 2018 2019 2020

All databasesNoSQL databasesSQL databases

Page 6: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

On the one hand there’s been a 26% increase in developers abandoning SQL altogether, reporting “We don’t use SQL.” On the other hand, there’s been a 20% decline in users who claim MongoDB is the only NoSQL technology they use.

The overall picture looks like a steady move toward polyglot database estates that deploy the right database for the job at hand. With this burgeoning data-Babel, it’s the big NoSQL workhorses of Redis, Elasticsearch, and Cassandra that are being co-opted alongside MongoDB, at the expense of MySQL and Microsoft SQL Server. AWS DocumentDB, launched in January 2019, has also grown fast.

6

Figure 1.03 SQL databases used alongside MongoDB by technology

MySQL

PostrgeSQL

Microsoft SQL Server

SQLite

Other

We don't use SQL20.1 %

21.9 %

13.5 %

25.4 %

37.7 %

45.4 %

16.0 %

21.3 %

15.6 %

26.2 %

30.7 %

52.7 %

20182020

Figure 1.04 NoSQL databases used alongside MongoDB by technology

Redis

Elasticsearch

DocumentDB/DynamoDB

Cassandra

Other

We only use MongoDB36.6 %

4.7 %

7.5 %

15.7 %

24.0 %

42.4 %

45.7 %

9.1 %

6.3 %

1.1 %

23.5 %

33.3 %

20182020

Page 7: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

Nonetheless, as quickly as MongoDB and similar NoSQL technologies have grown, and as optimistic as their forecasts often are, 79% of MongoDB users surveyed in 2020 report running at least one flavour of SQL database alongside.

MySQL remains the leading relational technology for MongoDB users, with PostgreSQL and Microsoft SQL Server coming after. Oracle, in the last two years, appears to have reversed a long slide against Microsoft SQL Server.

7

Opinions: Coexistence

Matthias Gelbmann, Co-Founder, DB-Engines

“There is, however, a stronger preference for open source systems (MySQL, PostgreSQL, Redis, Elasticsearch) among MongoDB users. The overall popularity, on the other hand, comparatively favors commercial systems (Oracle, SQL Server).

That is not surprising, as we see open source systems getting more popular, particularly among NoSQL systems (see https://db-engines.com/en/ranking_osvsc). Specifically, if we add up the popularity scores of Document Stores, it's 77.3% for open source systems, whereas the same figure for Relational DBMS is only 40.7%

When it comes to SQL vs NoSQL, I would like to add that we see more and more SQL systems also supporting non-relational data models. Of the SQL systems in the top 10 of our ranking [at DB-Engines], only SQLite is a pure relational system. All the others additionally support other models such as Document Stores and Graph DBMS, thus effectively turning them into multi-model systems.

What we also see is that MongoDB is still gaining popularity, which means this trend to multi-model systems does not make NoSQL systems obsolete. However, the strict distinction between SQL systems and NoSQL systems will probably make less sense in the future.”

Figure 1.05 Relational database technology usage over time

0 %

15 %

30 %

45 %

60 %

2017 2018 2019 2020

2.5 %2.1 %0.5 %0.4 %

15.6 %11.8 %

17.2 %18.9 %

25.4 %28.5 %

26.2 %

0.0 %

37.7 %

20.3 %

30.7 %

25.3 %

45.4 %

39.1 %

52.7 %

43.7 %

MySQLPostgreSQLMicrosoft SQL ServerOracleIBM DB2

Page 8: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

Polyglot data in the cloud

8

Page 9: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

Data differences in the cloud

Asia and the Pacific (APAC) are leading the move to the cloud, with more users hosting both their relational and non-relational data in the cloud than any other region. Latin America (LATAM) and North America (NA) aren’t far behind, while users in Europe, the Middle East, and Africa (EMEA) lag, with far fewer users hosting any data in the cloud.

More users report hosting all of their non-relational data in the cloud than relational data, suggesting that non-relational technologies like MongoDB might be moving to the cloud faster than their relational peers, or that query performance in the cloud might be easier to achieve with NoSQL data structures than with traditional tabular data structures used in SQL databases.

9

Opinions: Cloud Adoption

Andrew Davidson, VP, Cloud Products, MongoDB

“The biggest driver behind distributed databases like MongoDB is that they are better suited to the fundamental scale-out/distributed nature of the cloud—and as companies move, they are using this as the opportunity to totally migrate away from systems designed for scale-up, pre-cloud era.”

Figure 2.01 Proportion of all data in the cloud by region

Some or all data in the cloud

63.1 %

71.0 %

75.0 %76.0 % APACLATMNAEMEA

Figure 2.02 Proportion of relational data in the cloud

All21.0 %

Some47.4 %

None31.6 %

Figure 2.03 Proportion of non-relational data in the cloud

All29.9 %

Some44.5 %

None25.6 %

Page 10: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

Moving MongoDB to the cloud

Given that non-relational technologies like MongoDB might be moving to the cloud faster than their relational peers, and given the success of DBaaS offerings like MongoDB Atlas—due in part to existing MongoDB users moving to the cloud—a pressing question is: how fast are they moving?

While about 20% of current MongoDB users are already using a cloud-hosted MongoDB service, the remainder are split about their intentions. 38% of users reported no plans to move to a MongoDB cloud-hosted service, compared with 41% who plan to move at some point.

Again APAC leads the move to the cloud, with more users who don’t currently use a cloud-hosted MongoDB service planning to move in the next 6 months to 2 years. EMEA lags far behind, with far fewer users planning to move to a cloud-hosted MongoDB service, and with more users who have no plans to move. This could be in part due to stricter regulations in the region, like GDPR.

In fact, it likely is. When polled about their concerns regarding moving to a cloud-hosted MongoDB service, EU was the only region in which more users were concerned about compliance with data privacy laws and regulations than securing the data. By contrast, the US/Canada market is a lot more concerned about physical data breaches than legal compliance.

10

Figure 2.04 Intent to move MongoDB to the cloud by region

20 %

40 %

60 %

No plans or distant plans to move Planning to move within 2 years

APACLATMUS/CANEMEA

Figure 2.05 What is your primary concern about moving to a cloud hosting service?

Higher costs than anticipated

Compliance with data privacy laws and regulations

Securing the data

Increased latency

Establishing and enforcing data governance

Other 2.0 %

10.7 %

17.2 %

26.7 %

23.9 %

31.9 %

1.5 %

9.2 %

18.9 %

22.6 %

25.7 %

38.4 %

EUUS/CAN

Page 11: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

Concerns about cost vs. performance

Although concerns about high costs was the most often-cited concern about moving to the cloud amongst those still thinking about it, it was a minor concern for those who have already taken the plunge.

11

Opinions: Moving to the Cloud

Kimberly Wilkins, MongoDB Technical Lead, Percona

“MongoDB remains popular with developers who want flexible schemas and need to get their applications up and running quickly. It is also popular with architects looking for flexible datastores that can fulfill multiple functions for their overall infrastructure needs. Our own recent survey [at Percona] showed that architects and developers are now the primary decision makers when it comes to selecting which databases companies use. This is a huge shift, moving decision-making away from DBAs and IT Managers who previously had this responsibility.

MongoDB is great for putting together applications that don’t fit relational database models. However, there are some challenges when it comes to managing the service over time. Underlying problems may not be resolved during the early stages and then not located until too late. These issues are often glossed over by the autoscaling features in many public cloud offerings.

Another element that many users don't initially consider is that running databases in public clouds prevents you fine-tuning your overall OS and hardware stack, and adjusting individual parameters or components when needed to attain the best performance.”

Figure 2.06 Concerns with moving to the cloud by stage of adoption

20 %

40 %

60 %

80 %

Cost-relat

ed

Security

-relat

ed

Perform

ance

-relat

edOther

No or distant plans to movePlanning to move within 2 yearsAlready cloud-hosted

Page 12: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

Globally, those already cloud-hosted seem to be significantly less concerned about costs. Instead, current MongoDB users who employ a cloud-hosted service cite performance as their primary concern.

MongoDB Atlas and the DBaaS wave

For the fourth year in a row, MongoDB reported continued growth from their Database-as-a-Service (DBaaS) offering, MongoDB Atlas. Since its release in 2016, Atlas has continued to grow in revenue, accounting for a record 39% of MongoDB’s total revenue in 2020, or approximately $164.5 Million.

Indeed, 2020 saw a record high for Atlas usage, with 22.64% of users on a MongoDB Atlas paid tier. As Atlas usage has grown over the years, the use of third party hosting services has been steadily declining, with only 3.64% of MongoDB users hosting their MongoDB instance with a third party service in 2020.

While some of this growth in Atlas usage can certainly be attributed to the growth of MongoDB as a technology, the growth in Atlas usage can also be attributed to MongoDB users with existing cloud-hosted deployments moving from 3rd party hosts to Atlas.

12

Figure 2.07 What is your primary concern with your cloud hosted MongoDB service?

Performance at scale

Controlling costs

Lack of control of the underlying db

Migrations are hard

Connections are unreliable

Other 5.4 %

4.4 %

9.6 %

10.1 %

20.9 %

30.0 %

Currently hosted with Atlas or 3rd party host

Figure 2.08 Which edition of MongoDB do you use?

0.0 %

5.0 %

10.0 %

15.0 %

20.0 %

25.0 %

2017 2018 2019 2020

Atlas paid tier3rd Party host

Page 13: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

This finding is consistent with the MongoDB 2019 Annual Report, which attributed Atlas’s growth to: customers from their 2018 acquisition of 3rd party hosting service mLab, new adopters of MongoDB, and existing customers adding workloads to Atlas.

If we look strictly at paying MongoDB customers, as MongoDB’s annual reports do, the growth of paid Atlas users is even clearer.

It’s clear that usage of for-pay MongoDB solutions are on the rise, but the majority of MongoDB end-users prefer not-for-pay options, including free editions of MongoDB, free cloud hosting services, and free tools for working with MongoDB data. In 2020, only 23% of MongoDB users pay for an Atlas paid tier or MongoDB Enterprise edition. Of users who prefer free options, the downloadable Community Edition remains almost 10 times as popular as the hosted Atlas M0 tier.

13

Figure 2.09 Growth in Atlas paid tier customers among MongoDB paid editions

0 %

25 %

50 %

75 %

100 %

2018 2019 2020

51.4 %

40.4 %

27.9 %

Atlas paid tier usersMongoDB Enterprise users

Figure 2.10 Usage of MongoDB editions and services

3rd Party host2.8 %

Other Atlas tier11.9 %

MongoDB Enterprise11.2 %

Atlas M0 tier7.2 %

MongoDB Community66.9 %

Page 14: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

Methodology

14

Page 15: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

This report comprises data from seven online surveys conducted between 2017 and 2020 and distributed to users of Robo 3T and Studio 3T software products. Surveys were distributed via a link available to all users in-app and also via email to select recipients. Participants were entered into a prize draw upon completion. Prizes varied from survey to survey. Respondents have agreed to either the Robo 3T privacy policy or the Studio 3T privacy policy, both of which can be reviewed online at https://robomongo.org/privacy-policy and https://www.studio3t.com/privacy-policy. Over 18,000 survey responses constitute the data used to create this report.

The figures in this report use the entirety of this dataset, with the following exceptions. Figures 2.08 Which edition of MongoDB do you use? and 2.09 Growth in Atlas paid tier customers among MongoDB paid editions use only respondents from surveys distributed to Studio 3T users—users of a licensed IDE for working with MongoDB—to more accurately represent the commercial interests around MongoDB paid tools and services. The figures in the section Data differences in the cloud use only respondents from a survey distributed to Robo 3T users. Except where indicated otherwise, survey responses are presented as a proportion of responses against the total number of respondents in a given year. Where responses are not tracked over time, data is from the year 2020.

Survey Demographics

Respondents work all over the globe, with the largest number working in: India, United States, Brazil, United Kingdom, and Germany. In total respondents work in 126 countries.

Figure 3.01 Geographical attendance

• Afghanistan • Albania • Algeria • American Samoa • Andorra • Angola • Argentina • Armenia • Australia • Austria • Azerbaijan • Bahrain • Bangladesh • Belarus • Belgium • Belize • Bolivia • Bosnia and Herzegovina • Brazil • Bulgaria • Burkina Faso • Cambodia • Cameroon • Canada • Chile • Columbia • Congo • Costa Rica • Croatia • Cuba • Cyprus • Czech Republic • Denmark • Dominican Republic • Ecuador • Egypt • El Salvador • Estonia • Ethiopia • Finland • France • Georgia

• Germany • Ghana • Gibraltar • Greece • Guatemala • Honduras • Hong Kong • Hungary • India • Indonesia • Iran • Ireland • Israel • Italy • Jamaica • Japan • Jordan • Kazakhstan • Kenya • Kosovo • Kuwait • Kyrgyzstan • Latvia • Lebanon • Lithuania • Luxembourg • Macau • Macedonia • Madagascar • Malaysia • Malta • Mauritius • Mexico • Moldova • Monaco • Mongolia • Montenegro • Morocco • Myanmar • Nepal • Netherlands • New Zealand

• Nigeria • Norway • Pakistan • Panama • Paraguay • People's Republic of China • Peru • Philippines • Poland • Portugal • Puerto Rico • Qatar • Romania • Russia • Rwanda • Saudi Arabia • Serbia • Singapore • Slovakia • Slovenia • South Africa • South Korea • Spain • Sri Lanka • Sweden • Switzerland • Taiwan • Tanzania • Thailand • Trinidad and Tobago • Tunisia • Turkey • Uganda • Ukraine • United Arab Emirates • United Kingdom • United States • Uruguay • Venezuela • Vietnam • Wallis and Futuna • Zimbabwe

15

Page 16: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

By far the most common job title held by survey respondents was Software Developer or Engineer, with representation from all levels of their organizations.

Respondents work across 38 distinct industries, most commonly in Computer Software and IT Services. Their employers include both small and medium sized businesses as well as large enterprise organizations.

16

Software Developer/Engineer

Team Leader

DevOps Engineer

CTO

Systems Architect

Data Scientist

Product Manager

Database Architect

Founder 2.0 %

2.1 %

2.6 %

3.4 %

3.9 %

5.3 %

6.3 %

6.7 %

67.8 %

Figure 3.02 Attendance by job title

Team Member Manager/Team Leader Director or C-Level

6.4 %

19.1 %

15.7 %

Figure 3.03 Attendance by job position within organization

Self-employa

ed1 -

1011 -

50

51 - 200

201 - 500

501 - 1,0

00

1,001 -

5,000

5,001 - 10,000

10,000+

2.9 %

1.3 %2.7 %

2.0 %

4.6 %

7.7 %

10.8 %

8.1 %

1.9 %

Figure 3.04 Attendance by company size

Page 17: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

List of Figures

MongoDB and other databases 1.01 Size of largest database over time 1.02 Average number of different database technologies per MongoDB user over time 1.03 SQL databases used alongside MongoDB by technology 1.04 NoSQL databases used alongside MongoDB by technology 1.05 Relational database technology usage over time

Data differences in the cloud 2.01 Proportion of all data in the cloud by region 2.02 Proportion of relational data in the cloud 2.03 Proportion of non-relational data in the cloud

Moving MongoDB to the cloud 2.04 Intent to move MongoDB to the cloud by region 2.05 What is your primary concern about moving to a cloud hosting service?

Concerns about cost vs. performance 2.06 Concerns with moving to the cloud by stage of adoption 2.07 What is your primary concern with your cloud hosted MongoDB service?

MongoDB Atlas and the DBaaS wave 2.08 Which edition of MongoDB do you use? 2.09 Growth in Atlas paid tier customers among MongoDB paid editions 2.10 Usage of MongoDB editions and services

Methodology 3.01 Geographical attendance 3.02 Attendance by job title 3.03 Attendance by job position within organization 3.04 Attendance by company size 3.05 Attendance by industry

5 5 6 6 7

9 9 9

10 11

11 12

12 13 13

15 16 16 16 17

17

Other32.1 %

Games Development1.2 %

Information Services2.9 %

Healthcare3.4 %

Education3.8 %

Financial Services5.7 %

IT and Services13.0 %Computer Software

38.1 %

Figure 3.05 Attendance by industry

Page 18: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

Further Reading

Whitepapers

SQL Migration: MongoDB and SQL, Coexistence Made Easy https://studio3t.com/resources/whitepapers/mongodb-sql-coexistence-whitepaper

MongoDB Security Checklist: Essential Tactics Against Data Breaches https://studio3t.com/resources/whitepapers/mongodb-security-checklist

MongoDB DevOps Seven-Step Checklist https://studio3t.com/resources/whitepapers/mongodb-devops-checklist-whitepaper

MongoDB Database Testing: A Detailed Walkthrough of Best Practices https://studio3t.com/resources/whitepapers/mongodb-database-testing-best-practices

Articles

SQL Databases in Sports https://medium.com/@studio3t/sql-databases-in-sports-part-1-30d4e60fe0d5

NoSQL Databases in Sports https://medium.com/@studio3t/nosql-databases-in-sports-part-2-7649bd2d45-769149bd2d45

Detecting COVID-19 Trends Using MongoDB Atlas Data https://studio3t.com/knowledge-base/articles/mongodb-atlas-covid-19-data-trends

18

Page 19: The MongoDB Tools Company - Studio 3T...Andrew Davidson, VP, Cloud Products, MongoDB “The biggest driver behind distributed databases like MongoDB is that they are better suited

About 3T Software Labs 3T Software Labs—“The MongoDB Tools Company”—helps hundreds of thousands of MongoDB developers and administrators the world over, by providing the most widely used MongoDB tools on the market: Studio 3T, the #1  professional tool used by over 100,000 MongoDB developers and database administrators; Server 3T, the server-side automation workhorse; and Academy 3T, the education platform for MongoDB learning. Read more about 3T Software Labs online at www.studio3t.com.