secure sharding in mongodb
DESCRIPTION
Secure Sharding in MongoDB. Presented By: Anam Zahid. Agenda. Scaling MongoDB’s Approach Architecture Mechanism Proposed Architecture. Scaling. 64MB. 64MB. 64MB. 64MB. 64MB. 64MB. Scaling UP (Vertical Scaling) Addition of more CPUs and Storage Capacity. Scaling. - PowerPoint PPT PresentationTRANSCRIPT
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Secure Sharding in Secure Sharding in MongoDBMongoDB
Presented By: Anam Zahid
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
AgendaAgenda Scaling MongoDB’s Approach Architecture Mechanism Proposed Architecture
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Scaling Scaling UP (Vertical Scaling)
– Addition of more CPUs and Storage Capacity
64MB64MB
64MB64MB
64MB64MB
64MB64MB
64MB64MB
64MB64MB
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Scaling Scaling OUT (Horizontal Scaling)
– Distribution of data across multiple servers
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
MongoDB’s Approach Sharding (horizontal scalability)
– A method to store data across multiple machines/shards
– Supports deployments with very large datasets– Maintains high throughput operations
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
MongoDB’s Approach Sharding Advantages
– Reduces the number of operations each shard handles
– Reduces the Amount of data that each server stores
– Make the Cluster invisible for Clients– Ensures Cluster availability for reads and writes– Easy scaling out
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Sharding Architecture
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Sharding with Replica Set
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Mechanism Shards
– Master slave ,Replica Sets or Mongod instances Configuration Servers
– Exactly 3 for production– Contains meta-data
Routing Instances– Direct interface with client application– Can be many in number– Also act as a balancer for chunk migration
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Mechanism Sharding on per collection basis Based on shard keys Default chunk size is 64MB
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Mechanism Two basic operations
– Chunk Splitting
– Chunk Migration
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Proposed Architecture
Encryption/Decryption Layer
Key Distribution
Store
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Authentication
Authorization
Client Application
Key Distribution Store
Config. Server
Shard A Shard B Shard C
Config. Server
Config. Server
Encryption/Decryption Engine
Query Router
1
2 3
4
56
78
9
10
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Proposed Architecture1. Client Application sends Login Request to authentication Module
2. Authentication Module authenticates it on the basis of certificate and sends reply back to client application
3. Client application sends query to authorization server. The Authorization server generates policies and maps user policies with user profile. Then it verifies user query against user policy.
4. In case of successful authorization, authorization server sends the query to Query Router
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab
Proposed Architecture5. Query router get meta-data information about shards from the
configuration server/s
6. It then looks for appropriate shard/s on the basis of query parameters
7. Query router sends query request to encryption/decryption Engine
8. The encryption/Decryption Engine requests key distribution store for encryption key
9. The Engine requests data from appropriate shards, decrypt it and sends the response back to query router.
10. Query router forwards this data to Client Application
Department of Computing, School of Electrical Engineering and Computer
Sciences, NUST - Islamabad
KTH Applied
Information Security
Lab