Deploy a production Database in Kubernetes
Вставка
- Опубліковано 14 чер 2024
- Learn how to deploy a production-grade database in Kubernetes
In this step-by-step video, I'll guide you through the process of setting up a robust and scalable MySQL master-slave database architecture with High Availability & Durability guarantees!
Key Kubernetes Concepts that I cover:
1️⃣ Persistent Volumes
2️⃣ Stateful Sets
3️⃣ Headless Services
TIMESTAMPS ⏰
00:00 Intro
00:23 MySQL Architecture we'll deploy
00:40 Main tutorial we'll follow
00:54 Kubernetes Concepts you should already know
01:10 Kubernetes Persistent Volume (PV)
01:39 Kubernetes PV Static vs Dynamic Provisioning
01:54 Kubernetes Persistent Volume Claim
02:23 Kubernetes Stateful Set
02:43 Kubernetes Headless Service
03:33 Kubernetes Setup walk-through
04:14 Configurations walk-through
06:18 Create the database & play around
👉 REFERENCES
- Code repository 🔗 github.com/duaraghav8/SRE101/...
- K8s database tutorial 🔗 kubernetes.io/docs/tasks/run-...
- Minikube Cluster setup 🔗 minikube.sigs.k8s.io/docs/tut...
- Minikube PV setup 🔗 minikube.sigs.k8s.io/docs/tut...
- Persistent Volume 🔗 kubernetes.io/docs/concepts/s...
- Statefult Sets 🔗 kubernetes.io/docs/concepts/w...
- Headless Service 🔗 kubernetes.io/docs/concepts/s...
👉 SUBSCRIBE TO MY CHANNEL
/ raghavdua1
👉 CONNECT WITH ME
Linkedin 🔗 / raghavdua
Github 🔗 github.com/duaraghav8
Would you like me to cover a specific topic? Comment and let me know 👇
#kubernetes #database #devops
Thanks for your concise and informative illustration. It is so enlightening for me as a Kubernetes beginner!
Thanks a lot for your kind words :)
Thanks @Raghav for a clear explanation. Would like see more videos on DB scaling dynamically in K8s.
Hey Thanks for the feedback! I agree, I'm planning a part 2 for this where I cover more advanced topics like autoscaling, backups & disaster recovery, performance tuning, etc.
loved this video!! subbed right away!!
thank you!
great content and video 👍
Thank you Pankaj, means a lot
Excellent Video...Please keep doing the good work. 👍👍
Glad you found it useful :)
Thank you for the very useful video! Can you explain how the writer pod can be scaled if at all?
Great question. The writer Pod's scaling mainly depends on how you scale the writer node of MySQL.
IMO you have 2 options:
1. Vertical scaling: Keep adding more CPU, Memory, Disk, Network bandwidth resources to your writer machine until you reach the most expensive machine and any further addition only degrades the performance.
With vertical, you still don't have high availability in writer.
2. Horizontal scaling: Add more writer nodes and distribute the write queries among them. This can be accomplished using Sharding.
Some tricks you can also use:
1. people use to scale writer is to direct all read queries to read replicas so that the writer only needs to deal with write requests, so it can give higher throughput.
2. Use faster storage disk to increase data write speed. this can increase throughput.
How can i do this with an NFS server instead of stroring data locally
do we store the data of database pods in another instance outside of the cluster like aws rds or we just store everything in those storage classes itself in production?
Depends on the nature of your data.
Normally, people want high durability guarantees for their production data. So storing them on local machine disks is not a great strategy.
So for example, your Persistent Volume could connect to AWS EBS or S# under the hood and whenever your app writes some data to this PV, it is actually being written to the AWS storage.
If you want to store structured data in AWS RDS, you don't need a PV. In that case you application directly connects to the RDS database using its endpoint and credentials.
@sre101: can you suggest storage class which used and also suggest if you increase replica then seperate Pv will be provisioned as per value defined in pvc, and also suggest how we can do in AWS if we want to keep seperate Pv for each project then how
Storage class really depends on the nature of your data and what matters more to you.
eg- If you're storing payment transactions data, you want very strong guarantees that this data will not be lost, even if the latency is a bit high.
You want backups and disaster recovery on it.
You probably also want the database to be CP (in terms of CAP theorem).
So your storage class would reflect those needs.
@@sre101: can you please create a video or suggest as per my queries: 1: which storage class you used?, 2:if we want 3 replicas, then we would need to create pvc definition for each members in yaml files.?
3: how we can do in AWS if we want to keep seperate PV for each project?
@@Kk-rl7nv
1. Because I'm using minikube, I'm using the csi-hostpath-driver (see minikube.sigs.k8s.io/docs/tutorials/volume_snapshots_and_csi/)
2. No, that's not scalable. Instead, you can use VolumeClaimTemplate (github.com/duaraghav8/SRE101/blob/main/run-db-in-k8s/statefulsets.yaml#L157)
3. I believe you would need an appropriate Volume plugin to use AWS storage. Maybe this article can help you - aws.amazon.com/blogs/storage/persistent-storage-for-kubernetes/
Hi, I didn't get how replica set gets only pod's IP instead of service IP. Could you please explain in detail?
We Create a Headless service ( by setting clusterIP: None). This assigns all the pods a unique DNS name instead of just a cluster-level IP.
The mysql-0 dns name is always the writer because we write this logic in the statefultsets configuration file (if index == 0, apply master config else apply reader config)
kubernetes.io/docs/concepts/services-networking/service/#headless-services
In the video, the headless Service "mysql" is bound to the StatefulSet "mysql". This StatefulSet is also bound to the Service "mysql-read"? I mean the "serviceName" in the StatefulSet is set to "mysql" which is the headless service, not "mysql-read".
The StatefulSet is bount to the "mysql" service because that;s the main service controlling the cluster.