Scaling write-heavy OLTP systems with strong data guarantees - Gokulvanan V Velan

Order capture and Order management systems at Flipkart have had to scale by 10X volumes to cater to growth in eCommerce and user base.In addition, these systems need to scale for bursty traffic by 1000x for flash sale business model. These systems are write heavy and need strong data guarantees (Consistency, Data-availability, Durability etc). With scale, the data stores for these systems have outgrown capabilities provided out of the box by databases like MySQL and point solutions for each system in the ecosystem have resulted in fragmentation. This talk focuses on our journey in solving for our datastore needs holistically by customising Hbase at the source code level to support Strong Consistency in Write Heavy workloads, Transactional Change Propagation to enable Lamdba Architecture patterns, Basic index support and provide predictable Scalability using Tenant isolation. This talk will dive into details by introducing concept of regionserver groups (rsgroups) within an hbase cluster, tweaks to balancing algorithms in region rebalancing within rsgroup, ensuring no data loss in change propagation and mvcc style approach to support basic indexes over distributed transaction. We currently are live in production with a single multi-tenant hbase cluster that servers half a million QPS in Order capture and Order management flows.
Gokulvanan is an Architect for Order capture and Order management systems at Flipkart. Prior to Flipkart he worked as Senior Software Engineer for the Mobile team at a media advertising startup, Komli Media. He has close to 10yrs of experience working in Software Industry.

[@ryan-bo2xi]

this should be renamed as a Hbase tutorial :D