Skip to main content

Posts

Showing posts with the label datastore

Ahmedabad Shopping festival

Ahmedabad Shopping festival is solution created by  Tuzzin Infotech Pvt. Ltd . I worked as part of the development team My journey started as a part-time web developer. A MySQL database, Php for backend, VueJs and jQuery frontend, scss for css and I'm done. So easy! I was never thinking about scaling, rate limiting, timeout on those days, and normal site even does not need to care about such things unless they reaches that threshold. Luckily, I got chance to work on a product which was requiring all of this. It was the first time when I got exposure to real scalable application with spike in traffic, Ahmedabad Shopping festival 2019.   Brief Intro of Ahmedabad Shopping festival  Ahmedabad Shopping festival was first digitally managed shopping festival of India hosted in Ahmedabad, Gujarat. Every customer were getting rewards and coupons for chance to win prize on making shopping during shopping hours . Technical requirements Ahmedabad Shopping festival was con...

Datastore : Handling heavy insertions

Note: This is a case study to share what I faced, this might not be new to you and may have a better solution then I find. (You can share you solution here 😊) .  At first sight, datastore looks very easy to work with. And it is if insertions have to be done at a low rate. But when insertions are done at a high rate you come to find out certain issues which are not because your program is faulty but because you don't read the ( long detailed ) documentation of datastore, that itself has certain restrictions you might never hear before. Timeout Problems There are two kinds of timeouts 1. Call error 11: Deadline exceeded 2. API error 5 (datastore_v3: TIMEOUT) Call error 11: Deadline exceeded This kind of time out happens when your datastore operation takes time more than 60 sec to finish the operation.  For example, you are inserting too much data with a putMulti function. That takes time more then 60s insert. API error 5 (datastore_v3: TIMEOUT) This mos...