Skip to main content

Ahmedabad Shopping festival

Ahmedabad Shopping festival is solution created by Tuzzin Infotech Pvt. Ltd.
I worked as part of the development team


My journey started as a part-time web developer. A MySQL database, Php for backend, VueJs and jQuery frontend, scss for css and I'm done. So easy!

I was never thinking about scaling, rate limiting, timeout on those days, and normal site even does not need to care about such things unless they reaches that threshold.

Luckily, I got chance to work on a product which was requiring all of this. It was the first time when I got exposure to real scalable application with spike in traffic, Ahmedabad Shopping festival 2019.  


Brief Intro of Ahmedabad Shopping festival 


Ahmedabad Shopping festival was first digitally managed shopping festival of India hosted in Ahmedabad, Gujarat. Every customer were getting rewards and coupons for chance to win prize on making shopping during shopping hours.


Technical requirements

Ahmedabad Shopping festival was consisting a web portal for retailers, a web portal for administrators of shopping festival and mobile application for shoppers.

All of this modules were needing access to database, sms module and push notification module and requests to each service could scale from 0 to massive amount at any time. 

It was also required to be able to make deployment during shopping hours without having any downtime or any glitch in any of the module. 



Issues and Resolutions


Ability to make deployments during shopping hours.

Our first and for most issue was  ability to make a deployment during shopping hour without introducing any glitch. Glitch was not acceptable at all as It could affect result of lucky winner selection based on coupons issued. 

version defined in yaml

This issue was solved using versioning with app engine services and traffic migration in conjunction with warmup request. 

instance with warmup support


Rate Limiting

We were using sms and push notifications with all the modules. Sms service was capped by a rate limit per time period (X / sec ). Reaching that limit would have resulted in delivery failure of sms. 

Sms were used for notifications to retailers and admins but, most importantly sms were also used for OTP for signup/sign-in in shopper facing app. So neither late not failure of sms delivery was acceptable at all. 

Google cloud task was used as solution. All of the sms from different backend services were written to that queue which was optimised in way that it was sending messages at highest possible rate without exceeding limit.

google cloud task


Taking daily backup, loading data and processing to find lucky winner

This tasks were supposed to run daily on specified time. It was handled using cron jobs.




Giving as fast response as possible to every request

Shoppers were in count of thousands. During shopping hour request could making spikes that it reach the read limit per second of database. Therefor reading from the database was not a feasible one.

As solution memcache was used. For every incoming request we were first looking at memcache, if memcache does not have data then data was loaded to memcache and same time it was returned as response as well.

Here we found that data being sent to shopper were not going to be changed by any event or action of shopper other then his/her coupon counts. Most of the data was going to be static for the day,

There was only one request the first one of the day has to go to read the database, but we know that reading memcache is faster then reading database. So to make first request to read memcache as well, we started refreshing and reloading memcache before the first request of the day hits api.

So, technically were serving thorough only memcache through out the day without going to database for reading more then once in a day in normal scenario. 

Data sent to shoppers were static and going to be unchanged for day long. As result we were caching data to the devices local storage for day that was reducing request to the api and loading data faster at same time.

So there were two caches used, 
  • on cloud in form of memcache 
  • second local storage at device level which significantly improved response time.


Conclusion 

Architecture was designed to generate faster response with minimum usage of resource and without exceeding constrains imposed by third party service providers and cloud services (database & big-query ) itself.

Ahmedabad Shopping festival was the project which made me think as a person for whom things working is not enough, but the way it works is also matters.




Comments

  1. I generally check this kind of article and I found your article which is related to my interest. Genuinely it is good and instructive information. Thankful to you for sharing an article like this. product price history tracker

    ReplyDelete

Post a Comment

Popular posts from this blog

Apache : setup basic auth with apache in windows

Authentication is any process by which you verify that someone is who they claim they are. Authorization is any process by which someone is allowed to be where they want to go or to have information that they want to have. I will show here how to set up basic auth on the apache with windows. Pre-requests  Windows VPS Apache server ( That's it ) ( In windows it might be difficult to setup the Apache alone. So instead use something ling xampp , wamp or laragon .) RestClient (  I personally use the postman , but you can use your preferable client)  Windows VPS provider Steps  Enable the necessary modules in the Apache Create the password file Set the auth directives in the virtual host file. Verify basic auth. Enable the  necessary   modules  in the Apache Open the httpd.conf file in the apache's conf folder. httpd.conf file Enable the necessary modules to make the basic auth working. Necessary modules  mod_auth_basic

Firebase - update a spacific fields of single element of object of array in firestore

Firebase - update a spacific fields of single element of object of array in firestore  Its actully advisable to use map instead of array when ever it is possible. But, there are cetain cases where you don't have option to do so.  For example, you are directly saving the response from some outer source without any modification and they send you an array. In this case you will have array to work with. Firestore does not support array here is why  "bad things can happen if you have multiple clients all trying to update or delete array elements at specific indexes. In the past, Cloud Firestore addressed these issues by limiting what you can do with arrays " For more details information you can refer to Kato Richardson post Best Practices: Arrays in Firebase .  Firestore document having array [ used from stackoverflow question ] Suppose you have array of object something like shown in array. Now you want to update endTime field of the object on the index [1]

Sciter : GUI Application with Golang using HTML/CSS

GUI library for golang sciter This is the words from Sciter's Web site, Sciter brings a stack of web technologies to desktop UI development. Web designers and developers can reuse their experience and expertise in creating modern looking desktop applications. Various GUI frameworks offer different UI declaration and styling languages, such as QML and  XAML (Microsoft WPF) . On the contrary, Sciter allows using time proven, robust, and flexible HTML and CSS for GUI definition and GPU accelerated rendering.   Before using sciter I already tried other alternatives but none of them was satisfactory as an example first i tried andlabs / ui  library   i already have written a post on it. You can read it on post gui programming with golang .  But this library is still under construction and has no support for production apps. Secondly, I go for electron but the problem was my simple calc like the app was of size 150mb. Which is 15mb of go and other was the e