mchampaneri.in

Small team working on multi service product, With carrying more then one responsibilities. It's a big challenge to keep everything on track. Most importantly, When it comes to incident reporting. We can not automate every thing but, at least we can automate incident reporting to our project management service eg Gitlab / GitHub / Slack. ( Slack 😅 , for small team like us slack also work like project management tool. ) Google cloud log-sink Here comes google cloud log-sink handy to you. We can direct all of your logs with severity level of Error Crtitical Alert Emergency to a logs ink which log a incident on your project & issue tracking service ( For example Gitlab ) using cloud pub/sub topics. [ You can use firebase function for such pub/sub topics ] Inclusion filter for error logs severity='(ERROR OR CRITICAL OR ALERT OR EMERGENCY)' Next time when ever error log will be generated on your cloud logs. You will get an incident created on your gitlab. So Don't wait...

Ahmedabad Shopping festival is solution created by  Tuzzin Infotech Pvt. Ltd . I worked as part of the development team My journey started as a part-time web developer. A MySQL database, Php for backend, VueJs and jQuery frontend, scss for css and I'm done. So easy! I was never thinking about scaling, rate limiting, timeout on those days, and normal site even does not need to care about such things unless they reaches that threshold. Luckily, I got chance to work on a product which was requiring all of this. It was the first time when I got exposure to real scalable application with spike in traffic, Ahmedabad Shopping festival 2019.   Brief Intro of Ahmedabad Shopping festival  Ahmedabad Shopping festival was first digitally managed shopping festival of India hosted in Ahmedabad, Gujarat. Every customer were getting rewards and coupons for chance to win prize on making shopping during shopping hours . Technical requirements Ahmedabad Shopping festival was con...

Google Cloud Provides a ready made script to run scheduled export from datastore. I am using that export data as input to BigQuery. To get it done, I had created two crons, First for calling data export operation Second for loading that exported data to BigQuery Problem I was facing was that export was taking some time more time than I have assumed, So Bigquery loading cron was getting initiated before data export opertion get completed, and result bigquery loading was failing. Solution was to make those two cron job as a single one which works as chain. On successful completion of task one second gets called. So, I just modified the schedule export script provided by google, which exports files, and then checks if operation is successfully done or not fixed time intervals in loop, and if operation is found to be done successfully, it calls BigQuery load task. Here is gist 

Firestore returning no records yet, I see them in console Collection not retuning any element. ... ...  You might facing issues written above. Then it might be this case. You might getting empty response in this kind of query though there are elements in firestore. db.collection(collection1).doc(doc1) .collection(collection2).doc(doc2) .collection(collection3).get() .then( function (querySnapshot) { querySnapshot.forEach( function (doc) { console.log(doc.id, " = " , doc.data()); } ) I face the same problem. Though data was stored sucessfully ( I was guessing, which was wrong actully ) .  To verify that your data is not stored as you expceted you can do check this. Key in italics font means element is deleted If key of your data is in italic font then your data is not stored or is marked as deleted [ that was happening in my case ]. See in image. Now you might thinking, that I have created element and  not del...

copy datastore from one google project to another google project Its actually happens that we need to copy datastore values from one project to another project of google cloud platform. Requirements : Gcloud SDK on the local machine This process goes through 4 steps. Export datastore from origin project Give default service user of target project the legacy storage read role over the bucket  Transfer bucket data to target project's bucket  Import datastore file  Step 1  Export Datastore  From Feb 2019, you will not be able to use datastore admin as Google is going stop that support. So you have to use cloud datastore import-export for it. Cloud datastore import-export exports data in you specified storage bucket. To see how to setup automated datastore import-export refer this post. Step 2 Give default service user of targe projet the legacy storage read role over the bucket  Here we are giving read r...

If you are using the google app engine you might get this message from the google. Hello Cloud Datastore Customer, We're writing to let you know that the Datastore Admin backup feature is being phased out as of February 28, 2018, in favour of the generally available managed export and import for Cloud Datastore . Please migrate to the managed export and import functionality at your earliest convenience. To help you make the transition, Datastore Admin will continue to be available over the next 12 months prior to the shutdown date of February 28, 2019.  I am writing this post to show how to set schedule export. [ I am showing the steps described by the google at this link.  Scheduled-export  ] Enable the billing for google cloud service Ensure that you are using a billable account for your GCP project. Only GCP projects with billable accounts can use the export and import functionality.  You can Set your billing details at...

Cloud Datastore imposes limits on the number and overall size of index entries that can be associated with a single entity. These limits are large, and most applications are not affected. However, there are circumstances in which you might encounter the limits. Types of Datastore index Datastore composite indices Built-in indexes By default, Cloud Datastore automatically predefines an index for each property of each entity kind. These single property indexes are suitable for simple types of queries. Composite indexes Composite indexes index multiple property values per indexed entity. Composite indexes support complex queries and are defined in an index configuration file (index.yaml). Cost Counting for Datastore write In the current pricing model, inserting a new entity costs 2 write operations for the entity + 2 write operations per index. here index may be the composite index or built-in index ( single property index). You can remove ...