mchampaneri.in

Google Cloud Provides a ready made script to run scheduled export from datastore. I am using that export data as input to BigQuery. To get it done, I had created two crons, First for calling data export operation Second for loading that exported data to BigQuery Problem I was facing was that export was taking some time more time than I have assumed, So Bigquery loading cron was getting initiated before data export opertion get completed, and result bigquery loading was failing. Solution was to make those two cron job as a single one which works as chain. On successful completion of task one second gets called. So, I just modified the schedule export script provided by google, which exports files, and then checks if operation is successfully done or not fixed time intervals in loop, and if operation is found to be done successfully, it calls BigQuery load task. Here is gist 

If you are using the google app engine you might get this message from the google. Hello Cloud Datastore Customer, We're writing to let you know that the Datastore Admin backup feature is being phased out as of February 28, 2018, in favour of the generally available managed export and import for Cloud Datastore . Please migrate to the managed export and import functionality at your earliest convenience. To help you make the transition, Datastore Admin will continue to be available over the next 12 months prior to the shutdown date of February 28, 2019.  I am writing this post to show how to set schedule export. [ I am showing the steps described by the google at this link.  Scheduled-export  ] Enable the billing for google cloud service Ensure that you are using a billable account for your GCP project. Only GCP projects with billable accounts can use the export and import functionality.  You can Set your billing details at...

Authentication is any process by which you verify that someone is who they claim they are. Authorization is any process by which someone is allowed to be where they want to go or to have information that they want to have. I will show here how to set up basic auth on the apache with windows. Pre-requests  Windows VPS Apache server ( That's it ) ( In windows it might be difficult to setup the Apache alone. So instead use something ling xampp , wamp or laragon .) RestClient (  I personally use the postman , but you can use your preferable client)  Windows VPS provider Steps  Enable the necessary modules in the Apache Create the password file Set the auth directives in the virtual host file. Verify basic auth. Enable the  necessary   modules  in the Apache Open the httpd.conf file in the apache's conf folder. httpd.conf file Enable the necessary modules to make the basic auth workin...

There are many scenarios where we need some kind of proxy. We will consider one scenario from all where we need to use the forward proxy. Case : Suppose there is a service which is available for certain IP/ IPs only (These are mostly secured services). In this case, you have to forward the service requests through that specific.  Take an example : Entities playing role, Hosting App On Google App Engine ( Dynamic IP) White Listed Server ( X.X.X.X ) Resource Server ( Y.Y.Y.Y ) Here  Resource    Server ( Y.Y.Y.Y ) will just respond to the request come from the White Listed IP  Server   ( X.X.X.X ). So if we want to request   Resource   Server (Y.Y.Y.Y) we have to send it to the way Server feel like they have come from White Listed Server IP (X.X.X.X). Here Proxy server comes in the play. Proxy server takes the request and forwards it as they are initiated by itself. So what we have to do is we have to put a proxy ser...

Aim   The aim of this tutorial is to show how to migrate the local laravel 5.x project to the Ubuntu 16.04 vps. We will see how to setup the vps for hosting from scratch. We also see the ways you can copy your laravel project to the vps and at last we will see how to secure your project with ssl using the lets encrypt. we will also add a cron job to auto update the ssl certificate.   Prerequisites   To follow this guide along you will require this  Ubuntu vps with 16.04  laravel  project you want to host ssh client For windows you can use putty For Mac you can use terminal sftp client For windows you can use winscp For Mac you can use cyberduck Git Remote Git repository ( optional ) Steps Now after gathering the required elements you can follow these steps to host your laravel 5.x project. Prepare the vps for the web hosting Install Php Install Mysql  Install Apache Install Git Copy and setup the loc...

How To Configure Sendy on Ubuntu 16.04 with nginx Introduction Sendy is a self hosted email newsletter application that lets you send trackable emails via Amazon Simple Email Service (SES). You can host it on your vps. We are going to see how to host the Sendy on Ubuntu server having installed php , mysql and nginx Prerequisites Before you begin this guide you'll need the following: VPS with ubuntu 16.04 PHP MySQL nginx Step 1 — Get the licenced copy of sendy Sendy is a commercial product. You have to purchase it before using it. You can purchase it from   http://sendy.co After purchasing they will send you a copy of the sendy to your email id. Now transition to the next step by telling the reader what's next. Step 2 — Configer Sendy Extract the zip of sendy you get in the email Update the config.php file You will be able to find  config.php file inside the include folder. Update the varialbes inside config.php Set the App_PATH ...

Let assume that you have setup the linux server with mysql. Now you want to deploy the database to the server you have created on your developement enviroment.  You development enviroment may be windows or mac or linux with the gui but server are run only cli. So it is necessary to know the easy way to deploy database to the server with just cli Importing the mysql database to mysql using the cli 1) Export the sql file of the database from your development enviroment. 2) Copy the sql file to the serve using the sftp. 3) Now import the sql file to you mysql using this comman snippet     $ mysql -u [username] -p < [path/to/your/sql/file]      $ password : [your password ] [will be invisible on screen]     $  After sucessfull import of sql file your database will ready to be used. Note that if there is any error during the import than it will print the error on the screen but in case of successful import it will not ...

Jwt + Golang For Secured Api For jwt authnetication i am using the  dgrijalva/jwt-go package with my golang project.  First of all fetch packge from repository go get github.com/dgrijalva/jwt-go Now you are ready to use the jwt I am sharing here  my basic code to create secure routes using jwt Here you can see that we have used jwtcheck function as middleware. It processes the request before passing it forward. If request has a valid jwt token than that request get forwarded to the intended function otherwise that request does not forwarded. Hope you find it usefull. If you have any question just ask in comments.

Host Personal Cloud Torrent  cloud torrent This is the awesome project i found on the github.com.  project   jpillora / cloud-torrent What it does is it asks for the torrent or magnet link. After getting the link it downlods the torrent file on the vps and than you can download the file via idm or downloader to your pc from the vps This project is written in the go and uses go 1.9.1. If you are a person who don't want the complication of the compilation or cross compilation of this project you can use the file that is precompiled. here is the link for ubuntu binary cloud-torrent

golang has faciltity of cross compilation. So you can write a your application on any of supported  platform and deploy it over any supported platform. So will go step by step from compilation to deployment of the project over ubuntu server ( linux server ) Requirements 1. Linux Server I mostly use ubuntu but this also work with CentOs 2. FTP/sFTP Upload Tool  For windows I will recommand winScp Download For Mac I personlly use ftp uploader provided by the mozzila as firefox addon  Download 3. SSH Tool For windows i will recommand putty  Download For Mac I personally use FireSSH  Download Thats it. Compile the project for the linux (I am using the windows for this tutorial , but for mac the links are show where neccessary) Run this command to compile the project Set GOOS for cross Compilation (windows ) > set GOOS=linux  upper command will set the golang host architecture to linux ( this is temporary chang...

Virtual hosts on Apache Definition Virtual hosting is a method for hosting multiple domain names (with separate handling of each name) on a single server (or pool of servers). This allows one server to share its resources, such as memory and processor cycles, without requiring all services provided to use the same host name. You can read more here Wikipedia - Virtual Host If you have purchase vps than you can use virtual host to host multiple site from the single vps ( or form single droplet if you use digitalocean). Most of vps provider gives linux based os because they are free. I will explain how to create and setup virtual host on digitalocean lamp stack with ubuntu 16.04 Requirements   Ubuntu 16.04 with Apache Install Apache If you don't have already installed then run this commands to install the apache on you os sudo apt-get update sudo apt-get install apache2 Setup folders for site to be hosted Site that are hosted has to be inside the /var/www...