Automated Git Deployments on Azure Ubuntu VM from BitBucket With Nodejs

Posted by in Random, Softwares, Technology, Tutorials

Nobody needs an introduction to Git, its a basic skill you’ve to have these days if you’re into programming or programming related stuff. While Git makes a programmer’s life easier in multiple ways, it also has some perks. If you’re not a huge team, deploying your newest build can become redundant and boring super fast – especially if your production server is on the cloud (probably everyone’s production servers are now on the cloud!). A typical simple workflow would be like this:

  1. Push your commit to git
  2. Git fetch, git pull on your production server
  3. Restart/rebuild your application

You’re required to do these steps every time you push a change to your repo. What can be done? Automation of course! Recently during the development of our project IOStash, I came across this, we were pushing changes like crazy and every time a push is made, I’ve to fetch, pull and restart the application so that our volunteer beta testers can see our update/bug fix. I love Nodejs, ever since it was released, I was intrigued by its simplicity and ease of use and pretty much use Nodejs for almost all of my apps – IOStash is no exception. IOStash is (being) written in Nodejs with Redis and MongoDB for caching and persistence. We choose Ubuntu Server 14.04 on Azure to deploy it because I was already familiar with Azure dashboard and workflows (AWS is good, but not good enough in my views). We use Bitbucket for our private repo because it offers private repos with much better value for money than GitHub (sorry GitHub, but its true!). I use Windows and Mac interchangeably for development and I’m comfortable in both. We use SourceTree as our git client, again because it is simple, easy to use and works on both Windows and Mac. Back to the point duh!

By leveraging ‘Webhooks’ in BitBucket, we can automate the deployment process – whenever a new push is made to the repo, BitBucket will issue a POST request to a URL we specify with the details of the commit. We can issue a git fetch,git pull and application restart once we get this POST request. You can read more about BitBucket’s webhooks here.

Lets get into the details of how to achieve this. This tutorial assumes that you’ve basic knowledge about all the discussed technologies/softwares and hence you might see that I’ve skipped the obvious details in between.

Getting Started

The very first step is to setup SSH keys on your deployment machine so that BitBucket won’t ask for passwords (Assuming you’re using a private repo) when you try to issue a fetch programatically. So, to setup the SSH keys, follow the steps below:

cd to ~/.ssh
$ cd ~/.ssh

Generate the public and private keys
$ ssh-keygen -t rsa
Give a unique name for your keys or leave the default id_rsa as is. You’ll be prompted for a passphrase – enter one. After this, two keys will be generated, we’re interested in the one with a .pub extension.

Copy the contents of the key with .pub extension to clipboard
$ cat ~/.ssh/<YOUR KEY>.pub | pbcopy

Now login to your BitBucket Account and Click on your profile icon at the top right corner and click on BitBucket Settings
Screen Shot 2016-04-24 at 7.48.39 PM

 

Click on the SSH keys item on the bottom left side of the window

Screen Shot 2016-04-24 at 8.02.54 PM

 

Click on Add Key on the screen and something like this should pop up:

Screen Shot 2016-04-24 at 8.04.25 PM

 

Enter a label and Paste your key copied earlier and click on “Add Key”

Now, in your deployment server, add the host ‘bitbucket.org’ so that when we connect to it, the right security certificate will be presented. For that, issue this:

$ sudo nano ~/.ssh/config

This will open nano with the file ~/.ssh/config. Paste the following into nano and save it by pressing Ctrl+O and then Ctrl+X

Host bitbucket.org
 IdentityFile ~/.ssh/

Now, clone your repo into a directory of your choice.
git clone git@bitbucket.org:/.git [You can add --mirror flag which will tell git that this copy is not a working copy]
Enter your repo link and it should clone your repo without your intervention

Now that we have a working connection between BitBucket server and ours, lets setup the WebHook. We’re going to write a Nodejs application that will listen to webhook requests and upon a webhook request, will execute Git pull on our application directory and will restart the application (I’m using PM2 module for my application, it makes spawning and reloading apps a breeze).

You can use something like this for your use:

var express = require('express')
var app = express()
exec = require('child_process').exec
var sendgrid = require('sendgrid')('<YOUR SENDGIRD API KEY>')
 app.post('/webhook', function (req, res) {
 var timestamp = new Date().toISOString().replace(/T/, ' ').replace(/\..+/, '')
 var output;
 exec('git pull', {cwd: '/home/<yourdirectory>/<your local git repo directory>'}, function (err, stdout, stderr) {
 output = 'Time of webhook invoke: ' + timestamp + '\nOutput: ' + stdout || stderr
 exec('sudo pm2 restart <YOUR APPLICATION NAME>', function (err, stdout, stderr) {
 var email = new sendgrid.Email({
 to: '<your email address here>',
 from: '<your from>',
 subject: 'Subject',
 text: output + '\n' + stdout || stderr
 })
 sendgrid.send(email, function (err, json) {
 if (err) { return console.error(err); }
 console.log('[Core] Email Sent Status -> ' + json.message)
 })
 console.log(stdout || stderr)
 })
 res.send('[ok]')
 })
})
var server = app.listen(9091, function () {
 var host = server.address().address
 var port = server.address().port
 console.log('Webhook listening at http://%s:%s', host, port)
})

I’m using Sendgrid to send the output of webhook to my email address. Copy and paste the above code with necessary changes into a file in your deployment server and start it using PM2:

$ sudo pm2 webhookserver.js

This code will listen to port 9091. If you’re on Azure, make sure you add an End Point for it so that outside processes can access the service.

Now, in you BitBucket repo settings, click on WebHooks on the lower left part of the screen and then click on “Add Webhook”. You’ll be presented with something like this:

Screen Shot 2016-04-24 at 10.04.19 PM

Enter a title and the URL of your webhookserver. Make sure you check “Skip certificate verification” because we don’t have an SSL certificate in place. Click on Save and thats it! From now on, whenever you push changes to your repo, your application will reload automatically and you’ll receive an email from the webhook server.

Screen Shot 2016-04-24 at 10.11.50 PM

PS. This is a basic example to make the idea clear, I do not recommend using this on your production server as-is because it doesn’t have any security methods in place.

Cheers!