auto deploy from github to local server on git push

How to Automatically Deploy from GitHub to Local Server

As developers (oh yeah, as devops guys), it’s a no-brainer to automate our workflows to make life simple and easy for us. In this article, I’ll demonstrate how to automatically deploy a nodejs web app from GitHub to a production or test server on git push. You can follow the same instructions to deploy any application written in any language.

Tools used in this demo

We use quite a few tools in this demo. Each one has a specific role to play in the Continuous Deployment(CD) pipeline we are about to setup. Each of these tools are briefly explained below.

Nodemon

Monitors any changes in your node.js application and automatically restarts the server. Though Nodemon is typically used in development for quick re-spins, we’ll use it in production in this demo to simply explain you the overall concept.

Webhook

Incoming webhooks handler which can execute shell commands.

SocketXP

Enables us to receive webhooks from the internet and relay it into your local network without exposing the app to the internet. SocketXP eliminates the need to own a public IP.

auto deploy from github to local server on git push

Demo app

We’ll use the following simple nodejs web application for this demo.

var http = require('http');
var port = 8080
var handleRequest = function(request, response) {
  console.log('Received HTTP request for URL: ' + request.url);
  response.writeHead(200);
  response.end('Hello World!, v1.0');
};
var server = http.createServer(handleRequest);
server.listen(port);

You can find the source code used in this demo at the following github repo:
https://github.com/socketxp-com/webhook-autoupdate

# Git clone the demo app

First clone the demo app from GitHub.

$ git clone https://github.com/socketxp-com/webhook-autoupdate.git

# Install Nodemon

Change the working directory to the “webhook-autoupdate” git folder you just cloned above. Install nodemon globally and run it.

$ cd webhook-autoupdate

$ npm install -g nodemon
$nodemon
[nodemon] 2.0.4
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node http-server.js`

Curl the demo web app

Access the NodeJS web app running at http://localhost:8080 using curl.

$curl http://localhost:8080
Hello World!, v1.0

# Install Webhook app

To handle push notifications (or webhooks) from github (or any webhook sender application for that matter), you don’t need to implement a webhook handler yourself. There are apps available to offload the job. One such app is Webhook app.

Using the Webhook app easily create HTTP endpoints (webhooks handler) on your server, which you can use to execute configured commands. Webhook app also allows you to specify rules which must be satisfied in order for any hook to be triggered.

Download the Webhook app (choose one appropriate for your OS and Platform) from the github repo release folder

$wget https://github.com/adnanh/webhook/releases/download/2.7.0/webhook-darwin-amd64.tar.gz
$ls
webhook-darwin-amd64.tar.gz
$tar -xvf webhook-darwin-amd64.tar.gz 
x webhook-darwin-amd64/
x webhook-darwin-amd64/webhook
$ls
webhook-darwin-amd64		webhook-darwin-amd64.tar.gz
$cd webhook-darwin-amd64
$ls
webhook
$

You’ll find a “hooks.json” file in the “webhook-autoupdate” GitHub repository that you cloned in the previous section. Update the hooks.json file with the correct directory names and secret value, as shown by the marked lines below.

[
  {
    "id": "webhook",
    "execute-command": "/Users/gannygans/webhook-autoupdate/update-nodejs.sh",   // <<<=== Update this field to your directory path
    "command-working-directory": "/Users/gannygans/webhook-autoupdate",    // <<<=== Update this field to your directory path
    "response-message": "I got the payload!",
    "response-headers":
    [
      {
        "name": "Access-Control-Allow-Origin",
        "value": "*"
      }
    ],
    "pass-arguments-to-command":
    [
      {
        "source": "payload",
        "name": "head_commit.id"
      },
      {
        "source": "payload",
        "name": "pusher.name"
      },
      {
        "source": "payload",
        "name": "pusher.email"
      }
    ],
    "trigger-rule":
    {
      "and":
      [
        {
          "match":
          {
            "type": "payload-hash-sha1",
            "secret": "adsFgGdf1T23423",    // <<<<==== Update the secret 
            "parameter":
            {
              "source": "header",
              "name": "X-Hub-Signature"
            }
          }
        },
        {
          "match":
          {
            "type": "value",
            "value": "refs/heads/master",
            "parameter":
            {
              "source": "payload",
              "name": "ref"
            }
          }
        }
      ]
    }
  }
]

Use this hooks.json file to kickstart the webhook app.

$./webhook -hooks hooks.json -verbose
[webhook] 2020/06/25 16:27:42 version 2.7.0 starting
[webhook] 2020/06/25 16:27:42 setting up os signal watcher
[webhook] 2020/06/25 16:27:42 attempting to load hooks from hooks.json
[webhook] 2020/06/25 16:27:42 os signal watcher ready
[webhook] 2020/06/25 16:27:42 found 1 hook(s) in file
[webhook] 2020/06/25 16:27:42 	loaded: webhook
[webhook] 2020/06/25 16:27:42 serving hooks on http://0.0.0.0:9000/hooks/{id}

Note: The above webhook URL “http://0.0.0.0:9000/hooks/{id}” is a localhost url. Github cannot send notifications to this IP address because it is not a publicly reachable IP address. Github needs a publicly reachable public URL to send push notifications.

This is where SocketXP solution comes in handy. We can use the SocketXP Secure Reverse Proxy Tunneling service to relay webhooks notifications from Github to the localhost Webhook app.

# Install SocketXP agent

Download and install SocketXP agent on your system following the instructions here – SocketXP Downaload and Install.

After SocketXP agent is installed, request the agent to relay the github webhook to your local webhook URL as shown below.

$ socketxp relay http://0.0.0.0:9000/hooks/webhook

Connected.
Public URL -> https://webhook.socketxp.com/ganeshvelrajan-0er32gfj

Now copy the above SocketXP Public URL and use it to update the webhook settings in the Github page of your project repository as shown below. Also make sure you select the payload content type to “applicatin/json” in the dropdown box. Save the changes.

Automatically Deploy from GitHub to Server using Webhook

Begin Auto Deploy Demo

Let’s begin the demo by committing a code change into Github.

Git commit and push

Commit a code change into your Github project. I have bumped up the version string in my http-server.js” to version 2.0 and committed the code changes.

$git commit http-server.js -m "minor fix"
[master cae0d29] minor fix
 1 file changed, 1 insertion(+), 1 deletion(-)
$git push
Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 286 bytes | 286.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/socketxp-com/webhook-autoupdate.git
   f4f1516..cae0d29  master -> master

SocketXP webhook relay agent

Check if the SocketXP agent has received the webhook notifications from Github.

Webhook received:
POST /hooks/webhook HTTP/1.1
Host: webhook.socketxp.com
Accept: */*
Accept-Encoding: gzip
Content-Length: 8077
Content-Type: application/json
User-Agent: GitHub-Hookshot/3ea2330
X-Github-Delivery: b55dd9f6-b6d3-11ea-9b25-2831f872cb2c
X-Github-Event: push
X-Hub-Signature: sha1=43b9f9114ecf5c3da6e306976f95766f52c6bdd9

{"ref":"refs/heads/master","before":"714706d5f6f411581ec7abf1653c4a1e3cd6d53b","after":"49d0ec0d1310917338baf1dcda5e3bc99b2f2644","repository":{"id":274867773,"node_id":"MDEwOlJlcG9zaXRvcnkyNzQ4Njc3NzM=","name":"webhook-autoupdate","full_name":"socketxp-com/webhook-autoupdate","private":false,"owner":{"name":"socketxp-com","email":"64076796+socketxp-com@users.noreply.github.com",  ... }

Check the webhook app

Check if the webhook app has received the webhook notification from the SocketXP agent and processed the webhook notification.

$./webhook -hooks hooks.json -verbose
[webhook] 2020/06/25 16:27:42 version 2.7.0 starting
[webhook] 2020/06/25 16:27:42 setting up os signal watcher
[webhook] 2020/06/25 16:27:42 attempting to load hooks from hooks.json
[webhook] 2020/06/25 16:27:42 os signal watcher ready
[webhook] 2020/06/25 16:27:42 found 1 hook(s) in file
[webhook] 2020/06/25 16:27:42 	loaded: webhook
[webhook] 2020/06/25 16:27:42 serving hooks on http://0.0.0.0:9000/hooks/{id}
[webhook] 2020/06/25 16:35:00 [0fd63b] incoming HTTP POST request from 127.0.0.1:57435
[webhook] 2020/06/25 16:35:00 [0fd63b] webhook got matched
[webhook] 2020/06/25 16:35:00 [0fd63b] webhook hook triggered successfully
[webhook] 2020/06/25 16:35:00 [0fd63b] 200 | 18 B | 531.245┬Ás | webhook.socketxp.com | POST /hooks/webhook
[webhook] 2020/06/25 16:35:00 [0fd63b] executing /Users/gannygans/webhook-autoupdate/update-nodejs.sh (/Users/gannygans/webhook-autoupdate/update-nodejs.sh) with arguments ["/Users/gannygans/webhook-autoupdate/update-nodejs.sh" "49d0ec0d1310917338baf1dcda5e3bc99b2f2644" "socketxp-com" "64076796+socketxp-com@users.noreply.github.com"] and environment [] using /Users/gannygans/webhook-autoupdate as cwd
   057cadf..069d282  master     -> origin/master
Updating 057cadf..069d282
Fast-forward
 http-server.js | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Current branch master is up to date.

[webhook] 2020/06/25 16:35:02 [0fd63b] command output: Already up to date.
Current branch master is up to date.

[webhook] 2020/06/25 16:35:02 [0fd63b] finished handling webhook

Check Nodemon has restarted the web server

$nodemon
[nodemon] 2.0.4
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node http-server.js`
[nodemon] restarting due to changes...

Curl the web server

$curl http://localhost:8080
Hello World!, v2.0

The hello-world application displays “v2.0” in the output, which proves that the auto deployment has worked as we expected.

That’s all folks!

You can receive webhooks from any online service to your local server using SocketXP. SocketXP eliminates the need to own a public IP address to receive webhook notifications from online services. Moreover, SocketXP has a free-tier for developers.

Leave a Reply

Your email address will not be published. Required fields are marked *