General

(Free) cloud-based service monitoring

PTS

Jun 3, 2022 • 10 min read

It's a clunky title I know, but it was the best I could do. What does it mean? Well...

We all self-host services (99% of you wouldn't be reading this if you didn't, or if you weren't interested in beginning to self-host). Along with the services we self-host are other self-hostable applications which monitor those services, the idea being that if a service goes down, you get a notification. This is great, but what happens if the machine you host this reporting tool on has gone down completely? Not get any notification, that's what.

To get around this, an enterprising soul (reddit handle JPH94) has put together all the tools you will need to create a VM (Virtual Machine) in the cloud, completely separate from your host machine, which comes preloaded with your monitoring service. Please go ahead and check out his github project here, he's got a really comprehensive walk through. Having had a few hiccups in the setup myself (which he kindly helped me through) I wanted to put down some of my learnings here, but none of the following would have been possible without him.

Prerequisites

A basic understanding of how to navigate a Linux terminal. You can check out my SSH article to get a good idea of what this means, though if you're here then you've likely already got that knowledge
A basic understanding of spinning up a container using docker-compose (or docker compose as it seems to be migrating). I've written some articles on docker, getting started and docker tips, check them out if you need to
A google account to create a Google Cloud Platform. This can either be brand new or an existing one
A debit/credit card which will not be billed is necessary to set up the GCP account

💡

Note that if you have an existing GCP account which is in use, you won't be able to do the below for free. Please set up a new google account

Nice-to-haves

If you want to access the monitoring service via a domain name like https://example.domain.com, you will need to have a fully qualified domain name, a CDN (such as Cloudflare) and one of the following:

A suitable reverse proxy set up (such as SWAG)
The ability to create a Cloudflare Tunnel (recommended here, and I'll talk you through it as well)

OK let's get onto the setup.

Google Cloud Platform setup

💡

Skip this section if you already have an unused GCP account created

I'm assuming you can set up your own gmail account. With your credentials in hand:

Head over to https://cloud.google.com

fill out the relevant fields/check box and click continue

Depending on your country, you may be requested to provide your mobile/cell number to receive a confirmation SMS message

Go through and fill in your personal information, making sure you select Individual in the Account type drop down box
Fill in your billing details, then skip through the next few pop ups which ask you to try billable products

Click 'My First Project' and then `NEW PROJECT`

When you finally get to the home screen, click 'My First Project` top left, then in the pop up click 'NEW PROJECT'

Give your project a name, and the project ID will be automatically populated
Click CREATE to move on, then wait a few moments for the green tick to appear next to your new project

First up, we need to change the Network Tier to Standard. It's set to Premium by default, and that can result in charges. Note that we want to set the network tier first, because VMs take whatever is set when they're created, and can't be changed.

Click this link to take you to the correct page - as you're logged in, it should be pretty quick
Click CHANGE TIER, then in the pop-up modal click the radio button next to Standard, then CHANGE
You can now close this tab and go back to your project screen

Click Compute Engine (highlighted in the red box above) then ENABLE in the next screen. Give this a bit of time to enable, it will eventually refresh the screen, and show you this:

Setting up the VM documents

As per the added indications in the previous screenshot, we do NOT want to create our own VM instance manually. Instead, click the Cloud Shell button indicated top right, which will open a console in-browser at the bottom of your screen

You can resize this panel using the line indicated in turquoise

You should already be in your /home/[user] directory
Copy paste the following command to set up the directories you'll need:

cd ~/ && mkdir terraform auth compose_files startup .ssh

you can copy and paste this line exactly

If you now type ls and hit Enter you can confirm the directories have been made

Now we'll clone Joe's repository into the terraform folder which will copy the relevant files to our platform:

cd terraform && git clone https://github.com/Joeharrison94/terraform-gcp-ubuntu-container-ready-e2-micro-vm

copy pasta time

We now need to move some files around.

Open the editor using the button top right of the shell panel (if you are using an incognito or private browser, it may require you to open the editor in a new window or tab)

Start by moving the docker-compose.yaml file into the compose_files folder, and the startup.sh file into the startup folder (drag and drop)

you can check that this has worked back in your shell terminal by navigating to those folders and typing ls -a

Finally, drag and drop all the terraform .tf files into the terraform folder

At this stage, we want to modify our documents using the editor.

Open the docker-compose.yaml and modify the containers as necessary, paying close attention to the 'healthchecks' container

to be honest you could leave these containers as they are, and they will spin up just fine

Now for the .tf and .tfvars files inside the terraform folder. I'm not going to go through them all as they explain what to change pretty well, but some pointers:

IDENTIFIER must be changed in NETWORK-FIREWALL and NETWORK-MAIN files - it doesn't matter to what, but keep it short and keep them the same
The terraform.tfvars requires your project ID, project name, and for you to change the user. This user should be your email address using underscores, where me@pointtosource.com becomes me_pointtosource_com
I recommend not changing the GCP region or zone

We're now done with the editor, you can close it.

Navigate back to or reopen your shell terminal
To create the preshared SSH keys, make sure you're in your /home/[user] directory, and copy the following, changing the part in [ ] to something else:

ssh-keygen -t ed25519 -f ~/.ssh/sshkey -C [KeysForVPSAccess]

change the words inside the [ ] and remove the brackets

Follow the on-screen instructions
When done, you should have two files inside your .ssh folder, sshkey and sshkey.pub

Please now follow the instructions on the github page which look like this:

make sure to change the parts which require it

Given that each line requires at least one modification, I find it easier to paste these into notepad and make the required changes there, then paste them into the shell terminal.

💡

Note that you may need to authorize the API the first time you run one of the above commands

Having done all that you can now start to use terraform to create the VM:

Inside the shell terminal, navigate to your terraform folder
Type terraform init

If that's good, then type terraform plan, and when prompted type a name for your VM
Finally, type terraform apply, type in the same VM name, and when prompted type yes then hit Enter

The set up may take a bit of time, but if you've done everything correctly, your screen should show that 7 resources have been added successfully:

If you now go back to your compute engine and refresh the VM instances tab in the panel on the left, you should see that your VM has been successfully created:

We can now click the SSH button which should open a new browser window, and after thinking for a little it will show that we've successfully connected to our VM

Maximize this new window

Creating our docker containers

Navigate to the directory /mnt/disks/docker/projects/app and inside you should find your docker-compose.yaml

💡

I've experienced instances where the docker-compose.yaml hasn't been copied across for whatever reason. If that happens, just create it yourself in the /mnt/disks/docker folder with sudo touch docker-compose.yaml

Copy the docker-compose.yaml to the docker folder

sudo cp docker-compose.yaml /mnt/disks/docker

Navigate to the /mnt/disks/docker folder, then create the following folders:

sudo mkdir uptime-kuma healthchecks

You can now spin up your docker containers:

sudo docker compose up -d

Finally, type sudo docker ps to check that your containers started up and are running:

And that's it. Your containers are running.

Setting your firewall

But now you need to access them, and if you try and access them via your public IP and the port, it's not going to work. Why? You need to allow it through your VM's firewall.

Take note of the uptime-kuma port. If you haven't changed it, it's 3001

Back we go to the cloud portal, and the VM page (you can close the terminals now if you want).

Locate and click the button Set up firewall rules
Click CREATE FIREWALL RULE at the top of the page
Follow the steps below, adding the name, the correct network and the port:

Back in your VM instances screen, take note of your External IP
In a browser window type your external IP, followed by :3001 and press Enter

Congratulations! You've got access to your Uptime-Kuma instance!

But we're not very secure are we? First up, it's a public IP, and secondly it's http, not https. This is where Cloudflare comes in.

Setting up cloudflare tunnel

Create your user in Uptime-Kuma, click the icon top right, and select Settings

Hit the Reverse Proxy button, and you'll note that 'cloudflared` is installed, but not running. We need a tunnel token from cloudflare. Luckily, there's a handy link below which shows you how to set it up

💡

Note that for this to be possible, you do need to have your own Cloudflare account set up which manages an existing domain which you own

When complete, you should now be able to access your monitoring container via your domain name
Go ahead and start setting up your Monitors using either https or ping

There you have it, your own uber-monitor in the cloud.