Developing an url-shortener application and deploying it on k8s

Introduction

To enhance my skills in Golang and Kubernetes deployments, I decided to build a small containerized web application with dependencies on a database and a cache. An URL shortener application seemed like a suitable choice, as it fulfills these requirements while remaining simple and practical. The application exposes two endpoints: a POST endpoint to shorten a given URL and a GET endpoint to redirect the shortened URL back to the original one.

The flowchart for the shorten endpoint logic: alt

The flowchart for the redirect endpoint logic: alt

Code

Application code is available at https://github.com/andrei-don/url-shortener. I chose to develop the application using Gin, one of the most popular and lightweight web frameworks for Go. Gin provides a simple and intuitive API for defining HTTP servers and handling requests, making it easy to implement clean and maintainable web services with minimal boilerplate code. For the database and caching layers, I opted to use PostgreSQL and Redis, respectively. Both technologies are widely adopted in the industry, come with comprehensive documentation and have large active communities, ensuring plenty of resources and support if needed during development.

The code is instrumented with the following Prometheus metrics:

shorten_requests_total
shorten_request_latency_seconds
redirect_requests_total
redirect_request_latency_seconds
redis_cache_hits_total
redis_cache_misses_total

The CI pipeline runs on Github Actions. On new PRs, the unit tests and the integration tests are run. A Docker build command, along with a Docker Compose file, is used to build and spin up the application and its dependencies for running the integration tests. On merge to main a Docker image is built and pushed to the Dockerhub registry.

Deployment architecture

The architecture for the Kubernetes deployment: alt

The application and its dependencies are packaged with Helm. The chart can optionally deploy the Nginx ingress controller and the kube-prometheus-stack charts for a ‘batteries-included’ deployment experience. For the time being both Redis and Postgres are configured using simple deployments of the official Docker images and have no persistent storage. In a future iteration of this project, persistent storage will be used (a good opportunity to try and setup Longhorn, hopefully it will be easier to do that than Rook Ceph) and the Redis/Postgres operators will be used for improved management and scalability.

The database secret is handled using the helm-secrets plugin which essentially acts as an integration layer between helm and SOPS to seamlessly decrypt secrets at helm release install/upgrade time.

The application is exposing a /healthz endpoint which returns 200 HTTP code when the connections to both Redis and Postgres are successful. This endpoint is used by the ReadinessProbe to mark the application container as ready and allow traffic.

The helm chart also deploys a Prometheus ServiceMonitor which scrapes the application /metrics endpoint and makes the application metrics available in Prometheus.

Demo

The application helm chart was deployed together with its dependencies. I used my multi-k8s CLI to deploy a local k8s cluster. I deployed a metallb load balancer prior to installing the helm chart so that the ingress can expose the application and Grafana dashboard behind a load balancer.

After the helm release was installed, a few requests were sent to the application to simulate a couple of successful and failed shorten/redirect requests. Examples of such requests:

$ curl -X POST "http://k8s.local/url-shortener/shorten" \
     -H "Content-Type: application/json" \
     -d '{"url": "http://google.com"}'
{"short_url":"http://localhost:8080/qiI5wX"}

$ curl -X POST "http://k8s.local/url-shortener/shorten" \
     -H "Content-Type: application/json" \
     -d '{"ttp://google.com"}' 
{"error":"Invalid request"}

$ curl http://k8s.local/url-shortener/qiI5wX             
<a href="http://google.com">Found</a>.

$ curl http://k8s.local/url-shortener/qiI5w  
{"error":"URL not found"}

A picture of the Grafana Dashboard tracking the latency and request rates:

Next Steps

With the baseline application now deployed, the next phase of the project will focus on implementing persistent storage and deploying the cache and database using operators. It would also be valuable to observe how the application performs under heavy load and enhance monitoring to better track cache and database performance.

Alex Stan