Headscale + UI + Authelia

The Setup

I have been a long time (long time as 1 year) Tailscale user, and I absolutely love it. I’ve tried to use Wireguard only, and other VPN solutions, but nothing matches the ease of execution and consistency that Tailscale provides. In fact, my experience has only gotten better as I’ve upgrade my home network with a pfsense box, with built in Tailscale support (via a package).

The only gripe I’ve had with Tailscale is that it’s not a self-hostable solution. I don’t really have a reason for it be self-hostable, besides me wanting to maintain the extra level of privacy, and have a little fun while doing it.

That is where Headscale comes in. Headscale provides an open-source implementation of the only not open source part of Tailscale - the control plane. And seeing this, I wanted to try it out. And I have to say, I have mixed reviews. To add and remove nodes, you need access to the command line of your headscale node. Which is fine, but at the end of the day makes things very painful when you don’t have a computer with the right SSH keys (speaking from personal experience). It is totally doable to admin headscale from the commandline of your VPS node - in fact, after this experience I might recommend doing that - but it is not as user friendly.

One of the biggest benefits of using Tailscale is the UI, and the OIDC clients associated with it. It makes adding and doing admin on nodes super convenient. I wanted to try and make a similar experience using the many UIs for Headscale, along with and OIDC client I have a lot of experience with in my homelab, Authelia.

Config

The full config for this setup can be found in this GitHub repo. I used Digital Ocean as my VPS provider, mostly because I have the most experience with it. My DNS provider is Cloudflare, again because I have the most experience with it.

Prereqs

For this kind of set up, you need a small node on a VPS. There is really no way around that. I got things working pretty well on a 1 Gb/1 vCPU node from Digital Ocean - because things are using Docker, I wouldn’t recommend going any lower on the specs than that.

There are lots of other tutorials on setting up droplets on Digital Ocean, so I won’t go into detail here. Just make sure you have the following set up:

1. A droplet with either Debian 11 or Alma 9 linux. These are what I have tested with, I’m sure other distros will work.
2. A static IP. This is important for networking.
3. An A record on your DNS provider of choice. I’m choosing cloudflare here, but you can use whatever you like. You should have it point to your static IP, and use two domains - hs.yourdomain.com and auth.yourdomain.com.
4. Have Docker installed and configured to your liking. I like using firewalld as my firewall, this tutorial is a great way to configure docker to play nice. I will upload an Ansible playbook for installing docker and configuring firewalld when I get around to it.

And now we are set to go!

Docker Compose

My headscale ui of choice is headscale-ui, and the only criteria is that it looked the nicest (I’m a shallow man).

Here is my docker-compose for the setup:

version: '3.9'
 
services:
  headscale:
    image: headscale/headscale:latest
    pull_policy: always
    container_name: headscale
    restart: unless-stopped
    command: headscale serve
    networks:
      - frontend
    volumes:
      - ./headscale/config:/etc/headscale
      - ./headscale/data:/var/lib/headscale
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.headscale-rtr.rule=PathPrefix(`/`) && Host(`hs.flaskforge.com`)"
      - "traefik.http.services.headscale-svc.loadbalancer.server.port=8080"
      - "traefik.docker.network=frontend"
 
  headscale-ui:
    image: ghcr.io/gurucomputing/headscale-ui:latest
    pull_policy: always
    container_name: headscale-ui
    restart: unless-stopped
    networks:
      - frontend
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.headscale-ui-rtr.rule=PathPrefix(`/web`) && Host(`hs.flaskforge.com`)"
      - "traefik.http.services.headscale-ui-srv.loadbalancer.server.port=80"
      - "treafik.docker.network=frontend"
      - "traefik.http.routers.headscale-ui-rtr.middlewares=authelia@docker"
 
  traefik:
    image: traefik:v2.10.4@sha256:57b2516b7549c4f59531bb09311a54a05af237670676529249c3c0b8e58ad0f3
    container_name: traefik
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    networks:
      - frontend
    ports:
      - 80:80
      - 443:443
    secrets:
      - cf_email
      - cf_api_key
    environment:
      - CF_API_EMAIL_FILE=/run/secrets/cf_email
      - CF_API_KEY_FILE=/run/secrets/cf_api_key
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik/traefik.yml:/traefik.yml:ro
      - ./traefik/acme.json:/acme.json
 
  authelia:
    image: authelia/authelia:4.37.5@sha256:82831059ce5c1151d4ccd37f803cdf35fccbd488c80fe7f9f8de6b76adf40447
    container_name: authelia
    user: 1000:1000
    volumes:
      - ./authelia:/config
    networks:
      - frontend
    labels:
      - 'traefik.enable=true'
      - 'traefik.http.routers.authelia.rule=Host(`auth.flaskforge.com`)'
      - 'traefik.http.routers.authelia.entrypoints=websecure'
      - 'traefik.http.routers.authelia.tls=true'
      - "traefik.http.routers.authelia-secure.tls.certresolver=cloudflare"
      - "traefik.http.routers.authelia-secure.tls.domains[0].main=auth.flaskforge.com"
      - 'traefik.http.middlewares.authelia.forwardAuth.address=http://authelia:9091/api/verify?rd=https%3A%2F%2Fauth.flaskforge.com%2F'
      - 'traefik.http.middlewares.authelia.forwardauth.trustForwardHeader=true'
      - 'traefik.http.middlewares.authelia.forwardauth.authResponseHeaders=Remote-User,Remote-Groups,Remote-Name,Remote-Email'  
      - "traefik.docker.network=frontend"
    restart: unless-stopped
    environment:
      TZ: America/Denver
      AUTHELIA_JWT_SECRET_FILE: /run/secrets/authelia_jwt_secret
      AUTHELIA_SESSION_SECRET_FILE: /run/secrets/authelia_session_secret
      AUTHELIA_STORAGE_ENCRYPTION_KEY_FILE: /run/secrets/authelia_storage_encryption_key
      # AUTHELIA_DUO_API_INTEGRATION_KEY_FILE: /run/secrets/authelia_duo_api_integration_key
      # AUTHELIA_DUO_API_SECRET_KEY_FILE: /run/secrets/authelia_duo_api_secret_key
      # AUTHELIA_NOTIFIER_SMTP_PASSWORD_FILE: /run/secrets/authelia_smtp_password
    secrets:
      - authelia_jwt_secret
      - authelia_session_secret
      - authelia_storage_encryption_key
      # - authelia_smtp_password
      # - authelia_duo_api_integration_key
      # - authelia_duo_api_secret_key
 
networks:
  frontend:
    external: true
 
secrets:
  # Traefik
  cf_email:
    file: ./.secrets/cf_email
  cf_api_key:
    file: ./.secrets/cf_api_key
  # Authelia
  authelia_jwt_secret:
    file: ./.secrets/authelia_jwt_secret
  authelia_session_secret:
    file: ./.secrets/authelia_session_secret
  authelia_storage_encryption_key:
    file: ./.secrets/authelia_storage_encryption_key
  # authelia_duo_api_integration_key:
  # file: ./.secrets/authelia_duo_api_integration_key
  # authelia_duo_api_secret_key:
  # file: ./.secrets/authelia_duo_api_secret_key

So let’s talk through this:

• This is using the standard configuration for headscale/headscale-ui found here. I’m not doing anything fancy with Traefik because Headscale is in charge of it’s own SSL certificates. So as long as you set up your DNS entry, you should be okay.
• For traefik, I’ve set up Cloudflare as my DNS resolver. Even though headscale can take care of is SSL cert, authelia cannot. So we still need to get our Let’s Encrypt certificate. Make sure to add your email and api key into the secret files for this to work, and update the traefik.yml with your email address for ACME stuff.
• The authelia configuration is pretty standard for Traefik, you can find it in their documentation here. Authelia and Traefik work really well together, which is why I chose them for my proxy and authentication backbone.
• Obviously make sure to update your authelia secrets and keys - I recommend just generating 64 digit alphanumeric strings.
• I’ve commented out the duo authentication stuff for 2FA, mostly because this is not a permanent set up for me and I didn’t want to have to worry about it. But if you want push notifications from DUO, you can follow the documentation. This works really well for my personal homelab setup, and I highly recommend setting it up if you can. You can also use normal 2FA without the duo integration.
• Last thing, make sure to update all the domain entries in the traefik labels, or else you will have issues.

Authelia

This is not a tutorial on Authelia. If you want more information on the configuration, go check out the well maintained documentation of one of the many tutorials on Youtube. I will just point out a couple of things that you will need to adjust to make things work for your Headscale setup

In the configuration.yml:

• Adjust all yourdomain.com to your domain
• If you want to do OIDC for Headscale node authentication, uncomment this section. I had a lot of problems getting this to work, so be aware there will likely be a lot of troubleshooting involved.
• Additionally, if you want to set up an OIDC client you will need to generate a private key, this can be generated using the following commands:

openssl genrsa -out private.pem 4096
openssl rsa -in private.pem -outform PEM -pubout -out public.pem

Copy the contents of the private.pem file to the oidc section.

In the users_database.yml

• Update this file with you username, name and email. You will need to generate a secure value/password - use the following command:

docker run authelia/authelia:latest authelia crypto hash generate argon2 --random --random.length 64 --random.charset alphanumeric

The only thing that I will really note in my Authelia set up is the access_control section. The way headscale-ui works is by piggy backing off the SSL cert for headscale, so they are on the same domain. You do not want to put headscale behind authelia. There is no reason to, and you will block connections to the VPN. No good. We do however want to put the UI behind Authelia. So, we only set access rules for the /web postfix (I don’t know what that is actually called). If you want 2FA, change the policy to two_factor.

Headscale

Again, not a ton to change here. In the config/config.yml:

• Change server url to your domain. Leave the port number.
• If you want OIDC for node authentication, uncomment the oidc block. Obviously change the client id and client password, and make sure it matches whatever is in the authelia config. Use the command I showed above to generate the secure values.

And after all of those edits, you can run docker compose up -d and magically have headscale up with a UI behind auth proxy! Pretty cool. There will be some trouble shooting with this setup, just due to the general nature of networking. Make sure that if you are using firewalld, that you have opened up ports 80/tcp and 443/tcp, or else you will not have any internet traffic.

The final bit of configuration comes when you want to connect headscale to headscale-ui. To generate an API key with 90d expiry, use the following command:

docker exec headscale headscale apikeys create

Put this into headscale-ui, and you should be able to now create users and start to add nodes to your tailnet. The way headscale-ui works, it stores secrets in-browser as cookies, so do not clear your cookies for the site or you will loose your configuration.

Conclusions

So now let’s talk about why I am not going to keep this setup. Headscale-ui is a pretty new piece of software, and did not work as well as I had hoped. I also had some DNS issues with authelia and headscale, and was never able to get OIDC to work for headscale.

I am willing to mess with things like docker containers and new services because to me most of these things are non-essential. If they don’t work, I’m not going to have a terrible time. VPN is not one of those things (though Tailscale is not technically a VPN lol). I need to be able to connect to my home network, and I don’t want to have an additional worry with a self-hosted setup.

But for those of you who are willing to accept this risk, I think headscale provides a really cool solution. I would recommend running it as a systemd service before delving into this setup. I had the most consitent performance with it as a binary rather than a docker container. But to each their own!