Woodpecker CI Setup on NixOS

2024-07-18

I am currently making a custom ArmA3 mod for use by my current group of friends, with whom I get together to play weekly.

I ended up deciding on trying out codeberg as a git provider, considering that I haven't touched forgejo before, and I wanted to try something new.

Over the last couple of years or so there has been a decent push to get ArmA3 mod development working well on linux, and when it comes to the core aspects of it, It's possible to almost entirely avoid using Windows in lieu of Linux to write ArmA3 mods, despite the game's publisher only supporting windows for it's mod tools, and only having an older, incompatible version of the game published as a native build for Linux.

That being said, one of my biggest pain points when modding as of late was publishing. Every time i wanted to push an update out to the steam workshop, I'd have to boot up a windows VM, boot up the terribly buggy and slow publisher app from the Arma3 mod toolkit and use that to upload.

Luckily, the same folk who made modding ArmA3 on linux possible have also made it possibly to employ a CI pipeline for building and uploading ArmA3 mods directly from github (see here).

So, if it's possible to do it via github's CI system, it should be possible to do it via Woodpecker's CI system, but to get there I still have a lot of steps I need to take.

2024-07-21

Ended up deciding on this day as the one when I set up Woodpecker. I ended up configuring a freeDNS subdomain, alongside dyndns for my server a couple of days prior to this. I'm forced to use IPv6 for this, thanks to my ISP. Not that I'm complaining about it.

The setup for Woodpecker was relatively decent for NixOS. All I had to do was copy over the suggested nix config over from woodpecker's documentation, add in a bunch of environment variables, alongside other minor modifications and all it took after was a sudo nixos-rebuild switch.

I will note that I did run into a large snag at some point, which involved realizing that the Woodpecker server has two ports in use: One for incoming HTTP requests and one for incoming messages from it's agents. This resulted in a rather confusing error from the logs that seemed to suggest a bug in go's http library.

I later realized that the woodpecker agent needs to be configured to look for for port 9000 for it's server, and the port setting server-side didn't seem to exist, or, I never learned of it.

Once I got both the agent and it's server running without errors in it's logs, I proceeded to configure it for codeberg. The configuration was painless. First step was to do was generate an oauth2 client_id/secret pair and copy it over to the woodpecker config. Followed up by providing a proper authorization link to codeberg.

At this point I attempted to access the woodpecker web interface. I was able to get OAuth2 to work, but I was denied access. I forgot to add my username to woodpecker's admin whitelist.

After that, it was off to the races. Codeberg was seemingly able to access my woodpecker instance and vice versa.

2024-07-22

It was at this point that I decided to look into setting up automatic pipeline execution upon a change in the repository. I thought I had the configuration set up right, but, no matter what I typed in, I was unable to get the pipeline to trigger.

In an attempt to get to the core of the problem, I enabled firewall denied packet logging and loaded up tcpdump. I also figured out codeberg's IPv6 address. I also found out that codeberg has a very nice feature where you can test webhooks for connectivity with a single button press.

After a solid hour of digging, I was unable to find the culprit as to why Codeberg was unable to reach my home's computer.

It's possible that starlink has a firewall of their own setup somewhere, even on IPv6 It's also possible that Codeberg is refusing to send out IPv6, though, from what I read in their feedback repository, that doesn't seem to be the case.

For now, I'm not sure. Will investigate tomorrow.

2024-07-23

The problem turned out to be much simpler than expected. The issue lied within my router's firewall. All I had to do was allow for all incoming IPv6 connections from WAN into my server residing on my LAN. I also took the time to reflash my router to the latest stable build of OpenWRT.

The exact firewall rule is to accept all incoming IPv6 connections for my server's exact IP address. I do need to re-check the server's security, especially when it comes to services with a less than stellar track record, such as smb. Perhaps I should limit such services to locally assigned IPv6 addrs. I know for a fact that everything is at the very least behind a password. Perhaps I should only make ports 22, 80 and 443 accessible?

I guess that segues nicely into another gripe of mine -- my mobile internet provider is unable to assign me an IPv6 address for my phone. This made network testing difficult, as having a device that can operate outside your LAN within the physical confines of said LAN is extremely useful for discovering issues like this. I'm not sure what I can do about it just yet.

There was another tangent issue at play -- for some reason the woodpecker agent service was unable to properly acquire a TLS cert, seemingly for similar reasons, from what I could glean on the error logs. letsencrypt was attempting to dial to my server and was failing to do so. The updated firewall ended up fixing that as well, whereas before nginx was falling back onto a self-signed cert. This all ended up resolving itself after allowing incoming IPv6 connections.

And that's that. All in all, I wouldn't call this easy, but I gained some good experience when it came to setting up infrastructue through NixOS.