Been Learning Some Network Infra
What follows is a fairly lengthy yap about networking and being incredibly incompetent at a job no-one's paying me to do.
Read at your own risk.
The Situation
I've been tinkering with home networking / homelab stuff for a year or so now, massively accelerated in the last few months since I distributed the workload across two networked machines.
As time goes on I start to get new ambitions for it - media server becomes local cloud, maybe a VPN tunnel, I could use it as a DNS server and have a whole local domain, etc etc.
The latest idea came from my partner - she relies heavily on google docs to share and collaborate on text documents for her writing, and has lately lost further faith in the service after someone in her community got basically banned from accessing their own material. Something something platforms cannot be relied upon etc etc.
Combine that with the incredibly ramshackle implementation of the new UK Online Safety Act and it's become a bit of a priority to host as much as possible locally just so it's not taken from us outright, let alone misused or sold for AI.
I have a vague idea for how I want to do this, but I'm not a network guy so I'm learning all of this from scratch. I spent 6 hours on it the other day and made absolutely no progress, but a few hours last night and a bunch of the simple parts finally clicked.
The Plan
Here's the rough idea:
[ DNS ] -> [ My IP port 80/443 ] -> ( Auth/Security ) -> ( Reverse Proxy Homepage ) -> ( Services )
This was all purely theoretical, but I think it makes some sense. Literally all of this requires a bunch of research for me, but I think I've figured out some of the trickiest stuff. I'm using dockge on TrueNAS to manage docker-compose projects so I can iterate pretty quickly, but I'm also a moron with this stuff so iterations don't always advance knowledge.
First Failures
I started (foolishly?) with the reverse-proxy stuff. I figure if I need to open a bunch of my ports to the internet directly the idea's already cooked. I ended up cycling through several - Traefik, nginx, CADDY - but all had setup quirks that I couldn't get my head around, specifically that they pretty much all required SSL certs or they wouldn't function at all.
This led me down a massive rabbit-hole of manually specifying certs and failing because I don't actually control this URL's target server (something something should have made this blog.criminallyvulgar.dev...). I think I could have worked around this with some DNS nonsense on my provider's control page, but I'm far too stupid about this stuff to figure that out and my eyes were swimming at all the acronyms.
This whole experiment was just frustration, spinning up containers and having them fail relentlessly. I broke my TrueNAS UI by reallocating its port away from 80 to 8080, only to realise afterward that I already had a service on that and there's no error or warning like when docker tries to use an allocated port. That was a fun hour.
After a full day of this head scratching/banging, I called it quits. Then a few days later, I got a random article about a homelab enthusiast using HAProxy without https, so no SSL certs required.
This was exciting.
Actual Progress
I spun up a demo box for HAProxy's load-balancing, opened 400 tabs of config and setup examples, and after 20 minutes of docker-compose network bullshit I was able to route HAProxy ports to other container's open ports (which were otherwise invisible to my home network.)
Finally, a test box! This is all I was working towards previously, a fully local-to-this-machine example I could tinker with.
Next step was to set up a little NGINX HTML homepage to test different http link formats - just to reiterate, I do not understand networking so I need to see things to know they exist. I wrote a bunch of http://192.168.x.x:port or http://subdomain.domain example links, and none of them worked.
Good! My understanding was that they wouldn't with how I'd configured the containers. Score one for the world's worst net admin.
The Sketchy Part (advice always appreciated)
This was about the limit I could do without opening myself up to the wider internet.
To avoid it, I tried to setup a local DNS server but it appeared to conflict with my ISP, and I might be able to resolve that but I'm not breaking my partner's internet while she's on holiday - I might return to that idea once she's back to work. That was about an hour of tinkering, but rewarding stuff. I'm starting to recognise how things are going wrong.
So I bit the bullet, opened 80/443 and pointed a DDNS URL at it (I used duckdns.org, a DDNS is basically a free way to get an ugly-but-functional URL for projects/testing like this).
Instantly the simple NGINX page loaded at my URL - fucking glorious.
None of the links worked, cool. That also made sense.
Then I spent about 2 hours tinkering with the HAProxy config file, setting up literally dozens of variations of frontends and backend, until finally understanding how subdomains interact with the main domain and achieving a kind of nirvana - I will never have this problem again, because now I have achieved capital-u Understanding of something.
The real key was understanding that the only frontend (basically "what does the outside world see?") that matters for HTTP is the one on port 80. When a person visits website.duckdns.com, HAProxy:80 points them behind the curtain to the little page with the various links. If they click on a link taking them to service.website.duckdns.com, again HAProxy:80 again takes them behind the curtain, this time to wherever the service is.
I get that this is extremely rudimentary stuff for admins, but it's not something that I found to be well explained in any setup guide. I mentioned previously that networking guides always seem to assume more knowledge than I have, no matter how basic - this was a crucial step in understanding how to set up everything else.
At this point it was 1am so I went to bed.
Next Steps
Ok I closed the ports first - no need taking that kind of risk.
So what next?
If we look at the diagram above, we can see that I missed a couple of crucial steps. Namely, I have zero security and no actual proper homepage/portal to direct users to services. It's also a bit convoluted.
Now that I've come this far, getting HTTPS/SSL support should be simple - duckdns has a system for it, I can autorenew with a docker-compose service, and when the time comes to put it behind a proper URL it should be trivial to migrate. The issue for this was having to start with SSL before getting anything else running.
The little homepage with links is a simple and ugly solution, but might serve my purposes with some basic styling - I just need to hide it behind some Auth so we can vet who has access to it. I have absolutely no idea where to start with Auth, to be honest. Haven't researched it at all yet.
I might be able to solve both those problems with a Homepage service like Organizr. Though probably not Organizr - I tried it out and hated working with it in every way. Incredibly ugly design language.
The elephant in the room with my setup is that I use HAProxy to route traffic to NGINX - but NGINX can also do that. The config files are an order of magnitude less pleasant to work with, but it would be one less point of failure in a system I'll barely understand.
I think that'll also make it easier to bundle in Auth at the top level of access, but that may be a naive assumption.
Conclusion
So after over a dozen hours of fucking around, I've finally found something out. I have plans for what to do next, the resources to do it, and a euphoric sense that I can achieve anything if I set my mind to it!
I would not recommend it.