poVoq here. I am starting to think I am cursed, because everything works fine (or is at least fixable with a remote server restart) until the second I go on a longer business trip and can’t easily access the hardware.
Remote access is mostly lost, but I was able to triangulate the issue and of course it is something I had on my list of potential issues, but as a low priority item 🫠 At least it isn’t the same problem again as in 2025.
What exactly triggered the issue is unclear though. I was still able to access the reverse-proxy server yesterday (shortly before it went down) and tried to do some updates for that php security problem in a VM that runs on the same machine. I thought I had those covered, but there is a slight chance that it is related and someone chained another exploit to escape the VM.
Anyways, it looks like the realistic / not overly optimistic scenario is that the server will stay down until christmas. I did have someone who could restart the servers on site until this summer, but they had to relocate to mainland Portugal for job reasons (tourism sector) and so they will also only return for christmas.
So yeah, not great at all, and I will have to think more seriously about how compatible this kind of self-hosting is with my current job, which limits my access to the servers for a few weeks at least once a year. It was easier during the pandemic times when I started all this, as deployments were shorter and more seldom back then.
I think a more realistic option is to move my self-hosting ambitions down a notch and only focus on more redundand backend services that are too costly on a rented VPS.
You really just need a comically long stick to flick the power switch from the mainland. Simple as.
Or perhaps pay that Satyr, I mention in another comment.
In seriousness, is physically moving the server possible? I know absolutely nothing about servers at all. Or would we be able to host it virtually? I realize that probably defeats the idea of a self hosted instance. What about as a back up or something?
I’m completely just throwing spaghetti at a wall here, and it isn’t even half cooked.
Due to the LLM scrapers I have been thinking that I might have to move to a hybrid system of going through a VPS with only some storage and compute services on my home servers remaining. It is an annoying expense of about 20€/month that I was hoping to avoid, but it will give some other benefits like a fully fixed IP, IPv6 support and DDOS protection.
In such a setup the service would degrade better, like images breaking, but the main service staying up when my home server goes down.
But it is not something I can setup now that the service is already inaccessible.
poVoq here. I am starting to think I am cursed, because everything works fine (or is at least fixable with a remote server restart) until the second I go on a longer business trip and can’t easily access the hardware.
Remote access is mostly lost, but I was able to triangulate the issue and of course it is something I had on my list of potential issues, but as a low priority item 🫠 At least it isn’t the same problem again as in 2025.
What exactly triggered the issue is unclear though. I was still able to access the reverse-proxy server yesterday (shortly before it went down) and tried to do some updates for that php security problem in a VM that runs on the same machine. I thought I had those covered, but there is a slight chance that it is related and someone chained another exploit to escape the VM.
Anyways, it looks like the realistic / not overly optimistic scenario is that the server will stay down until christmas. I did have someone who could restart the servers on site until this summer, but they had to relocate to mainland Portugal for job reasons (tourism sector) and so they will also only return for christmas.
So yeah, not great at all, and I will have to think more seriously about how compatible this kind of self-hosting is with my current job, which limits my access to the servers for a few weeks at least once a year. It was easier during the pandemic times when I started all this, as deployments were shorter and more seldom back then.
Be really nice to build some local community so that you can both self host and have this kind of support if you need it
Would be nice, yes. But easier said than done 😓
I think a more realistic option is to move my self-hosting ambitions down a notch and only focus on more redundand backend services that are too costly on a rented VPS.
You really just need a comically long stick to flick the power switch from the mainland. Simple as.
Or perhaps pay that Satyr, I mention in another comment.
In seriousness, is physically moving the server possible? I know absolutely nothing about servers at all. Or would we be able to host it virtually? I realize that probably defeats the idea of a self hosted instance. What about as a back up or something? I’m completely just throwing spaghetti at a wall here, and it isn’t even half cooked.
Due to the LLM scrapers I have been thinking that I might have to move to a hybrid system of going through a VPS with only some storage and compute services on my home servers remaining. It is an annoying expense of about 20€/month that I was hoping to avoid, but it will give some other benefits like a fully fixed IP, IPv6 support and DDOS protection.
In such a setup the service would degrade better, like images breaking, but the main service staying up when my home server goes down.
But it is not something I can setup now that the service is already inaccessible.
Yeah fair enough. I agree, annoying on your end about the expense.
Bummed that you can’t seem to catch a break with the sever suffering from separation anxiety and throwing a fit every time you leave for work.
Artistic rendering. (had to use catbox since lemmy.cafe doesn’t seem to allow image uploads?)
Base image courtesy of @undthr@horp.sciops.net
You have no idea how much I wish I could fit this in my profile picture