Hello everyone,
I have an issue with my desktop, which I really don’t understand.
It is a desktop PC, with a Nvidia GPU, which I use for development (of AI, but like the image analysis part, not the slop part). I usually let it always run, and my previous machine was working fine like this.
However, on this machine, I get crashes after either a few days or a few weeks.
These “crashes” are weird : the machine is still up (and I can ping it), however I can’t ssh into it, I can’t connect to cockpit either, and can’t login physically on it (the display is frozen).
The only way to get it back up is to forcefully shut it down, wait a little and then powr it back up; but this isn’t ideal, since it interrupts computations etc
The only crashes shown in journalctl are related to irrelevant (I believe) programs : KDEConnect, DrKronqi, etc; nothing right before the crash.
Do you have any idea what could cause this ?
I tried logging temperatures, but nothing abnormal showed up. I ran memtest86+, nothing showed up either. Smart tells me everything is alright for my drives as well.
Right now I’m thinking about changing the alimentation or the motherboard, but frankly I don’t know what’s going on and it’s really puzzling me.
I did have an old laptop with bad SSD that would have similar problem - cannot ssh, but can ping, and desktop quickly gets unresponsive.
On root device issues kernel remounts root drive to read-only, and that breaks everything, but the machine still works - yet you get no logs because /var/log is also read-only.
I did some googling and it seems SSH requires writable /tmp, so you might setup some in-memory one, if you haven’t already - you could check if your machine uses tmpfs (in-memory tmp) to reject this hypothesis.