had this bizarre issue where a backup script ran fine monday through friday but failed every saturday. took me way too long to figure out.
turns out the script was calling an internal API that had rate limiting. during the week, the backup ran at 2am when nobody was using the system. on saturdays, some other batch process kicked off at the same time and ate up all the API quota.
the fix was dead simple — stagger the start time by 30 minutes. but finding the cause? that was a couple hours of my life I’m not getting back.
moral of the story: if your cron job is failing intermittently, check what else runs at the same time. crontab -l only shows YOUR crontab. other users and system crontabs in /etc/cron.d/ might be competing for the same resources.


Since we’re sharing troubleshooting weirdness:
My Active Directory admin account kept getting locked out on a seemingly random basis, about once every day or two. Had my bosses credentials, and yes, I know how dumb that is, so I’d unlock myself. No one else was affected, but it was an annoyance for me.
After two months of sporadic troubleshooting, I found it! When I first setup the domain (it was my first time, ok?!), I used my personal credentials to allow the DHCP server to poll the DNS server (something like that). Everytime some PC needed a new lease, it would use my personal creds 3 times and then fail. Because I had changed my password like a good little admin. 🙄