I have 2x20TB, 7x8TB, and 1x6TB spinners and 3x500GB SSDs, so a typical RAID setup isn’t really possible. I’m not buying any more hard drives for awhile.
I switched to unRaid from Windows/Drivepool/Snapraid because. Well I don’t really know why, wanted to try something new. Wish I had started with the trial instead of paying for the license, but hindsight is 20/20 as they say. Thankfully it wasn’t too expensive.
My big issue is just write speeds. I run sabnzb/sonarr/radarr/plex, and I’ve got everything configured properly, but sabnzb can barely handle 20 mb/s even though my 1gig cable isp pulls 100 mb/s.
I actually pulled the parity drive out of my array just to get better speeds because with parity you’re automatically 1/2-ing your disk write i/o. With WD shucks 5400 rpm’s I should be able to hit over 100 mb/s writes, but with parity plus sabnzb repairing I’m lucky to get 40 mb/s. It’s just abysmal.
I never had any slowdowns using windows and drivepool. Even during repair + download operations. Obviously snapraid is on demand so parity doesn’t play into things.
I have tried the cache drives, but then it just fills up and you’re in an even worse place where you have to wait on mover to move the data from the ssd cache to the array, but if you’re still downloading then you’re trying to download to the array and repair on the array too. That’s even slower than if you don’t have the cache setup.
I guess if I was downloading a tiny amount everyday it wouldn’t be such a big deal, but I’m trying to catch up on the time sonarr/radarr weren’t running while I moved my existing data to unRaid, as well as some new keywords for downloading x265 and upping some 720p content to 1080p. So think about a long, multi TB, queue in sabnzb.
So now I’m in the boat of thinking, maybe I should go back to windows. I’ve read about mergeFS and snapraid, but it seems like a fairly high learning curve when Windows worked fine for years. I wish I had never switched.
Am I missing something? I have reconstruct write “on” and took out the parity drive, and it does alright downloading, but a repair during download still brings it to a crawl. Is there something better out there I’m not thinking of?
Any tips would be appreciated.
Hard drives performance is never as good as SSD or even ISP. Give it a break, it’s mechanical, not solid state.
Raid parity calculations take time too, don’t underestimate the impact on performance of keeping your data safe.
Snapraid is a good solution if you can schedule your backups as it is like raid, but not continuous.
My suggestion mirrors more of the above, download to ssd then move to hdd afterwards.
I’ve been on mergerfs + snapraid for years and haven’t had any issues. I even tried downloading to a NVMe but then you just move the bottleneck from downloading to moving off the cache drive. And if you download more than your cache drive, you’re no better off.
Setup mergerfs and make sure and specify “most free space”. Then each file will actually get written to a different drive because it evaluates the space and rotates the writes. The n just run snapraid nightly to keep parity in sync. (This does mean you could lose something between download and sync, but if you just downloaded, you can probably grab again)
Now, the big assumption here is that most of your files are large media files. If you’re moving thousands of small files, you’ll probably notice a performance hit.
I can easily saturate my 2.5 gbps fiber connection with no issues. And as others have said, it’s just standard files and you’re not hosed it a single drive dies.
it’s about carefully tuning shares to what actually needs to be on the cache vs what can be directly written to disks. For example the downloaders can download to the cache shares but when the *arrs are moving the files, have them move to shares which are disk only. So you can get fast downloads and repairs, and a slower move to disk which should not impact performance much.
I agree it’s too much tuning which is needed, but once you do it, it works fine
Everybody says to use the trash guide for setup. So I setup Cache-> Array. I did exactly the same folder structure as the trash guide.
What confuses me, is won’t a completed movie that radarr imports just hardlink to the media/movie folder on the cache? What is telling it to move it to the array? If I have to wait for mover, then mover is wasting time moving my usenet/incomplete folder to the array, but the trash guide hammers home how hardlinking/atomic moves are really important.
It seems like the better setup is to have usenet only go to cache, or at least only incomplete. Leave the media as array only, no cache. And then let sabnzb move it to array/completed or let sonarr/radarr move it to the array/media. But then that isn’t an atomic move? Are atomic moves not really that big of a deal?
Normal operation I’m not filling up my cache drive, so its kinda moot, but what about the rare occasion where I queue 10 seasons of a show or something? Then I’d want stuff moving off cache asap instead of waiting on mover.