Basically, the fabric of the strap constantly rubs up against the case back, and when dirt gets trapped in between, over time, that causes scratches and even marring on the case back. According to a dedicated article by British GQ, this kind of marring cannot be prevented by simply cleaning the strap. They say no matter what, the fabric will cause wear to the case back. Are they right?
It’s technically plausible, but it’s not a practical concern. You might see significant marks after a couple hundred years of wear.