Linux

They don’t do anything.

They have lots and lots of redundancy, and when enough drive fails, they decommission the entire server and/or rack.

Them big players play at a very different scale than the rest of us.

We don't do maintenance, we just have redundancy, and backups, then replace failed components.

Hardware-backed RAID, with error monitoring and patrol read. iSCSI or similar to present that to a virtualization layer. VMFS or similar atop that. Files atop that to represent virtual drives. Virtual machines atop that.

Patrol read starts catching errors long before SMART will. Those drives get replicated to (and replaced by) hot spares, online. Failing drives then get replaced with new hot spares.

But all of that is irrelevant, because at the enterprise level, they are scaling their applications horizontally, with distributed containers. So even if they needed to do fsck at the guest filesystem level (or even if they weren't using virtualization) they would just redeploy the containers to a different node and then direct traffic away from the one that needs the maintenance.

They almost undoubtedly would. That wasn't the problematic statement.

Let's go over some fundamentals here.

fsck is a utility for checking and repairing filesystem errors. Some filesystems do not support doing so when they are mounted.

Why? At a high level, because:

The utility needs the filesystem to be in a consistent state on disk. If the filesystem is mounted and in-use, that will not be so: The utility might come across data affected by an in-flight operation. In its state at that exact moment, the utility might think there is corruption and might attempt to repair it.

But in doing so, it might actually cause corruption once the in-flight operation is complete. That is because the mounted filesystem also expects the disks to be in a consistent state.

Some filesystems are designed to support online fsck. But for OP's purposes, I assume that the ones they are using are not so (hence the reason for the post).

"I know!" said the other commenter. "RAID uses mirroring! So why not just take the mirror offline and do it there?"

Well, for the exact same reasons as above, and then some additional ones.

Offlining a mirror like that while the filesystem is in use is still going to result in the data on the drive being in an inconsistent state. And then, as a bonus, if you tried to online it again, that puts the mirrors in an inconsistent state from each other too.

Even if you wanted to offline a mirror to check for errors, and even if you were doing a read-only check (thus not actually repairing any errors, thus not actually changing anything on that particular drive), and even if you didn't have to worry about the data on disk being inconsistent... The filesystem is in use. So data on the still-online drive has undoubtedly changed, meaning you can't just online the other one again (since they are now inconsistent from each other).