SRE graph scrying test:
I'm backing up a bunch of files to S3, there's moments where the throughput goes down and moments where it goes up.
Assuming no network activity in a way that matters, why would the throughput number be going up and down like this?
@cadey is there some kind of rate limiting or throttling going on?
Kinda reminds me of https://www.schneems.com/2020/07/08/a-fast-car-needs-good-brakes-how-we-added-client-rate-throttling-to-the-platform-api-gem/
@Schneems Nope! The remote service (which is like S3 but not S3) is able to accept data at my NAS' line rate, and my drives are able to supply data also at line rate.
@cadey super weird.
@Schneems Biblical amounts of small files, there's more overhead in creating lots of little files than there are creating very few big files
