Retry dispersion

When Cloud Drive Mapper (CDM) is working with certain providers, it often receives a Retry-After header when throttling occurs. This header is primarily designed for server-based services, where all requests can adhere to the Retry-After interval in a coordinated manner. However, in client applications (such as CDM), each client operates independently and isn’t aware when another client has been throttled. As a result, each client must handle its own retry schedule, which can lead to extended recovery times as multiple clients retry at similar times.

To address this issue, CDM uses retry dispersion. Retry dispersion adds a randomized delay to the original Retry-After time, helping to distribute retries more evenly.

Specifically, CDM adds a random value of up to 50% of the original Retry-After interval to the retry time. For example:

If the Retry-After header specifies 60 seconds, CDM adds a random value between 0 and 30 seconds to the 60 seconds.
If the Retry-After header specifies 120 seconds, CDM adds a random value between 0 and 60 seconds to the 120 seconds.

Dispersed retry reduces the likelihood of simultaneous retries, helping to ease the load on providers.

In scenarios where CDM does not receive a Retry-After header, we artificially add one of 2 seconds, which exponentially increases up to 5 times.