Consistently high reliability and low latency are twin requirements
common to many forms of distributed processing;
for example, server farms and mirrored storage access.
To address them, we consider replication of requests with
canceling – i.e. initiate multiple concurrent replicas of a request
and use the first successful result returned, canceling
all outstanding replicas. This scheme has been studied recently,
but mostly for systems with a single central queue,
while server farms exploit distributed resources for scalability
and robustness. We develop an approximate stochastic
model to determine the response-time distribution in a system
with distributed queues, and compare its performance
against its centralized counterpart. Validation against simulation
indicates that our model is accurate for not only the
mean response time but also its percentiles, which are particularly
relevant for deadline-driven applications. Further,
we show that in the distributed set-up, replication with canceling
has the potential to reduce response times, even at
relatively high utilization. We also find that it offers response
times close to those of the centralized system, especially
at medium-to-high request reliability. These findings
support the use of replication with canceling as an effective
mechanism for both fault- and delay-tolerance.