Take the latency into account when evaluating whether or not to reuse a connection.