Fallacies of Distributed Computing: 2. Latency is zero

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

2. Latency is zero

The problem

Latency is defined as the time interval between a stimulus and the corresponding response. In network terms, it’s the amount of time required to move data from one location to another. Network traffic is not instant.

At one time remote objects were seen as a desirable design objective, encapsulating local or remote implementation details and allowing for so-called network transparency. For example:

// C#
var user = new User(1);
user.Password = "Password1";

The above operation may simply set the password on a User instance in memory or it may also initiate one or more remote calls to update a user password in a database, the implications are opaque.

Thankfully, the use of remote objects is, by and large, frowned upon these days in favour of other techniques such as DTOs which provide greater transparency. (It is always worth formalising the segregation of local and remote calls through effective application of appropriate design patterns.)

One technique frequently used with ORM technologies is deferred, or lazy, loading. With deferred loading, data is retrieved only when needed rather than prefetched.

A typical example might entail retrieving the orders and order line items associated with a customer account:

// C#
var svc = new OrderService();
var orders = svc.GetOrdersForCustomer(1); // Returns an IQueryable<Order>, deferred

foreach (var order in orders.ToList()) { // Enumerated, call
	var items = order.GetItems(); // Returns an IQueryable<OrderLineItem>, deferred
	// ...
}

Here, rather than fetching all the customer orders and associated line items in one call, the orders will be fetched when enumerated with the corresponding order line items also fetched on first use. This is a classic example of an unintended SELECT N + 1 query, resulting in multiple remote calls to the database or service providing the data.

Solutions

Reduce the number of remote calls

Given each remote call made will be several orders of magnitude slower than a local equivalent, you should design with the objective of optimising the number of remote calls. One way to achieve this by designing payloads to include all the data you might need to support a particular use case, removing the likelihood (or need) of additional remote calls to fetch data at more granular levels.

Locate the data closer to the client

As latency is ultimately a physics problem, locating data closer to the geographic location of the client will facilitate reduced latency (all other network factors being equal). All public cloud infrastructure providers such as AWS, Azure, or GCP support deployment into multiple availability zones.

Caching

Caching should also be implemented wherever possible to reduce unnecessary remote calls.

Derek Lawless