dereklawless.ie RSS feed

Towards Effective Standups

Thu, 03 Apr 2025 00:00:00 GMT

A guide I wrote for the Scrum development team in my previous startup and intended to foster actual team self-organisation vs. top-down Scrum.

Perhaps too dogmatic in places it is presented here for posterity if not value.

A sprint backlog is made up of PBIs (Product Backlog Items) plus a delivery plan (Sub Tasks & Estimates). The team will focus on delivering an increment of “Done” for every PBI based upon this delivery plan.

The daily standup is an “informal mini planning event” to ensure the team understands what is planned for the next 24 hour period of delivery (of the plan).

To understand the remaining sub tasks to deliver the aforementioned plan, the current state of the sprint must be known - hence why the team members inform each other of what they worked on in the previous 24 hours and ensure Jira remains up to date.

It also provides the following additional benefits:

A method of cross-checking progress with team members;
An accountability mechanism that has each team member accountable to other team members for their responsibilities;
A visible demonstration of the ability of the team to self-manage their project responsibilities.

Standup format

Answers to the following 3 questions only please:

What task(s) did you work on/accomplish yesterday? - Keep brief - the focus should be on what the team are planning for today
What task(s) do you plan to work on/accomplish today?
What issues are currently preventing progress?

Guidelines

These guidelines are not dogma - the Scrum team can decide what works best for them. All suggestions are welcomed.
Remember the standup is not a formal reporting meeting, nor is the team reporting to the scrum master (or other project stakeholders) - no minutes will be sent out.
To reiterate, it is an “informal mini planning event” for the team’s next 24 hour delivery period and not a solutioning workshop - no new ideas and no multitasking.
Jira should be updated prior to the standup for the previous 24 hours period - no updating should occur in the standup (for this period).
Try to stick to the questions at all times (except whereby the team has agreed to change).
Everyone should be on time and prepared.
The last person on the call will be responsible for commencing the standup conversations. Further the scrum master is not required to host the call (it is not a reporting meeting); in his/her absence the team should continue to have the standup.
Time limits may be enforced for conversations. Anyone in the team can insist on moving a conversation out of the standup - remember updates should be applicable to everyone.
Non team members (i.e. participants not responsible for delivery of sprint backlog) are welcome to the standup but do not have a formal speaking role.
The team is self organising, hence the Scrum team members are responsible for proactively ensuring the blocking issues are addressed/removed. Further the standup is not the sole means of communication - team members should not wait until the next stand-up to raise/resolve blocking issues.

Downloading Lambda layer versions using the AWS CLI

Thu, 16 Jun 2022 00:00:00 GMT

The packaged .zip for a Lambda layer can be downloaded using the AWS console (appropriate IAM permissions notwithstanding) - as long as you have access to the host AWS account of course.

If the Lambda layer originates in a different AWS account you don’t have console access to, you will need to use the AWS CLI.

Why would you want or need to do this though?

As with any third party dependency it makes sense to examine it rather than simply trusting it to be benign in nature
You want to override the layer’s behaviour in a specific way (i.e. by merging a later layer) and need to examine its implementation

Fortunately, the aws lambda CLI provides two useful options - one for downloading a layer version by name, the other by its ARN.

Downloading a Lambda layer version by name

To download version 99 of a Lambda layer called acme-corp-lambda-layer using its name, use the get-layer-version option:

URL=$(aws lambda get-layer-version --layer-name ars:aws:lambda:$AWS_REGION:$AWS_ACME_CORP_ACCOUNT:layer:acme-corp-lambda-layer --version-number:99 --query Content.Location --output text)

curl $URL -o acme-corp-lambda-layer.zip

Downloading a Lambda layer version by ARN

If you want to download version 99 of of the layer using its ARN instead, use get-layer-version-by-arn:

URL=$(aws lambda get-layer-version-by-arn --arn arn:aws:lambda:$AWS_REGION:$AWS_ACME_CORP_ACCOUNT:layer:my-lambda-layer:99 --query Content.Location --output text)

curl $URL -o acme-corp-lambda-layer.zip

Conditionally tagging resources in CloudFormation

Wed, 27 Apr 2022 00:00:00 GMT

While tagging resources in CloudFormation is straightforward, conditionally tagging them is a little non-obvious and requires use of conditions.

Use case: tagging a release version

I wanted the ability to conditionally tag a Secrets Manager secret with a release version when the secret was deployed into production. For lower environments, I didn’t care / want to specify a release version.

First, we define a parameter that will contain the release version:

Parameters:
	ReleaseVersion:
		Type: String
		Description: The release version e.g. 1.2.3
		Default: ''

Next, we define a condition to easily check whether a non-default value was provided for the parameter:

Conditions:
	HasNoReleaseVersion:
		!Equals [!Ref ReleaseVersion, '']

Finally, we tie it all together on the resource by conditionally setting the release tag depending on the evaluation of HasNoReleaseVersion:

Resources:
	MySecret:
		Properties:
			Tags:
				- Fn::If:
					- HasNoReleaseVersion
					- !Ref AWS::NoValue
					- Key: release
					- Value: !Ref ReleaseVersion

The AWS::NoValue pseudo parameter will ensure the tag is not created unless a non-default value for the release version was provided.

The unknown Type in TypeScript

Wed, 21 Apr 2021 00:00:00 GMT

Introduced in TypeScript 3.0, the unknown type exists to provide a type-safe counterpart to the any type. To understand why this was considered a necessary or desirable addition to TypeScript, we need to revisit any and why its use can be problematic.

The `any` type

The any type has been in TypeScript since its initial release and represents all possible JavaScript values (making it a top type of the type system).

any can be problematic because of its flexibility - it allows typing to be circumvented:

// TypeScript

let value: any;

// Any value can be assigned to an 'any' type:
value = true;
value = 0;
value = "Hello world";
value = [];
value = {};
value = null;
value = undefined;
value = Date.now;
value = new Error();
value = Symbol();
// ...

// No check on use:
value * value;
value[1];
value.foo.bar.baz();
new value();
// ...

While convenient, any undoubtedly increases the risk of errors or unexpected behaviour sneaking into code simply by negating one of the primary benefits of using TypeScript in the first instance - strong typing.

The `unknown` type

The unknown type is less permissive than any. While any value is assignable an unknown can only be assigned to itself (or toany).

Revisiting the previous example:

// TypeScript

let value: unknown;

// Any value can be assigned to an 'unknown' type:
value = true;
value = 0;
value = "Hello world";
value = [];
value = {};
value = null;
value = undefined;
value = Date.now;
value = new Error();
value = Symbol();
// ...

// An 'unknown' type can be assigned to 'any' or 'unknown':
const toAny: any = value;
const toUnknown: unknown = value;

// Assignments to other types will fail:
const toBoolean: boolean = value;
const toString: string = value;
// ...

Narrowing and asserting

Operations cannot be performed on an unknown type without first narrowing to a more specific type or asserting:

// TypeScript

let value: unknown;

// Operations including the following will error:
value[1];
value.foo.bar.baz();
new value();
// ...

One method of narrowing is with the typeof operator:

// TypeScript

const value: unknown = 'hello world';

if (typeof value === 'string') {
	// Narrowed to a string type, can safely call 'toUpperCase()'
	console.log(value.toUpperCase());
}

Types can be asserted using the as operator:

// TypeScript

const value1: unknown = 'hello world';
const value2: unknown = null;

console.log((value1 as string).toUpperCase()); // Ok
console.log((value2 as string).toUpperCase()); // Error

🔔 Type assertions should be used with caution as they may result in expected run-time issues.

Unions and intersections

In a union, an unknown absorbs everything else (the exception to this rule is a union with any).

With an intersection, the converse applies - that is, other types will subsume the unknown.

// TypeScript

// Unions:
type union1 = unknown | null;
type union2 = unknown | undefined;
type union3 = unknown | boolean;
type union4 = unknown | number;
type union5 = unknown | string;
type union6 = unknown | null | undefined | boolean | number | string;
// ...
type unionN = unknown | any; // any

// Intersections:
type intersection1 = unknown & null;
type intersection2 = unknown & undefined;
type intersection3 = unknown & boolean;
type intersection4 = unknown & number;
type intersection5 = unknown & string;
type intersection6 = unknown & null & string; // never
// ...
type intersectionN = unknown & any; // any

When to use `unknown`

Wherever possible specify an explicit type and avoid using any or unknown. Explicitly specifying types allows you to take advantage of TypeScript’s compile-time checking, provides a more explicit contract (something especially important for library writers to consider), and should result in fewer run-time issues.

Where this is not possible, prefer unknown over any as its characteristics require more deliberate and thoughtful use.

Nemawashi

Thu, 21 Jan 2021 00:00:00 GMT

Photo by Emma Gossett

Literally translating as “to go around the roots”, Nemawashi (根回し) historically described the farming activity of transplanting a tree from one location to another. Before transplanting, the tree’s roots are dug around and soil from the new location is introduced. This allows the tree to acclimate to its new environment prior to being moved, increasing its chances of survival.

In modern times, Nemawashi describes a process of laying the foundation for a proposal by first seeking consensus inside and outside the team informally - allowing others to give their opinions and contribute - before formally presenting the proposal. This increases the probability of the proposal being accepted and ultimately executed upon.

Although widespread in Japanese business, Nemawashi is perhaps most recognisable outside Japan as one of the 13 pillars of the Toyota Production System (from which Lean was partly derived).

Nemawashi and software development

Conversation about Nemawashi typically centres around the application of Lean to classical business activities and manufacturing. However, it’s also a technique that software development teams can make effective use of.

Getting buy-in

Consider a significant architectural change to an existing software system e.g. moving from a monolithic architecture to a microservices-based one.

An architect can dictate a design to the development team, handing it off to be implemented according to her vision or she can practice Nemawashi.

Nemawashi does not mean the architecture will be compromised as a result of design by committee, quite the opposite. By allowing the development team to examine the proposed new architecture in detail before architecture sign-off and provide their inputs and insights, buy-in amongst team members will be increased.

On a software development project of any complexity having a development team operating beyond the role of reluctant implementor can be a critical factor in maximising the probability of success.

Building consensus

Proposals may be organisationally or politically sensitive for a variety of reasons.

A proposal to move from manual infrastructure management on a project to Infrastructure as Code (IaC) might be viewed with a mix of suspicion, concern, and derision by an existing infrastructure team. Although there may be an element of self-preservation on the part of the infrastructure team, their views should not be dismissed as entirely unfounded.

Here, Nemawashi can be leveraged to build consensus with the infrastructure team - acclimating them to the proposal through positive engagement, assuaging their concerns, being open to feedback, and so on.

The goal

The goal is to see your proposal succeed.

Nemawashi provides a method for achieving this, recognising that by building consensus and offering inclusivity the art of persuasion is made easier.

Nemawashi is somewhat orthogonal to the model of traditional software team ‘leadership’, where forcefulness is viewed as a positive attribute and celebrated. (There can be little doubt this is partly because the software development industry is wholly under-representative of society.)

As development teams diversify - bringing new perspectives, approaches, and viewpoints - methods such as Nemawashi may become more prevalent and, I believe, recognised as a key attribute of high-performance software development teams.

Year in review

Wed, 13 Jan 2021 00:00:00 GMT

Photo by Juan Rumimpunu

2020

😱

Career

Many software development contractors I’ve worked with are poor at the business of it - not in their technical abilities, more because they’re individuals who care deeply about refining their craft and advancing software engineering. Admirable goals but ones that can be difficult to reconcile with roles where throughput of code is (typically) the primary metric of interest to a temporary employer who wants a problem solved yesterday.

I found myself at a crossroads professionally with my level of dissatisfaction increasing as 2020 dragged interminably on. Working as a ‘senior’ software engineer on a rolling contract, I was performing my assigned role well but felt capable of contributing much more and eager for change.

In September an opportunity to manage an engineering team presented itself once again. I jumped at the chance to get back to a role to where I could affect change, evangelise the role of engineering in product design, and foster a quality-focused engineering culture.

Website

For many years I specialised in frontend development and UX. Naturally, I failed to maintain a dedicated web presence of my own, it had felt too much of a busman’s holiday.

The impetus for this website was primarily to encourage me to write more - something I find enjoyable but rarely set aside time for. Having stepped away from frontend development a couple of years ago, it was also enjoyable to tentatively come back to it.

Another reason for this website is to give me a starting point for cultivating my personal brand (I apologise in advance). You should always own your own words.

Design thinking

I began a Certificate in Design Thinking third level course on a part-time basis in late 2019, and was awarded the accreditation in September 2020.

The techniques I learned on the course would have been incredibly useful when I was the CTO in a startup and trying to find that elusive product-market fit, but more on this another day.

Certifications

Being fortunate enough to work from home most of the year afforded me with additional free time that is otherwise spent commuting. I decided to use some of this time to focus on improving my marketability should the need arise - something which felt proactive given the situation that was unfolding globally.

I remain sceptical of the value placed on certifications in the software industry and believe they are no substitute for actual experience or, indeed, aptitude and inquisitiveness. More problematically, certifications can involve significant financial outlay and may constitute yet another obstacle to a more inclusive industry.

However, given I earn my living primarily from consulting they can be undeniably useful to possess and structured learning can lay the groundwork for developing deeper understanding.

I set myself the goal of attaining two certifications by the end of the year.

In August I sat and passed the AWS Certified Solutions Architect – Associate exam. As my daily work involves designing and developing serverless architectures on AWS it was the logical choice to complete first.

TOGAF certification was something I considered many times (more so when I was working in banking) but always deferred for one reason or another. In December after periodic and then concentrated study I sat and passed the TOGAF® 9 Certified exam.

2021

Professionally, I intend to be less dark matter this year. How that will manifest I’m not fully sure yet although I have some ideas for interesting projects and other collaborative things.

On this website I intend to write more about topics of specific interest to me - revisiting my previous life in a startup, the characteristics of high-performing teams and individuals, the confluence between design and development, cloud, computing history, and so on.

Further certifications are likely, but as the mood takes me. I think rebalancing after-work hours in favour of other interests will be more conducive to maintaining positive mental health given everything happening in the world.

Binary Search

Sat, 21 Nov 2020 00:00:00 GMT

A binary search employs a divide and conquer strategy to find the position of a target value in a sorted list. A binary search is also known as a half-interval search, logarithmic search, or binary chop.

For binary search to work the list must be sorted (sorting is a discussion for another time).

Given a sorted list from which we want to find the number 35:

Begin by splitting the list into two halves, disregarding the half that is less than, or greater than, 35:

Split the remaining half into two — again disregarding the half that is less than, or greater than, 35:

Repeating again finds 35 at position 5 in the list:

Complexity

🔔 Complexity is considered in terms of worst case.

Time

	Notes
Θ(log n)

Space

	Notes
Θ(1)	Requires three pointers to the list elements, regardless of list size

An implementation

// TypeScript
export interface IBinarySearchResult {
	position: number;
}

export function binarySearch<T>(values: T[] = [], target: T): IBinarySearchResult {
	let low = 0;
	let high = values.length - 1;

	while (low <= high) {
		const middle = Math.floor((low + high) / 2);
		if (values[middle] === target) {
			return {
				position: middle,
			};
		}
		if (target < values[middle]) {
			high = middle - 1;
		} else {
			low = middle + 1;
		}
	}

	return {
		position: -1,
	};
}

A recursive alternative

// TypeScript
export function binarySearchRecursive<T>(
	values: T[] = [],
	target: T,
	low: number = 0,
	high: number = values.length - 1): IBinarySearchResult {

	if (low <= high) {
		const middle = Math.floor((low + high) / 2);
		if (values[middle] === target) {
			return {
				position: middle,
			};
		} else if (target < values[middle]) {
			return binarySearchRecursive(values, target, low, middle - 1);
		} else {
			return binarySearchRecursive(values, target, middle + 1, high);
		}
	}

	return {
		position: -1,
	};
}

Code on Github

Sieve of Eratosthenes

Sat, 07 Nov 2020 00:00:00 GMT

The Sieve of Eratosthenes is an ancient algorithm for discovering primes in a consecutive sequence of natural numbers.

Given a consecutive sequence of natural numbers such as:

2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20

Starting with 2 (0 and 1 are not prime), eliminate every subsequent second number:

2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20

Starting with 3, eliminate every third number:

2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20

Repeat the process, each time starting with the next smallest number in the sequence that has not been eliminated. When the limit of the sequence is reached the numbers remaining are the primes:

2, 3, 5, 7, 11, 13, 17, 19

An implementation

// TypeScript
export class SieveOfEratosthenes {
	public static findPrimes(upperBound: number = 2): number[] {
		const sieve = new Array(upperBound + 1).fill(true);

		// 0 and 1 are not prime by definition.
		sieve[0] = false;
		sieve[1] = false;

		for (let i = 2; (i * i) <= upperBound; i++) {
			if (sieve[i] === false) {
				// The number is not a prime, therefore its multiples will also not be primes.
				continue;
			}

			for (let multiplier = 2; (multiplier * i) <= upperBound; multiplier++) {
				// A multiple of a prime is not a prime.
				sieve[multiplier * i] = false;
			}
		}

		const primes: number[] = [];
		for (let i = 2; i < upperBound; i++) {
			if (sieve[i] === true) {
				primes.push(i);
			}
		}

		return primes;
	}
}

// Test
import { SieveOfEratosthenes } from "../sieve-of-eratosthenes";

describe("SieveOfEratosthenes", () => {
	it("is exported", () => {
		expect(SieveOfEratosthenes).toBeDefined();
	});

	describe("findPrimes()", () => {
		it("returns all prime numbers in a given bound set", () => {
			const knownPrimes = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47];

			expect(SieveOfEratosthenes.findPrimes(50)).toEqual(knownPrimes);
		});
	});
});

Code on Github

Fallacies of Distributed Computing: 8. The network is homogeneous

Tue, 30 Jun 2020 00:00:00 GMT

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

8. The network is homogeneous

The problem

Networks typically must support a wide variety of connected clients — from desktop-class machines to mobile and IoT devices. These devices and the applications running on them may not be interoperable and a variety of standards-based and proprietary protocols and data formats may be in use.

The network is not homogenous, it is heterogenous.

Solutions

Interoperability is key.

Where possible, avoid applications that use proprietary protocols in favour of those leveraging standards-based ones. Favour data formats that enable interoperability e.g. JSON or XML over binary formats (fallacy 7 aside) and architectures such as REST or web services, particularly when integrating with third-party services.

Fallacies of Distributed Computing: 7. Transport cost is zero

Mon, 29 Jun 2020 00:00:00 GMT

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

7. Transport cost is zero

The problem

Whereas the second fallacy relates to the time taken to transport data over a network, the seventh fallacy relates to the resources required to do so.

Marshalling

There is a resource (and resultant latency) cost in marshalling data between the application layer (Layer 7 on the OSI Reference Model) and transport layer (Layer 4) in order to send it over the network. While this may be an abstract concern where a system is deployed on own hardware, in a cloud computing context where resource utilisation equals money, it can be a more concrete concern.

Computing time costs money.

Infrastructure

The infrastructure used to set up and run the system will incur initial and ongoing costs — for on-premises systems these may include the cost of physical hardware such as servers, switches, and routers to the salaries of the network operators who will hopefully manage everything.

For cloud computing, while physical hardware costs will of course not be a factor of concern, you will pay a premium for the resources you utilise.

Solutions

While the cost of infrastructure is mostly outside your control, optimising its use is within your domain. Reducing the time spent marshalling data between Layers 7 and 4 may be significant, depending on the functionality provided by the system. For example, the cost of transport for systems providing audio and video streaming, telephony, or realtime collaboration features can be brought closer to zero by leveraging an optimised data format e.g. protocol buffers vs. opting for JSON or XML.

Fallacies of Distributed Computing: 6. There is one administrator

Thu, 25 Jun 2020 00:00:00 GMT

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

6. There is one administrator

The problem

If your system is self-contained and deployed into environments you fully control and manage there may indeed be only one administrator.

However, it is more likely that any non-trivial system will depend on one or more in-house or third-party services for its operation and be deployed into environments you do not completely control or that differ from each other in any number of ways e.g. operating system and/or application framework versions, security patching policies, UAC, shared resources, firewall rules, etc.

Multiple in-house teams may be involved to deploy and support the system, with each operating outside the processes governing that system’s development. Third-party services — their availability, compatibility, and development cadence — will typically exist outside your control.

Solutions

Infrastructure as Code

Where possible, automate the provisioning and configuration of the environments into which deployments are made. Infrastructure as Code (or IaC) is a core element of DevOps, with infrastructure modelled in code (or DSL-as-template form) — versus manually managed — and are governed by the same quality processes as application source code e.g. revisioned using source control, unit tested, etc.

Ideally, regard IaC as a core competency — that is, embed DevOps capabilities in your development teams.

Logging and monitoring

Diagnosing issues can be non-trivial at the best of times and particularly complex for distributed systems. To gain better visibility into the behaviour of a distributed system ensure that centralised logging, metrics, and tracing (the Three Pillars of Observability) have been considered as key aspects of the system’s design.

Decoupling

Ensuring appopriate decoupling between system components allows for greater resiliency in the event of either planned (system upgrades) or unplanned (service failure) downtime.

Introducing queuing (with appropriate retry policies, exponential backoff, and DLQs) is one method of achieving decoupling.

Queues

Wed, 03 Jun 2020 00:00:00 GMT

A Queue is a linear data structure (i.e. its elements form a sequence) and can be modified by the addition of elements at one end of the sequence and removal from the other end. A queue is a FIFO (First In, First Out) data structure.

The queue data structure works in exactly the same way as its real life analogue — the first element in the queue is automatically at the front, with subsequent elements forming in line behind their precedent.

Queues can be used to control access to shared system resources e.g. a printer queue managing access to a printer on a first come, first served basis. Queues can also feature in any architecture involving asynchronicity e.g. purchasing event tickets online.

Typically, a queue implementation will provide at least three operations — enqueue, dequeue, and peek.

Enqueuing and dequeuing can be visualised at follows:

Complexity

🔔 Complexity is considered in terms of worst case.

Time

Insertion	Removal	Retrieval
Θ(1)	Θ(1)	Θ(n)

Space

	Notes
Θ(n)	The elements in the queue

An implementation

Here we’re using an array as a backing store for simplicity (although you can of course use a different approach such as a doubly linked list which can provide enqueing and dequeing in Θ(1) time).

Define a Queue<T> class with the following methods:

dequeue - dequeues an element, removing it from the queue
enqueue - enqueues an element, adding it to the queue
isEmpty - determines whether the queue is empty
length - returns the number of elements in the queue
peek - returns the front element in the queue without dequeuing it

// TypeScript
export class Queue<T> {
	private elements: T[];

	public constructor() {
		this.elements = new Array<T>();
	}

	public dequeue(): T | undefined {
		if (this.elements.length > 0) {
			return this.elements.pop();
		}
		return undefined;
	}

	public enqueue(element: T): void {
		this.elements.unshift(element);
	}

	public isEmpty(): boolean {
		return this.elements.length < 1;
	}

	public length(): number {
		return this.elements.length;
	}

	public peek(): T | undefined {
		return !this.isEmpty() ?
			this.elements[this.elements.length - 1] :
			undefined;
	}
}

Code on Github

Stacks

Thu, 28 May 2020 00:00:00 GMT

A Stack is a linear data structure (i.e. its elements form a sequence) where operations are performed at the ‘top’ of the stack. The order may be FIFO (First In, First Out) or more commonly LIFO (Last In, First Out).

An analogue in real life is a stack of plates — with a FIFO approach, you take a plate from the bottom of the pile whereas with a LIFO approach you take a plate from the top.

The stack data structure has a multitude of uses, an obvious one is your browser history (a LIFO stack). Typically, a stack implementation will provide at least three operations — push, pop, and peek.

Pushing onto the stack can be visualised at follows:

Popping from the stack can be visualised at follows:

Complexity

🔔 Complexity is considered in terms of worst case.

Time

Insertion	Removal	Retrieval	Notes
Θ(1)	Θ(1)	Θ(1)	Retrieval Θ(1) if popping, otherwise Θ(n) to locate

Space

	Notes
Θ(n)	The elements in the stack

A LIFO implementation

Here we’re using an array as a backing store for simplicity (although you can of course use a different approach such as a linked list).

Define a Stack<T> class with the following methods:

isEmpty - determines whether the stack has elements
length - returns the number of elements on the stack
peek - returns the top element from the stack without popping it
pop - pops from the stack
push - pushes onto the stack

// TypeScript
export class Stack<T> {
	private elements: T[];

	public constructor() {
		this.elements = new Array<T>();
	}

	public isEmpty(): boolean {
		return this.elements.length < 1;
	}

	public length(): number {
		return this.elements.length;
	}

	public peek(): T {
		return this.elements[this.elements.length - 1];
	}

	public pop(): T {
		if (this.elements.length > 0) {
			return this.elements.pop();
		}
	}

	public push(item: T): void {
		this.elements.push(item);
	}
}

Code on Github

Linked Lists, revisited

Tue, 26 May 2020 00:00:00 GMT

In a previous post I discussed the linked list data structure, how linked lists differ from arrays, and provided a sample implementation of singly linked list.

Recall that a singly linked list allows nodes to be traversed in one direction:

What if we want to be able to traverse the list in either direction? Presented with that requirement we can use a doubly linked list.

Doubly linked lists are a variant of linked lists that enable node traversal in either direction. While they do use more memory than singly linked listed (two pointers per node vs. one), they allow for easier node insertion and removal.

A doubly linked list can be visualised at follows:

An implementation of a doubly linked list

As before, define a class to encapsulate a node. Here, our LinkedListNode can hold a value of type T:

export class LinkedListNode<T> {
	public next: LinkedListNode<T>;
	public previous: LinkedListNode<T>;

	public constructor(public readonly value: T) {}
}

export default LinkedListNode;

Note the addition of a previous property — this will be used to link a node to the previous node in the list (or be assigned null if the current node is the ‘head’).

The LinkedList<T> class specification remains the same:

add - adds a node to the list
addFirst - adds a node to the beginning of the list
addLast - adds a node to the end of the list
interator - interates the list (implemented here as a JavaScript iterator for extra interview points)
removeAt - removes the node at the specified index

// TypeScript
import { LinkedListNode } from "./linked-list-node";

export class DoublyLinkedList<T> {
	public head: LinkedListNode<T> = null;
	public tail: LinkedListNode<T> = null;

	public constructor() {
	}

	public add(value: T) {
		this.addLast(value);
	}

	public addFirst(value: T): void {
		const node = new LinkedListNode(value);

		const originalHeadNode = this.head;
		this.head = node;

		// Link the nodes.
		this.head.next = originalHeadNode;
		originalHeadNode.previous = this.head;

		if (!this.tail) {
			// There is one node in the list, set the tail node to the head.
			this.tail = this.head;
		}
	}

	public addLast(value: T): void {
		const node = new LinkedListNode(value);

		if (!this.head) {
			this.head = node;
		}
		if (this.tail) {
			// Link the nodes.
			this.tail.next = node;
			node.previous = this.tail;
		}
		// Set the new tail.
		this.tail = node;
	}

	public *iterator() {
		let node = this.head;
		while (node) {
			yield node.value;
			node = node.next;
		}
	}

	public removeAt(index: number): void {
		let node = this.head;
		let i = 0;

		while (node && i++ < index) {
			node = node.next;
		}

		// Bounds checking.
		if (index >= i) { return; }

		if (!node.previous) {
			this.head = node.next;
		} else {
			node.previous.next = node.next;
			node.next.previous = node.previous.next;
		}
	}
}

export default DoublyLinkedList;

Code on Github

Fallacies of Distributed Computing: 5. Topology doesn't change

Mon, 25 May 2020 00:00:00 GMT

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

5. Topology doesn’t change

The problem

Network topology is volatile:

Environments may have differing network requirements — from quickly spun up development environments through to production environments supporting high availability, resiliency, and redundancy
Firewall rules are likely to change in response to security concerns or application requirements
Services may be introduced (e.g. federated authentication) or deprecated, changing network topology

The topology of the network is typically outside your control. Servers may be added or retired, network services may be upgraded or deprecated, planned and unplanned outages may occur.

In cloud computing scenarios — where elasticity is typically a key motivation for adoption — network topology will mutate in response to load and utilisation.

Network topology changes constantly.

Solutions

Given that network toplogy is indeed volatile, there are two broad approaches that can be taken to buffer against topology changes.

Abstract network specifics

Avoid referencing IP addresses directly, instead using DNS hostnames to reference resources
Consider using a service discovery pattern for Microservice architectures

Design for failure

Design your architecture to avoid irreplaceability, assume any server may fail
Introduce chaos engineering and test for system behaviour under infrastructure, network, and application failures

Fallacies of Distributed Computing: 4. The network is secure

Tue, 19 May 2020 00:00:00 GMT

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

4. The network is secure

The problem

Your ecommerce website is accessible over HTTPS only, you rest easy knowing that all communication between your customers and your website is securely encrypted.

However, some of the scripts embedded in your website for gathering customer analytics are using HTTP. Also, that third party service you call privately from your own API to validate customer addresses works over HTTP because HTTPS was judged to be “too expensive” in a dim and distant past.

In April 2014 the Heartbleed vulnerability was made public. The SSL standard includes a heartbeat option, which allows one party communicating over an SSL connection to send a short message to the other party to check if they are available.

Heartbleed was a buffer over-read exploit on OpenSSL servers - by sending malformed hearbeat requests that weren’t validated by OpenSSL using bounds-checking, attackers were able to trick OpenSSL into returning random bits of memory from the server - even for failed connections. Because these bits of memory were often near the code authenticating users, it was possible to obtain authenticated user data just by pinging the server.

A cursory browse of the have i been pwned? website reveals millions of user details that have been exposed by attackers exploiting vulnerabilities like Heartbleed, poorly configured application stacks, unsecured databases, and so on.

The network is not secure.

Solutions

Security should be built into the DNA of a system, not bolted onto it. As a quality attribute, it should be considered at the network, infrastructure, and application layers.

Data produced and consumed by the system should be identified and classified according to its nature (e.g. PII) and associated risk of exposure.

The Open Web Application Security Project (OWASP) maintain a top ten list of web application security risks. These can be considered as part of a broader threat modeling practice and (in some cases) evaluated on an automated basis.

Ultimately, the only truly secure system is one which cannot be observed or interacted with in any way by human or other means. As such, ensuring the security of a system based on identified risks and their probability while still meeting business goals in a timely fashion is frequently a non-trivial exercise.

Linked Lists

Tue, 12 May 2020 00:00:00 GMT

Linked lists. The interview topic every software developer will be quizzed on at some point in their career.

Historically, this was perhaps because linked lists conveyed knowledge of pointers and memory allocation and usage in languages like C on the part of the interviewee. Available memory was also at a premium on the constrained systems of the past and efficient use of resources was not only desirable, but frequently critical.

While many of us work with higher-level languages such as Java and C# (managed runtimes) or JavaScript (intepreted), knowledge of data structures for efficiently storing and retrieving data in memory is still entirely useful to possess.

What about arrays?

Consider the main characteristics of arrays:

Allocated a contiguous block of memory
Cannot be resized^*
Accessing an array element is Θ(1)
Insertion or removal of array elements is Θ(n)

Strings are perhaps the most common example of fixed size arrays in use (this is why strings are considered immutable).

* While many languages provide dynamic arrays, these are constructs that wrap arrays and allow resizing by allocating a new contiguous block of memory behind the scenes and copying the original array contents across.

Linked lists

Linked lists differ from arrays in that they can utilise non-contiguous blocks of memory, with each data element (or ‘node’) pointing to the next in the list. Characteristics include:

Memory allocation can be non-contiguous
Can be easily resized, there is no copy penalty
Accessing a node is Θ(n)
Insertion or removal of nodes is Θ(1)

While accessing a node is Θ(n) as the list must be traversed, insertion and removal are Θ(1). Given it is frequently impossible to know how the ‘correct’ amount of memory to reserve for data in advance, linked lists offer greater efficiency in terms of memory (re)allocation.

🔔 Remember, use the data structure that is more appropriate for your circumstance.

An implementation of a singly linked list

A singly linked list allows traversal of nodes in one direction, and can be visualised as follows:

First, define a class to encapsulate a node. Here, our LinkedListNode can hold a value of type T:

// TypeScript
export class LinkedListNode<T> {
	public next: LinkedListNode<T>;

	public constructor(public readonly value: T) {}
}

export default LinkedListNode;

The next property is the mechanism we can use to link one node to another.

Next, define a LinkedList<T> class with methods for manipulating and traversing the list:

add - adds a node to the list
addFirst - adds a node to the beginning of the list
addLast - adds a node to the end of the list
interator - interates the list (implemented here as a JavaScript iterator for extra interview points)
removeAt - removes the node at the specified index

// TypeScript
import { LinkedListNode } from "./linked-list-node";

export class SinglyLinkedList<T> {
	public head: LinkedListNode<T> = null;
	public tail: LinkedListNode<T> = null;

	public constructor() {
	}

	public add(value: T) {
		this.addLast(value);
	}

	public addFirst(value: T): void {
		const node = new LinkedListNode(value);

		const originalHeadNode = this.head;
		this.head = node;

		// Insert the rest of the list behind the head.
		this.head.next = originalHeadNode;

		if (!this.tail) {
			// There is one node in the list, set the tail to the head.
			this.tail = this.head;
		}
	}

	public addLast(value: T): void {
		const node = new LinkedListNode(value);

		if (!this.head) {
			this.head = node;
		}
		if (this.tail) {
			this.tail.next = node;
		}
		// Set the new tail.
		this.tail = node;
	}

	public *iterator() {
		let node = this.head;
		while (node) {
			yield node.value;
			node = node.next;
		}
	}

	public removeAt(index: number): void {
		let previous: LinkedListNode<T> = null;
		let node = this.head;
		let i = 0;

		while (node && i++ < index) {
			previous = node;
			node = node.next;
		}

		// Bounds checking.
		if (index >= i) { return; }

		if (!previous) {
			this.head = node.next;
		} else {
			previous.next = node.next;
		}
	}
}

export default SinglyLinkedList;

Code on Github

Fallacies of Distributed Computing: 3. Bandwidth is infinite

Tue, 05 May 2020 00:00:00 GMT

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

3. Bandwidth is infinite

The problem

Bandwidth is the capacity of a network to transmit data over a period of time. While bandwidth has increased significantly over the past few decades, it is not infinite.

Users may not be fortunate enough to enjoy reliable, high-bandwidth network connections or may experience highly-variable connectivity at times e.g. when using mobile networks. Further, as bandwidth-in-general has increased, so has the requirement to support large volumes of data such as audio and video streams.

Solutions

Although latency is ultimately a physics problem, bandwidth issues can be ameliorated by throwing more capacity at the problem. However, this can be a financially expensive approach with an upper-bound and may not be applicable to every usage scenario.

In terms of system design, architectural approaches such as CQRS coupled with a clear understanding of user journeys can assist when optimising data payloads and in establishing the resultant bandwidth requirements.

Recall that one solution to the Latency is zero fallacy is to transmit more data upfront, thus reducing the number of network requests required. This introduces a potential tension with solutions for fallacy 3 given that bandwidth reductions will require less data be sent over a network. Finding the correct balance between the two requires an appropriate level of demand modelling and knowledge of how the system will be used.

Fallacies of Distributed Computing: 2. Latency is zero

Wed, 22 Apr 2020 00:00:00 GMT

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

2. Latency is zero

The problem

Latency is defined as the time interval between a stimulus and the corresponding response. In network terms, it’s the amount of time required to move data from one location to another. Network traffic is not instant.

At one time remote objects were seen as a desirable design objective, encapsulating local or remote implementation details and allowing for so-called network transparency. For example:

// C#
var user = new User(1);
user.Password = "Password1";

The above operation may simply set the password on a User instance in memory or it may also initiate one or more remote calls to update a user password in a database, the implications are opaque.

Thankfully, the use of remote objects is, by and large, frowned upon these days in favour of other techniques such as DTOs which provide greater transparency. (It is always worth formalising the segregation of local and remote calls through effective application of appropriate design patterns.)

One technique frequently used with ORM technologies is deferred, or lazy, loading. With deferred loading, data is retrieved only when needed rather than prefetched.

A typical example might entail retrieving the orders and order line items associated with a customer account:

// C#
var svc = new OrderService();
var orders = svc.GetOrdersForCustomer(1); // Returns an IQueryable<Order>, deferred

foreach (var order in orders.ToList()) { // Enumerated, call
	var items = order.GetItems(); // Returns an IQueryable<OrderLineItem>, deferred
	// ...
}

Here, rather than fetching all the customer orders and associated line items in one call, the orders will be fetched when enumerated with the corresponding order line items also fetched on first use. This is a classic example of an unintended SELECT N + 1 query, resulting in multiple remote calls to the database or service providing the data.

Solutions

Reduce the number of remote calls

Given each remote call made will be several orders of magnitude slower than a local equivalent, you should design with the objective of optimising the number of remote calls. One way to achieve this by designing payloads to include all the data you might need to support a particular use case, removing the likelihood (or need) of additional remote calls to fetch data at more granular levels.

Locate the data closer to the client

As latency is ultimately a physics problem, locating data closer to the geographic location of the client will facilitate reduced latency (all other network factors being equal). All public cloud infrastructure providers such as AWS, Azure, or GCP support deployment into multiple availability zones.

Caching

Caching should also be implemented wherever possible to reduce unnecessary remote calls.

Fallacies of Distributed Computing: 1. The network is reliable

Mon, 20 Apr 2020 00:00:00 GMT

The fallacies of distributed computing are a set of assertions describing false assumptions made about distributed systems.

L. Peter Deutsch drafted the first 7 fallacies in 1994, with the 8^th added by James Gosling in 1997.

The 8 fallacies are:

1. The network is reliable

The problem

Many systems are interconnected in nature, that is to say in normal execution they may make calls to other systems over one or more networks to satisfy functionality requirements.

In any tiered architecture the underlying process and mechanism of communication will be (hopefully) abstracted away - for example taking a payment on behalf of a customer may be as straightforward as calling a method like so:

var svc = new PaymentService();
svc.TakePayment(paymentRequest);

This level of abstraction is both useful and desirable, however it does tend to pre-dispose our view towards success. What happens when an underlying network call to a third-party payments processor fails with e.g an HTTP timeout exception?

Depending on when and how such an error occurs, the request may be retryable without adverse side effect(s). However, for any retry to be deterministic in the affirmative the system must be indempotent. Without indempotence, retrying the above request N times may result in the customer being charged N times.

A network may be unreliable for any number of reasons:

A software issue
A hardware issue
A security issue
An environmental issue
A bad actor e.g. a Denial of Service attack
A combination of any or all of the above

A system architected for resiliance and correctness must consider the above factors in its design.

A solution

To mitigate network unreliability, a Retry and Acknowledgement design can be leveraged using a queuing system.

Queuing systems typically implement a pattern known as Store and Forward - they store the message locally and then forward it to the recipient(s). If forwarding fails, the queuing system will retry automatically. If failure persists past a specified number of attempts, the message will be moved to a Dead Letter Queue. The message is considered undelivered until the queue receives an acknowledgement of receipt from the recipient.

Implementing queuing requires moving from a synchronous request/response centric model to an ansychronous one. This is - typically - not a trivial excercise in a system of any complexity and may require re-evaluation of other aspects of your system, not least user experience.

Union Types in TypeScript

Tue, 17 Mar 2020 00:00:00 GMT

Consider the following contrived lunchbox example:

class Apple {
	eat() {
		console.log("Eating an apple.");
	}
}
class Sandwich {
	eat() {
		console.log("Eating a sandwich.");
	}
}
class ChocolateBar {
	eat() {
		console.log("Eating a chocolate bar.");
	}
}

const lunchbox: any[] = [];

const addToLunchbox = (item: any): void => {
	lunchbox.push(item);
}

const getFromLunchbox = (): any => {
	return lunchbox.pop(); // Treat the lunchbox as a stack for simplicity
}

In the example above, addToLunchbox() accepts an item: any, allowing a caller greater flexibility when adding items to the lunchbox:

addToLunchbox(new Apple());
getFromLunchbox().eat(); // "Eating an apple."

addToLunchbox(new Sandwich());
getFromLunchbox().eat(); // "Eating a sandwich."

addToLunchbox(new ChocolateBar());
getFromLunchbox().eat(); // "Eating a chocolate bar."

However, the loose typing also allows for unintended use:

class Brick {}

addToLunchbox(new Brick());
getFromLunchbox().eat(); // Error!

You can of course tighten up contracts by introducing a base class, say, Food:

abstract class Food {
	abstract eat(): void
}

class Apple extends Food { ...}

const addToLunchbox = (food: Food): void => { ... }
const getFromLunchbox = (): Food => { ... }
}

As long as Brick doesn’t extend Food the problem appears to be solved. However, there will be cases where the strategy breaks down e.g. we want to allow liquid to be added:

class Water extends Food { // Really?
	drink() {
		console.log("Drinking water.");
	}
}

While it’s obviously incorrect for Water to extend Food, the temptation is real in order to quickly dig yourself out of a hole vs. having to modify the existing function contracts. Alternatively, you may consider replacing Food with a more generic base class providing both eat() and drink() methods.

Both solutions are poor in terms of program design, understandability, testability, and numerous other quality metrics.

A better way to solve this problem is to use union types. A union type describes a value that can be one of several types.

Let’s modify both addToLunchbox() and getFromLunchbox() to accept either Food or Water:

const addToLunchbox = (item: Food | Water): void => {
	lunchbox.push(item);
}

const getFromLunchbox(): Food | Water {
	return lunchbox.pop();
}

addToLunchbox(new ChocolateBar());
addToLunchbox(new Water());
addToLunchbox(new Brick()); // Error!

With union types you can specify two or more types that can be accepted as arguments to, or returned from, a method. Note that it is still the responsibility of the caller to interpret the returned value correctly.

Errors

You may be considering including error types when specifying return types e.g.

class EmptyLunchboxError extends Error {};

const getFromLunchbox = (): Food | Water | EmptyLunchboxError => { ... }

You should avoid doing this, particularly when throwing an error - throwing represents an exceptional state in your program execution outside of normal program flow.

Enums in TypeScript

Fri, 28 Feb 2020 00:00:00 GMT

Enumerations are user-defined types consisting of named values (members) that behave as constants. They’re useful when you want to define a type consisting of a restricted set of values, for example the colours of the rainbow.

Numeric enums

By default, numeric enums start at zero and increment by 1 for each subsequent member:

enum Volume {
	Low,
	Medium,
	High
}

console.log(Volume.Low);    // 0
console.log(Volume.Medium); // 1
console.log(Volume.High);   // 2

You can change this if you wish and either start with a non-zero value or explictly assign values to members:

enum StoryPoints {
	Zero = 0,
	One = 1,
	Two = 2,
	Three = 3,
	Five = 5,
	Eight = 8,
	Thirteen = 13
}

console.log(StoryPoints.Eight); // 8

Numeric enums can be reverse mapped, in other words you can use the numeric value to retrieve the associated member:

console.log(StoryPoints[3]); // Three

String enums

TypeScript version 2.4 introduced support for string initialisers in enums. This can be useful for defining a restricted set of string values:

enum MimeTypes {
	CSV = "text/csv",
	HTML = "text/html",
	JSON = "application/json",
	XML = "application/xml"
}

console.log(MimeTypes.JSON); // application/json

As the release notes state, you cannot reverse map a string-based enum:

console.log(MimeTypes["application/xml"]); // Error!

Bitwise enums

Bitwise enums (also known as Flag enums) are useful when you want flexbility in combining enum members to represent new values. One such use might be where you want to elevate or remove file access permissions:

enum FileAccess {
	None = 0,
	Read = 1 << 0,  // Can also be expressed as Read = 1
	Write = 1 << 1, // Can also be expressed as Write = 2
	Delete = 1 << 2 // Can also be expressed as Delete = 4
}

While the FileAccess enum could be defined with explicit members for every combination, as that number increases the approach quickly becomes impractical.

As you may have noticed from the above example, for bitwise enums use values that are powers of 2 (2^{^}0, 2^{^}1, 2^{^}2, 2^{^}3, and so on).

And in action:

let access = FileAccess.None;
console.log(access); // 0

// Elevate to Read access
access = FileAccess.Read;
console.log(access); // 1

// Elevate to Read and Write access
access |= FileAccess.Write;
console.log(access); // 3

// Elevate to Read, Write, and Delete access
access |= FileAccess.Delete;
console.log(access); // 7

// Remove Delete access
access &= ~FileAccess.Delete;
console.log(access); // 3

// Evaluating incorrectly
console.log(access === FileAccess.Read); // false

// Evaluating correctly
console.log(FileAccess.Read === (access & FileAccess.Read)); // true

Note the |= and &= ~ operators. The |= operator allows to perserve any existing assigned value(s) - using = alone will overwrite. The &= ~ operator allows you to remove one or more assigned values.

`const` enums

If you wish to avoid the overhead of the generated code and additional indirection, you can use a const enum:

const enum Rainbow { // Note the const modifier
	...
}

With a const enum, the values are inlined with each use. Note that only constant enum expressions can be used, you cannot use computed values as in this contrived example:

const enum StoryPoints {
	Zero = 0,
	One = 1,
	Two = 2,
	Three = One | Two,
	Five = ".....".length() // Error!
	Eight = 8,
	Thirteen = 13
}

Tales from the computing frontier

Sun, 23 Feb 2020 00:00:00 GMT

I’ve always enjoyed reading about computing history, particularly tales of computing pioneers and I continue to regard Englebart, Kay, Shannon, Wozniak and others with awe. The books below are some of my favourites and ones I eagerly recommend to friends and colleagues.

(Links go to kennys.ie but are referral-free.)

The Soul of a New Machine

A Pulitzer prize winner, The Soul of a New Machine tells the story of an engineering team at Data General and their efforts to design a next-generation minicomputer (the Eclipse MV/8000) against almost insurmountable odds. Led by the enigmatic Tom West, the team run a skunkworks project (codenamed ‘Eagle’) as a Plan B to keep pace with Digital Equipment Corporation’s offerings should the favoured Data General project (‘Fountainhead’) fail.

Soul of a New Machine is a thrilling read, expertly told. It conjures a perhaps-romanticised tale of an engineering team pouring their very being into a project, willing it to completion in a cauldron of invention and creation.

I was a young, inexperienced software engineer when I first read Soul of a New Machine and it had an enormous effect on me. It galvanised me to seek out others who also viewed software engineering as craft and to work on projects that amounted to more than their constituent bytes. Revisiting it now with wiser eyes, I can of course see the negative aspects of projects like Eagle more clearly.

Hackers: Heroes of the Computer Revolution

Told in three parts, Hackers chronicles hackerdom from the 1950s through to the early 1980s.

As a child of 1980s personal computing and gaming the third part of the book, Game Hackers, resonates most for me (although all parts are excellent).

Ken and Roberta Williams found Sierra On-Line and enjoy meteoric rise developing and publishing games for the Apple II and Atari 8-bits, ultimately winning a legal battle with Atari over Pac-Man that leaves Williams with an uncertain victory. Game developer John Harris (who considers his portrayal inaccurate) features prominently as the archetypal game hacker, weaving magic on his beloved Atari 800.

If hardware engineers like Steve Wozniak or early game programmers like Harris approach rock star status in your eyes you’ll enjoy Hackers greatly.

The Idea Factory: Bell Labs and the Great Age of American Innovation

The legacy of Bell Labs is staggering. The birthplace of research and technologies that shaped the 20^th century and continue to shape the 21^st - transatlantic communication, the transistor, the laser, the communications satellite, UNIX, the C programming language… its importance cannot be overstated.

A comprehensive account of Bell Labs from its foundation through to its decline in the 1980s following the breakup of AT&T, The Idea Factory - while necessarily brief in places - superbly documents some of the extraordinary individuals who worked there and the contributions they made:

Claude Shannon, father of Information Theory
John R. Pierce, co-inventor of pulse code modulation and considered one of the fathers of communication satellites
William Shockley, co-inventor of the transistor (whose legacy has been tarnished by his repellent views on Eugenics)

Shannon, Pierce, and Shockley were giants of their field, and The Idea Factory explains why. That they worked collectively at the same research laboratory is mind blowing.

Docker spring cleaning

Fri, 21 Feb 2020 00:00:00 GMT

When using Docker you may find over time that disk space is being consumed rapidly by ophan images and/or volumes.

These commands were tested with Docker running on Ubuntu, your mileage with Docker on Windows or macOS may vary.

To find out how much disk space Docker is consuming type the following command in your terminal:

docker system -df

This will give you an overview of what resources are being used.

There are a number of ways to clean up unwanted Docker artifacts and reclaim disk space using your terminal, a few of which are outlined below.

Deleting exited containers

docker rm -v $(docker ps -a -q -f status=exited)

Removing unwanted ‘dangling’ images

docker rmi $(docker images -f "dangling=true" -q)

Deleting unwanted volumes

docker volume rm $(docker volume ls -qf dangling=true)

Pruning

With Docker v1.25 or later you can accomplish the above with the prune command:

docker system prune

To remove all unused images (vs. just ‘dangling’ ones), include the -a switch:

docker system prune -a

Automating spring cleaning

docker-gc-cron is an excellent Docker container provided by Spotify that can be used to automatically clean up unused containers and images.

You may want to consider modifying the default docker-compose.yml file provided with configuration values more appropriate to your needs e.g.

# Set the grace period
- GRACE_PERIOD_SECONDS=43200 # 12 hours

# Commenting out will reset the log output to UTC
# - TZ=America/New_York

Partial Types in TypeScript

Wed, 05 Feb 2020 00:00:00 GMT

The Partial<T> utility type (added in TypeScript 2.1) allows you to take an existing type and make all of its properties optional. This can help in situations where you want to have some flexibility without removing strictness across the board.

Consider the following interface:

interface ChocolateBar {
	readonly manufacturer: string;
	readonly name: string;
	readonly price: Money;
}

As specified, all properties of ChocolateBar are required for the interface contract to be satisfied. (Incidentally, don’t use the Number type for monetary values - introduce a type.)

Inevitably, there will occasions where strictness like this can make life awkward. What if you want to be able to selectively update the values of a chocolate bar? Well, you might write something along the lines of:

interface KeyValuePair {
	readonly key: string;
	readonly value: any;
}

const updateChocolateBar(bar: ChocolateBar, props: KeyValuePair[]): ChocolateBar {
	props.forEach(({ key, value }) => bar[key] = value);
	return bar;
}

While this may be sufficient, each KeyValuePair is ambiguous e.g. { "paws": 4 } is perfectly valid. Ambiguous code is more difficult to reason about, test, and maintain.

A better approach is to use a Partial type:

interface KeyValuePair {
	readonly key: string;
	readonly value: any;
}

const updateChocolateBar(bar: ChocolateBar, props: Partial<ChocolateBar>): ChocolateBar {
	return { ...bar, ...props };
}

Here, Partial<ChocolateBar> creates a type with all properties of ChocolateBar set to optional, effectively producing:

interface ChocolateBar {
	readonly manufacturer?: string;
	readonly name?: string;
	readonly price?: Money;
}

Why is this approach superior?

The ChocolateBar contract is only altered for this specific case
updateChocolateBar() communicates its own contract more effectively, is less ambiguous in use, and straightforward to test

Maintenance pages in NGINX without reload

Sun, 02 Feb 2020 00:00:00 GMT

Configuring a maintenance page in NGINX is quite straightforward and, if configured correctly, allows for live toggling on and off without requiring the NGINX instance to be reloaded or restarted.

location / {
	# Check for the existence of an enabled maintenance page
	if (-f {PATH}/{TO}/maintenance-page/index-on.html) {
		return 503;
	}

	# Include the rest of this location's configuration as normal
}

location /maintenance-page {
  root {PATH}/{TO}/{MAINTENANCE_PAGE_PARENT_FOLDER};
}

# Serve out the maintenance page for 503, 504 errors.
error_page 503 504 /maintenance-page/index-on.html;

For each inbound request matched by this location, NGINX will check for the existence of an index-on.html page - if found, it will be served with an HTTP 503 status code.

To toggle the maintenance page on or off, simply rename the maintenance page:

cd {PATH}/{TO}/maintenance-page
mv index-off.html index-on.html

cd {PATH}/{TO}/maintenance-page
mv index-on.html index-off.html

Null coalescing in TypeScript

Fri, 31 Jan 2020 00:00:00 GMT

The null coalescing operator has been a long requested feature in TypeScript. Indeed, when I began using TypeScript a few years ago, I assumed it was available and was suprised to find this wasn’t the case.

Null coalescing remained unsupported until recently because the TypeScript team (rightly) wanted to mantain compatibility with any eventual TC39 decisions regarding implementation and semantics.

Given the ECMAScript proposal is now at stage 4, the TypeScript team have responded by adding support for the ?? operator in TypeScript version 3.7.

The null coalescing operator is useful when you want to use a ‘default’ value in an expression. In TypeScript and JavaScript you will find the following pattern regularly used:

const a = b || c;

While this appears to achieve the same end (and in many cases, will) the logical OR operator (||) applies for any falsy value i.e. null, undefined, "", 0, NaN, false:

const b = 0;
const c = 42;
const a = b || c;

console.log(a); // 42

The null coalescing operator (??) is more strict; it only applies where the value is null or undefined:

const b = 0;
const c = 42;
const a = b ?? c;

console.log(a); // 0

This additional strictness can be useful in scenarios where subtle errors may otherwise be introduced due to unintended falsy evaluation i.e. where 0, NaN, or "" are valid values in your application data or state.

Renaming properties in a destructuring assignment

Thu, 30 Jan 2020 00:00:00 GMT

Destructuring is one of the more useful language features added in ECMAScript 6 (ES6). Destructuring allows you to unpack values from arrays, or properties from objects, into distinct variables.

Consider the following object:

const user = {
	emailAddress: "billg@microsoft.com",
	nickName: "Bill G"
}

Pre-ES6 you would typically access the object property values directly or assign e.g.

console.log(`${ user.nickName } (${ user.emailAddress })`); // "Bill G (billg@microsoft.com)"

const emailAddress = user.emailAddress
const nickName = user.nickName;
console.log(`${ nickName } (${ emailAddress })`); // "Bill G (billg@microsoft.com)"

ES6 introduced destructuring, enabling you to unpack into new variables in one assignment expression:

const { emailAddress, nickName } = user;
console.log(`${ nickName } (${ emailAddress })`); // "Bill G (billg@microsoft.com)"

One awkward scenario is where you want to destructure an object but the resultant variable(s) already exist in scope. Here, the unpacked variables collide with the function arguments:

const getDetails = (emailAddress) => {
	return user; // Fetch existing details from a database, etc.
}

const updateDetails = (emailAddress, nickName) => {
	const { emailAddress, nickName } = getDetails(emailAddress);

	console.log(`${ nickName } (${ emailAddress })`);
}

updateDetails("steveb@microsoft.com", "Steve B"); // Uncaught SyntaxError: Identifier 'emailAddress' has already been declared

Fortunately, this scenario was anticipated and you can rename variables in a destructuring assignment:

const updateDetails = (emailAddress, nickName) => {
	const { emailAddress: currentEmailAddress, nickName: currentNickName } = getDetails(emailAddress); // Renaming

	console.log(`${ nickName || currentNickName } (${ emailAddress || currentEmailAddress })`);
}

updateDetails(); // "Bill G (billg@microsoft.com)"
updateDetails("steveb@microsoft.com", "Steve B"); // "Steve B (steveb@microsoft.com)"

Securing endpoints in NGINX with Basic Authentication

Sun, 26 Jan 2020 00:00:00 GMT

Let’s begin by stating that Basic Authentication is likely not the authentication method you’re looking for. It has a number of shortcomings, not least of which is it transmits the password.

Granted, it base64 encodes the username and password first but this is reversible encoding.

Caveats aside, it does still have its uses - perhaps you want to restrict casual access to files or web applications available on your internal network only without the complexities of OAuth, Active Directory integration, or similar.

To secure endpoints in NGINX using Basic Authentication, follow these steps:

On the NGINX host, type the following in your terminal:

# Substitute {USERNAME} for the one you want to use e.g. 'admin'
printf "{USERNAME}:`openssl passwd -apr1`\n" >> .htpasswd

Enter an appropriate strength password when prompted. For simplicity, consider creating this file in an appropriate location e.g. /etc/nginx/sites-available.

Update the NGINX configuration(s) for each endpoint you wish to secure:

location / {
	# Display a title on the Basic Authentication browser challenge dialog (if supported)
	auth_basic "Administration Area"
	# The absolute path to the .htpasswd file
	auth_basic_user_file /etc/nginx/sites-available/.htpasswd;
}

Ensure the .htpasswd file is readable by NGINX workers:

chmod 644 .htpasswd

Verify the NGINX configuration and reload/restart:

sudo nginx -t -c /etc/nginx/nginx.conf
sudo service nginx reload

dereklawless.ie RSS feed

Towards Effective Standups

Standup format

Guidelines

Downloading Lambda layer versions using the AWS CLI

Downloading a Lambda layer version by name

Downloading a Lambda layer version by ARN

Conditionally tagging resources in CloudFormation

Use case: tagging a release version

The unknown Type in TypeScript

The any type

The unknown type

Narrowing and asserting

Unions and intersections

When to use unknown

Nemawashi

Nemawashi and software development

Getting buy-in

Building consensus

The goal

Year in review

2020

Career

Website

Design thinking

Certifications

2021

Binary Search

Complexity

Time

Space

An implementation

A recursive alternative

Sieve of Eratosthenes

An implementation

Fallacies of Distributed Computing: 8. The network is homogeneous

8. The network is homogeneous

The problem

Solutions

Fallacies of Distributed Computing: 7. Transport cost is zero

7. Transport cost is zero

The problem

Marshalling

Infrastructure

Solutions

Fallacies of Distributed Computing: 6. There is one administrator

6. There is one administrator

The problem

Solutions

Infrastructure as Code

Logging and monitoring

Decoupling

Queues

Complexity

Time

Space

An implementation

Stacks

Complexity

Time

Space

A LIFO implementation

Linked Lists, revisited

An implementation of a doubly linked list

Fallacies of Distributed Computing: 5. Topology doesn't change

5. Topology doesn’t change

The problem

Solutions

Abstract network specifics

Design for failure

Fallacies of Distributed Computing: 4. The network is secure

4. The network is secure

The problem

Solutions

Linked Lists

What about arrays?

Linked lists

An implementation of a singly linked list

Fallacies of Distributed Computing: 3. Bandwidth is infinite

3. Bandwidth is infinite

The `any` type

The `unknown` type

When to use `unknown`

`const` enums