How Software Is Built

01The shape almost everything has

Systems differ enormously in detail and barely at all in outline. Something runs near the user and cannot be trusted. Something runs on machines you control and makes the decisions. Something remembers. And there is a boundary between the first and the rest which is the single most important line on the map.

That is why the campus behind this page is drawn with a moat and exactly one bridge. Everything to the left runs on hardware you did not choose, on a network you cannot predict, in an environment a determined person can modify. Everything to the right is yours. Nothing that arrives from the left may be believed without being checked again on the right.

Most of the buildings on the map exist for one of three reasons: to make something faster (the edge, the cache), to make something safer (the gateway, identity, the vault), or to stop something slow from blocking something quick (the queue, the workers). A system with none of them is not wrong — it is just one where none of those pressures has arrived yet.

A useful discipline when reading any architecture diagram: for every box, ask what would break if you deleted it. If the answer takes more than a sentence, that box is doing more than one job.

02The client: the only part anyone sees

The frontend is the product, as far as every user is concerned. It is also the part of your system you have the least control over: it runs on a phone from 2019 on a train, in a browser with four extensions injecting scripts, on a connection that will disappear halfway through a form.

Modern frontend work is mostly the management of three things — the state the user can see, the state the server has, and the gap between them. Nearly every hard bug in a client is somewhere in that gap: a stale list, a double-submitted form, a spinner that never resolves because nobody wrote the failure branch.

Two properties matter more than the framework argument. How long until something appears, which is mostly a question of how many bytes have to arrive before anything can be drawn. And whether the thing that appears can be used — a page that renders in 400 ms and then blocks the main thread for two seconds is a page that feels broken.

Accessibility belongs in this section rather than in a box at the end. Semantic elements, focus order that follows the visual order, contrast you can read in sunlight, and everything reachable from a keyboard. It is the difference between a product some people cannot use and one they can — and in a lot of jurisdictions it is also the law.

03The edge: names, distance and the copy nearby

Before a request can be made at all, a name has to become an address. DNS does that, and it is cached at every level between the browser and the root — which makes it very fast and makes changes to it slow to take effect. The number attached to a DNS record, its time to live, is how long you have promised the internet it may keep believing an old answer.

Then there is distance. Light in fibre crosses the Atlantic in about sixty milliseconds each way, and no amount of clever code will beat physics. The fix is to stop making the round trip: put copies of anything that is the same for everybody in a few hundred locations, and answer from whichever is nearest. That is a content delivery network, and it is the highest-leverage thing on this entire map.

The interesting design question is not "should we use a CDN" but what fraction of your traffic can be made cacheable. Splitting a page into a shell that is identical for everyone and a small personalised fetch is often the difference between eighty-six per cent of requests never reaching your servers and none of them being cacheable at all.

04The gateway: one door, and it is watched

Everything that reaches your own machines arrives through one place. That is a deliberate choice, not an accident of topology: a single entrance can be instrumented, rate limited, patched and reasoned about, and twelve entrances cannot.

• TLS. Encryption ends here, so certificates live in one place and get renewed automatically. A certificate that expires on a Sunday is a rite of passage nobody needs twice.
• Load balancing. Requests are spread across whichever instances are currently healthy, which is also how a deployment happens without anyone noticing.
• Health checks. An instance that stops answering is taken out of rotation. This is the mechanism that turns "a server died" into a non-event.
• Rate limiting. A per-client budget. It is the cheapest security control that exists and it removes most of what the internet throws at you.
• Request filtering. Obviously hostile shapes — injection patterns, absurd payload sizes, known-bad clients — refused before any of your code runs.

A word about timeouts, because they belong here and almost nobody sets them until after their first incident. Every hop on this map should have a deadline. A request with no deadline does not fail; it waits, holding a connection, until the thing waiting on it also gives up. That is how one slow database becomes a total outage.

05Identity: two questions that are not the same

Authentication asks who is making this request. Authorisation asks whether they may have what they are asking for. They are answered by different code, they fail in different ways, and treating them as one thing is the most common serious security bug in applications that are otherwise well built.

Authentication is largely a solved problem you should not solve again. Sessions with a server-side store, or signed tokens with a short life, or — best of all for most products — delegating the whole thing to an identity provider so that you never store a password at all. Password storage in particular has exactly one correct answer, and it is a purpose-built hashing function that you did not write.

Authorisation is where the bugs live, because it is specific to your product and cannot be bought. The rule that saves you is simple and tedious: check ownership on every path, at the point of use. Not in the interface that hid the button. Not once, in the controller everyone remembers, while three other routes reach the same record. The classic failure is an endpoint that carefully verifies you are signed in and then hands you object number 9107 because you asked for it.

06The backend: where your product means something

This is the only building that knows what a customer, an order or a subscription is in your particular business. Everything else on the map is generic machinery you could swap for a competitor's. The services are not.

A request handler does roughly four things, in this order: work out who is asking, validate what they sent, apply the rules, and persist or fetch whatever that implies. Most codebases go wrong by letting those four blur into each other until the business rules are spread between a route handler, a database trigger and a piece of the frontend.

The perennial argument is one service or many. Splitting a system into services buys independent deployment and independent scaling, and it costs you a network between things that used to be a function call — which means latency, partial failure, versioned contracts and a distributed transaction problem you did not previously have.

Whatever the shape, the interface between the client and the backend is a contract that outlives both. Version it, document it, and remember that old versions of a mobile app will be calling it long after you have forgotten the endpoint exists.

07Data: the part you cannot rebuild

Every other building on this map can be reconstructed from a repository in an afternoon. The database cannot be reconstructed from anything except a backup you have actually restored. That asymmetry should govern how much care each part gets.

For most products the correct default is a single relational database, and it will remain correct for far longer than the internet suggests. Relational databases give you transactions, constraints that make certain wrong states impossible, and a query language that answers questions you had not thought of when you designed the schema. Those are not legacy features; they are the reason the category has outlived four waves of replacements.

A transaction is the promise that a group of changes either all happen or none do. It is what stops money leaving one account without arriving in another, and it is worth understanding properly rather than trusting by default, because the isolation level your database ships with probably permits more strangeness than you expect.

Backups deserve one blunt sentence. A backup you have never restored is not a backup; it is a file with a hopeful name. Restore one on a schedule, to a real environment, and time how long it takes — because that number is your actual worst case, and you will only ever discover it under pressure otherwise.

08Caching: the cheapest speed, at a price

A cache is a bet that the answer you worked out a moment ago is still good enough to hand to the next person who asks. It is the most effective performance tool available — a hit costs a fraction of a millisecond where the real work cost twenty — and it introduces an entire category of bug where the system is confidently, quickly wrong.

There are more caches in a running system than most people count. The browser holds one. The CDN holds one. Your application holds one in memory, and probably a shared one as well. The database holds a page cache. The same value can exist in five places, each with its own idea of how long it is valid.

Two failure modes are worth knowing by name. A stampede is what happens when a popular entry expires and a thousand requests all miss at once and all recompute the same thing — which usually takes down the very database the cache was protecting. And a poisoned cache is what you get when something user-specific is stored under a key that is not, so one person's data is served to everybody. The second one is a data breach, and it is more common than it should be.

The discipline that avoids most of this: cache derived things, not authoritative things, and be able to say out loud how stale any given answer is allowed to be. If you cannot answer that for a cache, you do not yet know what it is for.

09Queues: getting the slow part out of the way

Some work cannot happen while a user waits. Generating a report takes ninety seconds; sending an email depends on a third party having a good day; resizing a video is measured in minutes. Doing any of that inside a request means holding a connection open and losing everything if the browser closes.

So you write down the intent, put it on a queue, and answer immediately. A separate pool of workers pulls jobs off, does the slow thing, and records the result. The request that took ninety seconds now takes eleven milliseconds, and the ninety seconds happen somewhere nobody is watching a spinner.

What you have bought is responsiveness and the ability to survive a dependency being down. What you have paid is that "did it work?" is now a question with a delayed answer, and your users need to be told the ending somehow — a notification, an email, or a row that changes state when the page next asks.

10Putting a model in the product

A language model is a new kind of building on this map, and the useful way to think about it is as a dependency with an unusual profile: very slow by the standards of everything else here, expensive per call, and non-deterministic — the same input can produce a different output tomorrow.

That profile dictates the architecture. Two seconds is a good latency for a model call and fifty times the budget of every other hop on the map, so the answer has to stream and the interface has to be designed for text arriving progressively. The cost is per token, so caching is not an optimisation but part of the design. And the non-determinism means you cannot test it with assertions the way you test everything else.

Retrieval is most of what makes this work. Do not ask the model to remember your data; find the eight relevant rows yourself, put them in the prompt, and instruct it to answer only from those. A model given the facts is a summariser, which is a job it is genuinely excellent at. A model asked to recall facts is guessing, and it will guess fluently.

Finally, decide in advance what happens when the model is unavailable, slow, or returns something unusable — because all three will happen this month. An AI feature with no defined behaviour for "no answer" is a feature that will occasionally show a user a blank box and a spinner forever.

11Security: layers, not a wall

Security is not a feature you add near the end. It is a property of decisions made throughout — and the useful mental move is to stop asking "is this secure?" and start asking what would this let someone do, and how much would it cost?

That question has a name. A threat model is one honest page: what you hold that is worth taking, who might want it, how they would try, and what you have decided not to defend against. It is worth more than any tool, because it is what tells you which of the following actually matter for you.

Secrets get their own building on the map for a reason. Database passwords, API keys and signing keys should be issued by a store rather than configured by hand, scoped to a single job, given a short life, and rotated on a schedule that runs whether or not anyone remembers. The point is not that leaks become impossible; it is that a leaked fifteen-minute read-only credential is an inconvenience rather than a company-ending event.

One more, because it is where a lot of real compromises now begin: the pipeline that builds and deploys your software holds the credentials that can change production. Treat it as production infrastructure, because to an attacker it is the most valuable machine you own.

12Observability: finding out before your users tell you

Once software is running, most of the truth about it only exists in production. Observability is the ability to answer questions about a live system that you did not think to ask in advance — which is a higher bar than "we have some dashboards".

Measure the things users experience, not the things machines report. CPU at ninety per cent is not an incident if every request is still returning in eighty milliseconds; a healthy-looking fleet returning errors to one per cent of people absolutely is. And use percentiles rather than averages — an average latency of 90 ms can comfortably hide a p99 of four seconds, which is a real group of real people having a bad time.

Teams that do this well write down a service level objective — say, 99.9% of requests succeed within 300 ms over thirty days — which converts an argument about whether things feel slow into arithmetic. The failure the objective permits is an error budget: spend it on shipping while there is budget left, and spend it on reliability when there is not.

13Hosting: where all of this actually runs

Every building on this map is, in the end, a process on a computer that someone is paying for. The platform is the set of choices about which computers, how the processes get onto them, and what happens when one of them dies at three in the morning.

Two ideas underpin all four. The first is that a running service should be disposable: killable at any moment, replaceable by an identical copy, holding no state of its own that matters. That is what makes scaling a matter of arithmetic and deployment a matter of starting new ones. It is also why "stateless" is worth the inconvenience it causes everywhere else.

The second is that the arrangement should be written down as code. Infrastructure defined in files, in the repository, reviewed like anything else — so that the answer to "why is staging different from production?" is a diff rather than an investigation, and so that rebuilding the whole thing is a command rather than a memory test.

14What breaks, and what to do about it

Everything on this map fails. Machines stop, networks partition, disks fill, certificates expire, third parties have incidents on your busiest day. Resilience is not the absence of failure; it is the property that a failure in one building stays in that building.

The most instructive failure is not something dying — that is easy to detect and easy to route around. It is something getting slow. A slow dependency means every caller still starts, still waits, and still holds a connection, so the damage spreads upstream to buildings that are working perfectly. That is how one unhappy database becomes a total outage.

That last one is a product decision disguised as an engineering one, and it should be made calmly, in advance, rather than at 02:00 by whoever is on call. Which features may disappear? What may be shown stale, and how stale? What must never be wrong even if it means refusing the request? Write it down once and the incident becomes an execution rather than a debate.

15What it costs, and what it costs later

Every architectural choice has a bill attached, and the bill has two halves. There is the invoice — machines, storage, bandwidth, tokens — and there is the part nobody puts in the diagram: how much of your team's finite attention the arrangement will consume every week from now on.

The invoice half is usually dominated by a small number of surprising lines. Bandwidth leaving a cloud provider is priced far above bandwidth arriving. Object storage is cheap to hold and priced per request to read. Model calls are per token, in both directions, and a feature that quietly includes a large document in every prompt can cost more than the rest of the system combined.

The second half is the one worth being most careful about, because it compounds. Every component you add is a thing to patch, monitor, upgrade, understand and explain to the next person who joins. A system a small team can hold in their heads is faster to change than a technically superior one they cannot — and speed of change is, in the end, what almost every business is actually buying.

16Choosing: how to end up with less of this

Almost every building on this map was added because something hurt. The mistake that costs teams the most is adding them in advance, on the assumption that the pain is coming — and then paying for all of it for years while serving four hundred users.

1. Start with the smallest thing that could work. One deployable application, one relational database, one managed host. That configuration will comfortably serve more traffic than most products ever see, and it leaves you the option of anything else.
2. Add a building when you can name the pain. Not "we will need a queue eventually" but "this endpoint takes ninety seconds and users are closing the tab". The named pain also tells you what "fixed" looks like.
3. Prefer boring, widely-used technology. Its failure modes are documented, its bugs have been found by strangers, and you can hire people who already know it. Spend your novelty budget on the part that is actually your product.
4. Let the trust boundary be the one thing you never compromise. Every rule enforced on the far side of the moat, every time, whatever the client already checked.
5. Make the source of truth obvious. For each piece of data, one place is right and everything else is a copy. Systems become unmaintainable when nobody can say which is which.
6. Instrument before you optimise. Guesses about what is slow are wrong at a remarkable rate. A trace costs an afternoon and settles the argument.
7. Write down the decisions and the reasons. A short record of what you chose, what you rejected and why. It is what stops the same argument recurring every eighteen months, and it is the best thing you can hand a new joiner — or an AI agent working in the codebase.
8. Keep it small enough to hold in your head. The best architecture is not the most capable one; it is the most capable one your team can still reason about at three in the morning.

✳Glossary

→Go deeper

• MDN Web Docs — the reference for everything that happens in the client.
• OWASP Top Ten — the recurring shapes of application security failure, with the access-control ones first for good reason.
• OWASP Cheat Sheet Series — short, practical guidance on authentication, session management, secrets and the rest.
• Google SRE Books — service level objectives, error budgets, overload and incident response, free online.
• Use The Index, Luke! — how database indexes actually work, which is the highest-value thing most backend developers are missing.
• The Twelve-Factor App — dated in places, still the clearest statement of why services should be disposable and configured from the environment.
• OpenTelemetry — the vendor-neutral way to emit traces, metrics and logs.
• How Software Gets Made — the companion to this page: how a change gets from an idea to running here.
• How the Internet Works — what is happening in the wires between the client district and the bridge.