Availability

Quality Attribute · Chapter 4

Availability

Availability is the degree to which the system works when needed — the probability it is operating properly and not in unplanned downtime due to failure and repair. Often expressed in “nines” (99.9%, 99.99%, 99.999%).

Calculating availability

Based on time:

Availability = MTBF / (MTBF + MTTR)

• MTBF — mean time between failures (average time between two failures).
• MTTR — mean time to repair (average time to troubleshoot and restore).
• Only unplanned outages count as downtime.

Downtime budget per availability level (approx.):

Availability	Per year	Per month	Per week
99.0%	~3.65 days	~7.2 hours	~1.68 hours
99.9%	~8.76 hours	~43.2 min	~10.1 min
99.99%	~52.6 min	~4.32 min	~60.5 sec
99.999% (“five nines”)	~5.26 min	~25.9 sec	~6.05 sec

Each extra nine demands an order-of-magnitude improvement, costing time and money and crowding out other work (e.g. new features). Users often can’t perceive 99.99% vs 99.999% — network and device reliability may dominate anyway. Architect and stakeholders must weigh cost against benefit to pick the right level.

Based on request success rate: for globally distributed / cloud systems where some traffic is always served, a time-based calculation is less meaningful. Instead base availability on the ratio of successful to total requests. Time-based figures capture total failure duration (including repair) but don’t distinguish two 30-minute outages from one 1-hour outage; request-success rate does. Track both for a complete picture. See Cloud-native applications.

Faults, errors, and failures

• Fault — a characteristic in the code that can lead to a failure (alone or with other faults).
• Error — an erroneous system state caused by one or more faults.
• Failure — an event where the system does not behave as expected (user-visible), caused by one or more faults.

To keep outages within budget, the system can detect, recover from, or prevent faults.

Detecting faults

• Ping/echo — a monitor sends an ICMP echo request; no reply within a time window ⇒ report the target as failed.
• Heartbeat — a component periodically emits a heartbeat; a missing heartbeat ⇒ fault, take action.
• Timestamp — detect an incorrect ordering of events using timestamps or sequence numbers.
• Voting — e.g. triple modular redundancy (TMR): three components run the same process; voting logic compares outputs. A 2-of-3 majority outvotes and corrects the dissenter, reporting a fault on it.
• Sanity test — a simple check that a result is plausible; flags obvious faults quickly.
• Condition monitoring — watch conditions to catch a fault (or a developing one) early, sometimes before it occurs.
• Self-tests — built-in self-test (BIST/BIT): components carry logic to test their own correct operation, self-initiated or triggered by a monitor.

Recovering from faults

• Exception handling — catch and act on exceptions (return an error code, or an error object with code/message/stack trace); optionally correct the cause and retry, or show a friendly message.
• Retry strategy — for transient faults (network blips, temporary service unavailability, timeouts). Retry-interval types: regular (fixed wait), incremental (growing waits, e.g. 2s, 6s, 12s), exponential backoff, immediate (only once, then switch to another type), and randomization (jitter, to avoid synchronized retries from many clients).
• Redundancy / failover:
  • Active (hot spare) — a duplicate runs the same inputs; near-instant, transparent failover; most expensive.
  • Passive (warm spare) — active components send periodic state updates to backups; recovery in seconds to minutes; cheaper.
  • Cold spare — redundant components kept offline until needed; recovery can take hours.
• Rollback — return to a persisted checkpoint known to be good, then resume; combines with active/passive redundancy.
• Graceful degradation — drop non-critical functions to keep the critical ones running rather than failing entirely (e.g. when low on resources).
• Ignoring faulty behavior — deliberately disregard a known-ignorable fault and continue processing.

Preventing faults

• Removal from service — take a component offline in anticipation of faults; restore when healthy.
• Transactions — bundle steps so a failure undoes the whole bundle, avoiding incorrect saved state and race conditions on shared data.
• Increasing competence sets — broaden the states/conditions a component handles (e.g. handle a null argument instead of throwing) to reduce faults.
• Exception prevention — write code that avoids exceptions in the first place (e.g. bounds-check arguments and handle them gracefully).

Citations

• Software Architect’s Handbook (Packt, 2018), Ch.4 “Availability”, pp. 213-237.