Skip to main content

Software Design

Design/architect software applications/systems.

Related pages:

Notes​

Resources​

Sites​

  • Conway's Law (HN)
    • "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure."
    • Projects plans should follow how people communicate in the organization
  • Payments 101 for a developer
    • A guide for developers developing systems involving the transfer of money. The common terminologies and concepts.
    • Terminologies: merchant, issuer, acquirer, network or scheme, payment processor, PCI compliance, GDPR compliance, etc.
    • Concepts: intent to pay, data collection, authentication, authorization, status validation, dispute, refund, settlement, etc.
  • Imaginary Problems Are the Root of Bad Software (HN)
    • Imaginary problems are more fun to solve than the real problems
    • Optimizing over some details and losing sight of the real issues
    • The HN thread is also worth a read: it's the industry that encourages people to look for imaginary problems
  • Monitoring is a Pain | Mathew Duggan (HN)
    • Good monitoring requires a lot of work, costs a lot and often has a high noise-to-signal ratio
    • Tackle the 3 major aspects: logging, metrics and tracing
    • Common pitfalls, author's experience, benefits and drawbacks of each
  • Fast machines, slow machines | Julio Merino (HN)
    • Command prompt, notepad, file explorer and Paint all start faster in Windows 2000 than in Windows 11
    • While hardware has improved for faster machines, software degraded
    • Cross-platform technologies and interpreted language contribute to slow software, particularly startup time
    • The issue is we are capable to do it but deprioritizing it. Soon, the hardware advances are eaten away by poor software designs
  • Advice for Operating a Public-Facing API | Joshua Stein
    • Host the API on a separate hostname: for different scaling & security measures
    • Don't be too liberal in what you accept: strict API design, no workarounds, otherwise risk breaking changes down the road
    • Avoid OAuth and prefer static API tokens for simplicity
    • Stay on top of failures: hopefully, users can handle the failure themselves and fix their app
    • Others: log a unique ID, descriptive error responses, prefixed tokens,
  • Feature Flags: Theory vs Reality — Problems and possible ways to better manage feature flags
  • Common design pattern at Stripe
    • Patterns for designing APIs, part 4 of Designing APIs for Humans
    • Prefer enums over boolean (making impossible states impossible), nested object for extensibility
    • Do not enforce security through obscurity, make IDs unguessable, but prefix ID with an entity name
  • Squeeze the hell out of the system you have | Dan Slimmon (HN)
    • Increasing the complexity costs time, effort as well as attention required for all future decisions
    • Until reaching the fundamental limit of the current (dumb) system, try to optimize and squeeze as much as possible
  • Tracing: structured logging, but better in every way
    • Log levels is not useful, messages can be better if structured, stdout is mixed
    • No relationship between lines, no timing/duration but only timestamps, querying logs is ineffective and hard
    • Contains an example of how to evolve the system to use OpenTelemetry Tracing
  • Every infrastructure decision I endorse or regret after 4 years at a startup (HN)
    • Reflecting each product choice or architecture decision. Why they are good/bad
    • And as always, interesting opinions in the HN thread
  • Why, after 6 years, I am over GraphQL
    • Attack surface is large: authorisation, nested queries and rate limiting, query parsing
    • Performance issue: n+1, authorising with n+1, dataloaders boilerplate
    • Coupling business logic into the API, increasing complexity of API tests
  • OAuth from First Principles | Stack Auth
    • Access token to avoid disclosing the full credentials
    • Registered redirect URI to avoid impersonation
    • State to avoid CSRF, code and client secret to avoid eavesdropping
  • Bloom filters
    • Bloom filter will sometimes give false positive but never false negative
    • Store the mod of the hash of the data, to be used as an add only cache with little memory required
    • With interactive graphics on optimizing the number of bits and the number of hashes to use
  • Mistakes engineers make in large established codebases
    • The most common mistake is inconsistency, new code that is not consistent with the existing systems
    • Inconsistency creates surprises, makes future improvements harder
    • Understand the use cases and user behavior, avoid new dependencies, extra careful with removing code, etc.
    • Do not split up large codebase without understanding and shipping code with it
  • Boring tech is mature, not old (HN)
    • Boring tech is well understood, perform in predictable and reliable way, with no surprised

Emails

  • Email, explained from first principles
    • From basic concepts and terminologies of email systems
    • To the architecture, protocols and format for email to work
    • Issues with emails like spamming and privacy, and recent standards to address it
    • Alongside with many concepts introduced about the internet like TLS, replay attacks, CRAM, SCRAM, channel bindings, etc.
  • Understanding SPF, DKIM and DMARC: A simple Guide (HN)
    • SPF: a list of email friends that can send email on behalf of this domain
    • DKIM: public key to verify email are not modified
    • DMARC: rule book of how to handle SPF and DKIM checks

Queues

  • Load balancing algorithms (HN)
    • A little bit of Bartosz Ciechanowski style visualizing load balancing algorithms, with a playground at the end
    • Round robin, weighted random, weighted round robin, least connections, peak EWMA
  • Choose Postgres Queue Technology (HN)
    • Not specifically about Postgres queue, but the cult of prioritizing scalability over other things
    • Build boring technology: If and only if the boring technology is unable to meet demands should alternatives be considered
    • Build escape hatches: abstractions (adapters) to make it easy to replace
  • Retries
    • Visualizing how to design retry mechanisms, with exponential backoff and jitter
  • The Big Little Guide to Message Queues
    • What are message queues? When to and when not to use queues?
    • Delivery guarantees: at-least-once, at-most-once, exactly-once
    • Ordering vs parallelism, and parallel doesn't mean random
    • Fan out/in design, and the use of topics on top of queues
    • Consideration of poison pills and dead letters queue
    • Common implementation: AWS SNS/SQS, Redis Streams, Kafka, RabbitMQ, etc.
  • Queueing | Encore
    • Interactively visualizing queueing strategies: FIFO, LIFO, priority queue, active queue management
  • Kafka at the low end: how bad can it get? (HN)
    • At low volume, the job allocation between workers can be extremely unfair
    • More problematic when the number of partitions and producers are high
    • Kafka is not a good job queue, at least until queues for Kafka arrives

Algo