Distributed systems have a long history of technologies that were presented as the future, adopted quickly, and later understood with more caution. XML and SOAP are early examples. They were a response to a real problem, getting services on different platforms to talk to each other, and they delivered. The price was verbose messages, slow parsing, complicated tooling, and protocols that were painful to debug.

The pattern has repeated for decades. Each wave solved a real problem and came with costs that were easy to ignore while the wave was building.

A history of overpromise

Some past examples are:

• Distributed objects, including CORBA, DCOM, and Java RMI, promised that remote method calls could look like local method calls. The abstraction hid the fact that a remote call has different latency, failure behavior, security concerns, and versioning problems than a local call.

• XML promised universal, self-describing data exchange. It worked across platforms, but the price was verbosity, expensive parsing, complex schemas, and a wire format that was unpleasant to inspect by eye.

• SOAP and WS-* (WSDL to describe web services, WS-Security to add message security, WS_Addressing to encode routing information, etc.) promised standardized enterprise service integration through XML-based protocols for messaging, security, transactions, and service description. They produced large protocol stacks, complicated tooling, and message formats that were often harder to debug than the systems they replaced.

• Enterprise JavaBeans promised a standard component model for enterprise applications. In many systems, it introduced heavy containers, complex deployment, and unnecessary indirection.

• The first generation of service-oriented architecture promised clean separation between business services. It often evolved into distributed monoliths connected by brittle, complex XML interfaces.

• Java as the default enterprise language had real strengths: portability, maturity, and a large ecosystem. The mistake was treating it as the automatic answer for every problem, including systems where startup time, memory footprint, deployment complexity, or low-level control were the dominant constraints.

• Python as the default language is readable, productive, and has outstanding libraries. The mistake is assuming developer convenience always dominates runtime cost, concurrency behavior, deployment constraints, memory use, type safety, or integration with lower-level systems.

• NoSQL databases promised freedom from rigid schemas and the scaling that relational databases supposedly could not provide. Some delivered real benefits, but teams often rediscovered the value of transactions, secondary indexes, joins, schema discipline, and query languages.

• Document databases like MongoDB promised that schemaless storage would let applications evolve quickly. Early durability defaults acknowledged writes before they were on disk, secondary index behavior surprised teams used to relational databases, and most applications eventually wanted the constraints they had walked away from.

• Hadoop and big data platforms promised that every organization would become data-driven. Hadoop solved a real batch-processing problem, but many deployments became expensive clusters running jobs that a conventional database or a smaller analytics tool could have handled.

• Microservices promised independent deployment and team autonomy. They also turned local function calls into network calls, moved complexity into operations, and created new problems in observability, testing, versioning, and data consistency.

• Apache Kafka beyond its use case is what happens when teams adopt a high-throughput durable log because events feel modern. Kafka is excellent at the workload for which it was built. Most systems that deployed it did not have the throughput, retention, or replay needs that justified the operational cost of running it well.

• Containers promised packaging consistency. They helped enormously, but they did not remove the need to understand networking, storage, security, deployment, monitoring, or failure recovery.

• Kubernetes promised a common control plane for cloud-native systems. It is powerful, but it also creates a large operational surface area. Many small teams adopted Kubernetes before they had problems that justified it.

• Service meshes, including Istio and Linkerd, promised mutual TLS, observability, and traffic policy without changing application code. The cost was a sidecar proxy on every pod (which incurs overhead and complexity), a control plane to operate, sidecar versions to track, and failure modes that span both the mesh and the application.

• Serverless platforms, including AWS Lambda, promised that developers could stop thinking about servers. They changed the operational model, but cold starts, execution limits, debugging difficulty, vendor coupling, and performance variability still apply.

• GraphQL, a data query language created by Meta, promised that clients could request exactly the data they needed and stop chaining REST calls. The cost was N+1 query patterns when resolvers fan out to backing stores, harder rate limiting because every query is unique, complicated caching, and debugging that no longer follows a URL trail.

• Single-page application frameworks promised cleaner web applications. They also moved a large amount of complexity into the browser, increased build complexity, and sometimes delivered slower, less accessible sites than server-rendered pages.

• Blockchain promised decentralized trust for a wide range of applications. Outside a narrow set of use cases, projects struggled to justify the cost of consensus, low throughput, complex key management, and awkward user experience.

• “Real-time everything” encouraged teams to stream data when periodic batch processing would have been cheaper and more reliable.

• “AI for everything” is the current version of the pattern. Machine learning is useful, but it does not remove the need for correct data, clear requirements, security review, latency budgets, failure handling, and human accountability.

These technologies were (are) not useless. Many solved real problems. The lesson is that hype emphasizes the abstraction and downplays the cost.

What never took off

Some ideas were promoted as inevitable and never caught on:

• Distributed shared memory (DSM) (Munin, Ivy, TreadMarks) promised that programmers could write threaded code and have the runtime keep memory consistent across machines. False sharing, page-level coherence overhead, and the cost of moving pages over a network meant performance never approached explicit message-passing. Programmers learned to think in terms of shared-nothing architectures instead.

• Mobile agents (Aglets, Telescript) promised that computation would migrate toward data rather than the other way around. I recall some researchers working on them when I was at Bell Labs and didn’t get the point: it just felt like deploying little servers to deal with one specific problem. Security was the showstopper: untrusted code arriving on a machine and asking to execute is a hard problem to solve well. Java applets were the closest thing to a successful mobile-code platform; they had real adoption in the late 1990s and early 2000s and were defeated by the same security problem in slower motion, with browsers eventually dropping the plugin model around 2017. The idea survives in narrow forms (browser JavaScript, container schedulers) but never as the general framework it was promoted as.

• Process migration (Sprite, MOSIX, Charlotte) promised that running processes could move between machines for load balancing or fault tolerance. The implementation difficulty (open file descriptors, sockets, kernel state, locks held, kernel-level RPC) and the rise of stateless services and VM-level live migration meant process migration stayed in the research literature.

• Software transactional memory (STM) promised that concurrent programming would become as easy as wrapping code in atomic blocks. The model borrows from databases: a thread runs a block optimistically, the runtime tracks every memory location it reads and writes, and at the end it either commits the changes atomically or, if another thread touched the same locations, rolls them back and retries. The appeal was real, since lock-based programming is error-prone (deadlocks, priority inversion, lock-ordering bugs) and atomic blocks compose in a way locks do not. Two problems kept STM out of mainstream languages. Tracking reads and writes at runtime imposes overhead on every memory access inside a transaction, often more than careful locking would have cost. And I/O does not compose with rollback: you cannot un-send a packet or un-print a line, so any transaction that touches the outside world breaks the abstraction. The idea survives in specialized libraries (Clojure’s refs, Haskell’s STM monad) where the language constrains side effects enough to make the model work.

• Active networks promised that packets would carry code that routers executed, making the network programmable end to end. Security and operational concerns were too large to overcome, and software-defined networking took a different path that gave operators what they actually wanted (programmable control planes) without the safety problems.

Each of these tried to hide a hard property of distributed computing, and each was defeated by the property it tried to hide. The abstraction was too leaky to be useful, and patching the leaks cost more than starting from a less ambitious model.

Questions to ask before adopting

When you evaluate a new language, framework, database, RPC system, orchestration tool, or cloud service, ask engineering questions:

• What problem does it actually solve?

• What costs does it introduce?

• What does it make easier, and what does it make harder?

• What failure modes does it hide?

• What happens when traffic grows by 100x?

• What happens when debugging crosses process, machine, region, or vendor boundaries?

• Can the team operate it under stress?

• Is the abstraction honest about latency, failure, consistency, and state?

It’s easy to be smitten by hype. Good engineers are not reflexively conservative. They adopt new tools when a tool earns its place. They are skeptical of claims that a technology removes hard problems, because most of the time it relocates them somewhere else.

The lesson

Distributed systems punish wishful thinking. Latency still exists. Networks still fail. State still has to live somewhere. Serialization still costs something. Coordination still creates bottlenecks. Operational complexity does not disappear because a framework gives it a friendly API.

Use new technology, but do not outsource judgment to fashion. Ask what tradeoff you are accepting, and whether it is the right tradeoff for this system.