How AI Agents Are Changing Software Engineering and Modern Data Platforms

A review of modern data platform commit histories reveals a major shift over the past two years: the friction involved in writing code has declined dramatically. With coding agents, integrated development environments, containerized workflows, and automated tooling becoming more capable, the initial implementation of distributed streaming pipelines and complex API integrations is no longer the primary bottleneck.

AI agents can navigate repositories, write test coverage, inspect stack traces, recommend refactorings, and generate implementation plans. Given a plain-English request—such as building a Kafka-to-Iceberg data sink—an agent can often create a reliable starting point before an engineer opens every relevant file.

This changes the central question for software engineers.

If AI agents are becoming the primary authors of local system logic, what role will engineers play? Are we moving toward an industry of reviewers approving endless streams of plausible pull requests, or is software engineering shifting away from writing individual features and toward designing more reliable systems?

To answer that question, it helps to borrow a concept from thermodynamics. Thermodynamics provides a useful language for understanding directed work, feedback, inefficiency, and the boundaries required to keep complex systems stable.

AI Agents as Computational Heat Engines

When we remove the anthropomorphic illusion surrounding artificial intelligence, what remains is a computational engine. It receives direction and converts that direction into action.

Large language models running in data centers have enormous processing capacity, but they do not perform useful work until they receive intent. Prompts, business requirements, system instructions, failed tests, and data contracts provide that intent. Agents translate these signals into code, tool calls, database queries, tests, and changes to running systems.

Every engine experiences losses, and AI agent workflows are no different.

Anyone who has left an agent working against a complex repository has seen this process. The task may begin clearly, but the agent gradually follows outdated assumptions, treats symptoms instead of root causes, mistakes historical migrations for current behavior, and accumulates its own chain of decisions. After several tool calls, the context may contain plausible but contradictory details, making the next step less reliable than the first.

We can describe this operational entropy as the accumulation of outdated assumptions, branching contexts, and unresolved dependencies within an agent loop that is still attempting to move forward.

Human intervention helps because it introduces new and higher-quality information. Failed tests, accurate data contracts, deterministic tools, and precise evaluations also provide valuable feedback. Without these signals, an agent may continue producing output while moving further away from the desired result.

AI agents can generate movement quickly. The more important question is whether the systems surrounding them can convert that movement into reliable, profitable, and maintainable work.

The Infinite Monkey Theorem and the Expanding Search Space

The infinite monkey theorem offers a useful analogy for understanding repeated trials, finite constraints, and feedback-driven development.

The theorem suggests that if a monkey randomly pressed keys for an infinite amount of time, it would almost certainly reproduce the complete works of Shakespeare. Modern AI agents are considerably more capable than random monkeys. They have access to compilers, repositories, development tools, test suites, and feedback loops. Their actions are not random, and the results of each attempt influence what they do next. However, the underlying pattern—suggest, execute, observe, correct, and retry—remains familiar.

For well-defined tasks, this loop is extremely effective.

Give an agent a known input schema, a known target schema, a focused codebase, and tests that detect meaningful failures. It can inspect the code, make changes, run tests, interpret the results, and try again. The definition of done is visible, the search area is narrow, and the workflow has a strong chance of converging on the correct solution.

That level of clarity is rarely available in enterprise software systems. Real-time pricing engines may depend on mutable operational states, third-party APIs, delayed events, regional policies, and business rules that exist partly in code and partly in an employee’s memory. A data lakehouse can be physically consistent while remaining semantically incorrect. A data pipeline can pass every automated test and still produce numbers that finance does not recognize as valid.

The environment changes while the agent is still generating code.

The Three-Body Problem of Enterprise Software Logic

For this reason, the three-body problem provides another useful metaphor for enterprise software engineering.

With two celestial bodies, such as a planet and a star, it is possible to predict their movement using a relatively clear mathematical model. Adding a third body makes the problem far more difficult. There is no general closed-form solution, and some configurations display chaotic behavior. Small changes in one part of the system can produce dramatically different outcomes elsewhere.

Modern data platforms have a similar structure. Clickstream data changes as product behavior evolves. Operational databases change with customer activity. External APIs impose rate limits and introduce version changes. Schemas evolve. Security policies are updated. Legacy systems contain rules that were never formally documented because they were buried in exception-handling logic over many years.

Each system places pressure on the others. Changing one component can alter the meaning or behavior of another. What begins as a local feature request can quickly affect the broader data architecture.

Consider a simple example. An AI agent is asked to add a customer_tier field to a revenue model. It searches the production database, finds a field named status, maps that value into a transformation, and passes the existing type and nullability tests. The code is clean. The pipeline is green. The result is still incorrect.

A semantic data contract states that customer_tier must be derived from trailing 12-month spending, assigned to the appropriate business owner, and never populated from account status. That contract rejects the change before it reaches a dashboard or financial report.

The engineer’s contribution was not the transformation itself. It was the boundary that made the agent’s mistake visible, specific, and recoverable.

The New Mission: Designing Equilibrium

The role of a software engineer is no longer limited to writing every piece of application logic. AI agents will increasingly perform that work, often faster and at greater scale. The emerging mission is to design equilibrium: the conditions under which generated logic can be trusted.

When business requirements change faster than agents can absorb feedback, engineers must create containment fields. A strict semantic layer, immutable event logs, data contracts, idempotent APIs, and deterministic state machines do more than keep a data platform orderly. They reduce the number of assumptions an agent must make at the same time.

These architectural controls transform highly coupled problems into bounded domains with clear inputs, explicit rules, and reliable feedback.

Once those boundaries exist, AI agents become significantly more powerful. They can write data transformations, run automated tests, repair failures, and ship changes without having to infer the undocumented history behind every table, service, or business process.

The value of software engineering does not disappear when code generation becomes cheaper. Instead, it becomes more visible—and more important.

Autonomous systems will generate software, but software engineers will continue to design the contracts, feedback loops, and system boundaries that determine whether that software succeeds or descends into operational chaos.

Ananth Packkildurai is a data engineering leader, writer, and author of Data Engineering Weekly, where he shares insights on modern data platforms, large-scale data pipelines, and AI-driven software architectures.