The AI Organization, part I – by Omar Shams

If your job involves coding or writing, change is coming: Large Language Models (LLMs) will change the way you work, and it may happen very quickly.

The consensus opinion now is that LLMs are going to be as important as the printing press

or perhaps even the Industrial Revolution. Some are even saying it’s the end of the world as we know it.

Most of the discussion of AI today focuses on 1:1 interactions like ChatGPT, but we aren’t talking as much about how AI is going to change organizations. AI will dramatically enhance our collective communication abilities, and this will lead us to completely overhaul the structure of the corporation.

Throughout human history our technological development and scale of cooperation have been dictated by information and communication technologies

. LLMs erode many of the assumptions about information flow and coordination costs in the modern corporation. It’s a bit like dropping cell phones into a classic movie like Home Alone… Morning Alone — it doesn’t have quite the same ring to it. We predict an equally transformative evolution into what we call the AI organization. We humbly propose the following definition:

“

An AI organization is an entity that primarily uses AI to manage information flow within the organization and decide on team composition and function.

”

But before diving into how AI impacts organizations, we’ll first need to understand what corporations or firms are and how our pre-LLM communications technology grounds modern ones.

Look around you — almost everything you see was created by a corporation. Why? One answer is legal: governments make it possible to create legal entities that trade and sell goods & services as one unit. This creates a convenient unit for human action, especially when dealing with taxes and regulatory requirements. But the legal explanation does not tell the full story of why the market has adopted this vehicle and not another.

What prevents us from operating solely as independent contractors, or conversely, working for a single all-encompassing entity such as a giant company or the government?

The economist Ronald Coase

believed that transaction costs were the primary factor controlling the size of firms. For Coase, firms grow to the size where the cost of organizing an internal transaction in the firm (e.g. adding one extra person working to achieve a task) is higher than the same transaction on the open market (e.g. by hiring a contractor).

To understand this, Coase devotes a lot of time to understanding the following puzzle: Outside of a firm, market signals (i.e. price) seem to primarily guide decisions, while inside, human judgement (”management”) is the primary guiding force. Coase quotes economist Dennis Robertson’s description of firms in the sea that is the market: they are

“islands of conscious power in this ocean of unconscious co-operation like lumps of butter coagulating in a pail of buttermilk”

Organizations incur many transaction costs when they get help from entities outside the organization: it requires time to find the right outside organization or contractor, a well-defined spec of the work required, and, of course, time to negotiate contractual details. This is simply too much to do for every task, especially in the face of rapidly changing business needs.

Within a firm, these transaction costs disappear. There is devoted talent pool with deep organizational knowledge, capable of doing iterative work with a flexible project scope and time horizon.

Of course, the catch is that when we make decisions inside a firm all the market signals are gone and we must rely solely on our judgement. Inside a firm, especially for the largest and most complex companies, it becomes very difficult for even the founders & owners to understand their own organization, and we are plagued by high coordination costs.

In the past, limited communication and distribution networks constrained corporation size. If we asked a 18th century reader to look around them, it’s much more likely their possessions and services were NOT produced by a corporation.

Prior to the invention of the telegraph, information could only travel as fast as the fastest ship or horse, constraining the scale and degree of coordination a corporation could operate at. The British East India Company, while being the largest company of its time, was structured more as a collection of regional operations. Each operated with a significant degree of autonomy due to the difficulty of communication between different divisions. Transmitting information between London and Calcutta, for example, could take several months. It’s hard to imagine a product like the iPhone being created in these conditions – communicating the centralized vision of Apple’s executive team with an international manufacturing operation would prove immensely challenging.

The telegraph made instant communication possible within a firm, and railroads ensured a distributed market to offer their services. The companies providing services like railways were among the first to rise to this unprecedented scale. The New York and Erie Railway was one of the largest railroad companies in the 1850s. However, its general superintendent, Daniel McCallum, noticed that they were much more inefficient than smaller railways. The telegraph had led to an over-reliance on centralized control. Could these companies maintain their economies of scale while also preserving some of the efficiencies of smaller railways?

To address this challenge McCallum invented the modern org chart, which organized staff in a hierarchy with fungible role types like ‘Superintendent of Road’ and ‘Brakemen’. McCallum gave more responsibility to divisional superintendents, which allowed them to focus on their divisions and effectively operate as if they were smaller and more efficient rail lines. McCallum’s invention flourished and created the modern corporation that provides so many of the goods and services we rely on today.

With the innovation of the org chart McCallum eased the tradeoff between scale and efficiency that many organizations face as they grow. This new ‘operating system’ for firms operates even more effectively in modern times by leveraging the internet, but the tradeoff between scale and efficiency has not gone away. Structures and processes that work very well at small scales are difficult to maintain at larger scales. Firms today are still plagued by problems arising from scale

.

This tradeoff is not unique to organizations and is also found in natural settings. Natural beehives are much more efficient than industrial ones, but they are impossible to manage at scale. Small farms are also more efficient per unit of land because they can grow various crops side by side – unfortunately, it is infeasible to scale them into industrial operations

.

Many readers will recognize how this dynamic plays out in organizations. Small organizations have the advantage that everyone knows each other. Every member can share context with every other member of the organization. Processes and projects can be very specifically tailored to the makeup of the team and the current situation the organization faces. It’s also easier to keep a small group aligned & motivated to take on an ambitious mission – hello, startups!

A large org on the other hand has more staff (this can scale pretty far, Amazon has over 1M employees, for reference ancient Athens had around 150k people) but suffers from high coordination costs

. Executives deal with scaling pains by creating a reporting hierarchy via an org chart. This system allows information to be encapsulated and sub-organizations to focus on their own objectives without the friction of needing to interact with every other part of the org. This parallels programming practices, where we routinely divide codebases into modules, selectively concealing information through encapsulation. Notably, at Amazon, the mandate for teams to interact exclusively via APIs is believed to have catalyzed the creation of AWS.

In today’s organizations, management decides team composition and function. Unfortunately, we often see unnecessary duplication and a lack of sharing of hard-earned knowledge between teams. And sometimes how we originally organize teams can quickly become sub optimal and not reflect an organization’s changing needs. This is where the power of AI comes into play.

Stepping back, we see:

1. Firms reach an efficiency limit and practical maximum scale (e.g. VOC or British East India Company)

2. A new communication technology emerges boosting firm productivity — the telegraph (and later the internet)

3. Firms struggle to integrate that technology into the old corporate structure (traditional pyramid structure)

4. The firm is refactored into new structure that allows it to fully utilize the new technology — what we have today: the org chart, OKRs, stand-ups, and so on up to a new scaling limit (e.g. Amazon today)

We predict a similar evolution today

1. Firms at a practical efficiency and scaling limit with our present technology

2. New communication technology emerges: Large Language Models, boosting firm productivity — You are (probably) here

3. The new technology (LLMs) is running on an old corporate operating system and not fully benefiting the organization — this is the Copilot model we see today — You (might be) here

4. Refactoring the firm with a new structure, the AI organization — We will help you get here

Here are some familiar characteristics of the modern firm:

• Formal org charts with well-defined roles and levels (e.g. PM, level 6)

• Cooperation between teams, especially those in different divisions, incurs high coordination costs. Two teams working on similar problems in different divisions, as a rule, don’t talk and may only be vaguely aware of each other’s work. It’s interesting that at Google, the internal intranet search on its mono repo and document base functioned as a sort of mixer for the company because that was often the only way two teams working on similar topics could find out about each other.

• Technical personnel are often trapped in knowledge bubbles within their own divisions, unable to share their expertise with other teams facing similar problems.

• Use of OKRs to plan objectives and for teams to sync typically on a quarterly basis.

• Sprints and stand-ups within teams, going over basic facts about what they did and how close they are to achieving their objective.

As we’ll see, LLMs route around and reduce the need for every one of these.

> Formal org charts with well-defined roles and levels (e.g. PM, level 6)

> Cooperation between teams, especially in different divisions, has high coordination costs. Two teams working on similar problems in different divisions, as a rule, don’t talk and may only be vaguely aware of each other’s work.

Right now we’re hand-designing information flows and team structure. Instead, let’s use LLMs to share information between teams and help route important work to the right people.

• LLMs can summarize what work everyone does in an organization by parsing over their code, messages, and documents.

• LLMs in conjunction with other AI techniques can also identify common problems in an organization and rank them by severity.

• These models can then group the work of each team member by reviewing their code, messages, and documents, providing a comprehensive summary of their roles.

• We can then route important information to the right people in the organization who have the relevant expertise.

This way of organizing information effectively forms dynamic ‘flash’ teams that cut across traditional organizational boundaries.

>Technical personnel are often trapped in knowledge bubbles within their own divisions, unable to share their expertise with other teams facing similar problems.

By training LLMs on company code/docs and/or embedding company code/docs in a vector space we can capture institutional knowledge (‘tribal knowledge’) and spread it around the organization and safeguard it against loss due to personnel changes

.

> Use OKRs to plan objectives and for teams to sync typically on a quarterly basis

> Sprints and stand-ups within teams, going over basic facts about what they did and how close they are to achieving their objective

LLMs can breakdown objectives into tasks from all levels — executives, managers, engineers, or PMs — providing technical scope. This can occur continuously to ensure alignment. The need to ask “who should I approach for…” is eliminated and information that would otherwise be in an outdated wiki is continuously updated and readily accessible. As for executives, they receive a dynamic dashboard they can chat with offering insights into their organization. Meetings and stand-ups become less prevalent in part because you can interact with simulated versions of any team-member

.

So the AI organization will feature

• Universal dissemination of organizational know-how and context

• More fluid and less defined team boundaries.

• Accelerated development and execution

What’s the catch ? AI organizations will likely give up on some degree of legibility such as having a well defined org chart, but already the biggest and most important organizations are difficult for their founders/owners to grasp. The shift towards the AI organization is the natural next stage in the evolution of organizations.

To facilitate this transformative shift, we’ve created Mutable.ai.

To accelerate the emergence of AI organizations, we are building the foundation to support them, unifying how organizations communicate, build, and store knowledge on a single platform.

At Mutable.ai our mission is to build foundational AI tools that will accelerate the emergence of AI organizations. Our aim is to unify how organizations communicate, build and store knowledge on a single platform, all powered by the revolutionary capabilities of LLMs.

Given the central role of software creation in the global economy

, Mutable.ai is starting by helping software dev teams by building a multiplayer universal programming platform. Superficially, this will look like a unification of Slack, VS Code, and GitHub, but we aim to avoid vendor lock in and support integrations with your current tech (like GitHub) and also to include non engineers (especially PMs!) on day one.

While the Copilot model is a valuable starting point, it doesn’t meet the prerequisites for supporting AI organizations: unified AI driven information flow linking all stakeholders. To paraphrase Henry Ford, Copilot built a faster horse and we’re building a Ferrari.

We believe simply bolting on AI functionality to existing tools doesn’t create the seamless work flow we’d expect in an AI organization. Working in a true AI organization will feel almost effortless – imagine ops firmly out of the way, programming overhead eliminated, instant access to information you need from across the entire organization, and support from the colleagues best positioned to make you more effective.

So, instead of an AI copiloting the analog of an Airbus from the 70s, think of Mutable.ai as a Jaeger for your company, piloted by your team. This means shipping more features to your customers in less time, understanding what your team is building in real time, and fearlessly leveraging unfamiliar technologies without compromising the integrity of your systems.

To aid information flow within your organization, we’re offering:

• A codebase chat to ask questions about your codebase as well as pull requests or commits, TL;DR — too, didn’t review and soon we will expand to include your docs + messaging + your issues/ticketing system

• An auto wiki feature that serves as a living documentation of your codebase (and soon Notion/G-docs and Slack)

• An auto bug identification feature that identifies potential issues in your codebase and incorporates notes, issues, and active tickets and even suggests good people to investigate these issues — forming a sort of virtual flash team to solve your most pressing issues

• An auto stand up feature that serves as a snapshot and history of work across teams, utilizing all commits across company repos

We’re also breaking down technical barriers and piercing knowledge bubbles:

• Embedding the entire company corpus (code + other information) and in some cases fine tuning on the company corpus

• Multi-file editing with natural language that is aware of your codebase

• Automated test generation, that is aware of your testing style

These features come with privacy controls that give you complete control over how information is shared depending on role, team, and seniority. For example, execs will have a birds-eye view of their entire organization that will allow them understand what is happening at any desired level of granularity. The more information you put in, the better it will work, but you are in control of who sees what information.

It’s clear we’ve hit an inflection point in the history of the firm. LLMs and AI more generally will transform the modern organization as we know it. The AI organization, the next stage in the evolution of the company, holds tremendous promise to lower coordination costs and to increase firm velocity without compromising safety.

Just as one structure is being torn down, we now see the scaffolds of a grander future one. While we hope we’ve made it clear that this structure must exist, it’s not entirely clear yet how exactly it will look. In the coming months, we will do what we do best: talk to you, ship features, and provide more examples and case studies (perhaps featuring you!) about the structure of this new organization.

If you want to be the first organization to have its own AI-powered mecha custom-fit to your team, please reach out to [email protected]. We’d love to partner with you. We’re a small but mighty team whose members were formerly part of organizations like DeepMind, Google, Stanford, and AWS.

Our new codebase app is out today, check out app.mutable.ai for sneak peak of this future.

Best,

Omar Shams, Founder CEO