In this blog post Why Agent Legibility Will Matter More Than Better Prompting for CIOs we will look at why the next big challenge in enterprise AI is not writing smarter prompts, but making agent behaviour visible, understandable and governable. In my experience, that is the difference between an interesting demo and a system a leadership team can trust.

For the last two years, most AI conversations have focused on prompting. That made sense early on. If the model only had one job, a better instruction often produced a better answer.

But agents change the game. An agent does not just answer a question once. It can plan, call tools, read documents, update data, hand work to another agent, and keep going across multiple steps. Once that happens, prompt quality still matters, but it is no longer the main control point.

After more than 20 years in enterprise IT, working as a solution architect and enterprise architect across Azure, Microsoft 365, cybersecurity and now AI platforms such as OpenAI and Claude, one pattern keeps repeating. The systems that succeed are not the ones that look smartest in a workshop. They are the ones people can understand when something goes right, and especially when something goes wrong.

The shift from clever prompts to governable systems

When leaders ask whether an agent is ready for production, they are rarely asking about prompt writing. What they really want to know is much simpler. What did it do, why did it do it, what did it access, and how do we stop it from doing the wrong thing next time?

That is what I mean by agent legibility. It is the ability to inspect an agent’s actions, decisions, data access, tool use, memory and handoffs in a way a human team can follow. Not just the AI engineer. Risk, security, architecture and operations teams as well.

I often describe prompting as the opening instruction, and legibility as the operating window. A strong prompt helps the agent start well. Legibility helps the organisation stay in control after the agent starts moving.

What agent legibility actually means

At a high level, legibility means the agent is not acting like a black box. You can see its goal, the context it received, the tools it chose, the information it pulled in, the steps it took, and the basis for the output it produced.

In practice, I look for five things.

• Visible intent. The task, policy boundaries and success criteria are explicit, not hidden inside a tangled prompt.

• Traceable actions. Every tool call, retrieval step, handoff and output is recorded in a useful sequence.

• Explainable context. We know what documents, systems or memory entries shaped the result.

• Controlled permissions. The agent can only access the systems and functions it genuinely needs.

• Evaluable behaviour. We can test the agent against expected scenarios and measure drift over time.

If those elements are missing, the discussion quickly becomes emotional. One executive sees innovation. Another sees unacceptable risk. Legibility gives both sides something concrete to work with.

The technology behind it

Under the hood, most enterprise agents use a fairly simple pattern. A large language model receives a goal, reasons over the next step, decides whether to call a tool, observes the result, updates its working context, and repeats until it believes the task is complete.

That sounds straightforward, but each loop creates risk and complexity. The agent may use the wrong tool, misread a result, over-trust memory, or act on stale context. In a multi-agent setup, those problems multiply because agents can pass work, assumptions and errors to each other.

That is why platform teams are putting so much emphasis on traces, orchestration and evaluation. The industry is moving away from the idea that a prompt alone is enough. The more autonomous the system becomes, the more instrumentation it needs.

A simple way to picture the agent loop is this.

• Receive a goal and business context.

• Plan the next step.

• Select a tool, data source or sub-agent.

• Observe the result.

• Update working memory or state.

• Repeat or finish.

Legibility means each of those steps is inspectable. That usually requires structured logging, trace IDs, prompt and workflow versioning, tool schemas, policy checks, approval gates for higher-risk actions, and evaluation datasets that reflect real business tasks.

This is not just an engineering preference. It is architecture discipline. We learned the same lesson in cloud and cybersecurity years ago. If you cannot see the path of an action, you cannot govern it well.

Why prompting hits diminishing returns

I am not dismissing prompts. Good prompting still improves quality, especially when you need consistent tone, role clarity, output structure or domain-specific instructions. I use that every week.

But prompting has diminishing returns once the system becomes agentic. A beautifully written prompt cannot compensate for poor tool design, weak permissions, bad retrieval, unmanaged memory or missing auditability.

One pattern I keep running into is teams spending weeks refining the perfect system prompt while ignoring the tool layer underneath. Then the agent fails in production for a very ordinary reason. It pulled the wrong file, called the wrong API, interpreted a missing value as a valid answer, or mixed old context with new.

That is not a prompting problem. That is a legibility problem.

The fastest way to lose executive confidence is not a wrong answer on its own. It is a wrong answer that nobody can explain.

Why legibility matters to business outcomes

For CIOs and CTOs, legibility matters because it changes the economics of operating AI. Systems you can inspect are easier to test, easier to secure, easier to improve and easier to defend internally.

I have seen four business outcomes improve when teams prioritise legibility early.

1. Faster risk approval

Security and legal teams move faster when they can see data paths, action boundaries and human approval points. Ambiguity is what slows most enterprise AI programs down.

2. Better operational support

When an agent behaves badly, support teams need traces, not theories. Clear evidence shortens incident response and reduces blame-driven meetings.

3. Lower key-person risk

If only one prompt engineer understands how the system works, that is not maturity. A legible agent can be understood by the broader architecture, operations and governance teams.

4. Safer scaling

The first agent may only summarise documents. The second may update records. The third may trigger workflows across finance, HR or customer platforms. Legibility is what lets you scale from low-risk assistance to higher-value automation without losing control.

An Australian lens on trust and control

From Melbourne, working with organisations across Australia and internationally, I find the local context especially important here. Australian leaders are not just thinking about productivity. They are thinking about privacy, cyber resilience, regulatory expectation and public trust.

If an agent touches personal information, the Privacy Act and Australian Privacy Principles are immediately relevant. Leaders need to know what data entered the workflow, where it was processed, what was retained, and whether the use matched the original purpose and organisational policy.

The same applies to cybersecurity. Essential Eight will not make an AI agent safe by itself, but the mindset absolutely carries across. Least privilege, strong administrative controls, application control, patching discipline and secure identity foundations matter even more when software can take semi-autonomous action.

I would go further. In an agent world, identity becomes the new perimeter all over again. If an agent can read mailboxes in Microsoft 365, access files in SharePoint, query Azure data stores and trigger downstream workflows, its permissions model is now a board-level concern, not just a developer setting.

A practical checklist for leaders

When I review an agent design with executive or architecture teams, I usually ask a short set of questions.

• Can we see the full trace of what the agent did?

• Do we know exactly which tools and data sources it can access?

• Can higher-risk actions require human approval?

• Can we replay failures and test changes safely?

• Do we have clear ownership across architecture, security, operations and business teams?

• Can we explain the system to an auditor or executive sponsor in plain language?

If the answer to several of those is no, the issue is probably not model intelligence. It is design maturity.

My advice is to treat legibility as a first-class architectural requirement from day one. Not a reporting feature to add later. If you design for visibility early, governance becomes lighter, not heavier.

Final thought

As a published author, I have learned that clarity is not the enemy of sophistication. The same applies here. The most capable agent in the room is not always the most valuable one. The most valuable one is often the agent your organisation can understand, trust and improve with confidence.

Prompting will still matter. It just will not be the main differentiator for long. The leaders who get the best results from enterprise AI will be the ones who insist on agents that are legible by design. When your first truly important agent makes a questionable decision, will your team be able to explain what happened in minutes, or only after weeks of guesswork?

• The Hidden Risk in Enterprise Agentic AI and Silent Failures
• How to Evaluate Agent Platforms in 2026 with Identity First in Mind
• Don’t Buy Black-Box Agents and What Your Agentic AI RFP Needs
• MCP A2A OpenTelemetry and OAuth Every Architect Must Track in 2026
• Copilot Memory is Now Default and I’d Disable It in 3 Cases