How to Kick Off a Deep Research Agent Build
Our experience shows that the most successful Agentic AI projects begin with specific preparation steps. By exploring, defining and refining agent behaviour up-front, we’re able to set up the agent build phase for success.
In this blog, the first in our series on how to build AI agents, we're going to take a look at Veratai's process for designing deep research agents for the enterprise.
We call this step Envision.
What does it cover?
The purpose of the Envision phase of our deep research agent design process is to understand who needs the agent, what it needs to do, and how it might work. At the heart of Envision are two workshops: Kick-off and Analysis. Each workshop covers a significantly different aspect of the design phase.
-
If you have many stakeholders and a complicated research process, you will likely need to run the workshop multiple times to get enough detail. Conversely, if you have a very focused research task and small stakeholder pool you may be able to combine them.
The Kick-off Workshop
The Kick-off Workshop is where you set the ground rules for the rest of the project. You’ll identify critical stakeholders in the research process, select the experts you will use to help develop the agent’s behaviour, identify governance and guardrail requirements, and document at a high level how the agent will integrated into the business.
Who should be involved in this workshop? Certainly, the lead architects of the deep research agent. You will want the key sponsors from the business, and representatives from departments or teams who are going to be using the deep research agent.
You need four outcomes from the Kick-off workshop:
1. Stakeholder Identification
Ask yourself: who is going to be affected by the workflow that this agent is going to perform? Who is going to participate in it? Who is going to provide inputs or other forms of support? (This could be direct support or indirect support.) And who has a stake in the outputs?
For example, somebody who is not involved in producing a deep research report but is involved in using the information further down the line is a stakeholder. All stakeholders must be identified and considered, because the output of the deep research agent is going to have to be suitable for their work.
2. Expert Selection
The research process that you are automating or augmenting is probably subtle, nuanced, complex and likely requires expert judgement. What you want to know is: who is best placed in the business to be able to describe the research process, both as-is and to-be? These people will need to be substantively involved in the project.
Select a small number of these experts, people who are willing to put in a fair degree of time into the project - beyond just an initial input into the requirements and some evaluations of the outputs. These are people you will be coming back to when you have specific questions about nuances of behaviour.
3. Governance and Guardrails
What should your agent do and not do? For example:
Are there certain tasks that it should hand off to a human?
Are there certain aspects of tasks where a human will need to be involved?
Are there certain data sources which it will have access to but which it should not use, or which it should redact?
Are there certain kinds of outputs that it must produce or must not produce?
Where should humans get involved and where should the agent hand off to a human?
These are all sources of guardrails. It's good to think early on about what these might be, because it helps you with the design of the agent as you start to restrict its scope and you start to think about the various options you have for incorporating humans into the process.
4. Business Integration Pattern
We can split this into three sub-tasks.
Application type: What kind of application is required to interact with the agent? Do you need to build an end-user application, or will a simple workflow or integration with an existing system suffice?
Non-functionals: You'll need to agree the non-functionals for the system. How long can a research task take? How many will the system be expected to produce? Is there a cost limit on the amount of language model time that it can be using? Will there be concurrent users for the end-user application?
System constraints: You need to determine any system constraints that might affect deployment. These could be around security, governance and access controls. You'll also need to have a good idea as to where the system is going to be deployed (cloud, on prem?) and what it will have access to.
The Analysis Workshop
The second workshop is the Analysis Workshop. Depending on the number of experts you've identified and the number of end users who will have to provide input into the final solution, you may need to split this into several, focused workshops.
Here again, you have four objectives.
1. Process Mapping
You need to map the research process as it would be performed by human experts. This is a best-efforts attempt. You will probably find that it is very difficult for people, even the experts, to present you with a complete picture of the research task they perform.
More than likely, there will be a great degree of tacit knowledge - things they do that they are unaware of doing, or things they simply forget to tell you, or conditional factors: things they will only do if something else happens.
There are several techniques you can use to elicit this tacit knowledge:
Try and solicit this information from more than one expert, especially if there are handoffs in the process between different people.
Ask "what if" , "how", and "what else" questions to get them to keep speaking.
Be understanding that it is best efforts and they will be unlikely to get the process mapping correct the first time. The process map will evolve with each phase of the agent development. While it’s important to have an initial map, plan to iterate on the process as you gain knowledge of implicit steps. Later steps of our methodology involve testing and gathering feedback from both the experts and the business as a whole – it is here that missing steps will become apparent.
2. Resource Identification
You will need to identify the resources that the agent will use. Human experts use plenty of resources. They may use internal data sources. They may search the web. They may use various tools and databases. You have to understand what these resources are and how a machine could access the same. In your agent design, many of these will be translated into tools.
3. End-User Application Design
If you're going to be building an end-user application, you need to give it a design, so work with the experts to draft mock-ups for the tool.
Here's something very important: when you do this, think about what sub-units you can break your research outputs into. The more granularly you can get with this, the easier it will be to design schemas for your agent to complete, and the easier it will be to design an end-user application that allows users to give granular feedback on particular aspects of the agent's output.
This in turn provides you with a learning signal that you can use to improve the models during further iterations of the research agent. So think, as you design your end-user application: how are you going to gather granular feedback on the research agent's outputs from your end users so that you can use it in the future?
4. Output Specification
Finally, you will want to specify the research outputs themselves. Do they need to be long or short? Must there be tables, graphs, figures, summaries? How specific are you with the way it needs to look?
Make sure the views of all the affected stakeholders you identified in the Kickoff Workshop are represented. Even people who are two passes removed, further along the value chain, who use this research, should have a chance to say which elements of the research outputs are most important to them.
Next Steps
With these two workshops complete and all of these goals achieved, you should have a really good picture as to what your deep research agent needs to do. From there, you are ready to move on to the next of our five phases: Explore.