Loading...
The agent is designed help business user within the company to find accurate informations if they exists, within the company documents.
It must performant, reliable and trusted by the users to ensure adoption.
The Agent must be able to answer "I don't know" if he doesn't have the informations.
Depending on the level of accreditation the user have, he might have restricted access to some document. The Agent must respect that and not leak informations he is not supposed to do.
That imply that the metadata of the embedding and the chucking granularity strategy are well defined for that.
The Agent has access to the internal vector database containing the Embedding of the sources documents.
The Retriever will use hybrid search to optimize its research in it.
In order to help the Retriever, the Agent can have a tool that help him reformulate the user question to facilitate the semantic search.
LLM Judge can be used as well to validate the output generated (LLM judge using a different model than the one used to generate the answer)
Design the overall agent architecture. Choose an orchestration pattern (single agent, multi-agent, hierarchical). Identify key components and how they interact. Use the diagramming tool to illustrate the architecture.
To retrieval pertinent information given a specific question, the Agent will call the RAG to perform semantic search within the vector stores to find relevant informations.
We will keep 15 to 25% of the context free to generate the answer in order to avoid silenced truncate.
For the short term memory strategy (conversation) we will use summarization to balanced way in between cost and efficiency. It will summarize the first loops, and give the untouch last N loops. Another way to benchmark might be a sliding window over the N last loops.
Long term memory strategy (persistent):
Deep dive into the agent's reasoning strategy. How does it plan, decide, and self-correct? Discuss prompt engineering, chain-of-thought, reflection loops, failure handling, and tradeoffs.
What guardrails and safety mechanisms does the agent have? Define permission boundaries, human-in-the-loop checkpoints, content filtering, rate limiting, and how the agent handles adversarial inputs or unexpected states.