Loading...
What is the agent designed to accomplish? Define the core task and success criteria...
Agent is designed to provide the answers to the questions from with in the knowledge base of the enterprise. It will be a RAG agent which will store all the information of enterprise by semantic chunking in vector database.
When the query is made , using the hybrid approach which involves ANN search and the keyword based search, it will provide the chunks , which will be re-ranked and PII information is filtered out from the results which are presented as a result of this query
What level of autonomy does the agent have? What are the boundaries and constraints it must operate within?
The agent will have the document being chunked based on semantic meaning. The agent does not provide any inference for photos/pictures. It provides search for text information from the knowledge based extracted from the text in the documents. The agent should provide optimized cost, with low latency and high precision of the results , it presents as a result of this query
Define the tools and actions available to the agent. Specify tool interfaces, input/output schemas, and how the agent selects which tool to use...
Tools will be database searching from the meta-data table,
Keyword based searching from the database.
LLM to re-rank the searched results.
LLM(AI Gateway) to scan and screen the PII data from the results
Design the overall agent architecture. Choose an orchestration pattern (single agent, multi-agent, hierarchical). Identify key components and how they interact. Use the diagramming tool to illustrate the architecture.
Using the hybrid architecture where the keyword based approach and semantic search from the vector database is parallelly being done . The results are being passed through the re-ranking and AI gateway to filter and sort them in the order of relevance
How does the agent manage memory and context? Define short-term (conversation) and long-term (persistent) memory strategies. How is context window managed? What retrieval mechanisms are used (RAG, vector stores, summarization)?
Conversation and the previous results will be passed again for any follow up questions. Up to a max number of words like 2048 tokens will be allowed to be sent from the previous context
Deep dive into the agent's reasoning strategy. How does it plan, decide, and self-correct? Discuss prompt engineering, chain-of-thought, reflection loops, failure handling, and tradeoffs.
The model will be fine tuned with a golden dataset which will contain the possible questions and the responses. The testing will be done to check if the responses are ordered correctly, the citations are present along with responses, the hallucinations are not present in the document, the PII information is redacted
What guardrails and safety mechanisms does the agent have? Define permission boundaries, human-in-the-loop checkpoints, content filtering, rate limiting, and how the agent handles adversarial inputs or unexpected states.