# What modifications to the prompt or system might be needed to handle multi-step retrieval (such as giving the model the ability to output a follow-up question), and how to evaluate this new ability?

Source: https://zilliz.com/ai-faq/what-modifications-to-the-prompt-or-system-might-be-needed-to-handle-multistep-retrieval-such-as-giving-the-model-the-ability-to-output-a-followup-question-and-how-to-evaluate-this-new-ability
Author: zilliz

**Modifications for Multi-Step Retrieval** 
To enable multi-step retrieval, the prompt and system must explicitly support iterative interactions. The prompt should instruct the model to generate follow-up questions when information is incomplete or ambiguous. For example, adding a directive like, "If additional details are needed to answer, output a follow-up question in JSON format: `{'follow_up': '...'}`." The system must track conversation history to maintain context across steps, allowing subsequent retrievals to build on prior queries. A stateful architecture (e.g., caching prior interactions or using session tokens) ensures continuity. Additionally, the model might need fine-tuning on examples where follow-up questions resolve ambiguity (e.g., clarifying dates, locations, or technical terms in a query). 

**Technical Implementation Example** 
A practical approach is structuring the output format to separate answers from follow-up requests. For instance, the model could return: 
```json
{
 "answer": "Partial answer based on initial data...",
 "follow_up": "What specific version of the software are you using?"
}
``` 
The system would then process the follow-up question, retrieve new data, and feed it back into the model. Tools like LangChain or custom middleware can manage this loop, iterating until the model generates a final answer. To avoid infinite loops, a step limit or confidence threshold (e.g., stopping when no follow-up is generated for two consecutive steps) can be implemented. 

**Evaluation Strategies** 
Evaluation requires metrics for both the quality of follow-up questions and the overall process. For follow-ups, human judges can rate relevance (e.g., "Does the question address missing information?") and specificity. Automated metrics could track how often follow-ups lead to improved answers (e.g., comparing the final answer’s accuracy with/without multi-step retrieval). For end-to-end testing, create a benchmark dataset with ambiguous queries requiring iterative clarification (e.g., "How do I fix this error?" without context). Measure success rate, steps taken, and efficiency (time/computational cost). Additionally, use ablation studies to compare single-step vs. multi-step performance, isolating the value of follow-up capability. 

**Example Workflow** 
1. **Initial Query**: "How do I optimize database queries?" 
2. **Model Follow-Up**: Asks for database type (e.g., MySQL vs. PostgreSQL). 
3. **User Response**: Provides "PostgreSQL." 
4. **Final Answer**: Tailors indexing or caching advice to PostgreSQL syntax. 

This approach balances flexibility with structure, ensuring the model guides users toward precise solutions while remaining efficient.