Recall-at-k is a metric used to evaluate the performance of information retrieval systems, such as search engines or recommendation systems. It measures the ability of the system to retrieve relevant items from a collection when only a specified number of top results (k) are returned. Specifically, recall-at-k quantifies how many of the total relevant items are included within the top k results. This allows developers to assess whether the system is effectively surfacing the most pertinent information for users based on their queries.
To calculate recall-at-k, you first need to know how many relevant items are present in the dataset and how many of those are included in the top k results provided by the system. The formula for recall-at-k is: Recall-at-k = (Number of relevant items retrieved in top k results) / (Total number of relevant items in the dataset). For instance, if a search delivers 5 relevant documents out of a total of 20 relevant documents, and the search result set is limited to 10, the recall-at-10 would be 5 divided by 20, or 0.25, meaning 25% of the relevant documents were retrieved in the top 10 results.
Recall-at-k is particularly useful because it gives developers insight into how well their systems perform for users when the results are limited to a fixed number of items. For example, in an e-commerce application, if a user searches for "running shoes" and the system returns 15 results, recall-at-10 would measure how many of the most relevant shoe options were included in those top 10 results. By analyzing recall-at-k over different values of k, developers can gain a nuanced understanding of their system's effectiveness and make necessary adjustments to improve the relevance and quality of results provided to users.