Retrieval benchmarks sometimes feel benchmaxxed by models, so we wanted to find a way to tie it as close as possible to my objective: finding the article that answers a product question right. We worked on a new metric which seems more proportional to document relevance, and built up a benchmarking
