Would this dataset work as a retrieval or RAG benchmark?
I’m building a structured dataset of real-world locations represented in video games.
A retrieval system could use the data to answer questions such as:
- Which games feature locations in Japan?
- Which records refer to historical sites?
- Which locations have exact rather than representative coordinates?
- Which games contain fictionalized places connected to real-world geography?
A few records look like this:
{
"game_title": "Assassin's Creed",
"location_name": "Jerusalem",
"country_iso": "IL",
"location_type": "city",
"setting_status": "real",
"coordinate_method": "representative",
"verification_level": "verified"
}
{
"game_title": "Like a Dragon Gaiden: The Man Who Erased His Name",
"location_name": "Sotenbori",
"country_iso": "JP",
"location_type": "district",
"setting_status": "fictional_with_real_anchor",
"coordinate_method": "representative",
"verification_level": "corroborated"
}
I’d appreciate advice from people working with retrieval systems:
- Would row-based records like these be sufficient, or should each record become a natural-language document?
- Which values should be used for metadata filtering, and which should be embedded for semantic retrieval?
- Is the distinction between
realandfictional_with_real_anchoruseful for evaluation? - Are
coordinate_methodandverification_leveluseful for evaluation? - Would source URLs and evidence text be necessary for a meaningful benchmark?
- What kinds of test questions would make this more useful than a simple database lookup?
I am grateful for any feedback.