Skip to content

When the Music Stops: Tip-of-the-Tongue Retrieval for Music

License

Notifications You must be signed in to change notification settings

spotify-research/tot

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
 
 
 
 
 
 

Repository files navigation

When the Music Stops: Tip-of-the-Tongue Retrieval for Music

This repository contains data collected in the paper 'When the Music Stops: Tip-of-the-Tongue Retrieval for Music'. To appear at SIGIR2023, pre-print available on arXiv.

Abstract

We present a study of Tip-of-the-tongue (ToT) retrieval for music, where a searcher is trying to find an existing music entity, but is unable to succeed as they cannot accurately recall important identifying information. ToT information needs are characterized by complexity, verbosity, uncertainty, and possible false memories. We make four contributions. (1) We collect a dataset—$ToT_{Music}$—of 2,278 information needs and ground truth answers. (2) We introduce a schema for these information needs and show that they often involve multiple modalities encompassing several Music IR sub-tasks such as lyric search, audio-based search, audio fingerprinting, and text search. (3) We underscore the difficulty of this task by benchmarking a standard text retrieval approach on this dataset. (4) We investigate the efficacy of query reformulations generated by a large language model (LLM), and show that they are not as effective as simply employing the entire information need as a query – leaving several open questions for future research.

Authors

Samarth Bhargav (work done during internship at Spotify Research), Anne Schuth, Claudia Hauff

Cite

@inproceedings{bhargav2023,
  title = {When the Music Stops: Tip-of-the-Tongue Retrieval for Music},
  author = {Samarth Bhargav and Anne Schuth and Claudia Hauff},
  year = {2023},
  booktitle = {Proceedings of SIGIR'23},
  doi = {10.1145/3539618.3592086}
}

Description

Data can be downloaded using this link, which is a zipped JSONL file with one information need per line. You can view an example docuument at the bottom of this Readme. Each field and accompanying description is outlined in the table below:

Field Name Type Description
id Identifier (String) Unique identifier
wasabi_2.0_id Identifier (String) Corresponding item in the Wasabi 2.0 Corpus
wasabi_isrc Identifier (String) ISRC of the item, sourced from the Wasabi 2.0 Corpus
spotify_resolved Dict Denotes the matching Spotify song(s)
spotify_resolved.title String Title of the song
spotify_resolved.items List (Dict) Associated Songs(s) on Spotify (multiple recordings can exists, any are considered relevant)
spotify_resolved.items.IDX.url URL URL of song at index IDX
spotify_resolved.items.IDX.isrc List (String) ISRC(s)
spotify_resolved.items.IDX.album List (Dict) Album song appears on
spotify_resolved.items.IDX.album.name String Album name
spotify_resolved.items.IDX.album.uri URI Album URI
spotify_resolved.items.IDX.artists List (Dict) Associated Artists(s)
spotify_resolved.items.IDX.artists.IDX2.name String Artist Name of artist at index IDX2
spotify_resolved.items.IDX.artists.IDX2.name URI Artist URI
spotify_resolved.artists.IDX.uri String Artist URI on Spotify
raw_title String Raw title of the submission
title String Cleaned title of the submission
raw_text String Raw text of the submission
text String Cleaned text of the submission
query_types List (String) Type of query (can be multiple): descriptive, user_created, extant_media
split String Split (train, val, test) used in the original experimentation. This value is null if the query wasn't used during benchmarking
annotations List (Dict) Annotations per sentence. This value is null if the submission wasn't annotated.
annotations.IDX.id String ID of annotation at index IDX. Format is either title_<sentence number> (sentence was in title) or text_<sentence number> (sentence was in text)
annotations.IDX.text String Sentence text
annotations.IDX.labels List (String) Set of labels associated with the sentence. See Table 1 in the paper for possible values and their descriptions.
reformulations Dict Reformulations. null if query wasn't used during benchmarking
reformulations.keywords String List of keywords generated by Yake
reformulations.reform_<N> String Generated reformulation up to N (= 10, 25, or 50) words (see paper)
reformuations.reform String Generated reformulation (without N constraint)

Click here to view example document

{
  "id": "8sw82n",
  "wasabi_2.0_id": "5714dec425ac0d8aee38d04b",
  "wasabi_isrc": "USUM71506251",
  "annotations": [
    {
      "id": "title_0",
      "text": "pop song about being at the club but wanting to be somewhere else",
      "labels": [
        "Genre",
        "Story/Lyric Description"
      ]
    },
    {
      "id": "text_0",
      "text": "It's a recent pop song.",
      "labels": [
        "Genre",
        "Time Period / Recency"
      ]
    },
    {
      "id": "text_1",
      "text": "Maybe like 2 years old or something.",
      "labels": [
        "Uncertainty",
        "Time Period / Recency"
      ]
    },
    {
      "id": "text_2",
      "text": "Female singer.",
      "labels": [
        "Artist Description"
      ]
    },
    {
      "id": "text_3",
      "text": "I don't know the lyrics at all, but the basic gist of the song is about someone being at a club or party, and they're faking it, but in their mind they wish they were somewhere else.",
      "labels": [
        "Story/Lyric Description"
      ]
    },
    {
      "id": "text_4",
      "text": "I think the voice maybe sounds a bit like Halsey.",
      "labels": [
        "Vocals",
        "Relative Comparison",
        "Uncertainty"
      ]
    }
  ],
  "query_types": [
    "descriptive"
  ],
  "split": "train",
  "reformulations": {
    "keywords": "pop song, club but wanting",
    "reform_10": "Female pop song, Halsey-like, club/party, faking it.",
    "reform_25": "Pop song from the last two years, sung by a female artist with a Halsey-like voice, describing a person pretending to enjoy a club or party when they'd rather be somewhere else.",
    "reform_50": ".\n\nA recent pop song (2 years old or less) sung by a female artist, reminiscent of Halsey, about a person at a club or party faking enjoyment when they would rather be somewhere else.",
    "reform": "Recent pop song with female singer, sound like Halsey, about faking at a club or party."
  },
  "text": "It's a recent pop song. Maybe like 2 years old or something. Female singer.. . . I don't know the lyrics at all, but the basic gist of the song is about someone being at a club or party, and they're faking it, but in their mind they wish they were somewhere else.. . . I think the voice maybe sounds a bit like Halsey.",
  "raw_title": "[TOMT][song] pop song about being at the club but wanting to be somewhere else",
  "raw_text": "It's a recent pop song. Maybe like 2 years old or something. Female singer.\n\n\nI don't know the lyrics at all, but the basic gist of the song is about someone being at a club or party, and they're faking it, but in their mind they wish they were somewhere else.\n\n\nI think the voice maybe sounds a bit like Halsey.",
  "title": "pop song about being at the club but wanting to be somewhere else",
  "spotify_resolved": {
    "title": "Here",
    "items": [
      {
        "url": "https://open.spotify.com/track/5zUQZjVB6bfewBXWqsP9PY",
        "isrc": [
          "USUM71506251"
        ],
        "uri": "spotify:track:5zUQZjVB6bfewBXWqsP9PY",
        "album": {
          "name": "Know-It-All",
          "uri": "spotify:album:7HnbhIDKXIBhMR4EPGuMgu"
        },
        "artists": [
          {
            "name": "Alessia Cara",
            "uri": "spotify:artist:2wUjUUtkb5lvLKcGKsKqsR"
          }
        ]
      }
    ]
  }
}

About

When the Music Stops: Tip-of-the-Tongue Retrieval for Music

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published