Skip to content

Latest commit

 

History

History
29 lines (16 loc) · 1008 Bytes

Evalaution_metrics.md

File metadata and controls

29 lines (16 loc) · 1008 Bytes

Dataset Labeling and Query Preparation

Image Annotation

Annotate each image in the dataset with relevant metadata such as make, model, year, color, body type, and any other pertinent attributes.

Query Preparation

Query Pool

Create a diverse set of queries covering various aspects of cars, including make, model, color, condition, environment, and activity.

Query Difficulty

Categorize queries based on their difficulty level:

  • Easy: Straightforward queries
  • Moderate: Queries requiring some understanding of car attributes
  • Hard: Queries involving abstract or subjective concepts

Evaluation Procedure

Baseline Comparison

Compare the performance of the text-to-image search system against a baseline method, such as simple keyword matching or random retrieval, to establish a reference point.

Metrics

Use standard retrieval evaluation metrics such as Precision, Recall, and F1-score to quantify the system's performance across different queries and difficulty levels.