Welcome to the CAMEO pilot task, an initiative exploring cutting-edge challenges in multimodal product search for e-commerce. As online shopping continues to evolve into a complex information-seeking environment, traditional search methods often fall short.
CAMEO addresses these limitations by introducing two innovative and complementary subtasks: Composed Image Retrieval and Review-Based Question Answering.
About CAMEO
CAMEO is designed to tackle real-world challenges faced by online shoppers and aims to advance research in information retrieval, question answering, and multimodal understanding. Our focus includes supporting Vietnamese, a low-resource language, to promote language equity and develop tools that reflect global linguistic and cultural diversity, particularly within the growing Southeast Asian e-commerce market.
Our Subtasks
1. Composed Image Retrieval (CIR)
This subtask challenges participants to retrieve visually altered product variants by combining a reference product image with a textual modification. For example, a user might provide an image of a product and add "same type, but a larger size" to find suitable alternatives.
- Task: Given a product image and a textual modification, return a ranked list of product images that match the revised intent.
- Evaluation Metrics: Precision, Recall, and mean Average Precision (mAP) at K. Optional triplet loss-based metrics will also be considered.
- Example: "I want the same product, but made in Vietnam".
2. Review-Based Question Answering (QA)
This subtask focuses on extracting or generating answers to natural-language questions about products from customer reviews or product attributes. Unlike typical single-review datasets, this task encourages aggregating evidence across multiple user reviews, mirroring how consumers form opinions in real life.
- Task: Given a product ID or image and a user question, return a two-part answer: a short natural language answer to the question, and a list of supporting evidence sentences extracted from one or more product reviews.
- Evaluation Metrics:
- Short Answer: Exact Match (EM) and F1 score against manually curated reference answers.
- Evidence List: Sentence-level precision, recall, and F1, based on gold-standard evidence annotations.
- Example: "Is this suitable for dry skin?".
Dataset: ViEcom-Rec
Both subtasks leverage the ViEcom-Rec dataset, a rich resource containing images, metadata, and over 369,000 Vietnamese-language product reviews from a real e-commerce platform.
- Scope: The dataset primarily includes products in the category of face cleansers.
- Data Details: 369,099 reviews from 304,708 users for 2,244 distinct products.
- Publicly Available: Released under CC BY-NC-SA 4.0 license on GitHub.
- Language Accessibility: Reviews with high-quality English translations.
To view our dataset, please visit our Data Explorer page.
The dataset is also available for download on Huggingface: https://huggingface.co/datasets/allietran/CAMEO.
Timeline (2025–2026)
- July 10, 2026 (AoE): Registration deadline.
- July 20, 2026: Submission system closes (final runs due).
- August 1, 2026: Results and analysis released.
- September 2026: Paper drafting and internal reviews.
- October 2026: Task paper submissions (to NTCIR proceedings).
- December 2026: CAMEO Workshop at NTCIR-19 Conference.
How to Participate
Please register your team through the NTCIR-19 registration system here https://research.nii.ac.jp/ntcir/ntcir-19/howto.html.
Example of task questions and expected outputs for both subtasks will be provided in the coming weeks.
More details on participation guidelines, submission formats, and evaluation procedures will be provided soon. Stay tuned for updates!
Organizing Team
- Dr. Allie Tran, Dublin City University, allie.tran@dcu.ie
- Quang-Linh Tran, Dublin City University, linh.tran3@mail.dcu.ie
- Hoang-Bao Le, Dublin City University, bao.le2@mail.dcu.ie
- Prof. Binh T. Nguyen, University of Science, HCM, ngtbinh@hcmus.edu.vn
- Dr. Anton Louise P. De Ocampo, Batangas State University antonlouise.deocampo@g.batstate-u.edu.ph
Join Us!
We invite researchers in information retrieval, question answering, and multimodal understanding to participate in the CAMEO pilot task. This is an exciting opportunity to contribute to the advancement of e-commerce search and address critical challenges in multimodal information access.
For more information and to get involved, please stay tuned for updates on dataset release and participation guidelines.
Contact: allie.tran@dcu.ie