Welcome to the CAMEO pilot task, an initiative exploring cutting-edge challenges in multimodal product search for e-commerce. As online shopping continues to evolve into a complex information-seeking environment, traditional search methods often fall short.
CAMEO addresses these limitations by introducing two innovative and complementary subtasks: Composed Image Retrieval and Review-Based Question Answering.
About CAMEO
CAMEO is designed to tackle real-world challenges faced by online shoppers and aims to advance research in information retrieval, question answering, and multimodal understanding. Our focus includes supporting Vietnamese, a low-resource language, to promote language equity and develop tools that reflect global linguistic and cultural diversity, particularly within the growing Southeast Asian e-commerce market.
Our Subtasks
1. Composed Image Retrieval (CIR)
This subtask challenges participants to retrieve visually altered product variants by combining a reference product image with a textual modification. For example, a user might provide an image of a product and add "same type, but a larger size" to find suitable alternatives.
- Task: Given a product image and a textual modification, return a ranked list of product images that match the revised intent.
- Evaluation Metrics: Precision, Recall, and mean Average Precision (mAP) at K. Optional triplet loss-based metrics will also be considered.
- Example: "I want the same product, but made in Vietnam".
2. Review-Based Question Answering (QA)
This subtask focuses on extracting or generating answers to natural-language questions about products from customer reviews or product attributes. Unlike typical single-review datasets, this task encourages aggregating evidence across multiple user reviews, mirroring how consumers form opinions in real life.
- Task: Given a product ID or image and a user question, return a two-part answer: a short natural language answer to the question, and a list of supporting evidence sentences extracted from one or more product reviews.
- Evaluation Metrics:
- Short Answer: Exact Match (EM) and F1 score against manually curated reference answers.
- Evidence List: Sentence-level precision, recall, and F1, based on gold-standard evidence annotations.
- Example: "Is this suitable for dry skin?".
Dataset: ViEcom-Rec
Both subtasks leverage the ViEcom-Rec dataset, a rich resource containing images, metadata, and over 369,000 Vietnamese-language product reviews from a real e-commerce platform.
- Current Scope: The dataset primarily includes products in the category of face cleansers.
- Planned Expansion: Products in cosmetics such as sun cream, lipsticks, and face washers.
- Data Details: 369,099 reviews from 304,708 users for 2,244 distinct products.
- Data Collection: Crawled using Python scrapers (BeautifulSoup and Selenium).
- Key Attributes: Nine extracted by GPT-family models (item name, ingredient, product feature, skin type, capacity, design, brand, expiry, origin).
- Publicly Available: Released under CC BY-NC-SA 4.0 license on GitHub.
- Language Accessibility: Reviews with high-quality English translations; scripts and starter code in English.
Timeline (2025–2026)
- October 2025: Dataset preparation and triplet annotation.
- January 2026: Release of training data and baseline.
- June – September 2026: Submissions system online.
- September 2026: Results and analysis.
- October 2026: Paper submissions.
- December 2026: Workshop at NTCIR-19 Conference.
How to Participate
More details on participation guidelines, submission formats, and evaluation procedures will be provided soon. Stay tuned for updates!
In the meantime, if you are interested in participating in the CAMEO pilot task, please register your interest by filling out the form linked below. This will help us keep you informed about important updates and announcements related to the task.
Register Your InterestOrganizing Team
- Dr. Allie Tran, Dublin City University, allie.tran@dcu.ie
- Quang-Linh Tran, Dublin City University, linh.tran3@mail.dcu.ie
- Hoang-Bao Le, Dublin City University, bao.le2@mail.dcu.ie
- Prof.m Binh T. Nguyen, University of Science, HCM, ngtbinh@hcmus.edu.vn
- Dr. Anton Louise P. De Ocampo, Batangas State University antonlouise.deocampo@g.batstate-u.edu.ph
Join Us!
We invite researchers in information retrieval, question answering, and multimodal understanding to participate in the CAMEO pilot task. This is an exciting opportunity to contribute to the advancement of e-commerce search and address critical challenges in multimodal information access.
For more information and to get involved, please stay tuned for updates on dataset release and participation guidelines.
Contact: allie.tran@dcu.ie