We have compiled the following information to help analysts get to speed with this task type quickly. We find many analysts who are new to the task type are unsure of how to demote results. We hope you will find this helpful.
Book results
- When the result is the same book in a different edition, it should be graded Satisfying.
- When the result is the same book in a different language, the grade can’t be higher than Moderately Satisfying.
- When the result is a different book from the same series as the one in the image, it should be
graded Moderately Satisfying.
Visually similar objects
- When the input image and result are for different objects it’s easy to rate the result Not Satisfying. If they are visually similar you can rate it as high as Moderately Satisfying.
- Per the Moderately Satisfying section of the guidelines, some users would be satisfied with the result. It looks visually similar to the primary object but not the same thing.
- For example, the Input image is the Akita but the result is the Korean Jindo. These breeds are different, but they are visually similar. Other visually similar object examples are books in the same series, different versions of the same song, similarly looking bridges (the Williamsburg
Bridge and the Manhattan Bridge look similar).
Note: Map Results are not expected to contain an image or link. If the map result contains the correct address information that the user is looking for this is
considered Highly Satisfying.
Grouped Result
For grouped results, look at the similarity/usefulness of the images together. The image search grid has up to 8 images. To rate correctly you need to click on each of the thumbnail images and think about how
satisfying the user would be with this result. When you have reviewed all of the images you need to apply an overall rating.
- If all images are related to the input image rate Highly Satisfying (HS).
- If most of the images are related to the input image, more than half, rate Satisfying (S).
- If some of the images are related to the input image, about half, rate Moderately Satisfying (MS).
- If a few of the images are related to the input image, less than half, rate Slightly Satisfying (SS).
- If none of the images are related to the input image, rate Not Satisfying (NS).
Incorrect or missing research link
The research link is used to familiarize ourselves with the input image. Sometimes the research links are incorrect or missing which can lead to incorrect ratings. Clicking the input image (Note this only works if you click outside the red border) and selecting ‘Search Google for Image’ might return images which could help identify the object.