This document contains guidance to help you avoid mistakes when rating Search Ads tasks.
The rating is a two-step process:
In line with Guideline 3.1, concerning Query Type Classification, it’s crucial to identify the user’s query type. The objective is to discern if the user is searching for a specific app or is open to any app that meets their requirements. The classification options are Navigational, Functional, Mixed, and Ambiguous.
Examples:
As stipulated by GL 4 concerning Ad Relevance, once the user’s query intent is deciphered, we assess how relevant the displayed ad is. The grading scale for this includes Excellent, Good, Acceptable, and Bad ratings.
An Excellent ad is one where the app displayed aligns closely with what the user is searching for. These are highly likely to engage the user’s interest.
A Good ad displays an app with a reasonable link to the user’s search query. While it might engage the user’s attention, there could be more suitable alternatives. In cases where the query is Navigational and the displayed app serves as a companion to the intended app, the rating will also be Good. This is because the app is related to what the user initially sought, although a direct competitor would be a more desirable match.
An Acceptable ad reveals an app with a tenuous relationship to the user’s query. The ad wouldn’t be out of place, but it’s unlikely to generate much interest.
A Bad ad is entirely unrelated to the user’s search query, leading to a jarring experience that could adversely affect user satisfaction. One specific case warranting a Bad rating is when the query does not imply adult content, but the displayed ad promotes an adult-only app.
Examples:
The user is specifically looking for the Coursera: Learn new skills app, which offers a broad range of educational fields. The advertised app, however, is focused solely on coding and its subgenres. Despite this limitation, it shares the broader educational intent with Coursera and is designed for self-learning, much like the intended app.
Given the high demand for remote coding education in Israel, the ad is still likely to resonate with a substantial user base. Therefore, its relevance should be rated as Good.
The user is targeting the Coursera: Learn new skills app, which is designed for educational purposes and caters to a broad demographic. The advertised app, while also related to education, takes a divergent approach by being primarily a gaming app that focuses on problem-solving tasks. Although it shares the overarching theme of education and is intended for a similar audience, it is less aligned with the user’s specific search for a comprehensive learning platform like Coursera. Nevertheless, it retains some relevance to the general learning intent and should, therefore, be rated as Acceptable.
The user is explicitly looking for the Gmail app by Google. The advertised app is Google Maps, which is also developed by Google. The sole commonality between the two apps lies in their developer, which means the ad is not entirely irrelevant. However, given that the user’s specific intent was to find an email application, the advertised map application is less likely to captivate their interest. Because there exists a connection, albeit a tenuous one, between the intended app and the advertised app through their common developer, the relevance of the ad should be rated as Acceptable.
The user is searching for ‘photos,’ which is a vague term that could be fulfilled by a variety of apps in different ways. The app advertised is a duplicate cleaner that has the feature of cleaning duplicate photos, among other types of files. Importantly, this feature is highlighted in the description line below the app’s name, ensuring that the user is not surprised or displeased when they see the ad. If this specific feature were not prominently advertised in the short description, the relevance rating would have to be Bad, as the app would then appear largely irrelevant to the user’s query. Given the current presentation, the relevance should be rated as Acceptable.
The query contains a misspelling, yet it’s evident that the user is seeking the Adobe Lightroom app, known for its comprehensive features in photo editing, organization, storage, and sharing. The advertised app is a face-changing application that also includes some basic photo editing functionalities. While the user may have some level of interest in personalizing their photos, the advertised app is not an exact match for their likely intent. It is targeted at a different audience and offers fewer features compared to Adobe Lightroom. Therefore, the relevance of this advertised app should be rated as Acceptable.
The query ‘cards’ is open to multiple interpretations, creating a Functional query type. The most prevalent intent would likely be related to various types of card games. Another common intent could involve the supernatural aspect of card reading for fortune-telling. The advertised app aligns with this latter intent by using card equivalents to fulfill a similar purpose. Given this, and in accordance with Guideline 3 on Multiple Interpretations, the app’s relevance should be rated as Good. It’s worth noting that if the app were directly focused on using cards for fortune-telling, its rating would be promoted to Excellent
Query
Ad
The user is most likely searching for Final Cut Pro, a professional post-production software. While traditionally a Mac-only application, it’s worth mentioning that an iPad version has been introduced.
The advertised app, although within the realm of video editing, targets a different, usually younger, non-professional audience. Due to this mismatch with the user’s specific intent, the relevance rating should be Bad.
The user is specifically searching for the Call of Duty: Mobile app, which is known for its battle and strategy gameplay. The advertised app also centers around a battlestrategy theme but offers a different gaming experience and thematic elements. According to the guidelines, specifically section 4.3 pertaining to game queries, this result qualifies as Good. It is close enough to the intended play style and presentation to be a logical and potentially appealing option for the user, even though it isn’t a direct match to the original query.
The query is ‘Reminders,’ which falls under the Mixed category as it can refer to both a specific app (the default iPhone reminders app) and a general functional attribute related to scheduling and reminders. The advertised app is Calendar by Fantastical, which is a multi-functional calendar application that also includes advanced reminder functionalities. Despite its multi-functional nature, this app directly addresses both the specific and functional aspects of the user’s query. Therefore, its relevance should be rated as Excellent.
The query employs Hebrew slang to refer to WhatsApp Messenger, a universally recognized chatting application. Given the specific nature of the slang, this is a Navigational query. The advertised app is also in the chatting category but belongs to the ‘chat with strangers’ genre and is explicitly adult-only. Upon reviewing the app’s page, it’s evident that the age rating is 17+ and the content is sexual in nature. Since the original query does not suggest any interest in adult content, the ad is not aligned with the user’s likely intent and therefore should be rated as Bad.
The query is the Hebrew spelling of the game Fortnite and is, therefore, Navigational. The ad is for Clash Royale, another online multiplayer game. However, it is a strategy game that differs from the battle royale style of Fortnite. Although it might appeal to a similar user base interested in online multiplayer games, it is not a direct substitute for the queried game. Therefore, ratings of Acceptable and Good are accepted.
This query could be classified as Mixed according to the guidelines. “Suit” could refer to a playing card suit or a set of matching clothes. A user searching for apps related to card games might be interested in an app that helps visualize different suits, while a user interested in fashion could be looking for an app that suggests suites to buy or, to a lesser extent, helps them try on different suits. In this case, the result, a men’s suit photo editor, is relevant to one of the possible interpretations of the query. However, since it does not relate to the core query interpretation, we rate it as Acceptable.
The query “photoshop” is a navigational query seeking Adobe Photoshop, a widely known app for photo editing and graphic design. The result ad offers Procreate Pocket, which is an app for sketching and painting on iPhones. While Procreate Pocket is not directly related to photo editing like Photoshop, it is still within the creative and design realm. Both apps cater to users interested in graphic design and digital art, though the functionalities differ. According to the guidelines, since Procreate Pocket appeals to a similar user base and provides relevant creative functionalities, but does not directly match the user’s specific search intent for Photoshop, this ad should be rated as Acceptable.
The query “דיסני” (Disney) is a mixed query, potentially seeking either the Disney company, Disney-themed apps, or a specific Disney-related app. The result ad offers “Kung Fu Street Fighting Games,” which is a game focused on martial arts and fighting. This result does not align with the user’s potential intent to find Disney- related content, which would typically include family-friendly themes, characters, or entertainment associated with Disney.
According to the guidelines, a relevant app ad should match the themes and user interests implied by the query. In this case, the Kung Fu Street Fighting Games app does not share any thematic or functional connection with Disney content. It lacks the family-friendly, entertainment-focused nature typical of Disney-related queries, making the ad jarringly irrelevant to the user’s likely intent. Therefore, this ad should be rated as Bad due to its lack of relevance and potential to negatively impact the user’s search experience.