1. Introducion

Welcome to Autocomplete Evaluation. The Autocomplete feature is intended to save users time by showing them relevant content as they type. These guidelines will teach you how to use the evaluation tool and rate Autocomplete suggestions.

For each task, you will be presented with an autocomplete query and the suggestions that are provided to Maps users. The goals of this task are:

•Autocomplete and Autocomplete Relevance: To determine how well the suggestions satisfy the user’s query (relevance)

•Autocomplete only: To check the data accuracy (name and address) of the suggestion

You must:

•Determine how well each suggestion satisfies the user’s potential intent (suggestion relevance)

•Check the data accuracy of the suggestions (name and address) ^oPin accuracy data is not evaluated in this task

Since the user may not have completed the query, the query strings in this task will not always have a clear user intent. As the user continues to type the query the possible intent may become more clear or even change.

1.1. Rating Tool

The interface used in Autocomplete and Autocomplete Relevance is similar to the one used in other TryRating evaluations. This section highlights the differences. Please note that the term classification and category are used interchangeably going forward.

1.2. Rating Interface

The ratings and comments for each suggestion will be collected using the checkboxes and drop-downs. You may be asked to just provide the Relevance or Name/Address rating. This is expected and not a reason to escalate or release a task.

1.2.1.Sugges+on-Level Ra+ng Checkboxes

For each suggestion, you will be presented with two high-level questions. These will allow you to highlight any problems at the suggestion level.

High-level question	Explanation	Notes
Result name/title is in unexpected language or script	Suggestion is not in the language of the query, the test locale, or the suggestion region (the expected languages) or is in a mix of expected and unexpected languages (unless it is an official company, chain, or brand name).	If you check this box, your rating for this query will be complete and all other questions will become unavailable.
High-level question	Explanation	Notes
Business/POI is closed or does not exist	Research shows that the business is closed or that no such business exists at the address listed in the suggestion.	If you check this box, you will still rate the relevance of the suggestion as if it did exist. All other questions will become unavailable, and your rating will be complete. Use your best judgement to determine whether the suggestion is closed, has never existed or if the suggestion component(s) are merely incorrect. See Unclear Suggestions for further guidance.

1.2.2.Suggestinon Relevance Rating

The relevance rating will take into consideration user intent as well as distance/prominence. Relevance is always rated independently of any data (name, category or address) inaccuracies. This means that when rating relevance, we always assume that the suggestion exists (even if research reveals the location is closed) and that the data presented is correct.

Always rate against the real world: If there is a better result available but it is not shown, demote the existing result(s) considering the missing one(s).

Relevance Rating	Explanation	Notes
Excellent	A high-quality suggestion that clearly satisfies the user’s intent. Multiple suggestions can be considered high quality.	The best possible suggestions.
Good	Suggestions that could satisfy the user’s intent but are not among the best possible choices.	Any relevance rating of Good or below will require an indication of the reason(s) for the demotion: •User Intent Issue •Distance/Prominence Issue
Acceptable	Suggestions that technically satisfy the user’s intent but do so poorly due to lack of relevance or because they are farther away.
Bad	Suggestions that do not satisfy the user’s intent due to lack of relevance or great distance.

1.2.3.Suggestion Relevance Demotion Checkboxes

If you give a suggestion a relevance rating of Good or below, you will be presented with two checkboxes. Use them to select the reason(s) for your rating: ..

•User Intent issue

•Distance/Prominence issue

Checkbox	Explanation	Notes.
User Intent issue	Select when the demotion is due to a suggestion only partially fulfilling the query intent.	If both issues were reasons for your relevance rating, select both checkboxes.
Distance/Prominence issue	Select when the demotion is due to a distance issue based on the user or viewport location or when you demote for a suggestion that is less prominent based on the initial query intent.

1.2.4.Data Accuracy

After you’ve rated Relevance, you will evaluate two suggestion components for data accuracy:

•Name and Category Accuracy

•Address Accuracy

The data accuracy rating process will define the expected quality and any appropriate demotion(s) at each step.

Note: Pin Accuracy will not be evaluated in this task. Pins are shown only to indicate the approximate location of the suggestions.

1.2.5.Comments

Comments are mandatory for any relevance rating of Good or below as well as for any Name and

Address Accuracy issues not rated Correct and should explain the reasoning behind your rating. Be sure to:

•Include user intent: It’s useful to indicate what you think the user intent is. In Autocomplete the query is sometimes too short to indicate the precise intent.

•Refer to the guidelines: If the demotion is due to a guideline instruction, note the guideline section that guided your decision

•Include sources: If you find incorrect information in a suggestion, your comment should include the correct information and its source. Be sure that any links you include lead directly to the information you provide. If they don’t, include a link to the source you used and explain how to get to the information from there.

^oUse URL shorteners when inserting long links into your comments.

•Be concise: Don’t write elaborate comments. Be short and specific.

•Comment in English: Your comments will sometimes be reviewed or used by people who are not familiar with your test locale or its language. Comments in English are understandable to the widest possible audience.

Issue Identified

Expectations

Example Rating and Comment

Relevance Demotions

The comment should explain the reasoning behind your rating. For example, a demotion to a suggestion may be due to distance from the user’s location compared to similar suggestions/

potential suggestions, to secondary intent, and so on.

Relevance -> Good (Distance/

Prominence Issue)

Comment: Demoted -1 for a few

closer real world locations in the

Middletown/Anchorage area

Name and Address Issues

It is important to include a reason when rating Name or Address something other than Correct. This also applies to Business closed/does not exist.

Address Accuracy -> Incorrect

Comment: Correct street number: 332 W El Camino Real, Sunnyvale, CA 94087, see official webpage www.starbucks.com

2. Understanding User Intent

Before evaluating the suggestions, you will need to understand what the user wanted to find when they made their query. To determine user intent, use a combination of your own research and local knowledge plus the information found in the rating interface:

•User location

•User viewport

•Viewport age

User intent refers to the interaction between query, user location, user viewport, and viewport age.

2.1. User Query String

Autocomplete presents suggestions as a user is typing a query. Since the user may not have completed the query, the query strings in this task will not always have a clear user intent. As the user continues to type the query the possible intent may become more clear or even change.

Because Autocomplete works before the user is finished typing, some queries may be relatively complete, while others will be short and ambiguous. Many of the queries you’ll see will be incomplete and contain only parts of words or addresses. These queries are referred to as query strings. Identify the significant part of the query string and factor in all possible intentions:

Query String	Possible Suggestions	Notes
[starbu]	starbucks	It is likely the user is looking for the nearest location of the widespread coffee chain.
[7 e]	7 eleven	Intention could be to find the nearest location of this convenience store chain.
	7 east st	Intention could be to find a specific street.
	7 e 5th street	Intention could be to find a specific address.
[targ]	Target	Intention could be to find the nearest location of a well-known chain store.

2.1.1.Research Expectation

Research will help you understand the user’s query and intent:

•Use a search engine to investigate and understand the query intent

•Use official resources like businesses’ official websites, national postal service websites, and government websites, whenever possible

•Use your own local knowledge

•Use the information provided in the rating interface

2.2. Suggestion Types

Each query string will generate results in the form of autocompleted suggestions, which are presented as one of four suggestion types that include name/title and address details:

•Address suggestions

•Business/POI suggestions

•Query suggestions

•Category suggestions

The suggestion type is displayed in the rating tool and is used to pre-populate the n/a fields in Name and Address Accuracy. If this field is populated incorrectly, change the n/a rating to the appropriate Name and Address Accuracy of the suggestion.

The suggestion type will help you better understand what the suggestion is. For example, train station suggestions and locality suggestions often have similar names, but different levels of prominence.

If suggestion types are missing from the tool, use the following explanations and your best judgment to identify them.

Note that category and classification are used interchangeably throughout the guidelines.

2.2.1.Wrong Business/Address Type Displayed

In some situations an incorrect type will be displayed for a suggestion. When the type is incorrect, the suggestion should be rated as if it were the corrected suggestion type.

If a field is pre-populated incorrectly due to a wrong suggestion Type, change the ’n/a‘ rating to the appropriate rating.

For example, Imperial College London is a university, not an address, and should be Type: BUSINESS instead of Type: ADDRESS.

Because the Name and Category Accuracy has been pre-populated with

’n/a’, and Business/POI suggestions require a Name and Category

Accuracy rating, the ’n/a‘ should be corrected to the appropriate rating.

2.2.2.Query and Category Sugges+ons

Query and category suggestions are completions based on user queries that provide multiple search results when selected.

Query and category suggestions can be identified by their type and the fact that they don’t have coordinates or a pin shown on the map.

Both query and category suggestions use the rating criteria of suggestion expectation and search performance when rating relevance (see below for more details).

The Address Accuracy field for query and category suggestions is always rated n/a if not auto-populated. In these cases, only Name and Category Accuracy should be rated.

Suggestion Expectations

The purpose of a query or category suggestion is to support exploration of the map and its content by suggesting categories or groups of places and items. The only exception where a singular location is acceptable is for query suggestions that provide an alternate name for a query like “red bridge” for the Golden Gate Bridge. Rate the relevance Bad if a query or category suggestion does not fulfill these expectations.

Category type suggestions offer a search based on different categories of businesses and POIs, such as “Coffee & Tea” or “Pet Service.” No list of categories or category definitions will be provided and the suggestions should be rated using local knowledge.

Query type suggestions can be presented in multiple forms including:

•Category

•Chain Business

•Brands

•Products and services

•Official name of a place for a user searching with an alternate name

•Refined suggestions using location modifiers

•Attribute Searches like [restaurants open now]

Both of these types should NOT be used instead of an actual result that could be returned unless the query is searching with an alternate term. Examples of wrong usage of query or category suggestions are:

•Any unique location in the world (Rate Bad for relevance)

•Single POI or business (Rate Bad for relevance)

•City, street or address (Rate Bad for relevance)

Any query or category suggestion for a single place or location where the user query isn’t using an alternate name must be rated Bad for relevance.

It is possible for the suggestion to be identical to the query. This happens when the user continues typing instead of selecting the suggestion early on. No demotion is applied in such instances.

Location Modifier

Sometimes the query or category suggestion has a location modifier that appears on the second line, often triggered by the user query. It will provide a more narrow search around that point of reference and could be:

•Search Nearby – Provides results closest to the location intent (fresh viewport or user if stale)

•Address Components – For example the suggestion “cafe – Green Valley, SD” returns businesses within the neighborhood.

•Landmarks and POIs – Like in the example “restaurant – Jump Caze” the returned restaurants are close to the POI.

Search Performance

The search performance describes the usefulness of the suggestion when searched for in a flawless maps search engine. For it to be useful, it must provide results that are reasonably close and make sense for both the context of the query and the market/language region.

The query [papaya] returns a query suggestion for Papaya Clothing. When selected, it returns multiple locations around the user. Rate the relevance of the suggestion by comparing its closest result against all possible suggestions.

It is recommended to briefly research possible results before applying local knowledge to select the best/ closest result before arriving at the appropriate relevance rating.

2.3 Location Intent

A suggestion is considered most relevant when it is located in the area where the user expects to see suggestions. There are two types of Location Intent:

•Explicit: The query indicates a specific location or area where suggestions are expected.

•Implicit: The location expectation is not given by the query itself, so you must use context clues, in the form of User Location and viewport, to determine the area in which the user is expecting suggestions.

2.3.1 Explicit Loca+on

When there is an explicit location stated in the query, the user and viewport locations can be ignored because the user has stated exactly where they want to see suggestions. Explicit locations can include specific addresses or location modifiers, such as neighborhoods and localities.

Query	Possible Suggestions	Notes
[Boston museu]	Museums in the city of Boston	User and viewport locations should be ignored.
[kfc Phila]	The queried chain business within the stated locality.	User and viewport locations should be ignored.
[food near me]	Relevant POIs near the user’s location	Viewport location should be disregarded, even if fresh.
[nearby/nearest starbucks]	Relevant POIs near the user’s location	Viewport location should be disregarded, even if fresh.
[my location]	Suggestion placed on the user’s location	Viewport location should be disregarded, even if fresh.

Rate explicit location intents differently depending on how many possible suggestions exist within the requested area.

Query	Possible Suggestions	Notes
[Costco san francis]	One suggestion within the requested area	As there is only one Costco within San Francisco, the next closest Costcos outside the explicit location are still considered Good. The demotions are applied for distance gradually.
[IKEA san franc]	No suggestions within the requested area	Where no suggestions exists within the explicit area, like in the IKEA example, the closest one receives Excellent. Gradual demotions are applied for locations further out, like the one in Palo Alto.

Query	Possible Suggestions	Notes
[Starbucks san franc]	More than 100 cafes within the requested area	Because of the high number of Starbucks within the explicit area, any suggestion outside is considered too far and is rated Bad.

2.3.2 Implicit Loca+on

If the user’s location intent is not expressly stated in the query, for example [chinese fo] or [479 margarita a], use the user location, Viewport location, and viewport age to determine the area of expected suggestions.

Viewport	User	Location Intent
Fresh	Inside Viewport	When the user is within the fresh viewport, take the user location as location intent. Suggestions are generally expected in or near the viewport and suggestions inside the area cannot be rated Bad for distance alone.
	Outside Viewport	Suggestions are expected in or near the viewport area. All relevant suggestions inside the viewport are eligible for a rating of Excellent. If no suggestions can be found in or near the viewport, use the user location as a secondary location intent.
	Missing	When the user is missing, the viewport remains the location intent.
Stale	Inside Viewport	When the viewport is stale, consider only the user location as location intent.
Stale	Outside Viewport
.	Missing	Use the stale viewport as location intent when the user is missing.
Viewport Age Missing	Present or Missing	Consider the viewport fresh when the viewport age is missing.
Missing entirely	Present	The user location sets the location intent when the viewport is not present.
Missing entirely	Missing	When the user and viewport are missing, the test locale becomes location intent with a strong focus on prominent suggestions.

3. Rating ‘Suggestion Level’ Issues

For each suggestion, you must check to see whether:

•There is an issue with the language/script of the suggestion name/title

•The business/POI is closed or does not exist

3.1.Suggestion Name/Title in Unexpected Language or Script

A suggestion’s name or title is in an expected language or script when it is:

•In the language/script of the test locale

•In the language/script of the query

•In a language/script of the suggestion’s region

•Any combination of any of the languages or scripts described above

•An official company, chain, or brand name commonly used in the market, even if this is not any of the languages or scripts described above

When all or part of a suggestion’s name or title is in an unexpected language or script, users will not understand it. In these cases, select the Result name/title is in unexpected language or script checkbox. No further rating is required.

Note: When the address details, found below the name/title, are in an unexpected language or script, the suggestion can still be understood and rated for Relevance and Name and Category Accuracy. For more on what to do when the address details are in an unexpected language or script, see Language/Script Issue in Address.

Minor Differences in Spelling

Don’t consider minor spelling issues, such as missing or added diacritics (e instead of é), as

Unexpected Language/Script. This also applies to unnecessary or less relevant parts of a name or title. Use this checkbox only to report issues that interfere with understanding and those listed in your Country Specific Guidelines.

Expected Language

Explanation

Example

Query

The query gives clues as to what language the user is using. Any suggestion that matches the most specific part of the query or corrects a spelling issue in the query is considered to be a suggestion in an expected language. This also includes remote suggestions in foreign countries.

If the query is so short that the language cannot be identified, assume that the query language is that of the test locale.

But: Consider official suggestion name variations, and default to the test locale for very short queries.

Expected language/script:

•Test locale: en_US

•Suggestion in California, USA

•Query: [mystery]

•suggestion name: Mystery Spot Suggestion name is in English and satisfies the query.

Unexpected language/script:

•Test locale: en_US

•Suggestion in California, USA

•Query: [market]

•Suggestion name: ေစျးကွက်

The suggestion name is “Market” in Burmese script.

Expected Language	Explanation	Example
Test Locale and Query Locale	The language of the test locale is generally considered an expected language and script. .If the query is so short that the language cannot be identified, assume that the query language is that of the test locale. Note: When the test locale and the query locale differ, consider the query locale an expected language, as it provides more detailed information. For example, the query locale below is set to English in Indonesia. Therefore English, not Indonesian, is an expected language: •Query Locale: en_ID_ID •Test Locale: id_ID	Expected language/script: •Test locale: es_ES •Suggestion in California, USA •Query: [college] •Suggestion name: Colegio Skyline The suggestion name is in Spanish, which matches the language of the test locale, so this suggestion title is expected and acceptable. Unexpected language/script: •Test locale: en_US •Suggestion in California, USA •Query: [college] •Suggestion name: Colegio Skyline “Colegio” is in Spanish, but the test locale is in English, so the suggestion title is in an unexpected language.
Suggestion region	The language/script of the suggestion’s region is expected as the suggestion language.	Expected language/script: •Test locale: en_US •Suggestion in Rio de Janeiro, BR •Query: [beach] •Suggestion name: Praia de Copacabana Suggestion satisfies the category “beach” and the suggestion’s name is in Latin script and the Portuguese language, which is the language of Brazil. Unexpected language/script: •Test locale: en_US •Suggestion in Rio, BR •Query: [beach] •Suggestion name: Playa de Copacabana Though the suggestion name is in the same script as the test locale and query language, returning the name in Spanish is unexpected language because the language of Brazil is Portuguese.
Expected Language	Explanation	Example
Brand, chain, and business names	. Any official brand, chain, or business name in the language of the suggestion’s region or the official brand name commonly used in the market, even if it is not in the language of the query, test locale, or suggestion region, is expected. In bilingual areas, official names in any of the languages of the area are expected, unless otherwise specified in the Country Specific Guidelines.	Expected language/script: •Test locale: ar_SA •Suggestion in Riyadh, SA •Query: [زو] •Suggestion name: McDonald’s Suggestion name for this restaurant is in the language of the official brand name. Expected language/script: •Test locale: ar_SA •Suggestion in Riyadh, SA •Query: [زو] •Suggestion name: [زو] Suggestion name for this restaurant is in the language of the region. Unexpected language/script: •Test locale: ar_SA •Suggestion in Riyadh, SA •Query: [زو] •Suggestion name: [マクドナルド] The name of this restaurant is in Japanese, which is not the language of the query, the locale, the suggestion region, or the official brand name.

3.1.1. Query Sugges+on Language

A query suggestion has to be reasonable for the country (local language) and useful within the context.

Query	Test Locale	Suggestion	Rating	Explanation
[gr]	de_DE	Groceries	Result name/title is in unexpected language or script	Not a reasonable suggestion for the German market as this is an English word not used in Germany.
[aub]	...fr_FR	Hostels	Result name/title is in unexpected language or script	..Not a reasonable suggestion for the French market as a user here would expect to see “auberges” as the autocompletion of this query and not an English translation of it.

3.2. Business/POI is Closed or Does Not Exist

Use this checkbox after careful research, when you determine that a suggestion is:

•Closed/non-existent

•Randomly moving (like a food truck without a fixed schedule)

•A non-recurring past event with no further significance that does not carry any map intent

If you rate the suggestion as Business/POI is closed or does not exist, you must still give a relevance rating as if the place were open or did exist.

Remember, this rating can only be applied to business or POI suggestions, not to address type suggestions.

Use this checkbox to indicate a POI has closed only if there is evidence to prove the business indicated in the suggestion is no longer operational.

When you can’t make a decision about business closure due to lack of resources, assume the business could exist and rate the name and address accuracy as Can’t Verify (unless there is an obvious data issue like a missing mandatory address component).

Businesses and POIs can close for construction, remodeling, vacations, and other reasons. Treat all temporarily closed businesses as open when the closure is announced on the business/POI webpage or managed social media page. There is no limit to how long the closure can be.

3.2.1.Closed/Does Not Exist vs. Inaccurate Name and Address

It is important to understand what the suggestion really is prior to assigning your rating. If your research reveals that the business has an incorrect name and address, use the drop-down rating options to signal what precisely is wrong. Use the Business/POI is closed or does not exist checkbox only for the specified purposes. See “Unclear Suggestions” section for further guidance and examples.

3.2.2. Ra+ng Relevance of Closed/Non-existent Business/POI

Special consideration is required when rating a closed/non-existing business/POI. Please review section 3.3 carefully with regards to the status and permanently closed vs. open.

Never automatically demote a result to Bad.

3.3.Business/POI Status is PERMANENT_CLOSURE

The following chapter offers a high-level introduction to rating closed/non-existing businesses/POIs in relation to their status (displayed as part of the result information). It addresses Relevance and Data rating. Please also review the more specific guidance in the associated relevance chapter.

No matter what is displayed in the status section of the tool, you must always research the business/POI to determine whether it is open or not.

This chart shows the possible combinations of status displayed in TryRating plus a result’s actual state based on your research.

Real-World State Confirmed by Research	TryRating Status	Comments
Open/Exists	None	Rate as usual
Closed/Does not Exist	None	Rate as usual
Closed/Does not Exist	PERMANENT_CLOSURE	Rate relevance based on whether result is expected/ unexpected. See 4.16.
Open/Exists	PERMANENT_CLOSURE	Rate relevance based on whether result is expected/ unexpected. See 4.16.

3.3.1. No Status Shown

No result status shown or the status is blank. Research to determine whether the business/POI is open or closed/does not exist in the real world and then use the Business/POI is closed or does not exist checkbox when needed.

Rate the result relevance as if the business/POI were open or did exist. Use your best judgement to determine how well the result satisfies user intent within the area of expected results.

3.3.2.Status shown as PERMANENT_CLOSURE

When the result status is PERMANENT_CLOSURE, follow the research steps to determine if the business in fact is open in real world or not, and use the Business/POI is closed or does not exist checkbox when needed.

..The relevance rating for a result with PERMANENT_CLOSURE status depends on whether this result is expected or unexpected based on the specific relevance guidance in 4.16. Rating Relevance when

Status is set to PERMANENT_CLOSURE and 8.5. Rating Results with PERMANENT_CLOSURE Status

4. Relevance

Relevance for each suggestion is found by assessing:

•Matching Suggestion

•User Intent

•Distance/Prominence

Complete each step of this workflow before assigning your Relevance rating.

General Rating Rules

•Rate each suggestion individually and demote based on relevance and distance.

•Always rate against the real world: If there is a better suggestion but it is not shown, demote the existing suggestion(s) in relation to the missing one(s).

•Ignore the order of the suggestions and evaluate each one individually.

•Ignore duplicate suggestions and rate them individually. Do not demote for repetition.

•If there are suggestions that are extremely inappropriate or illegal, rate Bad. Identify such content by determining if the suggestion is likely to be embarrassing if seen by users.

Rate the relevance of each suggestion without considering the accuracy of the name or address.

When judging distance/prominence, disregard closed or non-existent suggestions.

Adult Content

Businesses and POIs with legal adult content are rated:

• Excellent to Bad (due to distance/prominence) if the query clearly requests this type of content • Bad (due to user intent) if the query does not imply such intent

4.1.Matching the Query String

Matching to the query string is the first step in the rating process and is the minimum requirement for a suggestion to be considered relevant. Once a match is confirmed, the relevance rating is rated in terms of prominence and distance. If there is no match between the query and the suggestion, rate the suggestion Relevance as Bad.

A suggestion can match the query word-for-word or be connected to the query without directly matching the query string through several types of indirect matches.

4.1.1.General Match

The query string can directly match the suggestion or it can match the most significant part of the suggestion.

Query string	Possible Suggestion	Explanation
[320 Finley]	320 Finley Ave Carnegie, PA	Match: Adding any street type to an address query without a street type is normal behavior and considered a match.
[marr]	San Francisco Marriott Union Square 480 Sutter St., San Francisco	Match: Although the suggestion doesn’t match the beginning of the query string, users in/near San Francisco are likely to start their search with the name of the brand “Marriott”.
[new york fore]	Forest Park New York, NY	Match: The suggestion matches the query string even though the words are in a different order. It completes the end part of the query [fore] to “Forest Park”.
[san die]	Target San Diego  8251 Mira Mesa Blvd., San Diego	No Match: The suggestion matches on nonessential information. The use of “San Diego” in the business name is extra information and not a part of the official branding for this business chain.
[bristol,]	Bristol Temple Meads Station Bristol	..No Match: The suggestion doesn’t take the “,” into consideration. Since the user has made the effort to type it, we expect suggestions to address the “,”. An example of a matching suggestion would be “Bristol, England”.

4.1.2.Prominence Match

..A query string that could be interpreted as a locality or sub-locality could return suggestions for an internationally prominent POI in that area. If the suggested POI is associated with the locality that matches the query string and is internationally known it can be considered a match through prominence as it may be useful to the user. This match type should be applied when the suggestion is not a direct match to the query string.

Query string	Possible Suggestion	Explanation
[Agr]	Taj Mahal Agra, India	Match: A possible intent for the query string is “Agra, India”. The suggestion is for a highly prominent and internationally known POI associated with this locality. This can be considered a prominence match.
[New Yo]	John F. Kennedy International Airport New York	Match: A possible intent for the query string is “New York City”. The suggestion is for a highly prominent transit hub located within New York City. This can be considered a match through prominence.
[Chicag]	The Magnificent Mile N Michigan Ave, Chicago	No Match: A possible intent for this query string could be “Chicago, IL”. The suggestion is for a locally known shopping district, however, it has no international prominence and would not be useful to a user potentially looking for the locality “Chicago”.

4.1.3.Abbreviation/Alternate Name

The user query is an abbreviation or alternate name, and the suggestion provides the full and currently accurate name.

Query string	Possible Suggestion	Explanation
[ewr]	Newark Liberty International Airport Newark	Match: The international airport code for Newark Liberty International Airport is EWR.
[general edward law]	Logan International Airport	Match: Although the query suggestion doesn’t match the query directly, it uses an accepted alternate name for the suggested airport.
[the shoe col]	Ohio Stadium 411 Woody Hayes Dr., Columbus	Match: The query string uses an alternative name for the stadium in Columbus, OH. The addition of ‘col’ in the query further implies this stadium may be the intent.

4.1.4.Category

The query string could be completed as a category. The suggestion satisfies that potential category.

Query string	.Possible Suggestion	Explanation
[foo]	La Ciccia 291 30th st, San Francisco	Match: A suggestion for an Italian restaurant should be considered a match for a category search “food”.
[ba]	Voodoo Lounge 718 N Rampart St, New Orleans, LA	Match: The lounge suggestion for the completed category search “bar” is a reasonable result.
[bart]	Richmond Station Richmond	Match: A suggestion for a specific station that is part of the BART transit system is a reasonable return for the category search “bart”. If the query string indicates a specific type of transit system, the suggestion must match that system.
[cable c]	Powell Street Station San Francisco	No Match: A likely intent for this query string could be the cable car transit system in San Francisco. The suggestion is for a station that belongs to another kind of transit system and does not satisfy the potential category intent.
[mal]	Bath&Body Works 2021 Westminster Mall, Westminster	No Match: A suggestion for a store does not satisfy a category search for “mall”, even if that store is located within a mall.

4.1.5.Spell Corrections

The user query may be misspelled or contains parts that may have been entered incorrectly, and the suggestion corrects that issue. This type of match should only be used for obvious issues, when no suggestion for the actual query exists at the user location or when the user intent clearly is satisfied by the autocorrected suggestion.

Query string	Possible Suggestion	Explanation
[sears tow]	Willis Tower	Match: The query suggestion provides the official name for the former name used in the query.
[aple sto]	Apple Store	Match: The query suggestion autocorrects the user’s obvious mistake.
[covfefe]	Coffee	No Match: This query string has no obvious spell corrections that would clearly satisfy the user intent. The returned spell-corrected suggestion should not be considered a match since user intent is obscure.
[fi]	5th Ave, New York	Match: This query suggestion converts the letters in the query string into numbers and can be considered an alternate name match. This is true the other way around, as well.
[23 Clair blv]	23 Clair St Great Neck, NY	Match: When there is no “Clair Blvd” within reasonable distance, we consider “Clair Street” to be a spell correction and a match to the incorrect user query.
[320 N Finley]	320 S Finley Rd Lombard, IL	.Match: Changing or adding the street direction is only considered a match when a suggestion matching the original query does not exist (320 N Finley) within or around the viewport/user. Consider “320 S Finley” to be a spell correction and a match to the incorrect user query.

4.1.6.Special Characters

Always consider the specific language conventions of a locale. In general, it’s a match when:

•The query contains a special character and the suggestion is a valid variation.

•The query does not include a special character but the suggestion with special characters adds value for the user and is therefore considered spell correction.

•The country or language does not have the special character and it is generally perceived as a form of expression or style (e.g. café in English).

Query string

Possible Suggestion

Explanation

[moe]

Möllersdorf Germany

Match: “oe” is a valid variation for the special character

“ö” in the country/language of the suggestion (Germany).

[pâ]

Paris, France

No Match: The suggestion disregards the special character used in French.

4.1.7.Article “The”

The suggestion matches a query that includes the article “the” when the suggestion is valid in both variations.

Possible Suggestion

Explanation

[the hom]

Home Depot

5125 Summit Ridge Ct., Reno

Match: The article can be disregarded because

Home Depot is also referred to as The Home Depot.

[the ik]

IKEA

No Match: The user query does not match the query suggestion because The IKEA is not a valid alternate name.

4.1.8.Address vs. POI

Consider a POI/business at the queried address as a match. However, if the query is for a POI, returning the address would not be considered a match.

Query string

Possible Suggestion

Explanation

[2333 S White rd, Sa]

Raging Waters San Jose

2333 South White Rd, San Jose

Match: The query string matches the address for the returned business. The POI suggestion located at that address can be considered a match.

[Raging Wat]

2333 South White Rd, San Jose

No Match: Suggestion is for the correct address, but does not include the business name. Users will have no way of knowing whether or not this address is connected to the business they asked for.

4.1.9.Address Sugges+on Match

When a query address and an address suggestion are not exactly the same, the kind of connection they have depends on their relationship:

•Street number is the same in both query and suggestion but the unit number is different or missing:

◦ If both the query and suggestion addresses exist in the real world, and neither address is a street extension, rate suggestion relevance Good when:

^▪The query contains a unit number and the suggestion does not.

^▪The suggestion contains a unit number and the query does not.

^▪The query contains one unit number and the suggestion contains another.

•The query is full address including street number and name and the suggestion is the street name only:

◦ Since this suggestion is an unlikely secondary intent, rate relevance as Acceptable.

•Query is for a street [Main Street, Pleasanton, CA] suggestion is just the locality (Pleasanton, CA).

◦ Rate the suggestion relevance Bad as it does not satisfy the user intent.

Query string	Possible Suggestion	Explanation
[357 western ave suite 1B]	357 Western Ave St. Johnsbury, VT	Match: The suggestion does not include the unit number requested in the query string and the relevance is therefore demoted by one to an initial rating of Good.
[357 western ave suite 1B]	Western Ave St. Johnsbury, VT	Match: Instead of returning the full address requested, the suggestion provides only the street name. The relevance is therefore demoted to an initial rating of Acceptable as it is an unlikely secondary intent.
[357 western ave st. johnsb]	St. Johnsbury, VT	No Match: Instead of returning the full address requested, the suggestion provides only the locality. This does not satisfy the user intent and is rated Bad.

4.2. User Intent

...Some user queries in Autocomplete can be relatively complete strings with a clear intent while others can have multiple interpretations of what the intent could be.

Part of assigning initial relevance, once there is a match, is establishing all possible intents for the query string to determine to what degree each suggestion satisfies those intents.

Local knowledge and the user/viewport location may impact the level of intent each suggestion represents. Use the information provided to determine which suggestions could be considered primary, secondary, or unlikely intent for each query string.

Tip: If you are unsure whether a suggestion would satisfy primary or secondary intent ask yourself:

•“Would I search for the suggested result by typing this query string?”

•“How likely is the user to be looking for this suggestion given the query and the location intent?”

User Intent	Definition	Highest Initial Rating
Primary Intent	A suggestion that is useful to the user, holds significant prominence, or directly satisfies a possible category intent of the query string. There can be more than one primary user intent suggestion per query string.	Excellent
Secondary Intent	A suggestion that is not as prominent as other potential suggestions or satisfies intent of a possible category query string to a lesser degree.	Good (User Intent)
Unlikely Intent	A suggestion which matches the query but is very unlikely to be useful to the user.	Acceptable (User Intent)
Non-Relevant Intent	There are issues that make the suggestion useless for the user, such as no match to the query string.	Bad (User Intent)

When determining intent, keep in mind that because we use queries made by actual users, there will be cases where a query string is gibberish or has no Maps intent. Do your best to determine what the intent could be, and see how well the suggestions fit any possible user intent. However, don’t be shy about assigning a Bad rating if a suggestion doesn’t fit any intent.

Query String	Suggestion	Intent	Highest Initial Rating	Explanation
[Hollywoo] User located in NY	Hollywood, CA	Primary	Excellent	This is an internationally known locality and should be considered a primary intent suggestion.
	The Hollywood Sign Los Angeles	Primary	Excellent	This is an internationally known landmark and should be considered a primary intent suggestion.
	Hollywood Heights, CA	Secondary	Good	.This suggestion is for a slightly less well-known neighborhood and not as prominent as other potential suggestions.
	Hollywood Fwy	Unlikely Intent	Acceptable	Although this suggestion is a major highway in the area it is not internationally or nationally known. This suggestion would unlikely be useful to the user located on the other side of the country.
	Arclight Hollywood 6360 W Sunset Blvd, Los Angeles	Non- Relevant Intent	Bad	This suggestion matches the query string on the location modifier part of the POI name, which is nonessential. Since there are many highly prominent suggestions available that also fit the query better, the suggestion is not relevant.

The rating determined by deciding how well potential intent is met is called Initial Rating. Distance and prominence will be considered next in order to establish the final rating.

4.3. Distance and Prominence

..Once you’ve made your initial rating by establishing that there is a relationship between the query and the suggestion and determining the user intent, you will need to consider the suggestion’s distance and prominence. Most important for this judgement is the concept of “all possible suggestions”.

4.3.1. All Possible Sugges+ons

The number of possible suggestions depends on the length of the query and the characters used. A query can range from one with a very clear user intent and only a few possible suggestions to one with an ambiguous intent and many possible suggestions. This impacts the rating.

In the example below, a user starts a search with [m], in a viewport showing Manhattan. To create the best user experience for a single-character query, we must return suggestions of high prominence within the viewport. Please see chapter 4.10 for rating short queries with small viewports. Because the query is ambiguous, various kinds of suggestions can be returned, including: business/POI, address, or query suggestions.

Once a query contains more characters and becomes more specific, the number of potential suggestions decreases. Only suggestions with the highest prominence should be presented.

In the example below, we have only a few prominent suggestions for [french] compared to all potential suggestions for the same query.

4.3.2.Distance

Generally, the farther away the suggestion is from the location intent, the less desirable it becomes. This is especially true for queries highly dependent on distance to the user/viewport, like chain businesses, hospitals, pharmacies, or grocery stores.

...Because a user can be provided with multiple similar suggestions, the closest entities providing the expected services should be considered the best options. As suggestions get farther away they should be considered less relevant and demoted accordingly. The context of each situation should be used to determine what constitutes a close or far suggestion. Factors which can affect distance demotions include:

•Number of possible suggestions in the real world

•Distribution of all possible suggestions

•Populations density (rural, urban, suburban)

Distance refers to the direct distance from one point to another and is measured via a straight line. There is no need to account for the actual distance required to travel from one point to the other, such as driving distance.

Do not demote a suggestion for distance when:

•It’s the closest possible suggestion, even if it is far away from the area of expected suggestions.

•Other, closer suggestions are closed or do not exist in real life.

Demote a suggestion for distance when:

•Other suggestions that satisfy the query and provide the same or similar service are significantly closer.

4.3.3.Prominence

..The prominence of a feature refers to its popularity, including the number of people visiting and media sources referencing it. Prominence can vary based on the test locale and even local knowledge. Consider the following list to get a general idea of the concept of prominence, ordered from the most prominent to the least prominent:

•The feature is known internationally

•The feature is known in the country

•The feature is known in the suggestion region

•The feature is known locally

•The feature may not even be known locally

Suggestions with high prominence may still be relevant even when far away from the user. Suggestions with low prominence, on the other hand, may not be as relevant even if they are close to the user.

Query String	Suggestion	Rating and Explanation
[Sydney]	Sydney, Australia	Relevance	Excellent
		The city of Sydney is a very well known and prominent locality. It can be considered a primary intent suggestion for this query string.
	Sydney Opera House Sydney, NSW, Australia	Relevance	Excellent
		The Opera House has overwhelming international prominence and should be considered a primary intent suggestion for this query string.
	Sydney Town Hall 483 George St, Sydney, NSW, 2000 Australia	Relevance	Good/Acceptable (Distance/Prominence)
		This suggestion directly matches the query string but is not as prominent as the locality or the internationally known Opera House.
	Sydney Buses Depot Randwick 34-36 King St, Randwick NSW 2031, Australia	Relevance	Acceptable/Bad (User Intent)
		The depot matches the query string but is unlikely to be the user’s intent and has very low prominence given how many other internationally prominent POIs Sydney has to offer.
[Agra] Fresh viewport over Agra	Agra Uttar Pradesh, India	Relevance	Excellent
		This suggestion directly matches the query string and is a locality in close proximity to the user and viewport.
	Taj Mahal Agra, Uttar Pradesh, India	Relevance	Good (User Intent)
		This suggestion is a highly prominent and internationally known POI associated with Agra, India, which is a likely intent for the query string. While not a direct match this can be considered a prominence match.
.	.	Relevance	Excellent
Query String	Suggestion	Rating and Explanation
[Jedd] Viewport over Saudia Arabia	Jeddah, Saudia Arabia	This suggestion directly matches the query string and is a prominent city in Saudi Arabia. It can be considered a primary intent suggestion for this query string.
	King Abdulaziz Airport Jeddah, Saudi Arabia	Relevance	Good (User Intent)
		This suggestion is a prominent and internationally known transit hub located within the locality of Jeddah. While not a direct match it can still be considered a match through prominence.
[10 Downing]  Fresh viewport over London City Airport	10 Downing Street  10 Downing Street, London	Relevance	Excellent
		The suggestion is the internationally known residence of the British Prime Minister and the most relevant suggestion for the user based on its prominence, even when there is a closer, less prominent suggestion within the fresh viewport.
	10 Downings London, England	Relevance	Excellent
		Suggested address is within viewport and rated Excellent even when not as prominent as the suggestion above.

4.3.4. Distance vs. Prominence

Once you’ve decided how relevant the suggestion is based on user intent, consider the suggestion in the context of all possible suggestions in the real world. Each suggestion is given a rating based on prominence and distance to the user or viewport. The table below shows how they interact: Far distance and low prominence receive low ratings, while prominent suggestions close to the user or viewport receive high ratings.

Distance Prominence	Close	Medium*	Far**
High	Excellent	Good	Acceptable
Medium	Excellent to Good	Good to Acceptable	Acceptable to Bad
Low	Excellent to Acceptable	Good to Bad	Bad

* Medium distance away compared to similar prominent suggestion ** Far distance away compared to similar prominent suggestion

Explanation of following examples

The following examples address common situations and explain how to evaluate the relevance of the suggestions. Please note:

•The examples are for illustration purposes only.

•All examples have a fresh viewport.

•Never rate pin accuracy. Pins are included only to indicate relative distance.

4.4. Many Possible Suggestions

Some queries, like [starbu], have many suggestions that satisfy the user. In cases like this, distance is more important than prominence, so suggestions that are closer to the user’s location/viewport should be rated higher than suggestions that are farther away. To illustrate the high density of Starbucks locations, possible suggestions are shown as purple dots.

4.4.1 User Inside Fresh Viewport

When the user is inside the viewport, consider all possible suggestions and demote by distance from the user.

4.4.2 User Outside Fresh Viewport

When a user is outside the viewport, suggestions are expected in or near the viewport. In cases where there are a large number of possible suggestions inside the viewport, rate Bad for those outside it.

4.5. Few Possible Suggestions

When the query has only a few suggestions in the real world, be more lenient when considering distance. Because there are only a few Zara clothing stores in Miami, suggestions for the query [zara] are demoted more leniently for distance than the many Starbucks locations in the example above example above.

4.6. Few Possible Suggestions and Greater Distance

..Leniency on distance is acceptable when there are only a few possible suggestions for the query in the requested area. For example, a user close to Berlin queries for [Wartestraße], which is a street name. There are only four streets in Germany called “Wartestraße” that can satisfy the user’s intent and all four streets are returned as suggestions. Given the limited number of suggestions, each suggestion’s relevance remains fairly high even over long distances.

4.7. Rural Areas

..Suggestions are often farther away in rural areas, but can still be considered relevant if there are no reasonable suggestions close to or within the fresh viewport. In a rural area, generosity can be applied to ratings with respect to distance.

4.8. Unspecific Location Intent and Prominence

A fresh viewport on half of a continent, or even the entire world with a user outside, makes it difficult to identify the specific location intent. In these cases, suggestions are generally rated by their prominence.

4.9. Rating for User Inside and Outside Viewport

..The following two examples show the differences in rating when a user is inside and outside a fresh viewport. The query is for a chain business called Great Wolf Lodge found across the United States. The examples show all possible suggestions within and around the large viewport.

When the user is outside the fresh viewport, no suggestion within the viewport is demoted for distance.

When the user is inside the fresh viewport, the user’s location is used as location intent and suggestions are demoted for distance.

4.10. Very Short Query and Small Viewport

As mentioned in chapter 4.3 Distance Prominence, when a query is very short, we generally expect to see highly prominent suggestions in the fresh viewport. This only works when the viewport is reasonably sized for the area. Very small viewports often don’t contain the most prominent possible suggestions.

..Rate all suggestions leniently. This means all suggestions within the viewport as well as the most prominent suggestions in the area are rated Excellent. Apply demotions for Distance/Prominence to less-prominent suggestions or suggestions that are farther away.

4.11.Location – User Intent Deviation

When the location intent and user intent don’t allow for an obvious suggestion, rate with a focus on high prominence. A user on the East Coast of the US in an en_US locale has issued the query [Hong]. The fresh viewport zooms in over Naples, where no suggestions can be found that allow for a primary intent.

Query String	Suggestion	Rating and Explanation
[hong] User and fresh viewport close to Naples, Italy	① Hong Kong	Relevance	Excellent
		Hong Kong is a very prominent location and in the case of this ambiguous query, has the highest chance of being the primary intent.
	② Hungary	Relevance	Good/Acceptable (User Intent)
		Hungary is also prominent and matches the query through autocorrect. It is a potential secondary intent.
	③ Hong En	Relevance	Acceptable/Bad (Distance/ Prominence)
		..This is one of the closest Chinese restaurants that matches the query, but is very far from the user given its prominence. This leads to low relevance.

4.12.Unexpected User Location

When a user is located in the middle of a desert or an ocean, it is difficult to understand intent when the query is not specific. Consider a user located on one of the islands in Channel Islands National Park typing [train]. It is difficult to speculate on user intent when all possible suggestions are outside the user’s fresh viewport and the user is possibly out at sea.

In such cases, demote with focus on prominence rather than distance.

4.13. Query and Category Suggestions

Query and category suggestions are rated by considering the user intent and the possible suggestions within the fresh viewport. Remember to check for the Suggestion Expectation: These types of suggestions should return multiple results unless the user is searching by an alternate name of a business or POI.

The following example explains how to rate relevance given all possible suggestions for the very vague query [su] using the amount of suggestions found inside the viewport.

Query String	Suggestion	Rating and Explanation
[su] Fresh viewport in San Jose, CA	① Subway	Relevance	Excellent
		Subway is a popular food chain with plenty of locations. This suggestion yields many results in and outside the viewport.
	② Sushi	Relevance	Excellent
		Sushi is a common food category that provides many suggestions inside and outside the viewport.
	⓷ Supermarket	Relevance	Excellent
		Supermarket is a common store category and there are suggestions inside the viewport as well as around it.
Query String	Suggestion	Rating and Explanation
[su] Fresh viewport in San Jose, CA	④ Supercuts	Relevance	Good (Distance/Prominence)
		Supercuts is a popular salon chain but there are no results inside the viewport. The closest branch is just outside the viewport, so the query suggestion is demoted to Good.
	⓹ Super 8	Relevance	Bad (Distance / Prominence)
		The Super 8 motel chain is fairly prominent, but there are no results within or around the viewport. The suggestion is therefore demoted to Bad.

Query and Category Suggestions need to contain or address all relevant parts of the user query. Leaving out such parts create a bad user experience as the example below with a user and fresh viewport over Bloomington IL shows.

Incomplete and Ambiguous Query Suggestions:

A query suggestion needs to fulfill a specific purpose (intent) to be useful for the user.

.....Depending on how ambiguous this purpose is, the rating can range from Excellent to Acceptable. In cases where the suggestion is gibberish, rate Bad. This also applies to query suggestions identical to the user query.

Query String	Suggestion	Rating and Explanation
[planet] Fresh viewport over Austin, TX	Planet Fitness	Relevance	Excellent
		Multiple Planet Fitness establishments are located within the viewport and the query suggestion presents a specific, valid intent for the user.
	planet	Relevance	Acceptable (User Intent)
		The query suggestion is identical to the query but does not fulfill a specific purpose and is therefore not a likely user intent.
[cheese] Fresh viewport over New York, NY	Cheesecake Factory	Relevance	Excellent
		This is a well-known chain restaurant with few locations in the area. Due to prominence of the chain business this is a useful suggestion with a valid intent.
	Cheesecake	Relevance	Excellent/Good (User Intent)
		The query suggestion fulfills a specific intent as category search. Most results however are cheesecake factories and could therefore be seen as less specific than expected. Rate primary or secondary intent.
	cheese	Relevance	Acceptable (User Intent Issue)
		This suggestion matches the query exactly, however, it is not likely to be very useful to the user as it’s not a very widespread search category. The suggestion is also very vague and doesn’t serve a specific purpose.

4.14.Missing Viewport

Rate queries without a viewport according to the user location with a focus on prominent suggestions. The suggestions are usually expected within the test locale. Always consider the specific query and situation.

4.14.1. Sugges+on Within the Test Locale

4.14.2. Sugges+on Outside the Test Locale

Suggestions in a country outside the user’s location need to be either very unique in name or very prominent to be relevant. In the query example [bieszcz] in an en_GB test, the suggestion is very unique and located in Poland.

4.15. Parking

When evaluating [parking] queries, keep in mind:

•Free and paid parking are equally relevant.

•Parking time limits do not affect relevance rating. Your rating should not be changed by the fact that a parking lot is long term, short term, or limited time (like a two-hour limit) parking.

•Parking for small cars, large cars, RVs, or motorcycles is equally relevant.

•If you can find evidence that a suggestion is for private parking that cannot be used by the general public, give it a rating of Bad.

Public parking vs Private Parking

Public parking is:

•Parking that is a mix of residential/staff/permit and visitor parking and is not in a gated community

•Parking that can be used only when visiting a certain business (limited-use parking)

Private parking is:

•Parking that is residents only, staff only, or by permit only

•Parking in a gated community

If you cannot confirm whether parking is public or private, give the suggestion the benefit of the doubt and consider it public parking.

4.16. Rating Relevance When Result Status is PERMANENT_CLOSURE

A business/POI that has its status set to

PERMANENT_CLOSURE requires special consideration when rating relevance.

Note: A result without any status indicated should be rated as usual, including when research indicates the business/POI is closed or does not exist.

There are some situations where you can expect to see results with the status PERMANENT_CLOSURE and others where such results are not expected. The following sections will describe how to identify and rate them.

4.16.1. Expected PERMANENT_CLOSURE Status

Sometimes you can expect to see results whose status is shown as PERMANENT_CLOSURE.

This can happen when:

•The permanently closed result satisfies the user intent completely.

AND

•There are no other result(s) that satisfy the actual user intent in the area of location intent.

Examples of results where PERMANENT_CLOSURE status is expected

•Single Result Queries:

•Queries that can only return a single result, which is permanently closed. This could be because the POI or business name is unique or the query contains location intents like locality or street components.

•Chain queries for closed chains:

•All chain locations within the area of location intent are permanently closed and the closest open chain result is significantly farther away.

Rating Approach

Rate results with an expected status of PERMANENT_CLOSURE as if they were open. This means that in most cases the rating will be Excellent, because even though they are closed, these are the best or only results that could ever be returned.

4.16.1.1.Expected PERMANENT_CLOSURE Status: Single Result Queries The user is looking for a unique business in a specific location.

Suggestions		RRating and Explanation
① 99 Bottles Of Beer On The Wall PERMANENT_CLOSURE 110 Walnut Ave, Santa Cruz, CA	Relevance	Excellent
	Business/POI Closed/Does not exist	Yes
Suggestions	Rating and Explanation
.	..This permanently closed restaurant is the only result that meets the user intent, which makes it expected. Choose the Business/POI Closed/Does not exist checkbox and rate the business as if it were open.
.② 99 Cents Only Stores 260 E 10th St, Gilroy, CA	Relevance	Bad (User Intent)
	The 99 Cents store does not match the user intent and is rated Bad.
⓷ 99 Bottles & Cocktails 8901 Katella Ave, Anaheim, CA	Relevance	Bad (User Intent)
	While also a restaurant with a name that starts with 99 bottles, it is not in Santa Cruz and does not match the user intent and is rated Bad.

4.16.1.2. Expected PERMANENT_CLOSURE Status: All chains closed in area

Chico’s has closed all of its locations in Canada. The closest locations to the user are across the border in the United States.

......

Suggestions	Rating and Explanation
① Chico’s PERMANENT_CLOSURE 100 City Centre Dr, Mississauga, ON	Relevance	Excellent
	Business/POI Closed/Does not exist	Yes
	.The permanently closed location for Chico’s is in the center of the fresh viewport. Since there are no open locations anywhere close to the area, this result is expected. Choose the Business/POI Closed/ Does not exist checkbox and rate the business as if it were open.
② Chico’s 5151 Main St, Williamsville, NY United States	Relevance	Excellent
	The open Chico’s closest to the user is just across the border. Because it is the closest open result, it is rated Excellent.
⓷ Chico’s 3349 Monroe Ave Ste 45, Rochester, NY United States	Relevance	Good (Distance/Prominence)
	As the second-closest open location, the Chico’s in Rochester is demoted to Good.

4.16.2. Unexpected PERMANENT_CLOSURE Status

These are results whose status is shown as PERMANENT_CLOSURE, but are not expected to be shown at all.

Results with the status PERMANENT_CLOSURE are unexpected when there are also open results in the area of location intent that would satisfy the actual user intent fully without changing or altering the user query.

Examples of situations where results with PERMANENT_CLOSURE status are not expected

•Category queries:

•There are always a large number of possible results for category queries like [restaurant] or [hair salon]

•Note: In cases where there is only one match for the category query within a large area and it is marked with the PERMANENT_CLOSURE status, consider it an exception and choose Business/POI is closed or does not exist, then rate it as if it were open. This should be extremely rare.

•Queries with several possible interpretations:

•These are queries could have several possible results and/or result types available, and do not depend on PERMANENT_CLOSURE results.

•Most chain queries:

•There are often multiple locations of a chain available within the area of location intent, and a permanently closed result does not need to be shown in these cases.

Rating Approach

Unexpected permanently closed results should be demoted by 2 for relevance which means the highest rating starts at Acceptable.

4.16.2.1. Unexpected PERMANENT_CLOSURE Status: Open OpFons are Nearby

There are many options available near the user for this category query, so there is no reason to show a permanently closed business.

......

Suggestions	Rating and Explanation
① Meeps Vintage 2104 18th St NW, Washington, DC	Relevance	Excellent
	Meeps Vintage matches the user intent and is close to the user within the fresh viewport.
② Miss Pixie’s 1626 14th St NW, Washington, DC	Relevance	Excellent
	Miss Pixie’s sells vintage clothing and is among the closest results for this category query.
Suggestions	Rating and Explanation
⓷ Buffalo Exchange PERMANENT_CLOSURE 1318 14th St NW, Washington, DC	Relevance	Bad or Acceptable (User Intent)
	Business/POI Closed/ Does not exist	Yes
	The permanently closed Buffalo Exchange with PERMANENT_CLOSURE Status did sell vintage clothing, but since there are other open stores that do this in the area of location intent, this result is unexpected. The relevance rating starts at Acceptable. Given the abundance of nearby results, this could be rated Bad, as well.

5. Name and Category Accuracy

The Name and Category Accuracy rating is used to evaluate the accuracy of a Business/POI name. Your final rating will take into consideration both the accuracy of the Business/POI name and the category assigned to it. Inaccuracies in either or both of these elements will result in rating demotions.

Business/POI Name and Category Accuracy can be rated as:

•n/a

•Correct

•Partially Correct

•Incorrect

•Can’t Verify

If you give a final Name and Category Accuracy rating of Partially Correct or Incorrect, you may be presented with two checkboxes:

•Name Issue

•Category issue

If they appear, use one or both them to select the reason(s) for your rating.

5.1.(N/A) for Address Type Suggestions

The n/a rating should be used for all address type suggestions, including residential addresses, streets, localities, and so on.

Unlike Business/POI type suggestions, address type suggestions do not actually have a suggestion name/title. Addresses type suggestions appear in the title field in large gray letters because this field is meant to present the most relevant part of the address information.

Since address suggestions do not have a name, the Name and Category Accuracy component cannot be evaluated. Address type suggestions will have Name and Category Accuracy pre-populated with an n/a rating. If this field is populated incorrectly as ’n/a’ for a suggestion that is not an address type suggestion change the ’n/a’ to the appropriate Name and Category Accuracy rating.

Suggestion	Rating and Explanation
1703 NW Taylor St Topeka, KS	Name and Category Accuracy	n/a
	...This is a residential address, not the name or title of a business or POI.
1600 Pennsylvania Ave, Washington, DC	Name and Category Accuracy	n/a
	Although this is the address of a well-known POI (the White House) it is still an address suggestion and contains no name or title. It cannot be rated for Name and Category Accuracy.
Paris, France	Name and Category Accuracy	n/a
	This is the name of a locality and country, not the name or title of business or POI.

5.2. Suggestion Name

Your final Name and Category Accuracy rating will take into consideration both the accuracy of the Business/POI name and the category assigned to it. First consider the name.

5.2.1.Correct Name

A business/POI suggestion name is accurate when it is used on the POI’s official website or on other official resources. Even if the official website does not use the name, if it is used on other official sources, the name is correct. Note that the name has to be used to refer to the particular POI. The corporate name by itself is not automatically the correct name.

Business Name

Every name that a specific business location uses to refer to itself is considered correct. These names can be listed in a variety of ways in different sections of the entity’s official website or official social media website:

•Chain business locator page on the official website

•About section on an entity’s official website

•Signs and storefronts seen on official websites or social media pages • Claimed social media pages

If these resources are not available, other reliable sources that can be used for researching business names include:

•Signs and storefronts seen on recent street or online imagery or crowdsourced review pages

•Recent articles in primary publications (newspapers or similar publications that do their own reporting)

•Official restaurant menus (user photos or scanned images from restaurants)

If the name cannot be confirmed with these resources and there is nothing objectively wrong with it, mark it Can’t Verify.

Corporate Name

The corporate name (and sometimes brand or product name) is often used to refer to a particular entity, but might be not the preferred name for a specific location. This depends on the particular company and the specific suggestion.

Some official websites have a local language and an international (English) version. Consider the name given in the international version only if English is the language/script of the test locale, the query, or the suggestion region, or if the official company, chain, or brand name is commonly used and understood in English in this market. If it is not, select the Result name/title is in unexpected language or script checkbox. No further rating is required.

Suggestion Name	Official Business Name	Rating and Explanation
McDonald’s	McDonald’s	Name and Category Accuracy	Correct
		The chain calls all stores McDonald’s and has the name in front of all locations as corporate branding.
Peet’s Coffee & Tea	Peet’s – Polk Street	Name and Category Accuracy	Correct
		The name is considered Correct, because it is used on the sign on the storefront. The store is officially called “Peet’s – Polk Street” on the website as it refers to the Polk Street location.
Best Buy	Best Buy San Jose	Name and Category Accuracy	Correct
		The chain Best Buy is also using a location modifier in their store names but their store sign (logo) is the corporate brand Best Buy and is therefore rated Correct.
.	.	Name and Category Accuracy	Correct
Suggestion Name	Official Business Name	Rating and Explanation
Microsoft	Microsoft Store	.The Microsoft store front shows only the logo without any text. On the webpage, it is referred to as “Microsoft Store” or together with a location modifier such as “Microsoft Store – South Coast Plaza”. Because of the close similarities between the suggestion name and the store name, rate Correct.

Location modifier or affiliation

Businesses can include the name of the locality or the larger complex or organization they belong to as additional information. These modifiers, even when not specifically used by the official resources, can be considered correct. A name with missing modifier can also be considered correct as long as the modifier isn’t crucial for understanding the particular suggestion. If a modifier is present, it must be spelled correctly. If it is misspelled, the name is partially correct.

See your Country Specific guidelines for cases where the location modifier is essential for understanding the suggestion.

Suggestion Name	Official Business Name	Rating and Explanation
Apple	Apple Valley Fair	Name and Category Accuracy	Correct
		The retail store is missing the location modifier (Valley Fair) given on the official website, but the suggestion is still considered Correct.
Peet’s Coffee	Peet’s – Ferry Bldg SF	Name and Category Accuracy	Correct
		The suggestion name is Correct because it is used as the name on the official website, while the store has the optional location modifier.
Old Navy – Belle Isle Station	Old Navy	Name and Category Accuracy	Correct
		.Old Navy doesn’t officially use the mall name as part of the store name, but the location modifier is correct and the business name therefore rated Correct.
Delgado Community College – Charity School of Nursing	Charity School of Nursing	Name and Category Accuracy	Correct
		The suggestion for the Nursing School includes the name of the college that it is affiliated with. Even though this form is not official, it is rated Correct.

Stylized logos and letters

A variation is still valid when a business uses stylized logos and letters (like the backwards “R” in Toys ” R” Us) that are outside the regular set of letters.

Suggestion Name

Official Business

Name

Rating and Explanation

Toys ‘R’ Us

Toys Я Us

Name and Category

Accuracy

Correct

The business uses a special character in its name, so the name is still considered Correct.

“The” is missing or added

•Missing: The use of definite articles depends on the individual suggestion. When a company using “the” is often referred to without it, the name is correct both with and without the article.

•Added: A business/POI that does not officially include a definite article in its name but has one included in the suggestion should be considered partially correct.

Suggestion Name

Official Business

Name

Rating and Explanation

Home Depot

The Home Depot

Name and Category

Accuracy

Correct

The business only uses The Home Depot, but the name is still considered Correct.

5.2.2.Partially Correct Name

A partially correct name differs from the official versions but can still be recognized by the user. Partially correct names can includes minor and moderate misspellings, service level mismatches, and missing or unnecessary name parts, including holding names/corporate structures. When the business name on the storefront does not include the corporate status (Ltd., Inc., etc.) seen in the suggestion, it is partially correct.

5.2.2.1.Minor and Moderate Misspellings and Missing or Extra Words

Consider names partially correct when they contain issues that don’t prevent the user from identifying the business:

•Name contains repeated, redundant or missing information

•Incorrect or missing punctuation or special characters

•Unnecessary or missing spaces

•Unexpected use of lower/upper case and ALL CAPS o Acronyms are often expected in ALL CAPS: YMCA, TGIF

•Minor and moderate misspellings that don’t influence the user’s understanding. See your Country-Specific guidelines for more information on misspellings.

•Mix of expected languages in the suggestion title. Expected languages are the languages of the query, test locale, or suggestion’s region, or an official business name.

Suggestion Name	Official Business Name	Rating and Explanation
Macys	Macy’s	Name and Category Accuracy	Partially Correct (Name Issue)
		The suggestion is missing the “ ‘ ”.
HM	H&M	Name and Category Accuracy	Partially Correct (Name Issue)
		Missing the “&” in H&M is a misspelling but the name is still recognizable.
uhaul uhaul	U-Haul	Name and Category Accuracy	Partially Correct (Name Issue)
		The suggestion name is duplicated and in all lowercase with no “-”.
Seven Eleven	7-Eleven	Name and Category Accuracy	Partially Correct (Name Issue)
		The name is in an unexpected form not used by the business itself.
GAMESTOP	GameStop	Name and Category Accuracy	.Partially Correct (Name Issue)
		The ALL CAPS is not expected for this business and demoted.

Suggestion Name	Official Business Name	Rating and Explanation
Mosjaw	Moosejaw	Name and Category Accuracy	Partially Correct (Name Issue)
		The outdoor store has two misspellings but is still identifiable.
Ecofuture Buildng Co	Ecofutures Building Inc	Name and Category Accuracy	Partially Correct (Name Issue)
		The business shows two misspellings and the incorrect business structure (Co. vs. Inc.).
Napoli Coffeehouse & Pastries	Napoli Coffee	Name and Category Accuracy	Partially Correct (Name Issue)
		The suggestion name has additional parts not used by the business. Because the additional parts are in line with the business and don’t make the suggestion ambiguous it is rated Partially Correct.
GAP Superstore	GAP	Name and Category Accuracy	Partially Correct (Name Issue)
		The additional “superstore” is not used officially, but also does not change the user’s ability to identify the well formatted and correct corporate name.
BerkeleyLaw Univerzitní of California	BerkeleyLaw University of California	Name and Category Accuracy	Partially Correct (Name Issue)
		The query [law] was issued in a cz_CZ (Czech) locale and the suggestion locale is the US (English). Rate Partially Correct due to the mix of expected languages: English and Czech.
Aqua Spa 夢幻⽔都	Aqua Spa – Ocean City	Name and Category Accuracy	Partially Correct (Name Issue)
		The query [aqua spa] was issued in the zh_Hant_HK (Hong Kong) test locale and the suggestion locale is the US (English). Rate Partially Correct due to the mix of expected scripts.
Suggestion Name	Official Business Name	Rating and Explanation
Nike	NIKELAB 21M NYC	Name and Category Accuracy	Partially Correct (Name Issue)
		Even though the Nike symbol appears on this store’s official website, the preferred store name for this specific result is not the general brand name but the NIKELAB name seen on the store sign and website.
University of California	University of California Irvine	Name and Category Accuracy	Partially Correct (Name Issue)
		Irvine is more than a location modifier for this university. The city name differentiates the Irvine location from other universities in the University of California network.
att	AT&T	Name and Category Accuracy	Partially Correct (Name Issue)
		URLs can give an indication of official names, but because they are lacking many formatting choices, it should generally not be used to confirm the official name. Similar restrictions exist for certain social media such as Twitter. The company’s name is confirmed by the URL (att.com ), but the name stated in the URL, “att”, is clearly poorly formatted, but still recognizable.
The Sears	Sears	Name and Category Accuracy	Partially Correct (Name Issue)
		Sears does not use “The” in any of their official webpages. Adding it to the company name gives the name a rating of Partially Correct.

5.2.2.2. Service-Level Mismatch

Chain businesses often indicate the level of service provided at that location in their name. If the suggestion name does not reflect the correct type, it is considered a service-level mismatch and is considered partially correct.

Suggestion Name	Official Business Name	Rating and Explanation
Patagonia Outlet	Patagonia	Name and Category Accuracy	Partially Correct (Name Issue)
		The suggestion name suggests that the store is an outlet, which it isn’t.
Delhaize	Delhaize Supermarkt	Name and Category Accuracy	Partially Correct (Name Issue)
		Depending on its size and the services available, this chain business comes in five varieties. The suggestion is missing part of the name indicating its service level.
Best Buy	Best Buy Express	Name and Category Accuracy	Partially Correct (Name Issue)
		A Best Buy Express is a vending machine and not a store. This is a service level mismatch.
Best Buy Express	Best Buy	Name and Category Accuracy	Partially Correct (Name Issue)
		The user is expecting a vending machine where there is a store. This is a service level mismatch.

5.2.2.3.Holding Names and Corporate Structures

Holding names or corporate structures are not considered correct if they are not explicitly used by the individual stores. Holding names or names that include corporate structure are considered partially correct or incorrect depending on how recognizable the name is in context with the particular business.

Suggestion Name	Official Business Name	Rating and Explanation
IT’SUGAR LLC	IT’SUGAR	Name and Category Accuracy	Partially Correct (Name Issue)
		The addendum LLC refers to the corporate structure and should not be used for the store, However, this name is still recognizable.
.	.	Name and Category Accuracy	Partially Correct (Name Issue)
Nordstrom, Inc.	Nordstrom	The addendum Inc. refers to the company’s corporate structure and should not be used for the store. However, the store can still be recognized.

.5.2.3.Incorrect Name

An incorrect suggestion name is one that is not recognizable with certainty because of severe misspelling or ambiguous or unnecessary/missing parts in the name.

Consider names incorrect when they contain issues issues including:

•Severe misspellings that prevent the user from identifying the business because of:

◦ Change in meaning

◦ Misspelling that results in an entirely different word or gibberish

•Holding names that are completely different than the recognized name

•Unnecessary or missing parts in the name that prevent the user from recognizing the business

•Slang and inappropriate language

•Using the previous name for the same entity that is not a variation of the current name and therefore cannot be identified

Note: If a suggestion name is incorrect, the final Name and Category Accuracy rating will always be Incorrect, even if the category is correct.

Suggestion Name	Official Business Name	Rating and Explanation
JAB Holding Company	Peet’s Coffee	Name and Category Accuracy	Incorrect (Name Issue)
		The holding company name is completely different than the store name.
Samsons Pizza	Seasons Pizza	Name and Category Accuracy	Incorrect (Name Issue)
		The misspelling changes the meaning of the word “Seasons” and makes this seems like a totally different business.
IEA	IKEA	Name and Category Accuracy	.Incorrect (Name Issue)
		.....In short business names, even small errors can prevent the user from recognizing the business.
.	.	Name and Category Accuracy	.Incorrect (Name Issue)

Suggestion Name	Official Business .Name	Rating and Explanation
Zatas Tacos + Tequila	Zacatecas Tacos + Tequila	.The misspelling is severe and does not allow the user to identify the business with certainty. The user is likely under the impression that it is a different restaurant.
Taco Bull	Taco Bell	Name and Category Accuracy	Incorrect (Name Issue)
		The misspelling changes the meaning, which makes the name ambiguous: It could be a “Taco Bell” chain store or a different business named “Taco Bull”.
UPS	USPS	Name and Category Accuracy	Incorrect (Name Issue)
		USPS (United States Postal Service) and UPS (United Parcel Service) are both in the delivery business, but are entirely different companies.
Walgreens Pizza	Walgreens	Name and Category Accuracy	Incorrect (Name Issue)
		By adding the unnecessary part “Pizza” the name becomes ambiguous and the user does not know if there is a pizzeria with the name Walgreens or a wrongly labeled Walgreens pharmacy.
Cheesecake	The Cheesecake Factory	Name and Category Accuracy	Incorrect (Name Issue)
		When missing the word “factory”, the suggestion name “Cheesecake” is not specific enough and the chain business cannot be identified with certainty.
Starbucks & Spencer	Marks & Spencer	Name and Category Accuracy	Incorrect (Name Issue)
		By switching Marks with Starbucks, the business can no longer be identified.
Mickey D’s	McDonalds	Name and Category Accuracy	Incorrect (Name Issue)
		..Mickey D’s is British slang for McDonalds brand, however it is not an official variation of the brand name, and not referenced by the official website nor in company signage.
.	.	Name and Category Accuracy	Incorrect (Name Issue)
Suggestion Name	Official Business Name	Rating and Explanation
Sommet Center	Bridgestone Arena	Returning a former name that is not used on the official webpage is not a good experience. Even when it might still be in use among users, if the result is a former name, rate it Incorrect.

5.3.Suggestion Category

Once you have determined the correctness of a suggestion’s name or title, you will have to evaluate the accuracy of its category in order to determine the final Name and Category Accuracy rating. The category of a suggestion refers to the category of the Business/POI.

Not every suggestion will include a categor but when one appears, you will find it below the address details

If the category is missing or listed as N/A, make your final Name and Category Accuracy rating based on the name alone.

5.3.1. Correct Category

When a business or POI is classified correctly, the category accurately reflects one of the main businesses, services, or functions of the entity. This includes broad or alternate categories that are not the preferred choice but are not misleading.

A correct name paired with a correct category is rated Correct for Name and Category Accuracy.

•If the category is missing or listed as ’N/A’, do not demote. Not every suggestion will include a category.

•Some categories may seem slightly too specific or too general. As long as the category is not misleading, consider it correct.

•Categories can vary for similar POIs and vary per market. Localization of categories should be taken into account when deciding if a category is appropriate.

Suggestion Name and Category	Rating and Explanation
Restaurant Toqué French Cuisine	Name and Category Accuracy	Correct
	...Restaurants are commonly classified by their respective cuisines. ‘French’ refers to French cuisine or a French restaurant and is an appropriate category for the suggestion.
Jocko’s World Famous Chicken and Seafood	Name and Category Accuracy	Correct
Suggestion Name and Category	Rating and Explanation
Seafood Restaurant	This category may seem a bit too specific but it is not misleading. Although this restaurant sells other items besides seafood, categorizing it this way still properly reflects the nature of this entity.
Walmart Grocery Store	Name and Category Accuracy	Correct
	As long as research shows that the particular Walmart location represented by the suggestion sells groceries, this would be an appropriate category. This is because grocery is a major department in this store and represents a large part of what it sells.
Walmart Discount Store	Name and Category Accuracy	Correct
	This category accurately represents this POI. Walmart is generally considered to offer merchandise at low prices.

5.3.2. Incorrect Category

When the category is wrong, the final Name and Category Accuracy is always rated Incorrect. This is true whether the suggestion name is Correct, Partially Correct, or Incorrect.

Consider the category incorrect when it is:

•Wrong

•Misleading

•Misspelled

•Incomplete (missing parts or has uncommon/odd abbreviations)

•In an unexpected language/script ^oThe category must match the language/script of the query, test locale, or suggestion region

Suggestion Name and Category	Expected Category	Rating and Explanation
Denver Zoo Active Life	Zoo	Name and Category Accuracy	Incorrect (Category Issue)
		Category is wrong since the suggestion is a zoo. This category does not represent the nature of the suggestion.
Macy’s Bar	Department Store	Name and Category Accuracy	Incorrect (Category Issue)

Suggestion Name and Category	Expected Category	Rating and Explanation
.	.	Category is wrong. The suggestion is a department store chain and should not be categorized as bar.
Macy’s Depart Store	Department Store	Name and Category Accuracy	Incorrect (Category Issue)
		Category is incomplete. The category is not a common abbreviation for “Department Store” and appears incomplete.
Macy’s Shoes	Department Store	Name and Category Accuracy	Incorrect (Category Issue)
		Category is misleading. The suggestion is a department store and should be categorized this way. Although Macy’s does sell shoes, this category does not reflect the wide variety of merchandise offered by the store. It’s misleading to categorize this suggestion as a shoe store.
Fry’s Electronics Electrnics	Electronics	Name and Category Accuracy	Incorrect (Category Issue)
		Category is misspelled. Any type of misspelling in the category will make the Name and Category Accuracy rating Incorrect.
Pondok Djaja Daging Ayam translation of category: Chicken Meat	Restoran Ayam translation: Chicken Restaurant	Name and Category Accuracy	Incorrect (Category Issue)
		Category is misleading. Pondok Djaja is a restaurant in Jakarta famous for its chicken dishes, but the category ‘Daging Ayam’ or ‘Chicken Meat’ is more appropriate for a poultry butcher than a restaurant. The category “Restoran Ayam” or “Chicken Restaurant” is more appropriate.
.	.	Name and Category Accuracy	Incorrect (Category Issue)

Suggestion Name and Category	Expected Category	Rating and Explanation
Aral Gas Station	Tankstelle	Test Locale: Germany Category is in unexpected language or script. The category “Gas Station” is in English instead of the expected German “Tanskstelle.” English should be considered an unexpected language in this test locale (even if it could be understood by a user in that locale).
Macys Bar	Macy’s Department Store	Name and Category Accuracy	Incorrect (Category Issue)
		The suggestion name should be considered partially correct because it does not include the proper punctuation: Macys should be Macy’s. When the name is partially correct and the category is wrong, rate Name and Category Accuracy Incorrect.

5.4. Can’t Verify

When the business or POI name cannot be confirmed due to a lack of available data online, rate Can’t Verify. This can be due to the lack of official resources or street imagery or can be the case for small businesses that don’t maintain their own webpage or social media profile.

5.5. Transit Names

Transit suggestions, including stations, airports and shipping ports, are often named after the locality or street where they are located and can include a transit indication such as “Station” or “Subway”. The transit indication in the suggestion name/title is considered optional and can be missing and still be considered correct. The user query for [san mateo station] can therefore return the name San Mateo Station or simply San Mateo to be Correct. The category of the suggestion will help you identify the correct interpretation of the suggestion as categories for transit POIs should be a reference to transit.

Keep in mind that some transit POIs located on street intersections may have a suggestion title that is the name of the street intersection (e.g. “Lincoln Way & 19^th Avenue”).

If there are multiple transit systems operating out of the same station, any of the systems can be included in the name to be Correct. If transit stations are close to each other, but users have to walk outside to get from one building to another, consider the suggestions for the two stations as two separate POIs.

Query	Suggestion	Rating and Explanation	Rating and Explanation
[millbr]	Millbrae Millbrae, CA Category: Train Station	Name and Category Accuracy	Correct
		“Millbrae” has no transit indication in the result name, but the category indicates it is a train station.
	Millbrae BART Millbrae, CA Category: Train Station	Name and Category Accuracy	Correct
		“Millbrae BART” has a transit indication for the BART system in the suggestion name.
	Millbrae Amtrak Millbrae, CA Category: Train Station	Name and Category Accuracy	Incorrect (Category Issue)
		“Millbrae Amtrak” has an incorrect Amtrak transit indication. Only BART and Caltrain operate from Millbrae station.

5.6. Parking Names

Parking lots may or may not have names. Confirm this on the lot’s official webpage:

•The generic names “Parking,” “Garage,” or “Lot” (or any other words that indicate parking can take place at the location) are considered Correct when no official name can be found.

•If an official name can be found, rate the generic names “Parking,” “Garage,” or “Lot” (or any other words that indicate parking can take place at the location) as Partially Correct.

•If the parking lot is specialized (for instance, motorcycle-only parking), and the suggestion’s name doesn’t reflect that specialization, the suggestion should be rated Incorrect for Name and Category Accuracy.

Accept the following combinations of names as Correct if no official name is found:

•(Business name/Plaza name/Shopping area name) + (parking/ garage/lot)

•Without the addition of parking/garage/lot, the name is Incorrect

•Parking/ Garage/Lot (or any other words that indicate parking can take place at the location)

Query	Suggestion	Rating and Explanation
.	Union Square Garage	Name and Category Accuracy	Correct
		This garage’s official website confirms its name.
	Parking (Suggestion is for the Union Square Garage)	Name and Category Accuracy	Partially Correct (Name Issue)
		..Research shows that the official name for this parking location is Union Square Garage
Query	Suggestion	Rating and Explanation
[Parking] User and viewport in San Francisco, CA	Safeway Parking (Suggestion is for the parking lot attached to a ...Safeway store in San Francisco)	Name and Category Accuracy	Correct
		.This parking lot has no official website or name, but is associated with a Safeway store.
	Parking (Suggestion is for the parking lot attached to a Safeway store in San Francisco)	Name and Category .Accuracy	Correct
		This parking lot has no official website or name.
	Safeway (Suggestion is for the parking lot attached to a Safeway store in San Francisco)	Name and Category Accuracy	Incorrect (Name Issue)
		Without the words “Parking,” “Garage,” or “Lot,” users will have no way to know they can park at this Safeway.

5.7. Query and Category Suggestions

Query Suggestion	Official Business Name	Rating and Explanation
ikea	IKEA	Name and Category Accuracy	Partially Correct (Name Issue)
		The query suggestion is in all lower case while the official business name is in all capitals.
T Mobile	T-Mobile	Name and Category Accuracy	Partially Correct (Name Issue)
		The query suggestion is missing the character “-“ used consistently on the official business website.
..Haagen-Dazs	Häagen-Dazs	Name and Category Accuracy	Partially Correct (Name Issue) ....
		...The query suggestion is missing the diacritic that is used on the official business website.

...The accuracy of query and category suggestions is rated using the guidelines for business/POI Name and Category Accuracy.

....

6. Address Accuracy: Components and Issues

Address Accuracy is split into two main parts: Components & Other Issues and Suggestion Expectation. This section will discuss Components and Issues and the next section will discuss Suggestion Expectations.

This section covers the issues that can be associated with the address components as well as the possible issues that can be associated with the address itself. It refers to the checkboxes presented in the rating interface when Incorrect is selected. Checkboxes should be marked for either incorrect or missing information. Issues can occur within parts of the address or the entire address.

6.1. Address Components

Address components are the elements used to form the address details for each suggestion.These components can include street number, unit/apt, street name, locality, region/state, postal code, and country. They are defined in these guidelines with a global approach in mind. There may be variations for specific markets that are not addressed here. See your Country Specific guidelines for more information on the address component requirements for each type of suggestion.

Some address components are not necessary in Autocomplete suggestions. This will be further explained in Address Accuracy: Suggestion Expectations

Choose all incorrect components you identify. If the issue is not tied to individual components but to an issue pattern, choose the appropriate issue instead.

Never select issues and components at the same time to report the same problem. That means, do not rate Incorrect —Language/Script issue for the entire address and also Incorrect—Street Name because street name is in an incorrect language.

Component problems that are not connected to the issues can get their own ratings. For instance, if you select language issue for entire address, you can also mark the street number as incorrect, since this is an error that’s not language related.

6.1.1. Street Number

If the suggestion address has an incorrect or missing street number rate Address Accuracy as Incorrect – Street Number.

Suggestion	Official Address	Rating and Explanation
7-Eleven 836 E Fremont Ave, Sunnyvale ...	834 E Fremont Ave, Sunnyvale, CA 94087, USA .	Address Accuracy	Incorrect – Street Number
		Select the Street number checkbox because the street number in the suggestion address is incorrect.
. Defour ..Potgieterstraat, Amsterdam	. Potgieterstraat 47, 1053 XS Amsterdam, Netherlands	Address Accuracy	Incorrect – Street Number
		....Select the Street number checkbox because the street number is missing in the suggestion address.

The way street numbers are assigned varies by country and area. Here are some patterns you may see:

Address Range

Address ranges are used to assign numbers to a stretch of road. Given the nature of such ranges, not all numbers will have a building assigned to them.

When an address range is given by the query or returned as a suggestion, consider it correct when:

•The street number is within the queried range

•The suggestion address range exists and contains the queried street number

•The range or number matches the odd/even numbering system for left/right side of street required by the country

In case of discrepancies between the title of an address and the address details, consider only the title and ignore the difference in the details.

Suggestion	Official Address	Rating and Explanation
The Crutched Friar 39 Crutched Friars, London	39-41 Crutched Friars, London EC3N 2AE, United Kingdom	Address Accuracy	Correct
		No need to select the street number checkbox since the street number in the suggestion is within the official address range.
The Crutched Friar 45 Crutched Friars, London		Address Accuracy	Incorrect – Street Number
		Select the Incorrect – Street Number checkbox because the street number in the suggestion is outside of the official address range.
26 Winchester Avenue Burpengary East Qld 4505 Australia .	25-27 Winchester Avenue, Burpengary East Qld 4505 Australia	Address Accuracy	Incorrect – Street Number
		Select the Incorrect – Street Number checkbox because the number 26 is even while the official address range is odd.
.	.	Address Accuracy	Incorrect – Street Number

.The same criteria apply to POI addresses.

Suggestion

Official Address

Rating and Explanation

Apartamentos Oporto

Rua de Dom Hugo 1-45 Porto

Rua de Dom Hugo 17

Porto

Portugal

.......Select the Incorrect – Street Number checkbox because the street range in the suggestion does not exist. This is a random address range and should be considered incorrect even if it contains the official street number.

...Street Number Extensions

Some markets use a street number extension system. Street number extensions are part of the street number, not a further breakdown of it like building or unit numbers. If a street number extension is incorrect or missing from a POI address it should be treated as an incorrect/missing street number. Refer to Country Specific Guidelines for more details.

Suggestion	Official Address	Rating and Explanation
Elkjøp Halden Walkersgate 8, Halden	Walkersgate 8c, 1771 Halden, Norway	Address Accuracy	Incorrect – Street Number
		In Norway, the street number extension is part of the street number, not a unit number.The street number in the suggestion is missing a street number extension ‘c’ which is listed as part of the official street number.
REMA 1000 Tynset Elfengveien 4 A, Tynset	Elfengveien 4 B, 2500 Tynset, Norway	Address Accuracy	Incorrect – Street Number
		In Norway, the street number extension is part of the street number, not a unit number. The street number in the suggestion has an incorrect number extension ‘A’ when ‘B’ is listed as the official street number extension.

6.1.2. Unit/Apt

A street number can be further split into unit, apartment, or building numbers. A common form is the use of letters: A, B, C as well as the character # in front of a number like: #420. Less common are forms like ½.

A unit number should be correct and present when the official address of a business/POI contains a unit number.

When a business does not list a unit number and the unit number cannot be confirmed, no unit number should be given in the suggestion. Rate Incorrect – Unit/Apt when unit number is present in the rating tool.

Reminder: Only evaluate the address provided in the suggestion, which is displayed immediately under the

Suggestion Name. Do not evaluate the information provided in the “Extra Information Not Rated” section, as the information may differ, as shown in the image on the right. The rating for this suggestion would, therefore, be Incorrect – Unit/Apt as the suggestion address is missing the Unit Number provided on the official business site.

Suggestion	.Official Address	Rating and Explanation
Free People 333 na Row Suite #1110 San Jose	. 333 Santana Row Suite #1110 San Jose, CA 95128	Address Accuracy	Correct
		This is a correct address as listed on the official website of the business.
Free People 333 Santana Row San Jose		Address Accuracy	Incorrect – Unit Number
		This suggestion address is missing the unit number that is listed on the official website.
Free People 333 Santana Row Suite #2490 San Jose		Address Accuracy	Incorrect – Unit Number
		This suggestion address has a different unit number than the one listed on the official website.

6.1.3.Street Name

If the suggestion address has an incorrect or missing street name rate Address Accuracy as Incorrect – Street Name.

Suggestion	Official Address	Rating and Explanation
Van Gogh Museum Museumplein 6, Amsterdam	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Correct
		This is a correct address as displayed on the official website of the business.
Van Gogh Museum Van Baerlestraat 6, Amsterdam		Address Accuracy	Incorrect – Street Name
		This address has the wrong street name. Street name checkbox should be selected.
Van Gogh Museum 6, Amsterdam		Address Accuracy	Incorrect – Street Name
		This address has a missing street name. Therefore, the street name checkbox should be selected.

Alternative Street Names

A street can have alternate names or route numbers, as well as multiple names in different languages in bilingual areas. Consider a street name correct when the name used is still valid and in the language expected given the test language, suggestion’s region, or user query. If the street name has changed and the older name is no longer in use, consider the old name to be Incorrect – Street Name if .returned.

Misspelled Street Names

.Suggestion	Official Address	Rating and Explanation
Van Gogh Museum Musuemplein 6, Amsterdam	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	.Incorrect – Street Name
		The suggestion street name is misspelled (“ue” .instead of “eu”) and so the street name checkbox should be selected
Bozego Ciala Wrocław, Poland	Bożego Ciała, Wrocław, Poland	Address Accuracy	Incorrect – Street Name
		The suggestion street name is missing diacritics in two letters (ż and ł) which should be treated as a misspelling in the Polish market. The street name checkbox should be selected.

Misspellings in the street name, including missing diacritics (e instead of é, l instead of ł) may be rated Incorrect – Street Name. See your Country-Specific guidelines for more information.

Street Directions and Types

Suggestion	Official Address	Rating and Explanation
Van Gogh Museum Museumkade 6, Amsterdam	Museumplein 6, 1071 DJ Amsterdam	Address Accuracy	Incorrect – Street Name
		This address has the wrong type of street: both “plein” (square) and “kade” (quay) are street types. The official street type is listed as ‘plein’, therefore the type of street is considered wrong and the checkbox for street name should be selected.
118 El Camino Real Sunnyvale, CA	118 E El Camino Real, Sunnyvale, CA 94087	Address Accuracy	Incorrect – Street Name
		This address is missing the direction, “E,” and the street name checkbox should be selected.
Blue Stones 3530 E El Camino Real, Santa Clara	3530 El Camino Real, Santa Clara, CA 95051	Address Accuracy	Incorrect – Street Name.
		.This suggestion address has additional direction, “E,” which is incorrect.

Street directions (North (N), East (E), South (S), West (W)) as well as street types (Street (St), Boulevard (Blvd), Avenue (Ave), etc.) are part of the street name. If they are incorrect or missing, the street name is wrong and rated Incorrect – Street Name. Exceptions are when such elements of a street name are used interchangeably for the same street.

6.1.4.Sub-Locality and Administra+ve Subdivisions

Some countries or areas require additional subdivisions like sub-locality for correct address information. Consult with your Country Specific guidelines and apply the given guidance. The general guidance is to rate a wrong sub-locality as Incorrect – Sub-Locality. The same applies if the sub-locality is required but missing.

6.1.5.Locality

Misspellings within the locality name, including missing diacritics (e instead of é, l instead of ł) may be rated Incorrect – Locality. See your Country-Specific Guidelines for further guidance.

Every suggestion requires the correct locality component within its address detail. In case of local government reforms or other changes in naming, the suggestion has to reflect the official name. Exceptions are made when alternate names are accepted.

Suggestion	Official Address	Rating and Explanation
Van Gogh Museum Museumplein 6, Utrecht	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Incorrect – Locality
		This address has the wrong locality. Locality checkbox should be selected.
Van Gogh Museum Museumplein 6		Address Accuracy	Incorrect – Locality
		This address has a missing locality. Locality checkbox should be selected.
Jana Matejki Lódz	Jana Matejki 90-237 Łódź Polska	Address Accuracy	Incorrect – Locality
		The locality within the street suggestion is missing diacritics in two letters (Ł and ź) which should be treated as a misspelling in the Polish market.

6.1.6. State or Region

Suggestion

Official Address

Rating and Explanation

1384 Navarro Dr,

Sunnyvale

1384 Navarro Dr,

Sunnyvale, CA

94087, USA

Address Accuracy

Incorrect – Region/State

.This address is missing the state, which is a mandatory component in the USA. Region/State checkbox will be selected for missing state.

The State/Region component may only be provided in address type suggestions. When it comes to a state or a region, each country has its own understanding of what is expected (correct) or unusual (incorrect). For example, in the US only address type suggestions are required to have a state/region component. Refer to your Country Specific Guidelines for more information.

Blue Stones

3530 El Camino

Real, Santa Clara

3530 El Camino

Real, Santa Clara, CA 95051

Address Accuracy

Correct

This address is missing the state, but this is not a mandatory component for a business/POI type suggestion in the USA.

6.1.7. Postal Code

Postal codes are not a mandatory component in any suggestions type unless the query or the suggestion itself is a postal code. If a suggestion contains a postal code, rate Correct as long as the postal code applies to at least part of the feature (street, locality, large POI without an expected address, etc). See your Country Specific guidelines for more information.

Consider any country specific conventions when it comes to the use and correctness of shortened postal codes. As an example, the US postal code does not need the four-digit extension. Ignore it if it appears.

Postal codes are often not displayed in Autocomplete. Suggestions with and without postal codes are rated Correct as long as the postal code is not wrong.

6.1.8. Country (Address Type Suggestions Only)

The country is a “nice to have” piece of information, but should always be included when the suggestion is located in a country other than the test locale. If missing, rate Incorrect and select the Country checkbox.

Test Locale	Address in Suggestion	Address in reality	Rating and Explanation
US	1384 Navarro Dr, Sunnyvale, CA	1384 Navarro Dr, Sunnyvale, CA 94087	Address Accuracy	Correct
			This is a correct address since the user’s location is the USA and so country is not required. The postal code is not mandatory in address type suggestions.
DE	1384 Navarro Dr, Sunnyvale, CA, United States	1384 Navarro Dr, Sunnyvale, CA 94087, USA	Address Accuracy	Correct
			This address includes the country name since it is outside of Germany, which is the user’s location.
. US	Museumplein 6, Amsterdam	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Incorrect – Country
			..The country is mandatory in this example because the address is outside of the US, the user’s location.

6.2. Other Address Issues

Other Address issues encompass any issue with address accuracy that is not directly covered by the drop down list of address components. Issues can occur within parts of the address or the entire address. There may be variations for specific markets that are not addressed here. See your Country Specific guidelines for more information on address requirements for each type of suggestion.

6.2.1 Address Does Not Exist

In rare cases, there can be an existing street address found in a different locality. Always mark such cases as Incorrect – Address does not exist and not as Incorrect – Locality.

6.2.2. Language/Script issues

The address details must be in a language and script that’s useful to the user, given the test language (like en_US), the user query, and the suggestion region.

Use the Address Accuracy Incorrect – Language/Script Issue checkbox if you find issues of this nature in any of the address components in the address details of any suggestion. (For business/POI names and address titles in unexpected language, please refer to the chapter on Suggestion name/title in unexpected language or script.) Chapter 3.1 Suggestion Name/Title in unexpected language or script

Any combination of unexpected language or script is considered Incorrect. An exception can be made when only special characters are added that are not used in the expected language.

Suggestions located in a market using different scripts are hard to verify. Use the normal research steps to confirm the address. It is not expected that you understand markets and address formats outside your local knowledge expertise, so do the best you can without doing more research than normal.

Rate mixed languages and scripts of any address details as Language/script Issue when they include a language or script different from the one used in the query, test locale, or suggestion region.

6.2.3. Country Specific issue

The variety of address formats and distinctive country specific issues is broad. In general, use the checkbox for Country Specific Issue when there is an address problem that is not covered by the available checkboxes. This includes things like instances when an address also requires a sub-locality or municipality in addition to locality and state, but those components are either missing or incorrect. Please see your Country Specific guidelines for more details.

6.2.4. Other Issue

Issues not mentioned in these guidelines are reported using the Incorrect — Other Issue checkbox. If you choose this option, be sure to leave a comment as well as links to resources for further investigation. The checkbox is also used also for:

•Duplicate address components

•Name of POI reappears in the address details

•Natural Features that contain street address elements (see chapter on Natural Features)

•Addresses that point to a different location or building, such as management offices, P.O. Boxes, mailing addresses, or shared office spaces, are not valid and are rated Incorrect – Other Issue.

You may also use this checkbox if you notice formatting issues along with any Incorrect components you’ve already selected. This is because once you mark a component Incorrect, you can no longer choose Correct with Formatting issue.

Suggestion

Official Address

Rating and Explanation

John F. Kennedy

International Airport

4 World Trade Center,

150 Greenwich St, New York

4 World Trade Center,

150 Greenwich St.,

New York, NY 10007

Address Accuracy

Incorrect – Other Issue

The address listed on the official airport site is for The Port Authority of NY & NJ corporate office ,which is not the location of the actual airport. Even though the suggestion address matches the address listed on the official site, it is a different location and does not refer to the location of the suggestion POI.

6.3. Correct with Formatting Issue

When all address information is correct and present, but not in the format expected, mark it as Correct with formatting issues. Formatting issues include the order of address components and other minor issues that are not necessarily considered wrong. They does not include misspellings which are addressed under the relevant individual components. Reasons to rate Correct with formatting issues include:

•Components are in unexpected order

•Non-required but correct address components

•Extra spacing

•Double comma [,,] ^o Rate Incorrect if a required component is missing.

•Valid but redundant or unnecessary components (See your Country Specific Guidelines for more information)

If a component has already been discussed in this chapter, apply the guidance provided in the applicable section, not the Correct with formatting issues rating. If your market has additional address components, consult the Country Specific guidelines before demoting.

6.3.1. Country Specific Issue vs. Correct with Formadng Issues

Any additional components beyond the regular address format for a country are rated:

•Correct with formatting issues if correct

•Incorrect – Country specific Issue if very unusual or wrong

6.4 Can’t Verify

An address is rated Can’t Verify when it cannot be confirmed as Correct or Incorrect. This is usually the case when there is a lack of resources or an unexpected address format is given on the official website.

•Lack of resources:

◦ No official webpage found o No official address listed o General lack of official resources o Lack of street imagery

•Unexpected address formats used by official resources:

◦ Intersection address (Main St and 2^nd St)

•Exit address (Exit 5, Hwy 101)

•Descriptive address (Main St between 6th and 7th Streets)

Ask yourself: Is the location given by the suggestion address at least not wrong given all information I have found? Use the following examples to learn how to reach a conclusion.

Note: If you find that an address has missing or incorrect components, you can confirm it as Incorrect. Mark the individual incorrect components. See your Country Specific guidelines for more information on missing or incorrect components.

1.1.1Can’t Verify for Street Number

The following example shows address details for a business called Utopia. No information online confirms the address and only a few street numbers can be confirmed with official resources.

1.1.2Can’t Verify for Street Name

The following examples give different address details for a business called Aztec. The official address is “Granite Ave & W Babcock St, Bozeman, MT” and therefore in a different format than the suggestion addresses.

7. Address Accuracy: Suggestion Expectations

There are three kinds of suggestions that contain address details:

•Business/POI suggestions

•Address type suggestions

•Features without an expected address

Each suggestion type has different expectations regarding its address details. For example, the Amazon River has no street address but is accompanied by the country, while the fast food place “Wienerschnitzel” shows the street number, name, and city. This chapter will help you form the appropriate expectations in order to rate address accuracy.

7.1.Business/POI Suggestion

Business/POI type suggestions in Autocomplete generally include:

•Street number

•Street name and type

•Locality

The address given is confirmed Correct when it points to the same location as the address listed on the official webpage or on other official resources and all mandatory components for your market are present. See your Country Specific guidelines for details on mandatory address components.

Business/POI suggestions don’t include a state, postal code, or country. These addresses are rated Correct even when such components are missing. If the components are present, rate them as usual.

7.1.1. Business/POI Address Research and Resources

The official webpage and other official resources have an impact on what components are expected and in what form. Other official resources are used to confirm the address when the official webpage does not provide a full address or if there is a conflict with the address in the suggestion. Such resources should be reliable and may include:

•Social media sites claimed by the business/POI and updated within the last 6 months

•Street imagery (any recent online imagery showing streets and buildings)

•Postal authorities (for postal codes and locality names)

•Government business and property registries

If no official resources can be found, rate according to consensus from multiple reliable sources. Reliable resources include:

•Articles in primary publications (newspapers or similar publications that do their own reporting)

•Crowdsourced user review sites

When no official resources are available and address components are missing or clearly incorrect, check to see if those components are required in the market or region of the suggestion and select the appropriate checkboxes under Incorrect. See your Country Specific guidelines for more information.

Some resources, like data aggregators, spammy directory sources, etc., are unreliable, and consensus between unreliable resources should be disregarded. Rate conflicting information between similarly reliable sources Can’t Verify. See your Country Specific Guidelines for more details.

•Valid Alternate Names: When valid alternate names for streets or localities are used interchangeably, consider all versions Correct.

•Street Addresses: When valid alternate addresses are returned, consider all versions found on the official web page or confirmed by official sources as Correct.

When the address listed by the official web page or resources is in a different format than the one seen in the suggestion, confirm if the address given is Incorrect or Can’t Verify. See Can’t Verify for more information.

If there is any obvious error in an official source, like a misspelled country or city name, and the error appears in the suggestion, rate the erroneous component Incorrect.

7.1.1.1.Department Addresses

When the official webpage does not give a complete address with street number and name for a large entity (mall, university, etc.):

•Rate the result address Correct when it matches the address of any department or entity at the same location complex or campus.

•Rate the result address Correct when it lists the correct locality for the entity. If the exact locality is unclear because the entity borders multiple localities, rate Can’t Verify.

•The locality is always expected, even when no complete address is provided. If missing, rate Incorrect – Locality.

•Region and postal code must also appear in countries where they are mandatory components. See your Country Specific guidelines for more information.

Suggestion	Official Address	Rating and Explanation
Dartmouth College 10 N Main St, Hanover, NH 03755	Dartmouth College Hanover, NH 03755 USA	Address Accuracy	Correct
		Dartmouth College has no official street address listed on their website. The address in the suggestion belongs to the Office of Admissions, which is part of the larger university, therefore the address is Correct.
Dartmouth College Hanover, NH 03755		Address Accuracy	Correct
		The college lists only the locality as its official address and the result matches that address and is therefore Correct (even though it is missing a street number and name).
Dartmouth College NH 03755		Address Accuracy	Incorrect – Locality
		A locality is expected in the address details for most POIs. Rate Incorrect – Locality when the locality is given by the official webpage but missing in the result.

In addition to a street address, a store or department can have the name of the complex or campus that contains it and be rated Correct.

Suggestion	Official Address	Rating and Explanation
Stanford School of Law Stanford University, Stanford, CA 94305	559 Nathan Abbott Way, Stanford, CA 94305	Address Accuracy	Correct
		This suggestion is a department within Stanford University. The address is Correct because the larger entity name and correct locality is listed as the address.
The Cheesecake Factory Westfield Valley Fair, Santa Clara, CA 95050	3041 Stevens Creek Blvd Santa Clara, CA 95050	Address Accuracy	Correct
		This suggestion is a business within Westfield ..Valley Fair shopping mall. The address is Correct because the larger entity name and correct locality are listed as the address.

If the entity has an official address and the result address belongs another entity in the same complex or on the same campus, rate Incorrect.

Suggestion	Official Address	Rating and Explanation
. Stanford School of Law 450 Serra Mall, Stanford, CA 94305	559 Nathan Abbott Way, Stanford, CA 94305	Address Accuracy	Incorrect – Street Number & Street Name
		The suggestion address belongs to the larger entity (Stanford University). The department has an official address that is different from the university’s address, so the result address is rated Incorrect.
Stanford University 559 Nathan Abbott Way, Stanford, CA 94305	450 Serra Mall, Stanford, CA 94305	Address Accuracy	Incorrect – Street Number & Street Name
		The suggestion address belongs a smaller entity within the university (Stanford School of Law). The university has an official address that is different, so this address is rated Incorrect.
The Cheesecake Factory 2855 Stevens Creek Blvd Santa Clara, CA 95050	3041 Stevens Creek Blvd Santa Clara, CA 95050	Address Accuracy	Incorrect – Street Number
		The suggestion address belongs to the larger entity (Westfield Valley Fair mall) that contains the restaurant. The Cheesecake Factory’s official website lists a different address, so the address Incorrect.
Westfield Valley Fair 3041 Stevens Creek Blvd Santa Clara, CA 95050	2855 Stevens Creek Blvd Santa Clara, CA 95050	Address Accuracy	Incorrect – Street Number
		The suggestion address belongs a business (The Cheesecake Factory) within the mall. the shopping mall has an official address that is different, so the address is rated Incorrect.

7.1.1.2.Different Address Formats

When the official address and the suggestion each use a different address format, confirm that the address given by the suggestion is at least not wrong. If that is the case, rate Can’t Verify. Please confirm the appropriate rating for different address formats in your Country Specific Guidelines.

Suggestion	Official Address	Rating and Explanation
Bunnings Anketell St. & Oakden St. Greenway	Anketell St. & Oakden St. Greenway	Address Accuracy	Correct
		This suggestion is for a hardware chain. The suggestion address and the official address are both using the identical intersection address format. Therefore rate Correct.
Bunnings 320 Anketell Street Greenway		Address Accuracy	Can’t Verify
		The suggestion address and official address are using a different format. Research shows that both addresses point to the same location and the suggestion address is at least not wrong, therefore rate Can’t Verify.
Bunnings Greenway		Address Accuracy	Incorrect – Street Name
		Because the official address consists of two intersecting roads, rate Incorrect – Street Name. Rate the same way if only one street is missing.

7.1.2. Businesses/POI with moving loca+ons

Moving entities can either be on a fixed schedule or moving at random. In some cases, entities appear to be moving because they are located in a truck or similar entities, but are actually permanently located at the same location.

Random location

The location of randomly moving entities cannot be predicted and should not be shown on the map. Rate such businesses as Result is closed or does not exist.

Fixed schedule

Some entities, such as food trucks, can move on fixed schedules. Their stops and schedule are usually posted on their official webpage. The most accurate location for these entities is the address of the longest stay over a full cycle of the given schedule (i.e. if a business has a weekly cycle and stays in one spot for four days, that spot will be the only acceptable address for it). Mark any other address as Incorrect – Other Issue. In cases of a tie, consider any of those locations as Correct.

Fixed location

Businesses that operate out of a truck or bus that is stationary are evaluated like a standard business. The vehicle does not need to be parked at the location after closing hours to be considered at a fixed location.

7.1.3.Back Offices and Businesses with No Physical Location

These type of suggestions are most often rated Bad because they are either not useful to the user or have no map intent at all.

.For these type of suggestions, rate the address accuracy according to the official website or consensus of three sources, otherwise rate Can’t Verify. If the entire address is missing, rate Can’t Verify.

Business Type	Example	Rating and Explanation
No Physical location	Mobile locksmith	Businesses with no physical location, like mobile locksmiths or mobile dog groomers, have no relevance for maps and are rated Bad.
Back Office	Administrative offices of a company	Back offices are usually not open to the public. Rate them with a lower prominence. In most cases the relevance rating will be Bad unless these offices match the specific user intent.
Home Business	Home based calligrapher	Unless your Country Specific guidelines state otherwise, give businesses running out of a home very low prominence. These businesses see customers by appointment only and have no regular opening hours or a brick and mortar business (like a store with regular hours where customers can drop by and choose from a large selection of merchandise). The most likely relevance rating is Bad, unless there is a likely user intent or the suggestion is similar to a bed and breakfast or other business that a user may want to visit.

For these type of suggestions, rate the address accuracy according to the official website or consensus of three sources, otherwise rate Can’t Verify. If the entire address is missing, rate Can’t Verify.

7.1.4. Events

Some events can be very prominent but happen only a few days a year, others are regularly recurring.

Regularly recurring (weekly/monthly)

Rate events like weekly farmer’s markets like any other business.

Annual Events

Events that happen once a year, such as Burning Man or the Sziget Festival, only occur for a short time each year, but attract massive amounts of people and media attention. Users can be interested in these kinds of events outside the time they are held.

If the query and location intent are clear, assign an appropriate relevance rating, otherwise rate Bad. Treat seasonal stores (like Spirit Halloween store) and seasonally open POIs (like California’s Great America theme park), the same way as annual events.

Depending on its location, an event may not have an official address. This kind of event should be treated like a landmark or natural feature.

Singular or Non-Established Events

Non-recurring events from the past don’t carry any map significance and are rated Business/POI closed or does not exist, unless the place itself became a POI due to its historical importance that carries the name of the event further

7.2. Address Type Suggestions

Address type suggestions include all suggestions not considered a business/POI or natural feature. Examples are full addresses or locality suggestions.

7.2.1. Address Suggestions

An address usually includes a street number, street name, locality, and state/region. It can be the same as the mailing address or other addresses provided by official resources. Use the chapter on Address Format below, as well as your local knowledge, to identify the necessary components and format. The country is expected for an address type suggestion when the suggestion is in another country than the user location.

Postal codes are not a mandatory component in Autocomplete for most markets. See your Country Specific guidelines for more information.

•The address given has to be associated with a building or a plot of land officially assigned that address.

•P.O. boxes are not valid addresses in the map context. Rate such addresses as Incorrect – Other Issue.

•Postal codes are only required in Autocomplete suggestions for postal code queries. If they are missing for any other suggestion type, this is Correct behaviour.

If your research shows that the address does not exist, see Address Does not Exist.

Address	Mandatory components	Explanation
13 Navarro Dr, Sunnyvale, CA	Street Number, Street Name, Street Type, Locality, State	Mandatory components vary by country. See your Country Specific guidelines for
.	.
Museumplein 5, Amsterdam	Street name, Street Type, Street Number, Locality	more information.

7.2.2. Address Formats

Various address formats are used in different countries. The sequence and presentation of the components is adapted to match the local needs. For example, the street number comes before the street name in the United States but after in Germany. See your County Specific Guidelines for more information.

It might be impossible to confirm an address on official resources when another format is used. Therefore rate Can’t Verify if the address points to the same location using an alternate format that is commonly used in the market.

7.2.3. Street Suggestion

Depending on the length of the street, different address components are expected. Mandatory components may vary per country, but here are a few examples with an explanation for their respective markets. Check the country specific guidelines for a complete list.

•Local roads: Local roads that stay within a certain locality or county should carry the appropriate locality and state name.

•Highways/Interstates: Longer roads travel between cities or even states don’t require a locality or state component. If such components are present, they have to be accurate.

U.S. Route 395 covers almost the entire distance from Canada to Mexico. It passes through four states and connects countless towns, such as Big Pine, CA. Within that locality, it carries for a short distance the name N Main St. The two suggestions have different expectations regarding components.

There are several valid forms that are appropriate for the suggestion U.S. Route 395 because of its length: 

•U.S. Route 395 or US 395

•U.S. Route 395, California

•US 395, Big Pine, California, United States

For the suggestion N Main St, there is only one expected form because the road is short and stays within the locality:

•N Main St, Big Pine, CA United States

7.2.4. Locality Suggestion

Address	Mandatory components	Explanation
Sunnyvale, CA	Locality, State	Mandatory components may vary per country. Check your country specific guidelines for a complete list.
Amsterdam, Netherlands	Locality

.A locality suggestion can be a city, village, municipality or neighborhood. In the U.S., the suggestion usually includes the state it is part of.

7.2.5. Postal Code Sugges+on

Postal codes are returned when the user query matches the specific numbers or letters. Because there might be more than one postal codes for the same locality, various postal codes are suggested.

Autocomplete suggestions currently show postal codes only as suggestions and not in their address details.

7.2.6. State/Region/Territory Suggestion

If the user makes the query from a country outside the state or region suggestion, that suggestion must be accompanied by the country name.

Address .	Mandatory components	Explanation
California.	State name	If the suggestion is outside of the test locale, country name is a mandatory component.
North-Holland	Region name
Puerto Rico	Territory name

7.2.7. Country Sugges+on

The country is the highest administrative level and has no other address components. Rate Incorrect – Other Issue whenever other components are added.

.Address	Mandatory components	Explanation
Netherlands	Country	The country is the highest administrative level and has no other address components.
Romania	Country

7.3.Features Without an Expected Address

There are several kinds of features where no address is expected by the user. This could mean no street address or address or, in some cases, no components of any kind, are expected to be shown. These types of suggestions have a different rating criteria when it comes to Address Accuracy in that they do not require an address.

7.3.1. Natural Features

Natural features are specific landforms or ecosystems like rivers, mountains, jungles, and other geological features. They have names that can be rated, but typically do not have a street address, so natural feature suggestions should be returned without any street address components. If a street address is present, it is considered Incorrect – Other Issue, even when pointing to a building that is associated with the feature (like a ranger station or visitor center).

•A locality may be an acceptable address if it is appropriate for the size of the feature. It can help locate a small feature, like a hot spring, within the locality. For larger features, such as mountains, a locality may not make sense and would therefore be rated Incorrect – Other Issue.

•The state the feature is in can also be provided as the feature’s address and is rated Correct.

•A feature that is clearly inside a state but does not show a state is rated Incorrect – Region/ State

•A feature that is overlapping several states may return any or none of those states.

•If a postal code is included, it must be correct.

•If a very large or expansive natural feature, like a river or mountain, spans more than one country, any one of these countries can be a Correct address. If no country at all appears, this is also Correct.

Parks and other protected areas are artificial boundaries created to help protect the natural feature. Those POIs are often named after the natural feature they contain. In cases where a feature can be considered a POI, such as Mount Rushmore or a state park beach, the official address can be returned. There might be other natural features within a park that reference the park name as part of their address details.

Treat land masses, like islands and continents, as natural features with no expectation of an address. If a street address is returned, rate it Incorrect – Other Issue.

Suggestion	Official Address	Rating and Explanation
Wreck Beach Vancouver	No official address	Address Accuracy	Correct
		Research confirms that this beach is located within Vancouver.
Wreck Beach NW Marine Dr, Vancouver		Address Accuracy	Incorrect – Other issue
		The address contains the name of a street that provides access to the beach, which is unnecessary.
Wreck Beach W King Edward Ave, Vancouver		Address Accuracy	Incorrect – Other issue
		The suggestion contains the name of a street that is nowhere near the beach. The street is not expected and therefore rated Incorrect – Other Issue.
Mendenhall Glacier Juneau, AK		Address Accuracy	Correct
		Research confirms that the glacier is located within the borough of Juneau.
Mendenhall Glacier 8510 Mendenhall Loop Rd, Juneau		Address Accuracy	Incorrect – Other issue
		This address belongs to the ranger station associated with the glacier. The glacier itself should not have this address.
Mount Rushmore 13000 HWY 244 #31-1, Keystone	13000 Highway 244 Building 31, Suite 1 Keystone, SD 57751	Address Accuracy	Correct
		Mount Rushmore is both a mountain and a monument within its own national park. The address given on the official National Parks Service website confirms the suggestion address.

7.3.2 POIs Without an Expected Address

POIs without an expected address differ from regular POIs and businesses in that the user would not expect a street name and number to be shown.

These types of POIs include, but are not limited to:

•Parks (golf courses and theme parks not included)

•Monuments

•National Landmarks

•Park-like heritage sites

•Bridges

•Squares

•Parking lots

•Transit POIs: These can include airports (including airport terminals), ferry ports, subways, bike share stations, train and bus stops

The rating criteria listed in the sections below apply to all features without an expected address.

7.3.2.1. Minimum Address Component

Whether there is an official street address or not, POIs without an expected address do not need to show it. The locality must be returned whenever a POI is small enough to fit entirely within it. A POI that covers an area overlapping several localities does not require a locality component.

Suggestion	Official Address	Rating and Explanation
Quincy Station Chicago	220 S. Wells St, Chicago, IL 60606	Address Accuracy	Correct
		This is a transit POI so no street address is expected. The suggestion address is just the locality. The POI is physically located within the given locality and it matches the one listed within the official address.
Stonehenge Amesbury	None	Address Accuracy	Correct
		This is a heritage site so no street address is expected. The suggestion address is just the locality. There is no official street address but since the POI is physically located only within given locality it can be considered correct.
Tongariro National Park New Zealand	None	Address Accuracy	Correct
		.This is a park and heritage site POI so no street address is expected. No locality is listed for this suggestion. Since this POI covers an area that overlaps several localities no locality component is required.
Suggestion	Official Address	Rating and Explanation
Founders Park Virginia	351 N Union St, Alexandria, VA 22314	Address Accuracy	Incorrect – Locality
		This is a park POI so no street address is expected. No locality is listed for this suggestion. Since this park fits into only one locality, Alexandria, the missing locality component must be considered as incorrect.

7.3.2.2. Official Address Present

Official address is a full street address:

Some of these features have an official street address listed on their website or found on other official sources, but users wouldn’t necessarily expect the address to be shown. Since a full address is not expected, the suggestion does not need to contain a full address even if there is one found via official sources. Do not automatically consider any missing street address components as incorrect.

Full Address: When the suggestion contains a full address it should be rated Correct if:

•The suggestion and official addresses are identical

Street: When the suggestion address is just a street it should be rated Correct if:

•The suggestion address street is the same as the street within the official address

Suggestion	Official Address	Rating and Explanation
Embarcadero Station 298 Market St, San Francisco	298 Market St, San Francisco, CA 94111	Address Accuracy	Correct
		The suggestion has a full address that is the same as the address listed on the official website.
Embarcadero Station Market St, San Francisco		Address Accuracy	Correct
		The suggestion address is a street name that matches the street listed within the official address.
Embarcadero Station 388 Market St, San Francisco		Address Accuracy	Incorrect – Street Number
		The suggestion has a full address., but the street number does not match the number listed within the official address.
Embarcadero Station Davis St, San Francisco		Address Accuracy	.Incorrect – Street Name
		The suggestion address is just a street name, but the street is not the same as the street listed within the official address.
Suggestion	Official Address	Rating and Explanation
Embarcadero Station	.	Address Accuracy	Incorrect – Locality
		.The station requires at least a locality and is flagged as incorrect if missing.

Official address is a partial street address:

When the suggestion address is more specific than the official address it can be rated as Can’t Verify or Incorrect.

Full Address: When the suggestion contains a full address it should be rated Can’t Verify if all of the following apply:

•It does not belong to another POI/entity

•The street within the suggestion address matches the street listed as the official address

If the full address belongs to another building not associated with the POI it should be rated Incorrect.

Suggestion	Official Address	Rating and Explanation
St. Nicholas’ Church Nikolaikirchplatz 3, Berlin	Nikolaikirchplatz 10178 Berlin, Germany	Address Accuracy	Can’t Verify
		The suggested POI is a landmark church and museum. The suggestion has a full address. The street number is not associated with any other POI or entity and the street within the address matches the official street address.
St. Nicholas’ Church Nikolaikirchplatz 5, Berlin		Address Accuracy	Incorrect – Street Number
		The suggestion has a full address. The street number belongs to a building not associated with the suggestion POI and should be considered incorrect. The street within the address matches the official street address.
St. Nicholas’ Church Poststraße 1, Berlin		Address Accuracy	Incorrect – Street Number, Street Name
		The suggestion has a full address. The street name does not match the official street name, so both ddress components should be considered incorrect.

7.3.2.3 No Official Address

If no official street address is found, the suggestion street address can be rated as Can’t Verify or Incorrect unless another official source can be found to confirm the returned address. When there is no official address the returned street address cannot be confirmed and therefore cannot be considered correct, unless otherwise stated in the Country Specific Guidelines.

.Full Address: If the full address belongs to another building not associated with the POI (even if they are on the same street) it should be rated Incorrect. When the suggestion contains a full address it should be rated Can’t Verify if all of the following apply:

•It belongs to a building associated with the intended POI/entity

•The street provides access to the intended POI

Street: When the suggestion address is just a street it should be rated Can’t Verify if:

•The street provides access to the intended POI

Providing Access

A person must be able to reach the POI directly from the given street by walking or driving. This includes:

•Streets providing access by following the boundaries of the feature or leading into it.

•T-intersections:

•A street ending at an intersection does not provide access to the feature on the other side

of it.

•A street provides access to a feature when it continues through the feature (even as a different road type with or without a different name) as long as it keeps its original name on the other side of the feature, and by doing so provides physical access.

•Ignore access restrictions when considering whether a street provides access or not.

Suggestion	Rating and Explanation
Grosvenor Square Garden Grosvenor Square, London	Address Accuracy	Can’t Verify
	The suggestion address is the street in front of the suggestion POI. Like the example illustrated above, this street provides direct access to the suggestion POI.
..Grosvenor Square Garden N Audley St. London	Address Accuracy	Incorrect – Street Name
	The street ends in a T-intersection with the park on the other side. The street does not continue on the other side as it becomes S Audley St. and is therefore rated Incorrect.
Charles de Gaulle Airport Rue A. Parreux, Mauregard	Address Accuracy	Incorrect – Street Name
	The suggestion address is just a street. The street does not provide direct access to the suggestion POI and should be considered Incorrect.
Charles de Gaulle Airport 4 Rue de la Fossette, Mauregard	Address Accuracy	Incorrect – Street Number
	The suggestion has a full address. The street provides direct access however, the street number belongs to a building not associated with the suggestion POI and should be considered incorrect.
Charles de Gaulle Airport 64 Rue de Claye, Mauregard	Address Accuracy	Incorrect – Street Number, Street Name
	The suggestion has a full address. The street does not provide direct access to the suggestion POI and the street number is associated with another entity. Both address components should be considered Incorrect.
Zurich Airport	Address Accuracy	Incorrect – Locality
	Since the airport is associated with a distinct locality “Kloten” it requires at least a locality name and is rated Incorrect – Locality if this is missing.

8. How to Rate Suggestions

This chapter provides a variety of examples and discusses how to rate them in more detail. In some cases the examples will discuss addresses that do not exist in real life. The symbols below are used to indicate if there is a building associated with the given address or if the address is non-existent in real life.

Additionally, possible suggestions are added as purple dots in some examples to illustrate the high density of possible suggestions.

For each suggestion, you will need to answer questions about:

•Relevance

•Name and Category Accuracy

•Address accuracy

8.1. Full Address Suggestions

Full address suggestions usually contain a street name and street number and are associated with a specific building or complex.

8.1.1.Address does not Exist

Relevance is always rated independently of any data issues the suggestion may have. This includes suggestions for addresses that do not exist in real life.

When assessing the distance for valid, existing address suggestions, they should be evaluated only in relation to other existing addresses, including any potential unreturned valid addresses. Non-existent address suggestions should be disregarded, even if closer to the user/VP location.

However, when assessing the distance for non-existent address suggestions, it should be evaluated only in relation to valid address suggestions, including any potential unreturned valid addresses.

For example, a user in Half Moon Bay, CA is looking for [2373 chestnut] and is presented with five residential addresses. Only the address in San Francisco is a valid address. The suggestions in Redwood City, Daly City and Oakland are not supported by the postal authorities and no evidence can be found that the suggested street addresses in those localities exist.

Query String	Suggestion	Rating and Explanation
[2373 chestnut] User and viewport location near Half Moon Bay, CA	① 2373 Chestnut St Redwood City, CA, United States Type: ADDRESS	Relevance	Excellent
		..Name and Category Accuracy	n/a
		Address Accuracy	Incorrect — Address does not exist
		This is the closest suggestion, however, this suggested address is not supported by the postal authorities and no evidence can be found that it exists. Since this suggestion is closer than the nearest possible valid address it should not be demoted for distance. The Relevance should not be demoted for suggesting an invalid address.
	② 2373 Chestnut St Daly City, CA, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Incorrect — Address does not exist
		This is the second-closest suggestion. However, this address is not supported by the postal authorities and no evidence can be found that it exists. As the closest existing match is further away, this should be rated Excellent and not be demoted for distance.
	③ 2373 Chestnut St San Francisco, CA, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		This suggestion is for the closest valid address. No other valid addresses have been suggested and there are no possible closer valid suggestions in the real world.
	.	Relevance	Good/Acceptable (Distance/Prominence)

Query String	Suggestion	Rating and Explanation
.	④ 2373 Chestnut St Oakland, CA, United States Type: ADDRESS	Name and Category Accuracy	n/a
		Address Accuracy	Incorrect — Address does not exist
		This suggested address is not valid and does not appear to exist. It is significantly farther away from the user/ VP location and farther than the closest valid matching address in the area. Since there is a closer existing address this suggestion should be demoted for distance.
	⑤ 2373 Chestnut St Livermore, CA, United States Type: ADDRESS	Relevance	Good/Acceptable (Distance/Prominence)
		Name and Category Accuracy	n/a
		Address Accuracy	Incorrect — Address does not exist
		This suggested address is not valid and does not appear to exist. It is significantly farther away from the user/ VP location and farther than the closest valid matching address in the area. Since there is a closer existing address this suggestion should be demoted for distance.

8.1.2.Partial Address Query and Mixed Intent

.Address query strings are rated according to the distance from the user or viewport. Judge the distance relative to all possible suggestions in the real world, even those that are not included in the suggestions. Note that the closest suggestion in rural areas may be several miles away from the user, while in urban areas, suggestions may be found only blocks away.

Query String	Suggestion	Ratings & Explanation
[12112 sugarloaf k] User and viewport location in Tampa, FL	① 12112 Sugarloaf Key St Tampa, FL, United States Type: ADDRESS	Relevance	Excellent
		.Name and Category Accuracy	n/a
		Address Accuracy	Correct
		This suggestion is for a valid address that is close to the user/viewport location.
	② Camden Westchase Park Apartments 12112 Sugarloaf Key St Tampa Type: BUSINESS	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		The suggestion’s address matches the the query string and gives the POI located at that address. Because of the Address to POI type match this is considered a relevant suggestion.
	③ Sugarloaf Key St Tampa, FL, United States Type: ADDRESS	Relevance	Acceptable (User Intent)
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		This suggestion does not include all parts of the query string and should not be considered a match. The address however is rated on the suggestion and therefore correct.
	④ 12112 Sugarloaf Key St Type: QUERY	Relevance	Bad
		Name and Category Accuracy	Correct
		Address Accuracy	n/a
		This query suggestion is for a single location and is therefore not an expected result even when the returned address is primary intent.

Query String	Suggestion	Ratings & Explanation
[7 e] User and viewport location in Hawthorne, CA	① 7-Eleven 4800 120th St, Hawthorne Type: BUSINESS	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		The query string is ambiguous. The user could be looking for a chain business or a unique address. This suggestion is for the nearest location of a convenience store chain: 7Eleven. This is a widespread chain business therefore it is likely to be the user’s intent.
	② 7 East Pacific Coast Highway Hermosa Beach, CA Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		This suggestion fits the query string and is the closest existing address to the user’s location.
	③ 7 Explosion Blvd Los Angeles, CA Type: ADDRESS	Relevance	Good (Distance/ Prominence)
		Name and Category Accuracy	n/a
		Address Accuracy	Incorrect — Address does not exist
		Addresses are more unique than chain businesses therefore the distance demotions are less harsh. This suggested address is not valid and does not exist. It is farther than the closest valid matching address in the area (7 East Pacific Coast Hwy). Since there is a closer existing address this suggestion should be demoted for distance.
	.	Relevance	Good (Distance/ Prominence)

Query String	Suggestion	Ratings & Explanation
.	④ 7 East Valley Blvd Alhambra, CA Type: ADDRESS	Name and Category Accuracy	n/a
		Address Accuracy	Correct
		It is farther than the closest valid matching address in the area (7 East Pacific Coast Hwy). Since there is a closer existing address, this suggestion should be demoted for distance.
	⑤ 7-Eleven 5495 Sepulveda Blvd, Culver City Type: BUSINESS	Relevance	Bad (Distance/ Prominence)
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		Research shows there are many 7- Eleven locations in the real world close to the user that we fail to return. Therefore, this suggestion should be demoted to Bad for distance.
	⑥ 7-Eleven 3845 E 3rd St, Los Angeles Type: BUSINESS	Relevance	Bad (Distance/ Prominence)
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		Research shows there are many 7- Eleven locations in the real world close to the user that we fail to return. Therefore, this suggestion should be demoted to Bad for distance.

8.1.2.1. Different Street Directions

..Adding a cardinal, or other directional qualifier (like street direction) to a query string when there is no real world suggestions to satisfy the original query, is expected behavior. Even if an exact match does exist, you should also consider any additional suggestions with an added direction as a possible match to user intent, when within a reasonable distance.

Query String	Suggestion	..Ratings & Explanation
[1109 lamar] User in San Marcos, TX (Viewport is stale)	① 1109 Lamar Ave San Marcos, TX, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Incorrect — Address does not exist
		This address suggestion should be considered an exact match to the query string. This is the closest suggestion to the user, however, this suggested address is not supported by the postal authorities and no evidence can be found that it exists.
	② 1109 Lamar San Antonio, TX, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		This is a suggestion for an existing address that is a direct match to the query. This is one of the closest possible matches to the query string.
	③ 1109 S Lamar Blvd Austin, TX, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Correct

This is a suggestion for a valid address that includes a cardinal direction. The suggested address is within a reasonable distance and should still be considered a match to the query string. Since it is one of the closest possible matches no further demotions should be applied.

Query String	Suggestion	Ratings & Explanation
[200 michigan av] User and fresh viewport in Chicago, IL	① 200 S Michigan Ave Chicago, IL, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		200 Michigan Avenue does not exist without a cardinal direction included with the street name. This should be considered a match through spell correction.
	② 200 N Michigan Ave Chicago, IL, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		200 Michigan Avenue does not exist without a cardinal direction included with the street. This should be considered a match through spell correction.

Changing or omitting the street direction is only considered a match when a suggestion matching the original query does not exist.

8.1.2.2. Different Street Types

Changing the street type is only considered a match when a suggestion matching the original query does not exist within a reasonable distance.

Query String	Suggestion	Ratings & Explanation
[23 Clair blv] Fresh viewport location over Long Island	① 23 Clair Ave Holbrook, NY, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		There is no existing address that matches the query string within a reasonable distance to the VP location. The suggested address on Clair Avenue is located within the VP and should be considered a spell correction match to an incorrect user query.
	② 23 Clair St Clifton, NJ, United States Type: ADDRESS	Relevance	Good/Acceptable (Prominence/ Distance)
		Name and Category Accuracy	n/a
		Address Accuracy	Incorrect — Address does not exist
		The suggested address is not supported by postal authorities and there is no evidence it exists. This suggestion is outside of the VP and should be demoted for distance since there is an existing relevant address inside the viewport.
	③ 23 Clair Ave Brick, NJ, United States Type: ADDRESS	Relevance	Good/Acceptable (Prominence/ Distance)
		Name and Category Accuracy	n/a
		Address Accuracy	Incorrect — Address does not exist
		...The suggested address is not supported by postal authorities and there is no evidence it exists. This suggestion is outside of the VP and should be demoted for distance since there is an existing relevant address inside the viewport.

8.1.3.Street Type Suggestions

Query String	Suggestion	Ratings & Explanation
.	① Colfax Ave Denver, CO, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Incorrect — Address does not exist
		This suggestion is an exact match to the query string. Research shows that the
[colfax av] User and viewport location in Denver, CO	.	suggested street does not actually exist in the real world without an ‘E’ or ‘W’ cardinal direction, so the Address Accuracy should be rated accordingly. Any data issues should not affect the relevance rating.
	② W Colfax Ave Denver, CO, United States Type: ADDRESS	Relevance	Excellent
		Name and Category ccuracy	n/a
		Address Accuracy	Correct
		Research shows that ‘Colfax Ave’ only exists with the cardinal directions of ‘E’ or ‘W’ included.
	③ E Colfax Ave Denver, CO, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		Research shows that ‘Colfax Ave’ only exists with the cardinal directions of ‘E’ or ‘W’ included.
	Colfax Avenue Type: QUERY	Relevance	Bad
		Name and Category Accuracy	Correct
		Address Accuracy	n/a
		The suggested street is considered a single place and is therefore not a good query suggestion and rated Bad.

Query String	Suggestion	Ratings & Explanation
[stevens creek bl] User and viewport location in Cupertino, CA	① Stevens Creek Blvd Cupertino, CA, United States Type: ADDRESS	Relevance	Excellent
		.Name and Category Accuracy	n/a
		Address Accuracy	Correct
		This suggestion matches the query string and is located in close proximity to the user and viewport. Note: Stevens Creek Blvd runs through both Cupertino and San Jose, CA.
	② Stevens Creek Blvd San Jose, CA, United States Type: ADDRESS	Relevance	Good (Distance/ Prominence)
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		This street spans across more than one locality. The suggestion is for the same street as the first suggestion but within a locality slightly farther away from the user and viewport, therefore demote for distance.
	③ Stevens Creek Drive Mountain View, CA, United States Type: ADDRESS	Relevance	Bad (User Intent)
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		This suggestion does not match the query string and research shows there are two Stevens Creek Bouldevards that do exist. Changing the street type is only considered a match when a suggestion matching the original query does not exist.
	.	Relevance	.Bad (User Intent)
		Name and Category Accuracy	n/a
		Address Accuracy	Correct

.8.1.4.Locality Sugges+ons and Prominence Match

8.1.5.Postal Code Suggestions

8.2.Business/POI Suggestions

Before you rate a POI/business suggestion, consider:

1.Multiple interpretations – Are there possible interpretations of the query terms other than a business name? French Laundry is both a famous restaurant in Napa and a type of laundromat/ cleaners.

2.Number of locations – How many locations of this business exist? If there are many of them (i.e. Starbucks) then the user expects to find locations in or very close to the user’s location intent.

3.Population density – Is the area of the business suggestion urban, suburban or rural? City dwellers are less willing to travel longer distances than suburban and rural users.

Query String	Suggestion	Ratings & Explanation
.	① Mount Rushmore National Memorial 13000 S Dakota 244, Keystone Type: BUSINESS Category: Landmark	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		This suggestion matches the query string and is a nationally known POI near
[mount rushm] User and viewport location near Keystone, SD	.	the user and viewport location. Both the Name and Address Accuracy are verified and rated Correct.
	② Mount Rushmore Gift Shop 13000 SD-244, Keystone Type: BUSINESS Category: Gift Shops	Relevance	Good/Acceptable (Distance/ Prominence)
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		This suggestion is the gift shop inside Mount Rushmore National Park. Its relevance is demoted because the feature is less prominent than the park itself. Both the Name and Address Accuracy are verified and rated Correct.
	③ Mount Rushmore KOA at Palmer Gulch 12620 Highway 244,  Hill City Type: BUSINESS Category: Campground	Relevance	Good/Acceptable (Distance/ Prominence)
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		This suggestion is a campground within the Mount Rushmore National Memorial Park. The relevance is demoted because this feature is less prominent than the park itself. Both the Name and Address Accuracy are verified and rated Correct.

8.2.1.Non-Chain Businesses

Often there are multiple unrelated businesses with the same name in the same area. In this case, rate each business independently considering its prominence and distance from the implied location intent. For example:

Query String	Suggestion	Ratings & Explanation
[lighthou]  User and Viewport in Dearborn, MI (suburb of Detroit)	① The Lighthouse 4454 W Jefferson Ave, Ecorse Type: BUSINESS Category: Pub	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	Incorrect – Street Number
		The article can be disregarded because the suggested business is also referred to as Lighthouse Pub, therefore this would be considered a match. This is one of the closest suggestions that fits the query string. This suggestion is about the same distance from the user’s location as the suggestion below (Lighthouse Ministries Church). Both suggestions are non-chain businesses and are equally prominent. They should receive the same relevance rating of Excellent. Both the Name is Correct but the Address should be 4474 instead or 4454 and is rated Incorrect – Street Number.
	.	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	Correct

Query String	Suggestion	Ratings & Explanation
.	② Lighthouse Ministries of Free Methodist Church 2069 Chandler Ave Lincoln Park Type: BUSINESS Category: Church	This is one of the closest suggestions that fits the query string. This suggestion is about the same distance from the user’s location as the suggestion above (The Lighthouse). Both suggestions are non-chain businesses and are equally prominent. They should receive the same relevance rating of Excellent. Both the Name and Address Accuracy are verified and rated Correct.
	Lighthouses Search Nearby Type: QUERY	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	n/a
		The query suggestion is for a category of buildings and fits the query string. There is a lighthouse, Milliken State Park Lighthouse, near the user’s location, therefore this query suggestion should receive a rating of Excellent. The query suggestion is spelled correctly. Name and Category Accuracy is Correct.
	③ Lighthouse Liquor 13651 W Nine Mile Rd, Oak Park Type: BUSINESS Category: Liquor Store	Relevance	Good (Distance/ Prominence)
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		This suggestion fits the query string, but is farther away from the user’s location compared to other possible suggestions. Therefore, it should be demoted due to Good for distance. Both the Name and Address Accuracy are verified and rated Correct.

8.2.2.Back Office and Businesses with No Physical Loca+on

For such suggestions, rate the address accuracy according to the official website or consensus from multiple sources, otherwise use Can’t Verify. If the entire address is missing, rate Can’t Verify.

Query	Suggestion	Ratings & Explanation
[flower sho] User and fresh viewport in Provo, UT	① Wendy’s Flower Affair 1175 E 930 N, Provo Type: BUSINESS Category: Wedding Planning	Relevance	Bad (User Intent)
		Name and . Category Accuracy	Correct
		Address Accuracy	Correct
		Research shows that this business operates out of a residential home. It has no real maps intent as walk-in clients are usually not received. The relevance rating for this location should be Bad. The address for this business has been confirmed as Correct. This business will not be rated Business/ POI is closed/does not exist because it as been confirmed as existing.

8.3.Chain Business Query

You’ll need to determine the user’s intent based on the query string:

•Primary Intent refers to suggestions that are prominent and match the query string terms or satisfy a potential category intent.

•Secondary Intent refers to businesses that match the query string or fit the potential category intent but would not be the user’s first choice. Secondary intent suggestions will receive a demotion of -1 or an initial rating of Good.

•Unlikely Intent refers to suggestions that technically match the query string terms or potential category intent, but do so poorly. They receive a demotion of -2 or an initial Relevance rating of Acceptable.

8.3.1.Chain Business Query

Query String	Suggestion	Ratings & Explanation
.	③ Whole Foods Market Express Cafe 12357 Riata Trace Pkwy, Austin Type: BUSINESS Category: Grocery	..This is a Whole Foods cafe and not a full service Whole Foods Market. As such this suggestion is less prominent than other full size Whole Foods in the area and is rated Acceptable.
	whole foods 365 Type: QUERY	Relevance	Acceptable (Distance/Prominence)
		Name and Category Accuracy	Correct
		Address Accuracy	n/a
		This query suggestion matches the query string but is not as prominent as a traditional Whole Foods Market. The closest possible location for this suggestion is farther from the user than multiple other suggestions matching the query string.
	④ 1105 N Lamar Blvd, Ste 200, Austin Type: Address	Relevance	Bad (User Intent)
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		The suggestion is the address of the regional office for Whole Foods. Rate addresses returned instead of POI or business suggestions Bad.

Query String	Suggestion	Ratings & Explanation
.	⑤ Starbucks 1899 Union St, San Francisco Type: Business	Name and Category Accuracy	Correct
		Address Accuracy	Correct
		There are lots of closer Starbucks locations closer to the user and within the viewport (purple dots) and is therefore not rated higher than Acceptable.
Query String	Suggestion	Ratings & Explanation
[taco del] User and fresh viewport in Austin, TX	① Tacodeli 12001 Burnet Rd, Austin Type: BUSINESS Category: Mexican	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		While the query string has a space between ‘taco’ and ‘del’, there are no suggestions that fit the query string exactly, therefore we can consider this suggestion a spell correction. The user is outside of the viewport and the viewport is fresh. Therefore the ratings should be based on the fresh viewport. This suggestion is within the fresh viewport and a prominent chain business.
	.	Relevance	Acceptable (Distance/ Prominence)
		Name and Category Accuracy	Correct
		Address Accuracy	Correct

Query String	Suggestion	Ratings & Explanation
.	② Tacodeli 1500 Spyglass Dr Austin Type: BUSINESS Category: Mexican	While the query string has a space between ‘taco’ and ‘del’, there are no close suggestions that fit the query string exactly, therefore we can consider this suggestion a spell correction. The suggestion is outside the fresh viewport and there are 3 closer locations to the fresh viewport (1 returned and 2 in the real world we do not return). Therefore it should be demoted for distance.
	taco bell Type: QUERY	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	n/a
		The query string has a space between ‘taco’ and ‘del’, there are no suggestions that fit the query string exactly, therefore we can consider this suggestion a spell correction. The user is outside of the viewport and the viewport is fresh. Therefore the ratings should be based on the fresh viewport. This suggestion would yield several locations within and near the fresh viewport.

8.4.Category Query

Category queries are the broadest query type because they can return the greatest variety of suggestions. Some category queries are straightforward and may have suggestions that satisfy an intent exactly, like the category suggestion “italian restaurant”. However, sometimes categories are very broad or have multiple interpretations, so several different types of suggestions could satisfy the user’s intent.

It’s often difficult to identify all possible suggestions belonging in any one category, so you’ll have to research the individual suggestions provided to make your rating decisions as well as doing general research in the area of location intent.

Because of the general type of query, rating with regards to distance is more lenient as described in the following examples. This is to allow a more diverse selection of suggestions other than what is close to the user or viewport.

Partial Queries

Autocomplete queries are often short. Use your local knowledge to understand the user intent, and if there is a good chance the user is typing a category query, then apply more leniency towards distance when rating the results. Please also consult chapter 4.3 and the example 8.4.6 and examples 4.10 for very short queries as the rating remains the same in such cases.

Significant Increase in Distance

Given how many suggestions usually are within close proximity to the location intent, suggestions for category queries are not demoted for distance simply because there is something closer (in the suggestion set or in real life). Instead, use significant increases in distance to group possible suggestions for the appropriate demotions.

8.4.1.Demotion for Distance with User Inside Viewport

.................

Suggestions	Ratings and Explanation
① Domino’s Pizza 930 W Hamilton Ave Campbell, CA Category: Pizza	Relevance	Excellent
	The pizza place is among the closest suggestions to the user, who is in an area with many pizza options, and is about 1km away. Other suggestions in this general range would also receive this rating.
② Luigi’s Pizza and Pasta 2495 S Winchester Blvd, Campbell Category: Pizza	Relevance	Good (Significant Distance)
	Luigi’s is significantly further away from the user is and is demoted for distance, as are all other suggestions in this general range.
Suggestions	Ratings and Explanation
③ Pizza My Dear 2590 S Bascom Ave, San Jose, CA Category: Pizza	Relevance	Acceptable (Significant Distance)
	Pizza My Dear is over 4km away from the user, which is significantly further than many other results. It is demoted -2 for distance.

8.4.2.Demotion for Distance with User Outside Viewport

..........

Suggestion	Ratings and Explanation
① EDC TX 964 N Walnut Ave, New Braunfels, Tx Category: Gun Store	Relevance	Excellent
	This suggestion is inside the very specific viewport and carries weapons and other accessories.
② Freddy’s Pawn & Jewelry 671 S Seguin Ave, New Braunfels, TX Category: Pawn Store	Relevance	Excellent
	The pawn shop is very close to the fresh viewport and sells guns and related items.
③ Guntrap 850 Schneider, Cibolo, TX Category: Gun Store	Relevance	Good (Significant Distance)
	This weapons store satisfies the user intent but is significantly further away from the fresh viewport in a more rural area, with fewer such stores close by.
④ River City Pawn & Jewelry 417 Main St, Scherz, TX Category: Pawn Store	Relevance	Good (Significant Distance)
	The pawn shop sells guns and other related equipment but is also significantly further away from the fresh viewport.

8.4.3.Demotion for Distance with Fewer Possible Suggestions

...........

Suggestions	Ratings and Explanation
① Deerfield Park 2401 Deerfield Dr, Kennesaw, GA Category: Park	Relevance	Excellent
	Deerfield Park is among the closest suggestions to the user. There are other suggestions that are within an acceptable range given the number of other possible suggestions that are within 13km.
② Allatoona Creek Pitner Rd NW, Acworth, GA 30101 Category: MTB Trails	Relevance	Good (Significant Distance)
	..These mountain bike trails are about 25km away from the user and significantly more distant than the closest group of results. The result, and others in the same general range, is demoted by -1 for distance.
Suggestions	Ratings and Explanation
③ Iron Hill Trail Head Cartersville, GA 30121 Category: Hiking Area	Relevance	Acceptable (Significant Distance)
	...Iron Hill Trail Head is about 30km away. It is still within the Acceptable group in terms of distance to the location intent given the smaller number of possible results in the area, and gets a demotion of -2 for distance.

8.4.4.Category Queries and User Intent

8.4.5.Category Queries and Prominence

Query String	Suggestion	Ratings & Explanation
[museu] No user location (viewport is fresh)	① Solomon R. Guggenheim Museum 1071 5th Ave New York Type: BUSINESS Category: Museum	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		This suggestion matches the query string through a category match and is a highly prominent museum likely to fill the primary intent for the user. Both the Name and Address Accuracy are verified and rated Correct.
	② 9/11 Memorial & Museum 180 Greenwich St New York Type: BUSINESS Category: Museum	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		...This suggestion matches the query string through a category match and is a highly prominent museum likely to fill the primary intent for the user. Both the Name and Address Accuracy are verified and rated Correct.

.	③ Houdini Museum 421 7th Ave New York Type: BUSINESS Category: Museum	Relevance	Good/Acceptable (Distance/ Prominence)
		Name and .Category Accuracy	Correct
		Address Accuracy	Correct
		This suggestion is a small museum that is much less prominent than other possible suggestions. Both the Name and Address Accuracy are verified and rated Correct.
	④ Museum of the Moving Image 36-1 35th Ave Astoria Type: BUSINESS Category: Museum	Relevance	Bad (Distance/ Prominence)
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		This suggestion is a small museum that is much less prominent than other possible suggestions, and is located outside of the viewport with multiple other options available inside of the viewport. Both the Name and Address Accuracy are verified and rated Correct.

8.4.6.Partial Category Query

A query string may be interpreted as a category query, POI, or even an address query.

8.4.7.Category Query: Parking

“Parking” generally refers to:

•Parking lots

•Parking garages

•Parking decks

Example Type	Query	Suggestion	Ratings & Explanation
Specific Parking	[parki]  User and viewport in San Francisco, CA	Parking 123 O’Farrell St, San Francisco Type: BUSINESS Category: Parking	Relevance		Excellent
			Name and Category Accuracy		Partially Correct (Name Issue)
			Address Accuracy		Correct
			Research reveals that the official name for this parking location is Ellis-O’Farrell Garage. The generic name “Parking” is demoted to Partially Correct.
Generic Parking	[parki]  User and fresh viewport in San Francisco, CA	Parking 121 Spear St, San Francisco Type: BUSINESS Category: Parking	Relevance	Excellent
			Name and Category Accuracy	Correct
			Address Accuracy	.Correct
Example Type	Query	Suggestion	Ratings & Explanation
.	.	.	This suggestion is the parking garage for the Rincon Center, which does not have a specific name according to official resources. Because of this, we will accept the generic name “Parking” as Correct.
Airport Parking	[parking]  User and viewport in San Francisco, CA	Parking San Francisco Type: BUSINESS Category: Parking	Relevance	Excellent
			Name and Category Accuracy	Correct
			Address Accuracy	Correct
			This suggestion refers to the parking lot of San Francisco International Airport. The generic name “Parking” is considered Correct. Since the larger POI that this parking lot belongs to is a POI without an expected address, providing just a locality is appropriate and rated Correct without a street number and street name.
Private vs. public parking	[parki] User and viewport in Santa Clara, CA	Parking 3600 Flora Vista Ave, Santa Clara Type: BUSINESS Category: Parking	Relevance	Bad (User Intent)
			Name and Category Accuracy	Correct
			Address Accuracy	Correct
			The suggestion in this example is for a residents-only parking lot. This should be rated Bad because the suggestion is not useful for the general public.

8.5.Rating Results with PERMANENT_CLOSURE Status

When rating results whose status is PERMANENT_CLOSURE, be sure to understand whether the results are expected or unexpected. For more information on the differences between expected and

unexpected results, see section 4.16. Rating Relevance When Result Status is

PERMANENT_CLOSURE.

8.5.1.Expected PERMANENT_CLOSURE Status

When a result with PERMANENT_CLOSURE status is expected, choose the Business/POI is closed or does not exist checkbox and rate as if the location were open.

8.5.2.Unexpected PERMANENT_CLOSURE Status

When many branches of a chain are available in the area of intent, we would not expect to see any locations with a status of PERMANENT_CLOSURE, whether that status is correct or not.

........

Suggestion	Rating and Explanation
① McDonald’s PERMANENT_CLOSURE 615 N Mathilda Ave, Sunnyvale, CA 94085	Relevance	Acceptable
	Business/POI closed/ does not exist	Yes
	This permanently closed restaurant (confirmed by research) has its status set to PERMANENT_CLOSURE. It is considered unexpected because there are many other locations of this chain in the area. This means the highest rating it can get is Acceptable. Name, address, and pin will not be rated.
② McDonald’s 550 Lawrence Expy, Sunnyvale, CA 94086	Relevance	Excellent
	Business/POI closed/ does not exist	No
	Name and Category Accuracy	Correct
	Address Accuracy	Correct
Suggestion	Rating and Explanation
.	This open chain location is among the closest to the user and rated Excellent.
⓷ McDonald’s PERMANENT_CLOSURE 556 E El Camino Real, Sunnyvale, CA 94086	Relevance	Acceptable
	Business/POI closed/ does not exist	No
	Name and Category Accuracy	Correct
	Address Accuracy	Correct
	This chain location is showing a status of PERMANENT_CLOSURE. Research shows that it is, in fact, open. Its relevance is rated independent of whether it is really open or closed in the real world. The result is considered unexpected and rated Acceptable. Do not mark this result as Business/POI closed/does not exist, and continue rating the result for relevance plus name, address, and pin details.
④ McDonald’s 1082 E El Camino Real, Sunnyvale, CA 94087	Relevance	Good (Distance)
	Business/POI closed/ does not exist	Yes
	The chain location does not exist according to official resources and should be rated as Business is closed or does not exist. The result does not show the status as PERMANENT_CLOSURE, so it should be rated as if it were open, which is Good given the distance.

8.6.Unclear Suggestions

There may be times when it is difficult to determine exactly what the suggestion is, as the business name can’t be identified with certainty or the business does not correspond with the given location and it cannot be confirmed as closed. When this happens, use all provided information (including information that is not rated, such as URL and phone number – never call a business) to reach a conclusion.

If there is a tie in the evidence or you are rating Autocomplete Relevance Only, rate Business/POI is closed/does not exist and then rate for relevance as if the entity did exist. 

Query	Suggestions	Ratings & Explanation
. [dinosa] User and large, fresh viewport over Texas	Dinosaur Inc 8655 Jones Rd Houston Additional info: Phone: 1-281-531-6500 Category: n/a	Business/POI is closed/does not exist		☑
		Relevance	Excellent
		Research on the suggestion information shows that the address belongs to Trails at Corinthian Creek Apartments. The suggestion phone number belongs a different apartment complex, Domain West Apartments. Finally, the purposed business, Dinosaur Inc, has no information that it ever existed. Therefore, based on the information listed we should rate the suggestion as does not exist. You will still need to rate relevance as if the business were open. The suggestion matches the query intent and is within the large, fresh viewport. The suggestion should receive a rating of Excellent.
[Salon di jalan tebet timur dal] Translation: [Salons on a street named Tebet Timur Dal…]	Lowongan Guru Kimia SMA SMU Di Jakarta Jalan Danau Sunter Utara No. 1 Tanjung Priok Category: Education Suggestion Title Translation: [Job Opening for High School Chemistry Teacher in Jakarta] Additional info: Lat/long: -6.13799,106.86598	Business/POI is closed/does not exist		☑
		Relevance	Bad (User Intent)
		The query is for a category with a specific location modifier. The suggestion is for a job opening which does not satisfy the intent and is not a POI. There is a possibility the suggestion is for a particular high school, however, the suggestion address is not tied to any high school and no specific school is named in the suggestion title. Since the suggestion data is ambiguous and does not represent a single POI this entity should be marked as does not exist. You will still need to rate relevance as if the business were open. The user’s intent is for a Salon on a specific street. Our suggestion does not match this intent and should be rated as Bad.

Name	Screenshot	Description
Pop-up Information Box	. .	Selecting an individual suggestion or pin causes a pop-up box to appear over the pin. The text within the box may show additional information and should only be used for research purposes to help clarify the suggestion: •Distance to user •Distance to viewport •Phone number (for research only, never call a business) •URL •Coordinates (latitude/longitude) Remember: Do not evaluate the information in the pop-up box.Evaluate only the information in the suggestion.
Conflicting Information	Different address in title and address details:	In case there is a disagreement between the information shown in the title and the address details, ignore the equivalent components in the address details and rate the title instead. Information rated in example: •1027 170 Street, Edmonton, AB T6W 0S4, Canada Information ignored in the example: •1027 170 St NW (equivalent address details)
Release Survey		The button in the top right corner allows you to skip a survey in rare cases when a technical or other issue prevents you from rating. Reasons are: •Adult Content •Not trained on task yet •Technical Issue •Map or map features not loading •Cannot submit ratings •Other •Anything that prevents you from rating

Suggestion Type	Explanation	Example
Address	Address suggestions can be one of multiple address types, for example, a full address, a street or a locality. Instead of a business or POI name, this kind of suggestion will show the suggested address in the top field, followed by a type. Address type suggestions will not include a category. Note: If a suggestion does not include a category, don’t assume the suggestion type is an address suggestion. You must still do research to verify it is an address suggestion, as not all POI results will have a category. Address type suggestions should have Name and Category Accuracy prepopulated as n/a.
Business/POI	Businesses and POI type suggestions will show a name and address in the top field, followed by a type. Businesses and POI suggestions will often also include a category. If included, the category will be listed twice, before the address in the top field and below the suggestion type.
Category	Category suggestions are suggestions for category queries that directly match the query string or may be useful based on the query string. They have no category or pin information. They may include partial address information. Note: Category type suggestions should have Address Accuracy pre-populated as n/a.
Suggestion Type	Explanation	Example
Query	Query suggestions support the exploration by providing search term suggestions. There may not be a category and Address/Pin are missing. .Examples are product or category searches and alternate names for POIs that match the query string. Query type suggestions are rated N/A for Address Accuracy if not pre-populated. The second line “Search Nearby” follows the usual location intent based on fresh/ stale viewport.	Ignore “No Results Nearby” note and base your rating on your research.

Query and Screenshot	Suggestion	Rating and Explanation
[starbucks] User and fresh viewport in San Francisco, CA	① Starbucks  865 Market Street	Relevance	Excellent
		Rate Excellent for Starbucks locations that are in close proximity to the user.
	② Starbucks  170 O’Farrell St	Relevance	Good (Distance/Prominence)
		Rate Good for locations that are a bit farther away from the closest relevant locations to the user.
	③ Starbucks  264 Kearny St	Relevance	Acceptable (Distance/ Prominence)
		Rate Acceptable for relevant locations that are even farther away from the user but still inside the viewport.
	④ Starbucks 580 California St	Relevance	Bad (Distance/Prominence)
		When there are many locations close to the user, rate Bad for locations that are significantly farther away and outside the viewport.
	➄ Starbucks  140 Mason Street	Relevance	Excellent
		Rate Excellent for Starbucks locations that are in close proximity to the user, even if they are outside the viewport.

Query and Screenshot	Result	.Rating and Explanation
[starbucks] User in Alameda with fresh viewport in San Francisco, CA	① Starbucks  865 Market Street	Relevance	Excellent
		Rate any Starbucks within the fresh viewport Excellent when the user is outside the viewport.
	② Starbucks  170 O’Farrell St	Relevance	Excellent
		Rate any Starbucks within the fresh viewport Excellent when the user is outside the viewport.
	③ Starbucks  264 Kearny St	Relevance	Excellent
		Rate any Starbucks within the fresh viewport Excellent when the user is outside the viewport.
	④ Starbucks  580 California St	Relevance	Bad (Distance/Prominence)
		When there are many possible suggestions within the viewport, including those not returned for rating (purple dots), rate Bad for suggestions outside it.
	➄ Starbucks  140 Mason Street	Relevance	Bad (Distance/Prominence)
		When there are many possible suggestions within the viewport, including those not returned for rating (purple dots), rate Bad for suggestions outside it.

Query	Suggestion	..Rating and Explanation
[train]	①Surf Station Amtrak 2Santa Barbara Amtrak Station 3Ventura Amtrak 4Union Station Amtrak	Relevance	Excellent
		All train stations returned are equally prominent and reasonably distanced to the user to be rated Excellent.
	⑤ Gaffey / 9th Bus Stop	Relevance	Bad (User Intent)
		The bus stop does not match the user query and has therefore no user intent. Rate Bad.

Rule and Explanation	Example
The suggestion address has to provide access and be within the possible range of street numbers. W Babcock St provides access to the business and the number appears to be within the range of addresses we can confirm. Rate the address Can’t Verify.
Granite Ave provides access to the business and the number appears to be within the range of addresses we can confirm. Rate the address Can’t Verify.
Ravalli St does not provide access to the business. The address is therefore rated Incorrect – Street Name.

..Query String	Suggestion	Ratings & Explanation
. [mal] User and viewport location in Los Angeles area	mall Type: CATEGORY	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	n/a
		There are a large number of malls in the area and “mall” is a very commonly searched category. It is very likely this could be the users intent and should be considered a primary intent category suggestion for this query string.
	① Malibu CA, United States Type: ADDRESS	Relevance	Excellent
		Name and Category Accuracy	n/a
		Address Accuracy	Correct
		This is a suggestion for a very well known locality in the area. Since this is a prominent locality within a reasonable distance from the user location, it should be considered a primary intent suggestion.
	② Beverly Center 8500 Beverly Blvd, Los Angeles Type: BUSINESS Category: Shopping Center	Relevance	Excellent
		Name and Category Accuracy	Correct
		Address Accuracy	Correct
		A reasonable primary intent for this query string would be the category “mall”. The suggestion is for a well known shopping center in the area. Because of this, it can be rated Excellent even though there are closer malls to the user. The name and address can be confirmed on the official website.
	.	Relevance	Bad (User Intent)
		Name and .Category Accuracy	Correct
.	③ Sephora 8500 Beverly Blvd, Los Angeles Type: BUSINESS Category: Beauty Supplies	Address Accuracy	Correct
		A reasonable primary intent for this query string would be the category “mall” . The suggestion is a store inside the Beverly Center which is a large shopping center in the area. Rate Bad due to user intent. The name and address can be confirmed on the official website.

Autocomplete Evaluation Guidelines

1. Introducion

1.1. Rating Tool

1.2. Rating Interface

1.2.1.Sugges+on-Level Ra+ng Checkboxes

1.2.2.Suggestinon Relevance Rating

1.2.3.Suggestion Relevance Demotion Checkboxes

1.2.4.Data Accuracy

1.2.5.Comments

2. Understanding User Intent

2.1. User Query String

2.1.1.Research Expectation

2.2. Suggestion Types

2.2.1.Wrong Business/Address Type Displayed

2.2.2.Query and Category Sugges+ons

Suggestion Expectations

Location Modifier

Search Performance

2.3 Location Intent

2.3.1 Explicit Loca+on

2.3.2 Implicit Loca+on

3. Rating ‘Suggestion Level’ Issues

3.1.Suggestion Name/Title in Unexpected Language or Script

Minor Differences in Spelling

3.1.1. Query Sugges+on Language

3.2. Business/POI is Closed or Does Not Exist

3.2.1.Closed/Does Not Exist vs. Inaccurate Name and Address

3.2.2. Ra+ng Relevance of Closed/Non-existent Business/POI

3.3.Business/POI Status is PERMANENT_CLOSURE

3.3.1. No Status Shown

3.3.2.Status shown as PERMANENT_CLOSURE

4. Relevance

General Rating Rules

Adult Content

4.1.Matching the Query String

4.1.1.General Match

4.1.2.Prominence Match

4.1.3.Abbreviation/Alternate Name

4.1.4.Category

4.1.5.Spell Corrections

4.1.6.Special Characters

4.1.7.Article “The”

4.1.8.Address vs. POI

4.1.9.Address Sugges+on Match

4.2. User Intent

4.3. Distance and Prominence

4.3.1. All Possible Sugges+ons

4.3.2.Distance

4.3.3.Prominence

4.3.4. Distance vs. Prominence

Explanation of following examples

4.4. Many Possible Suggestions

4.4.1 User Inside Fresh Viewport

4.4.2 User Outside Fresh Viewport

4.5. Few Possible Suggestions

4.6. Few Possible Suggestions and Greater Distance

4.7. Rural Areas

4.8. Unspecific Location Intent and Prominence

4.9. Rating for User Inside and Outside Viewport

4.10. Very Short Query and Small Viewport

4.11.Location – User Intent Deviation

4.12.Unexpected User Location

4.13. Query and Category Suggestions

4.14.Missing Viewport

4.14.1. Sugges+on Within the Test Locale

4.14.2. Sugges+on Outside the Test Locale

4.15. Parking

Public parking vs Private Parking

4.16. Rating Relevance When Result Status is PERMANENT_CLOSURE

Examples of results where PERMANENT_CLOSURE status is expected

Rating Approach

4.16.2. Unexpected PERMANENT_CLOSURE Status

Examples of situations where results with PERMANENT_CLOSURE status are not expected

Rating Approach

5. Name and Category Accuracy

5.1.(N/A) for Address Type Suggestions

5.2. Suggestion Name

5.2.1.Correct Name

Business Name

Corporate Name