Introduction

Welcome to Maps Search Evaluation. These guidelines will teach you how to use the evaluation tool and rate Maps Search results.

For each task, you will be presented with a search query and the results that are provided to Maps users. The goals of this task are:

•Search Relevance and Search 2.0: To determine how well the results satisfy the user’s query (result relevance)

•Search 2.0 only: To check the data accuracy (name, address, and pin location) of the results

Note: Examples shown in this document reflected real locations at the time of publication. The examples continue to demonstrate the rating principles described even if a business or point of interest has changed its name.

1.1. The Rating Tool

This is the web-based rating tool used to rate Search 2.0 and Search Relevance queries.

Anatomy of the Tool

Name	Screenshot	Description
Query Header		The query header shows: ..Query: The term the user entered into the search bar. Viewport Age: How long ago the user moved the viewport. •A fresh viewport is indicated by the word FRESH displayed in green text. •A stale viewport is indicated by the word STALE displayed in red text. Locale: The country and language combination this query is for. Country: Should match the country in the Locale field. If it does not, defer to the country found in the Locale field. User Lat,Lng: The user’s location.
Top-Level Question		A top-level question that must be answered once for each query.
Result		The result returned for a user query. The color of the result heading corresponds to the color of the pin on the map that marks that result’s location. (See Result Pin on page 9.) The result address, category and status (if included) will be shown beneath the result name/title. You will evaluate the relevance of each individual result. If you are rating Search 2.0, you will also rate the name/address/ pin accuracy for each result. Note: A query can generate more than one result.

1.2.Rating Workflow

Complete each step of the following workflow before you assign a rating.

Evaluate and rate each rating drop-down independently of the others. Rate result Relevance based on whether the result fits the query intent, even if the result is closed or does not exist or has incorrect Name, Address, or Pin data. If you are rating Search Relevance you will stop after you’ve rated result relevance. If you are rating Search 2.0, you will continue by rating Name and Category Accuracy, Address Accuracy, and Pin Accuracy individually.

Note: For any result in a Search 2.0 project, you may be asked to make all of the possible ratings, some of them, just one, or none. This is expected and not a reason to escalate or release a task.

1.3. Rating Interface

Your ratings and comments will be collected in the rating interface shown below. You will first answer one top-level question about the query itself, then rate each individual result using the other checkboxes and drop-downs.

1.3.1. Query-Level Navigational Result Question

You will answer a top-level question for each query to indicate whether there are any results in the real world that would completely satisfy the user intent.

Query-level question	Explanation	Notes
Is there a navigational result for this query?	Are there any results in the real world that can completely satisfy the user intent?	You will only need to answer this question once for each query.
Query	Answer to Query-Level Question	Explanation
[Eiffel Tower]	Yes	There is only one result in the real world that can completely satisfy the user intent.
[Starbucks]	No	There is not only one result in the real world that can satisfy the intent of this query.

Note: If you are presented with a query that has no results, you must still answer this question and submit your answer.

1.3.2. Result-Level Rating Checkboxes

For each result, you will be presented with two high-level questions. These will allow you to highlight any problems with the result.

High-level question	Explanation	Notes
Result name/ title is in unexpected language or script	Result is not in the language of the query, the test locale, or the result region (the expected languages) or is in a mix of expected and unexpected languages (unless it is an official company, chain, or brand name).	If you check this box, your rating for this result will be complete (all other questions will be unavailable).
High-level question	Explanation	Notes
Business/POI is closed or does not exist	Research shows that the business is closed or that no such business exists at the address listed in the result.	If you check this box, only the relevance needs to be rated. Pay attention to the status of the POI. If the result’s status is PERMANENT_CLOSURE, refer to 5.19 Rating Relevance when Status is set to PERMANENT_CLOSURE and 10.8 Permanently Closed Results for rating instructions. If the status is blank or displays any other text, rate the relevance of the result as if it did exist. Use thorough research and your best judgement to determine whether the result is closed, has never existed, or the result component(s) are incorrect.See Unclear Results for further guidance.

1.3.3. Result Relevance Rating

The relevance rating will take into consideration user intent as well as distance/prominence.

Relevance is always rated independently of any data (name/category, address, or pin) inaccuracies. This means that when rating relevance we always assume that the result exists (even if research reveals the location is closed) and that the data presented is correct.

Always rate against the real world: If there is a better result available but it is not shown, demote the existing result(s) while considering the missing one(s).

Relevance Rating	Explanation	Notes
Navigational	The most likely result implied by the user’s query, location, and/or viewport that completely satisfies the user’s distinct intent. A distinct user intent is defined by a combination of the following characteristics: •Extreme prominence (e.g. Eiffel Tower) •Uniqueness (e.g. complete address) •Proximity (e.g. incomplete address that points to an unambiguous option that is is extremely close)	A rating of Navigational for one result does not prevent other results from satisfying the query to a lesser degree.
Excellent	A high-quality result that clearly satisfies the user’s intent. Multiple results can be considered high quality.	Highest possible initial rating for ambiguous queries as well as for queries that are not eligible for a Navigational rating.
Good	Only partially satisfies the user’s intent due to relevance, prominence, or distance.	Any relevance rating of Good or below requires an indication of the reason(s) for the demotion: ^•User Intent Issue ^•Distance/Prominence Issue
Acceptable	Technically satisfies the user’s intent but does so poorly due to relevance or distance.
Bad	Does not satisfy the user’s intent due to lack of relevance or great distance (when closer satisfying results are available).

1.3.4. Result Relevance Demotion Checkboxes

If you give a result a Relevance rating of Good or below, you will be presented with two checkboxes. Use them to select the reason(s) for your rating:

•User Intent issue

•Distance/Prominence issue

You must also leave a comment explaining your rating.

Checkbox	Explanation	Notes
User Intent issue	Select when the demotion is due to a result only partially fulfilling the query intent.	If both issues were reasons for your relevance rating, select both checkboxes.
Distance/ Prominence issue	Select when the demotion is due to a distance issue based on the user or viewport location or when you demoted for a result that is less prominent based on the initial query intent. You can also select this option when the query asks for something at a specific location and the result is not at or near the requested location.

1.3.5. Data Accuracy

If you are rating Search 2.0, once you’ve rated relevance, you’ll evaluate three result components for data accuracy: Name, Address, and Pin.

1.3.6. Comments

Comments are mandatory for any result relevance rating of Good or below and for any data issues not rated Correct/Perfect and should explain the reasons for all rating demotions. Be sure to:

•Include user intent: Indicate what you think the user intent is.

•Refer to the guidelines: If the demotion is due to a guideline instruction, note the guideline section you used to make your decision.

•Include sources: If you find incorrect information in a result, your comment should include the correct information and its source. Be sure that any links you include lead directly to the information you provide. If they don’t, include a link to the source you used and explain how to get to the information from there.

•Use URL shorteners when inserting long links into your comments

•Be concise: Don’t write elaborate comments. Be short and specific.

•Comment in English: Your comments will sometimes be reviewed or used by people who are not familiar with your test locale or its language. Comments in English can be understood by the widest possible audience.

•

Issue

.Expectations

Example Rating and Comment

Relevance Demotions

The comment should explain the reasoning behind your rating. A demotion may be due to distance from the user’s location compared to similar results/ potential results, secondary intent, or other reasons.

Relevance -> Good (Distance/

Prominence)

Comment: Demoted -1 for a few closer real world locations in the Middletown/

Anchorage area

Data Issues

Include a comment when you rate a Name, Address, or Pin something other than Correct or Perfect or you give a rating of Closed/Does not Exist or Unexpected Language.

Address Accuracy -> Incorrect — Street

Number

Comment: Correct street number is: 332 W El Camino Real, Sunnyvale, CA 94087 as found on the official site: https:// www.patelco.org/locations/sunnyvale

Pin Accuracy -> Wrong

Comment: Wrong pin: correct location is:

52.368067, 4.895389

2.Understanding User Intent

Before evaluating any results, you’ll need to understand what the user wanted to find when they typed in their query. To determine user intent, use a combination of your own research and local knowledge plus the information found in the rating interface, which includes:

•User location

•User viewport

•Viewport age

User intent refers to the interaction between query, user location, user viewport, and viewport age.

2.1. Query Types

The queries you are evaluating are real queries submitted by users. Because they are real, they can:

•Be misspelled

•Contain mistakes

•Be inconsistent

•Be ambiguous

•Be badly formatted

•Be, or contain, emojis

•Have no maps intent

The first step in the rating process is to make sure you understand the query.

If you see a foreign-language query, do not release the survey. Research the query or use an online translation tool to find out what it means, then rate it as usual.

You will see several types of queries, including:

Address queries

Address queries contain all or part of a complete address, including street number, street name, locality, state, country, and postal code.

Query	Explanation
[717 E El Camino Real, Sunnyvale, CA 94087]	This is an address query in the form of a full address. It contains a street number, street name, locality, state and postal code.
[Stevens Creek Blvd, Cupertino CA]	This is an address query for a street. The user is looking for the location of this particular street.
[Ireland]	This is an address query for a country. The user is looking for the location of the country of Ireland.
[New York]	This ambiguous query could refer to New York city or New York state. You will find more information on prominence, user location, and the viewport in the following sections to help you determine the intent of this query.

Point of Interest (POI) queries

A point of interest (POI) is a location that people find interesting or useful. POIs can be locally or internationally known. These guidelines use a very broad definition of POI in which businesses are also considered POIs.

Query	Explanation Explanation
[London Bridge]	This is a query for a famous POI in London.
[Danube River]	This is a query for a river in Europe.
[Charing Cross Station]	This is a query for a transit station in London.
[Union Square, SF]	This is a query for a location in San Francisco.

Business queries

Query	..Explanation
[Zola Palo alto]	This is a business query with a location modifier (Palo Alto). The user is looking for a specific restaurant (Zola) in Palo Alto, CA. Queries can include location modifiers, like city names, that make them more specific.
[Bookasaurus]	.This is a business query with a request for a specific store in Sunnyvale, CA. The viewport and user location can help pinpoint the desired location for such a query.
[Starbucks, 7 Boulevard Poissonnière, 75002 Paris, France]	This is a query for a chain business followed by a specific address. Chain businesses are businesses that have more than one location, and include everything from national chains, like Starbucks, down to small local chains, like Burrito Boyz. Target, Auchan, Albert Heijn, and Boots are all chain businesses.
[Target sunnyvale]	This is a business query with a request for the chain business Target and location modifier of Sunnyvale, CA.

usiness queries contain the name of a specific business.

Category queries

A category refers to a group of entities that share a set of characteristics.

Query	Explanation
[fast fod]	This query is for locations where fast food is served. Assume the user wanted the correct version of the query when the query contains a simple misspelling.
[bus stop]	This query requests the locations of bus stops.
[coffee shops]	This query requests the locations of businesses that sell coffee.
[gym]	This query requests the locations of fitness centers.
[gas San Francisco]	This query requests the locations of gas stations with a location modifier of San Francisco.

Product and Service Queries

A product or service query is a query about something that can be purchased at or is offered by a business or POI.

Query	Explanation
[vanilla latte]	This query requests the locations of businesses that sell a specific type of coffee drink.
[deep-tissue massage ]	.This query requests the locations of businesses that offer a unique style of massage.
.[school supplies]	.This query requests the locations of businesses that sell pencil, pens, notebooks, and everything else students need for school.
[truck tire repair]	This query requests the locations of businesses that offer truck tire repair services.
[ATV tours]	This query requests the locations of businesses where ATV tours are available.
[perms]	Not every hair salon offers perms, but this query requests the locations of ones that do.
[deep dish pizza detroit]	This query requests the locations of places that sell a specific type of pizza in a particular city.

Coordinate and “My location” queries

Some queries are simply a set of coordinates or the words ”my location” (or something similar).

Query	Explanation
[36.082857, -115.172916]	The user has specified a location using lat/long coordinates.
[my location]	The user has typed these (or similar) words to find out where they are.

Emoji Queries

Some queries are not made using words. Emojis represent the category they picture. Use the most literal meaning.

Query	Explanation
⛽	The fuel pump represents a search for places to get gas.
🍕	The pizza slice stands for the query “pizza.”
☕	The coffee cup represents the query “coffee.”

Queries with no maps intent

Some queries have no maps intent. They include, but are not limited to, queries that:

•Do not refer to a physical location

•Do refer to a physical location but have an information intent rather than a location intent

•Refer to a brand or company with a predominantly online intent, like Groupon or LinkedIn

•Ask about the time or weather in a specific location Rate all results for these queries Bad.

Query	Explanation
[eureka temperature]	This is a weather query with no maps intent. Rate all results Bad.
[time in new york city]	This is a “time in” query with no maps intent. Rate all results Bad.
[facebook]	This is a query about a company that is mainly used online. It does not have maps location intent. Rate all results Bad.
[is cucumber a fruit or a vegetable]	This is an information-seeking query that has nothing to do with finding a location. Rate all results Bad.

2.1.1. Research Expectation

Research will help you understand the user’s query and intent:

•Use a search engine to investigate and understand the query intent

•Use official resources like businesses’ official websites, national postal service websites, and government websites whenever possible

•Use your own local knowledge

•Use the information provided in the rating interface

2.2. Result Types

A query can return one of three types of results.

2.3. Location Intent

A result is considered most relevant when it is located in the user’s area of expected results. There are two types of Location Intent:

•Explicit: The query is made clearly and indicates a specific location or area where results are expected.

•Implicit: The location expectation is not as clear, so you must use context clues, in the form of user location and viewport, to discover the area in which the user is expecting results.

2.3.1. Explicit Location

Query	Expected results	Notes
[bubble tea tully road san jose]	Places to buy this drink on Tully Road in San Jose.	User and viewport locations should be ignored.
[Boston museums]	Museums in the city of Boston	User and viewport locations should be ignored.
[kfc Philadelphia]	The queried chain business within the stated locality.	User and viewport locations should be ignored.
[food near me]	Relevant POIs near the user’s location	Viewport location should be ignored, even if fresh.
[nearby/nearest starbucks]	Relevant POIs near the user’s location	Viewport location should be ignored, even if fresh.
[my location]	Result placed on the user’s location	Viewport location should be ignored, even if fresh.

When there is an explicit location mentioned in the query, you can ignore the user and viewport locations. The user has told you exactly where they want to find results. If the query contains words like “near me” or “nearest,” the user’s location, not the viewport location, should be considered the explicit location intent.

2.3.2. Implicit Location

If the user’s location intent is not expressly stated in the query, for example [chinese] or [479 margarita ave], use the user location, viewport location, and viewport age to determine the area of expected results.

Viewport	User Location	Location Intent
. Fresh	Inside Viewport	When the user is within a fresh viewport, take the user location as location intent. Results are generally expected in or near the viewport, and results inside the area cannot be rated Bad because of distance alone.
	Outside Viewport	Results are expected in or near the viewport area. All relevant results inside the viewport are eligible for a rating of Excellent. If no results can be found in or near the viewport, consider the user location a secondary location intent.
	Missing	When the user is missing, the viewport is the location intent.
Viewport	User Location	Location Intent
Stale	Inside Viewport	When the viewport is stale, consider only the user location as location intent. .
	Outside Viewport
	Missing	.Use the stale viewport as location intent when the user location is missing.
Viewport Age Missing	Present or Missing	Consider the viewport fresh when the viewport age is missing.
Missing entirely	Present	The user location sets the location intent when the viewport is not present.
Missing entirely	Missing	When the user location and viewport are missing, the test locale becomes the location intent with a strong focus on prominent results.

3.Rating the Query-Level Navigational Result Question

Once you have identified user intent, you will need to answer a query-level question. You must decide whether the intent is unique and clear enough that there is only a single (navigational) result in the real world that could fully satisfy it.

You will answer this question before you rate any results, since you don’t need to know what the results are in order to learn if there’s a real-world result that could fully satisfy the user’s intent:

•If there is a real-world navigational result for the query, answer Yes

•If there are no possible real-world navigational results for the query, answer No

You will answer this question once for each query, whether or not any results are shown.

4. Rating Result-Level Issues

For each result, you must check to see whether:

•There is an issue with the language/script of the result name/title

•The business/POI is closed or does not exist

4.1.Result Name/Title in Unexpected Language or Script

A result name or title is in an expected language or script when it is:

•In the language/script of the test locale

•In the language/script of the query

•In a language/script of the result region

•Any combination of any of the languages or scripts described above

•An official company, chain, or brand name commonly used in the market, even if this is not any of the languages or scripts described above

Below Left: Result name is Walmart Supercenter (in yellow box), address details are below (in green box).

Below Right: Result is an address, so the first line of the address appears in the yellow box as the Address title. All the address details appear below it in the green box.

When all or part of a result name or title is in an unexpected language or script, users will not understand it. In these cases, select the Result name/title is in unexpected language or script checkbox. No further rating will be required.

Note: When the address details, found below the name/title, are in an unexpected language or script, the result can still be understood and rated for relevance and name accuracy. For more on what to do

when the address details (not the name or title) are in an unexpected language or script, see Language/Script Issue in Address.

Minor Differences in Spelling

Don’t consider minor spelling issues, such as missing or added diacritics (e instead of é), as

Unexpected Language/Script. This also applies to unnecessary or less relevant parts of a name or title. Use this checkbox only to report issues that interfere with understanding the name/title and issues listed in your Country Specific guidelines.

4.1.1. Name of Business/POI Results

..Pay special attention to the language and script of brand and chain names. See your Country Specific guidelines for more information.

Expected Language

Explanation

Example

Query

The query gives clues as to the language in which the user expects results. Any result that matches the most specific part of the query or corrects a spelling issue in the query is considered to be a result in an expected language. This also includes remote results in foreign countries.

But: Consider official result name variations and default to the test locale for very short queries.

Expected language/script:

•Test locale: en_US

•Result in California, USA

•Query: [mystery]

•Result name: Mystery Spot

Result name is in English and satisfies the query.

Unexpected language/script:

•Test locale: en_US

•Result in California, USA

•Query: [market]

•Result name: ေစျးကွက်

The result name is “Market” in Burmese script.

Test Locale and

Query Locale

The language of the test locale is generally considered an expected language and script.

If the query is so short that the language cannot be identified, assume that the query language is that of the test locale.

When the test locale and the query locale differ, consider the query locale an expected language, as it provides more detailed information.

In the example below, the Query

Locale is set to English in Indonesia. Therefore English is an expected language: • Query Locale: en_ID

•Test Locale: id_ID

Expected language/script:

•Test locale: es_ES

•Result in California, USA

•Query: [college]

•Result name: Colegio Skyline

The result name is in Spanish, which matches the language of the test locale, so this result title is expected and acceptable.

Unexpected language/script:

•Test locale: en_US

•Result in California, USA

•Query: [college]

•Result name: Colegio Skyline

“Colegio” is in Spanish, but the test locale is in English, so the result title is in an unexpected language.

Expected Language

Explanation

Example

Result region

The language/script of the result’s region is expected as the result language.

Expected language/script:

•Test locale: en_US

•Result in Rio de Janeiro, BR

•Query: [beach]

•Result name: Praia de Copacabana Result satisfies the category “beach” and the result’s name is in Latin script and the Portuguese language, which is the language of Brazil.

Unexpected language/script:

•Test locale: en_US

•Result in Rio, BR

•Query: [beach]

•Result name: Playa de Copacabana Though the result name is in the same script as the test locale and query language, returning the name in Spanish is unexpected language because the language of Brazil is Portuguese.

Expected Language

.Explanation

Example

Brand, chain, and business names

Any official brand, chain , or business name in the language of the result’s region or the official brand name commonly used in the market, even if it is not in the language of the query, test locale, or result region, is expected.

In bilingual areas, official names in any of the languages of the area are expected, unless otherwise specified in

the Country Specific Guidelines.

Expected language/script:

•Test locale: ar_SA

•Result in Riyadh, SA

•Query: [زو]

•Result name: McDonald’s

The name of this restaurant is in the language of the official brand name.

Expected language/script:

•Test locale: ar_SA

•Result in Riyadh, SA

•Query: [زو]

•Result name: [زو]

The name of this restaurant is in the language of the region

Unexpected language/script:

•Test locale: ar_SA

•Result in Riyadh, SA

•Query: [زو]

•Result name: [マクドナルド]

The name of this restaurant is in Japanese, which is not the language of the query, the locale, the result region, or the official brand name.

4.1.2. Title of Address Results

Unlike business/POI results, address results do not have an actual name. Instead, the first line of the address appears as the title. Rate Result name/title is in unexpected Language or Script when the title (first line of the address) is in an unexpected language or script.

Localized and Non-Localized Components

Address results often cannot be translated and are accepted in the original version of the result location. Some features, however, are translated into the test locale’s language, especially for bilingual or English-speaking areas. Although the preferred language is the test locale’s language, the query language and language of the result’s location are also accepted.

Localized components may include:

•City

•State

•Country

Appropriate localization depends on the result itself and the language it is translated into. Research and local knowledge will help you decide if the translation has been done appropriately.

Components that are not typically localized include:

•Feature type (square, bridge, roundabout, or intersection)

•Street names and street addresses including building names

Use the Address Accuracy Incorrect – Language/Script Issue checkbox if you find issues of this nature in any of the address components in the address details (not the address title) of any result.

4.2. Business/POI is Closed or Does not Exist

Use this checkbox when, after careful research you determine that a result is:

•Closed/non-existent

•Randomly moving (like a food truck without a fixed schedule)

•A non-recurring past event with no further significance that does not carry any maps intent

Important: If you rate a result as Business/POI is closed or does not exist, you must still give a relevance rating as if the place were open or did exist.

Remember, this rating can only be applied to business or POI results, not to address type results.

Use this checkbox to indicate a POI has closed only if there is evidence to prove the business indicated in the result is no longer operational.

When you can’t make a decision about business closure due to lack of resources, assume the business could exist and rate the name, address, and pin accuracy as Can’t Verify (unless there is an obvious data issue like a missing mandatory address component or a pin in the ocean).

Note: Businesses and POIs can close for construction, remodeling, vacations, or other reasons. Treat all temporarily closed businesses as if they were open when the closure is announced on the business/POI webpage or managed social media page. There is no limit to how long the closure can be.

4.2.1. Closed/Does Not Exist vs. Inaccurate Name and Address

It is important to understand what the result really is before making your rating. If your research reveals that the business has an incorrect name or address, use the drop-down rating options to signal precisely what is wrong. Use the Business/POI is closed or does not exist checkbox only for the specified purposes. See Unclear Results for more guidance and examples.

4.2.2. Rating Relevance of Closed/Non-Existing Business/POI

Special consideration is required when rating a closed/non-existing business/POI. Please review section 4.3 carefully with regards to the status and permanently closed vs. open.

Never automatically demote to Bad simply because of closure. (You may find other reasons to demote to Bad, but closure should not be the sole reason for the demotion.)

4.3. Business/POI Status is PERMANENT_CLOSURE

The following chapter offers a high-level introduction to rating closed/non-existing businesses/POIs in relation to their status (displayed as part of the result information). It addresses Relevance and Data rating. Please also review the more specific guidance in the associated relevance chapter.

Note: No matter what is displayed in the status section of the tool, you must always research the business/POI to determine whether it is open or not.

This chart shows the possible combinations of status displayed in TryRating plus a result’s actual state based on your research.

.Real-World State Confirmed by Research	TryRating Status	Comments
Open/Exists	None	Rate as usual
Closed/Does not Exist	None	Rate as usual
Real-World State Confirmed by Research	TryRating Status	Comments
Closed/Does not Exist	PERMANENT_CLOSURE	....Rate relevance based on whether result is expected/ unexpected. See 5.19.
Open/Exists	PERMANENT_CLOSURE	Rate relevance based on whether result is expected/ unexpected. See 5.19.

4.3.1. No Status Shown

When no result status shown or the status is blank, you must research to determine whether the business/POI is open or closed/does not exist in the real world. Use the Business/POI is closed or does not exist checkbox when needed.

Rate the result relevance as if the business/POI were open or did exist. Use your best judgement to determine how well the result satisfies user intent within the area of expected results.

4.3.2.Status Shown as PERMANENT_CLOSURE

When the result status is PERMANENT_CLOSURE, follow the research steps to determine if the business in fact is open in real world or not, and use the Business/POI is closed or does not exist checkbox when needed.

The relevance rating for a result with PERMANENT_CLOSURE status depends on whether this result is expected or unexpected based on the specific relevance guidance in 5.19. Rating Relevance when Status is set to PERMANENT_CLOSURE and 10.8. Permanently Closed Results

5. Relevance

Relevance for each result is found by assessing the relationship between:

•Satisfying user intent

•Prominence

•Distance

Complete each step of this workflow before assigning your relevance rating.

General Rating Rules

•Rate each result individually and demote based on relevance and distance.

•Always rate against the real world: If there is a better result available but it is not shown, demote the existing result(s) considering the missing one(s).

•Ignore the order of the results and evaluate each one individually.

•Ignore duplicate results and rate them individually. Do not demote for repetition.

•If there are results that are extremely inappropriate or illegal, rate Bad. Identify such content by determining if the result is likely to be embarrassing if seen by users.

Rate the relevance of each result without considering the accuracy of the name/category, address, or pin.

When judging distance/prominence, ignore closed or nonexistent results.

Adult Content

Businesses and POIs with legal adult content are rated:

•Navigational to Bad if the query clearly requests this type of content

•Bad if the query does not imply such an intent

5.1. Query-Result Connection

For a result to satisfy a query to any degree, there must be some kind of relationship between them. Your first task is to understand what this connection is.

Satisfying the user intent is only the first step in the rating process and covers the minimum requirements for a result to be considered relevant.

5.1.1. General Connection

.User Query	Result	Type of Connection	Description
[marriot]	San Francisco Marriott Union Square	General	Satisfies Intent: Users are likely to start their search with the name of the brand “Marriott”. Result satisfies user intent by providing a Marriott close to a user/viewport located in the San Francisco area.
[Houston airport]	George Bush Intercontinental	General	Satisfies Intent: The user is looking for an airport in the city of Houston. This is an airport in Houston, which satisfies user intent.
[london, brighton]	London	General	Satisfies Intent: The query contains two different cities, so it’s likely a routing query. Returning either of the individual results is expected and should be rated Excellent.

A result can satisfy user intent for multiple reasons, but most often the result satisfies the most specific part of the query’s intent. This kind of relationship between query and result is called a General Connection.

5.1.2. Abbreviation/Alternate Name Connection

The query is an abbreviation or alternate name and the result provides the full and currently accurate name.

User Query

Result

Type of Connection

Description

[ewr]

Newark Liberty

International Airport

Abbreviation

Satisfies Intent: The international airport code for Newark Liberty International Airport is EWR.

[sears tower]

Willis Tower

Alternate

Name

Satisfies Intent: The result is the correct name of what was formerly known as the Sears Tower.

5.1.3. Category Connection

The query is a category and the result satisfies that specific category, which may or may not be displayed within the information provided in the result.

User Query	Result	Type of Connection	Description
[food]	La Ciccia	Category	Satisfies Intent: A category search for “food” in a viewport over San Francisco returns an Italian restaurant there.

5.1.4. Spell Correction Connection

The query is misspelled and the result corrects that issue. This kind of connection only applies to obvious issues when no result for the actual query exists at the user’s location or when the user intent is clearly satisfied by the spell-corrected results.

This kind of correction also happens when a query address does not exist but the result address does, correcting the issue and creating a match.

User Query	Result	Type of Connection	Description
[aple store]	apple store	Spell Correction	Satisfies Intent: The result corrects the user’s spelling mistake.
Lodz	Łódź	Spell Correction	Satisfies Intent: This query suggestion for the Polish market adds the diacritics the user may have missed when typing the query.
[23 Clair street]	23 Clair Boulevard	Spell Correction	Satisfies Intent: When there is no “Clair St” within or around the viewport/user, consider a “Clair Boulevard” that is close to the viewport/ user as the most likely intent.

5.1.5. Transit Intent Connection

The result satisfies the transit intent of the query. If the query indicates a specific transit system, the result must match that system. There is also a transit intent connection when a station provides access to more than one transit service and the queried one is included.

User Query	Result	Type of Connection	Description
[bart]	Richmond BART Station	Transit Intent	Satisfies Intent: Result satisfies user intent by providing a close station that runs the requested transit service for a user/viewport located in the Richmond area.

5.1.6. Special Character Connection

Always consider the specific language conventions. In general, user intent is satisfied when:

•The query contains a special character and the result is a valid variation (if applicable).

•The query does not include a special character but the result with special characters adds value to the user and is therefore considered spell corrected.

•The country or language does not have the special character and it is generally perceived as a form of expression or style (like “café” in English).

User Query

Result

Type of Connection

Description

[möllersdorf]

..Moellersdorf

Special Characters

Satisfies Intent: “oe” is a valid variation for the special character “ö” in the country/language of the result (Germany/German).

5.1.7. Address-Result Connection

..When a query address and a result address are not exactly the same, the kind of connection they have depends on their relationship:

•Street number is the same in both query and result but the unit number is different or missing:

•If neither address is a street extension, rate result relevance Good when: ▪ The query contains a unit number and the result does not.

▪ The result contains a unit number and the query does not.

▪ The query contains one unit number and the result contains another.

•The query is full address including street number and name and the result is the street name only:

•Since this result is an unlikely secondary intent, rate relevance as Acceptable.

•Query is for a street [Main Street, Pleasanton, CA] result is just the locality (Pleasanton, CA).

•Rate the result relevance Bad as it does not satisfy the user intent.

User Query	Result	Type of Connection	Description
[357 western ave suite 1]	357 Western Ave St. Johnsbury, VT	Address	Satisfies Intent: The result does not include the unit number requested in the query so its relevance is rated Good.
[357 western ave suite 1]	357 Western Ave Suite 2 St. Johnsbury, VT	Address	Satisfies Intent: The result has the same street number but a different unit number from the address in the query, so its relevance is rated Good.
[357 western ave suite 1]	Western Ave St. Johnsbury, VT	Address	Satisfies Intent: Instead of returning the full address requested, the result provides only the street name. This is an unlikely secondary intent, so relevance is demoted to an initial rating of Acceptable.
[357 western ave st. johnsbury]	St. Johnsbury, VT	Address	..Does not satisfy user intent: Instead of returning the full address requested, the result provides only the locality. This does not satisfy the user intent, so relevance is rated Bad.

5.1.8. Lack of Connection

.If the result does not satisfy the user intent either because there is no relationship between the query intent and the result or because the connection between the query intent and result will not be immediately obvious to the user, rate the result relevance Bad.

User Query	Result	Type of Connection	Description
[airport]	Santa Cruz Boardwalk Santa Cruz	None	Does not satisfy user intent: Research shows that there is no connection between the query and the result. Rate Bad.
[Raging Waters 2333 South White Rd San Jose]	2333 South White Rd San Jose	None	Does not satisfy user intent: Result is for the correct address, but does not include the business named in the query, so users will have no way of knowing if this is truly connected to the business or not. Rate Bad.
[Raging Waters]	2333 South White Rd San Jose	None	Does not satisfy user intent: Query is the name of a business and result is for the correct address, but does not include the business name. Users will have no way of knowing whether or not this address is connected to the business they asked for. Rate Bad.
[valley fair mall]	Macy’s 2801 Stevens Creek Blvd, Santa Clara	None	Does not satisfy user intent: Result is for a store, not a mall. This store is inside the requested mall, but does not satisfy the intent of the query, which is the whole mall. Rate Bad.
[macy’s]	Westfield Valley Fair 2855 Stevens Creek Blvd, Santa Clara	None	Does not satisfy user intent: Result is for a mall, not a store. This mall contains the requested store, but this result is the whole mall, not the single store the user asked for. Rate Bad.
[costco]	Costco Gasoline 1601 Coleman Ave, Santa Clara	None	Does not satisfy user intent: Result is for a Costco gas station, not a Costco store. The store and the station are at the same address and share a brand name, but the result is not what the user asked for and does not satisfy the query intent. Rate Bad.
[costco gas]	Costco 1601 Coleman Ave, Santa Clara	None	Does not satisfy user intent: Result is for a Costco store, not a Costco gas station. The store and the station are at the same address and share a brand name, but the result is not what the user asked for and does not satisfy the query intent. Rate Bad.

5.2. Satisfying User Intent

The user’s intent can be determined by considering the query, the user’s location/viewport, and your local knowledge. Sometimes queries are ambiguous or can have multiple interpretations.

In order to determine a query’s primary and secondary intent, look at the relationship between the query and the result. Ask yourself:

•Is there a logical relationship between the query and the result?

•How likely is the user to be looking for this result given the query and the location intent?

Intent Type	Definition	Highest Initial Rating
Primary Intent	Result satisfies the most obvious and likely user intent	Navigational or Excellent
Secondary Intent	A result which is less likely to be the user’s intent. Results matching secondary intent are often not as prominent as the originally queried entity but still satisfy the intent of the query.	Good (User Intent)
Unlikely Intent	A result which matches the query but is very unlikely to be the user’s intent.	Acceptable (User Intent)
Non-Relevant Intent	There are issues that make the result useless for the user.	Bad (User Intent)

For any Relevance Rating of Good or below, you must select the appropriate check box(es) to indicate the reason(s)for demotion: User Intent and/or Distance/Prominence. If both reasons apply, use both checkboxes.

When determining intent, keep in mind:

•It is rare that a navigational query will have multiple primary intents.

•Rating one result Navigational and one Excellent within the same set of results is also rare, but possible (see examples in Navigational Results for Category Queries).

•A navigational query will most often produce one result that satisfies the distinct intent and multiple other results that satisfy a secondary intent or do not satisfy the intent at all. This means there will be one result rated Navigational and all other results will be rated Good or Acceptable for secondary intent or Bad for not satisfying user intent.

•Most non-navigational queries will produce results that are rated from Excellent to Bad.

•Since the queries were made by actual users, there will be cases where a query is gibberish or has no maps intent. Do your best to understand it and see how well the results fit the user’s intent, but don’t be shy about assigning a Bad rating if a result doesn’t fit.

Satisfying the user intent is only the first step in the rating process and covers the minimum requirements for a result to be considered relevant. The rating determined by deciding how well intent is met is called the Initial Rating. Distance and prominence will be considered next in order to establish the final rating.

5.3. Prominence

Once you’ve made an initial rating by establishing that there is a relationship between the query and the result and determining the intent type, consider the result’s prominence.

The prominence of a feature refers to its popularity, including the number of people visiting and media sources referencing it. Prominence can vary based on the test locale and even local knowledge. Consider the following list to get a general idea how to establish prominence, ordered from the most prominent to the least prominent:

•The feature is known internationally

•The feature is known in the country

•The feature is known in the region

•The feature is known locally

•The feature may not even be known locally

A result that may not directly appear to be user intent can be promoted for being internationally prominent.

Query	Result	Rating and Explanation
[Sydney]	Sydney, Australia	Relevance	Navigational
		The city of Sydney is clearly the user’s primary intent.
	Sydney Opera House Sydney, NSW, Australia	Relevance	Good (User Intent)
		The Opera House does not satisfy primary intent. However, it is promoted to secondary intent due to its international prominence.
	Sydney Town Hall 483 George St, Sydney, NSW, 2000 Australia	Relevance	Bad (User Intent)
		The Sydney Town Hall does not satisfy primary intent. While the civic center is historically important, it is not internationally prominent and will not be promoted to secondary intent.
.	Sydney Buses Depot Randwick 34-36 King St, Randwick NSW 2031, Australia	Relevance	Bad (User Intent)
		The depot matches the query string but is unlikely to be the user’s intent and has very low prominence given how many other internationally prominent POIs Sydney has to offer.
.	Agra Uttar Pradesh, India	Relevance	.Navigational
		....Query is for a small locality in India and the result satisfies the user intent.
Query	Result	Rating and Explanation
[Agra] .	Taj Mahal Agra, Uttar Pradesh, India	Relevance	Good (User Intent)
		Result is for a prominent site in the queried locality. The Taj Mahal does not satisfy the user’s primary intent. However, it is promoted to secondary intent due to the international prominence of the feature in the queried locality.
[japan] (User is in Seattle)	Japan	Relevance	Navigational
		The country of Japan is clearly the user’s primary intent.
	Sarku Japan 401 NE Northgate Way Seattle, WA 98125	Relevance	Bad (User Intent)
		A chain restaurant with “Japan” in its name does not satisfy the query’s primary intent (the country of Japan) or any secondary intent (entities with international prominence).

5.4.Distance

Generally, the farther away the result is from the area of expected results, the less desble it becomes. This is especially true for queries highly driven by distance to the user/viewport, such as chain businesses, hospitals, pharmacies, or grocery stores.

Because the user can be offered many possible results that are all very similar, the closest entities providing the expected service can be considered the best options. Results which are farther away are less relevant and should be demoted according to their additional distance. Defining what is close and what is far depends on the context of each individual query. Factors affecting distance include:

•Number of possible results in the real world

•Distribution of all possible results

•Population density (rural, urban, or suburban)

Distance refers to the direct distance from one point to another and is measured via a straight line. There is no need to account for the actual distance required to travel from one point to the other, such as driving distance.

Do not demote a result for distance when:

•This is the closest possible result, even if it is far away from the area of expected results.

•Other, closer results are found to be closed or non-existent.

Demote a result for distance when:

•Other results that satisfy the query and provide the same or similar service are closer.

5.5. Distance vs. Prominence

Once you’ve decided how relevant the result is based on user intent, consider the result in the context of all possible results in the real world. Use the distance to the user or viewport and the prominence of the result to decide if a demotion should be applied.

A result’s fit to user intent, prominence, and distance should all be relative to the query and the context around the query. Before deciding whether to demote because of prominence or distance, it’s important to look at:

•The type of area around the result – rural, urban, or suburban

•The number of businesses that satisfy the query

•The distance to the user or viewport

The following examples of multiple query/result pairs will demonstrate how to interpret the relationship between distance, prominence, and satisfying the user’s intent. The examples address common rating situations and explain how to evaluate the relevance of the results.

5.6. Many Possible Results

Some queries, like [starbucks], have many results that could satisfy the user. In cases like this, distance is more important than prominence, so results that are closer to the user’s location/viewport should be rated higher than results that are farther away. To illustrate the high density of Starbucks, possible results are shown as purple dots.

User Inside Fresh Viewport

When the user is inside the viewport, consider all possible results, including real-world results not returned for rating (purple dots), and demote by distance from the user.

User Outside Fresh Viewport

....When a user is outside the viewport, results are expected in or near the viewport. In cases where there are a large number of possible results inside the viewport, even if they are not returned for rating (purple dots), rate Bad for those outside it.

5.7. Few Possible Results

When the query has only a few results in the real world, be more lenient when considering distance. Because there are only a few Zara clothing stores in Miami, results for the query [zara] are demoted more leniently for distance than the many Starbucks locations in the example above.

5.8. Few Possible Results and Greater Distance

Leniency on distance is acceptable when there are only a few possible results for the query in the requested area. For example, a user close to Berlin queries for [Wartestraße], which is a street name. There are only four streets in Germany called “Wartestraße” that can satisfy the user’s intent and all four streets are returned as results. Because of the limited number of results, each result’s relevance remains fairly high even over longer distances.

5.9. Rural Areas

Results are often farther away in rural areas, but can still be considered relevant if there are no reasonable results close tr within the fresh viewport. In a rural area, generosity can be applied to ratings with respect to distance. The criteria for rating are similar to the [zara] example in the Few possible results section, but with an emphasis on distance. Here is an example for the query [american legion] in North Dakota.

5.10. Location – User Intent Deviation

When the location intent and user intent do not allow for a clear result (because either the user or viewport are in an atypical location, like the middle of the ocean), rate with a focus on high prominence. For example, the query [Hong Kong] in a test locale set to en_US with the viewport and user location around Naples, Italy. There is no real change in rating if the user is inside or outside of the viewport, or if the viewport is fresh or stale.

5.11. Rating for User Inside and Outside Viewport

The following two examples show the differences in rating when a user is inside and outside a fresh viewport. The query is for a chain business called Great Wolf Lodge found across the United States. The examples show all possible results within and around the large viewport.

In general, when the user is outside the fresh viewport, no result within the viewport is demoted for distance. The only exception to this is when the viewport is large enough to include an unusually large area, like an entire continent, and the results within it would not realistically be useful or interesting to a user with this query.

(Note that even though the viewport in the example below is large, because of the limited number of

Great Wolf Lodges with it, the results still might be useful or interesting to a user with this query.)

When the user is inside the fresh viewport, the user’s location is used as location intent and results are demoted for distance.

5.12. Partial Address Does Not Exist

Relevance is always rated independently of any data (name, address, or pin accuracy) issues the result may have. This includes results for addresses that do not exist in real life.

When judging distance for existing addresses do not factor in any returned non-existent addresses. Evaluate distance separately for existent and non-existent addresses. Do not demote existing address results when closer, non-existent addresses are returned.

For example, a user in Half Moon Bay, CA is looking for [2373 chestnut] and is presented with three residential addresses. The address in San Francisco is the only one that actually exists. The results in Oakland and Redwood City are not supported by the postal authorities and no evidence can be found that the addresses in either city exist.

Note: For cases where the query is a unique full address that does not exist and the result matches the query, see Query Address does not Exist.

5.13. City vs. Municipality

In many cases, a city and a state/county/municipality (or other similar market-specific components) have the same name, but the city is typically more prominent. In this case, the chances that a user is looking for the state/county/municipality and not the city, are very small, but still a possibility. Therefore, states, counties, and municipalities will be treated as a secondary intent and given an initial rating of Good. Consider the relationship between the city and the municipality in terms of their locale and adjust if necessary. On the other hand, if the query specifically requests the state/county/ municipality, that result’s relevance should be rated as Navigational and the city should be rated Bad.

See your Country Specific guidelines for more information.

Query	Result	Rating and Explanation
[new york]	New York City New York, USA	Relevance	Navigational
		The result is for the city, which is considered more prominent than the state.
	New York State New York State, USA	Relevance	Good (Distance/Prominence)
		Result is for the state, which is less prominent than the city.
[santa clara]	Santa Clara Santa Clara, CA	Relevance	Navigational
		Query is for the city, which is considered more prominent than the county.
	Santa Clara County Santa Clara County, CA	Relevance	Good (Distance/Prominence)
		Query is for the city, which is considered more prominent than the county.
[santa clara county]	Santa Clara County Santa Clara County, CA	Relevance	Navigational
		Query is specifically requesting the county, so the result for the county will be rated Navigational.
	Santa Clara Santa Clara, CA	Relevance	Bad (User Intent)
		Query is for the county and a result for the city is considered irrelevant for the query.

5.14.Unexpected Results

Straightforward queries can have unexpected results which at first glance might not fit the query intent. For these kinds of results, consider:

•Is there a logical relationship between the query and the result?

•How likely is it that the user is going to be looking for this result given the query and location intent?

A result can partially satisfy a secondary intent due to:

•Prominence of the feature

•Transit intent

Promoting for prominence should only occur for internationally prominent POIs.

..In the examples below, two ratings are provided for some results: Good and Acceptable. The difference between the two ratings rests in how well the result satisfies user intent. Choose Good if there is a strong likelihood that it will satisfy user intent and Acceptable if it is less likely to satisfy.

Query	Result	Rating and Explanation
[mission & 14th street] User and fresh viewport in San Francisco, CA	Mission St & 14th St San Francisco, CA	Relevance	Navigational
		Query is for an intersection and the result satisfies the user intent.
	Mission St & 14th St Bus Stop San Francisco, CA	Relevance	Good/Acceptable (User Intent)
		Result is for the bus stop at the queried intersection. The bus stop carries the name of the intersection and users tend to look for intersections when seeking transit POIs.
[soho] User and viewport in London, UK	Soho London, UK	Relevance	Navigational
		Query is for a neighborhood in London and the result satisfies the user intent.
	Mildreds Restaurant 45 Lexington St, Carnaby, London W1F 9AN, UK	Relevance	Bad (User Intent)
		Result is for a specific restaurant in Soho. It is very unlikely to satisfy the user intent.
[stevens creek blvd cupertino] User and fresh viewport in Cupertino, CA .	Happy Lamb Hot Pot 19062 Stevens Creek Blvd, Cupertino, CA, 95014	Relevance	Bad (User Intent)
		The query is for a street and the result is a specific business on that street. This specific result is unlikely to satisfy the broad intent of the query.
	19062 Stevens Creek Blvd Cupertino, CA, 95014	Relevance	Bad (User Intent)
		.The result includes a street number when the query is for a street, making this result too specific for the more general query.
Query	.Result	Rating and Explanation
[The Slanted Door One Ferry Building #3 San Francisco, California 94111]	One Ferry Building #3 One Ferry Building #3 San Francisco, CA 94111	Relevance	Bad (User Intent)
		Result is for the address and not the business requested in the query, so it does not satisfy the user intent.
[One Ferry Building #3 San Francisco, California 94111]	The Slanted Door One Ferry Building #3 San Francisco, CA 94111	Relevance	Navigational
		The query is for an address and the result is the business at this address. Including the business name provides additional correct information.
[san francisco] User and fresh viewport in San Francisco, CA	San Francisco San Francisco, CA	Relevance	Navigational
		Query is for the city of San Francisco and the result satisfies the user intent.
	San Francisco International Airport San Francisco, CA	Relevance	Good (User Intent)
		The result represents the airport within the queried locality, which satisfies the secondary transit intent.
The Pruneyard	The Pruneyard 1875 South Bascom Ave Campbell, CA	Relevance	Navigational
		The query is for a mall and the result is the mall the user asked for.
	Pruneyard Cinemas 1875 South Bascom Ave Campbell, CA	Relevance	Bad (User Intent)
		The query is for an entire mall and the result is for a cinema inside the mall. Even though the cinema shares the mall’s name and address, it does not match the query intent.

5.15. Multiple Transit POIs with the Same Name

Many large transit POIs include multiple means of transportation at the same location. For example, large airports can have metro or train service within the same location, or train stations can be a hub for trains, metros, and buses. Other transit POIs at the queried transit POI should be treated as secondary intent with an initial rating of Good, but can also be considered Acceptable if the result is less likely to satisfy the user’s intent. Additional demotions can be applied based on the prominence of the returned feature.

Query	Result	Rating and Explanation
[hamburg airport] User and fresh viewport in Germany	Hamburg Airport 22335 Hamburg, Germany	Relevance	Navigational
		Result is for the queried airport.
	Hamburg Airport Station 22335 Hamburg, Germany	Relevance	Good (User Intent)
		Result refers to the train station at Hamburg Airport.
	Lufthansa Basis 22335 Hamburg, Germany	Relevance	Bad (User Intent)
		The result is for a small bus stop that seems to service the Lufthansa training center. The result does not share the same name as the queried transit. Therefore, the result is not relevant.
[Embarcadero Station] User and fresh viewport in San Jose, CA	Embarcadero Station San Francisco, CA	Relevance	Navigational
		The query is for a unique transit station. The result (a unique BART station) fully satisfies the user’s intent.
	Ferry Building & Embarcadero Station San Francisco, CA	Relevance	Acceptable (User Intent & Distance/Prominence)
		The result (a historic streetcar) name includes the queried station name, but has low prominence, given that there is a unique Embarcadero Station. Therefore, the result should be demoted based on user intent.

5.16. POIs and Transit Intent

A non-transit POI can be associated with a transit POI with the same name. You can establish the relationship between the two entities by using local knowledge, researching prominence, and understanding how well the result satisfies user intent. This way you will learn whether the query has a distinct navigational intent or if the prominence of the two POIs that share a similar name is so strong that both can be the intent.

Query	Result	Rating and Explanation
....[mall of America] User and fresh viewport in Minnesota	Mall of America 60 E Broadway, Bloomington, MN 55425 United States	Relevance	Navigational
		Result is POI requested in the query.
Query	Result	Rating and Explanation
.	Mall of America Station 8240 24th Ave S Minneapolis, MN 55425 United States	Relevance	Good/Acceptable (User Intent & Distance/Prominence)
		Even though the station has the same name as the POI, it is not prominent and therefore demoted.
[Piccadilly circus] User and viewport in London, UK	Piccadilly Circus London, UK	Relevance	Excellent
		Result is for famous square in London. The square and the train station are both equally prominent and local users would be looking for the station while tourists would be looking for the square. As both features are equally prominent, it is hard to establish a distinct user intent.
	Piccadilly Circus Station London, UK	Relevance	Excellent
		Result is the famous station that is just as prominent as the square.
[dam square] User and viewport in Amsterdam	Dam Square Dam Square, Amsterdam, Netherlands	Relevance	Navigational
		The result is the famous square in Amsterdam.
	Dam Square Line 2 Dam Square, Amsterdam, Netherlands	Relevance	Good/Acceptable (Distance/Prominence)
		Result is for the tram stop for line 2. The stop is on a street adjacent to Dam Square. However, it is not prominent and should be demoted.

5.16.1. Transit Queries

The criteria described above should be applied to understand the transit queries. Additionally, if it is determined that a query has a clear navigational intent, all other results will be Bad (see [12th st oakland bart] example below). If a result is promoted to Navigational, other results that could potentially satisfy the user intent should be demoted further for distance and prominence (see [BART daly city] and [stockport station] examples below). In general, the fewer the results that satisfy the user intent, the farther away the results can be and still be considered relevant. The more available results that can satisfy the user intent, the closer they need to be.

Query	Result	Rating and Explanation
[12th st oakland bart] User and viewport in California	12th St. Oakland Station Oakland, CA	Relevance	Navigational
		The query is for a specific BART station with a unique name that is not the same as the name of the locality where the station is found. (Compare this to the examples below). This result matches the query exactly and fully satisfies the user’s intent for this specific station.
	19th St. Oakland Station Oakland, CA	Relevance	Bad (User Intent Issue)
		The query intent is for a specific BART station with a unique name that is not the same as the name of the place where the station is located. The result returned is for a different BART station and does not satisfy the user intent. This result should receive a rating of Bad.
[BART daly city] User and viewport in California	Daly City Station Daly City, CA	Relevance	Navigational
		The query intent is for a BART station in Daly City. Research shows that there is only one BART station in Daly City and its name is Daly City Station, so this result fully satisfies the user’s intent.
	Colma Station Colma, CA	Relevance	Good (Distance/Prominence)
		The query intent is for a BART station within Daly City. This result fits intent as it is a BART station. This station is in a neighboring locality just outside of the requested locality. Even though the station is outside of the requested locality, it provides a choice of BART stations in the area for the user. This result is demoted due to distance.
	San Francisco International Airport Station San Francisco International Airport, CA	Relevance	Acceptable (Distance/Prominence)
		The query intent is for a BART station in Daly City. This result fits intent as it is a BART station. This station is in a neighboring locality, but is farther away than the Colma station and is demoted due to distance.

Query	Result	Rating and Explanation
[stockport station] User and viewport in Stockport, England	Stockport Station Stockport, England	Relevance	Navigational
		The query intent is for a station in Stockport. Research shows that there are several stations in Stockport, but only one named Stockport Station, so this result fully satisfies the user’s intent.
	Davenport Station Stockport, England	Relevance	Excellent
		The query intent is for a station in Stockport. There is one station with a name that matches the query exactly, which receives the Navigational rating. All other stations within the queried locality should receive Excellent.
	Heaton Chapel Station Stockport, England
	Burnage Station Manchester, England	Relevance	Good (Distance/Prominence)
		This station is just outside of the requested locality, in a neighboring locality. Even though the station is outside of the requested locality, it provides a choice of stations in the area for the user. The fewer choices there are available, the farther away the results can be and still be relevant. This result is demoted due to distance.
[san jose station] User and viewport in California	San Jose Diridon Station San Jose, CA	Relevance	Excellent
		The query intent is for a station in San Jose. There are several stations in San Jose, but no stations with a name that matches the query exactly, so we should rate every station within the queried locality equally.
	.
.	Tamien Station San Jose, CA	There are two stations that match the user’s intent, so both results should be rated Excellent.
	Lawrence Caltrain Station Sunnyvale, CA	Relevance	Good (Distance/Prominence)
		This station is just outside of the requested locality, in a neighboring locality. Even though the station is outside of the requested locality, it provides a choice of stations in the area for the user. The fewer choices there are available, the farther away the results can be and still be relevant.
Query	Result	Rating and Explanation
.	Sunnyvale Caltrain Station Sunnyvale, CA	Relevance	Acceptable (Distance/Prominence)
		This station is just outside of the requested locality, in a neighboring locality. Even though the station is outside of the requested locality, it provides a choice of stations in the area for the user. The fewer choices there are available, the farther away the results can be and still be relevant.
	Palo Alto Train Station Palo Alto, CA	Relevance	Bad (Distance/Prominence)
		Result is far from the queried station. The farther away the result is from the queried station, the less likely it is that it will be considered relevant, so the relevance rating should be lowered.

5.16.. Stops vs. Stations

A transit stop is a location typically designated by a bench or covered waiting area, a posted sign, or street markings. Transit stations, on the other hand, are typically larger areas associated with major structures including, but not limited to, platforms, ticketing offices, enclosed waiting/seating areas, or terminal buildings/structures encompassing the transit area. The term “station” is primarily used to refer to transit locations that are larger and typically a terminal housing multiple routes. All stations are stops, but not all stops are stations.

Since all stations are stops, but not all stops are stations, returning a station for a stop query is ok, but returning a stop when the query is station should result in a rating of Bad with the User Intent checkbox selected. Try to identify these corresponding terms in the test locale and understand the relationship between them.

For the category [bus stops], relevance should be based mainly on distance from the user location or fresh viewport and the physical size of the bus stop. If the result is a small bus stop in a rural area and the bus stop is among the closest ones to the user’s location intent, it should be rated Excellent. On the other hand, if the result is a large bus hub that is a bit farther away from the user, it should still receive an Excellent because of its size, regardless of the fact that there are closer, smaller bus stops. Results close to the user should not be demoted because there is a larger bus stop farther away, instead, the large bus stop should be promoted because of its importance.

Query	Result	Rating and Explanation
[bus station in san francisco] User and viewport in San Francisco, CA	Greyhound Lines 200 Folsom St San Francisco, CA 94105	Relevance	Excellent
		This is a major bus station in San Francisco where a user can take Greyhound buses.
	Main St & Folsom St San Francisco, CA 94105	Relevance	Bad (User Intent)
		These results are small local bus stops and do not satisfy the user intent for a bus station.
	5th St & Howard St San Francisco, CA 94103
	Sacramento Bus Station 420 Richards Blvd Sacramento, CA 95811	Relevance	Bad (Distance/Prominence)
		This result is another Greyhound station, but it is in Sacramento, quite far from San Francisco. There are closer real world results we could return. This result is too far and should be demoted due to distance.
[bus stop] User and viewport in Chicago, IL	Chicago Bus Station 630 W Harrison St Chicago, IL 60607	Relevance	Excellent
		The query intent is for a bus stop. This result is a Greyhound station, which fully satisfies the user’s intent. This result is close to the user.
	W Harrison St & S Jefferson St Chicago, IL 60607	Relevance	Excellent
		The query intent is for a bus stop. This result is a Greyhound station, which fully satisfies the user’s intent. This result is close to the user.
	Harlem Ave & W 115th St Worth, IL 60482	Relevance	Bad (Distance/Prominence)
		The query intent is for a bus stop. This result is a bus stop and matches the user’s intent, but there are many bus stops closer to the user in the real world we should be returning instead. Therefore, this result should be demoted due to distance.

5.17. Parking Intent

When evaluating [parking] queries, note that:

•Free and paid parking are equally relevant.

•Parking time limits do not affect relevance rating. Your rating should not be changed by the fact that a parking lot is long term, short term, or limited-time (like a two-hour limit) parking.

•Parking for small cars, large cars, RVs, or motorcycles is equally relevant.

•If you can find evidence that a result is for private parking that cannot be used by the general public, give it a rating of Bad. Public parking vs Private Parking

Public parking is:

•Parking that is a mix of residential/staff/permit and visitor parking and is not in a gated community

•Parking that can be used only when visiting a certain business (limited-use parking)

Private parking is:

•Parking that is residents only, staff only, or by permit only

•Parking in a gated community

If you cannot confirm whether parking is public or private, give the result the benefit of the doubt and consider it public parking.

5.18. Relevance: Service-Level Mismatch

Sometimes a query specifically requests a certain level of service, which may or may not be reflected in the result. Common requests include:

•Specific stores (e.g. Walmart Supercenter)

•Outlet stores (e.g. J. Crew Factory)

When the query requests a specific level of service which is not reflected in the result, give an initial rating of Good and demote as necessary depending on how well the result would satisfy the user intent.

When the query requests a type of business with a generic level of service, providing a result that offers more service than requested still satisfies user intent and should receive an initial rating of Excellent.

When the query does not specify the type of service, any relevant result can be rated Excellent.

Query	Result	Rating and Explanation
[walmart]	. Walmart Supercenter 3435 East Broadway Blvd, Tucson, AZ 85716	Relevance	Excellent
		Query is for a generic Walmart and the result is a Walmart Supercenter store that offers more services than the average Walmart store. User intent has been met and there is no need to demote for service level.
	Walmart Neighborhood Market 5500 E 22nd St, Tucson, AZ 85711	Relevance	Excellent
		The query does not specify the type of Walmart, and any Walmart can be rated Excellent. Demote further for distance.
[walmart supercenter]	Walmart Neighborhood Market 5500 E 22nd St, Tucson, AZ 85711	Relevance	Good (User Intent)
		Query is for a store with specific level of service. Walmart Supercenters are very large stores that have a full range of grocery and general merchandise. A Walmart Neighborhood Market is generally smaller with a more limited array of grocery and general merchandise.
[nordstrom rack]	Nordstrom 55 E Grand Ave, Chicago, IL 60611	Relevance	Good (User Intent)
		The query is for an outlet store, but the result is for a full-price store. The result satisfies the query intent, but to a lesser degree.
[bank of america]	Bank of America 10 00 Orangewood Blvd, Orlando, FL 32821	Relevance	Excellent
		When the query is for a bank or a general bank brand, an ATM or a branch location of that specific brand are equally relevant.
	Bank of America ATM 10900 Orangewood Blvd, Orlando, FL 32821

Query	Result	Rating and Explanation
[ATM]	Citibank 3333 E Tropicana Ave Las Vegas, NV 89121	Relevance	Excellent
		The result is a bank branch location. Since many branches contain ATMs, and would be able to provide the same level of service as an ATM, this result usually fulfills the user’s intent. Note that if research indicates the result does not offer cash services, rate it Bad.
	Citibank ATM 7-Eleven 3421 E Tropicana Ave Las Vegas, NV 89121	Relevance	Excellent
		This is an ATM located inside a 7-Eleven convenience store. It satisfies the primary intent of the category.
[burger king]	McDonald’s 6875 Sand Lake Rd, Orlando, FL 32819	Relevance	Bad (User Intent)
		A user has asked for a specific fast food restaurant and the result is another fast food restaurant that offers similar food and service. Even though the service level in the two restaurants is similar, this result does not match the user intent for the named restaurant.

5.19. Rating Relevance When the Result Status is PERMANENT_CLOSURE

A business/POI that has its status set to

PERMANENT_CLOSURE requires special consideration during relevance rating.

Note: A result without any status indicated should be rated as usual, including when research indicates the business/POI is closed or does not exist.

There are some situations where you can expect to see results with the status

PERMANENT_CLOSURE and others where such results are not expected. The following sections will describe how to identify and rate them.

5.19.1. Expected PERMANENT_CLOSURE Status

Sometimes you can expect to see results whose status is shown as PERMANENT_CLOSURE.

This can happen when:

•The permanently closed result satisfies the user intent completely.

AND

•There is/are no other result(s) that satisfy the actual user intent in the area of location intent.

Examples of results where PERMANENT_CLOSURE status is expected

•Navigational queries:

•Queries that can only return a single result, which is permanently closed. This could be because the POI or business name is unique or the query contains location intents like locality or street components.

•Chain queries for closed chains:

•All chain locations within the area of location intent are permanently closed and the closest open chain result is significantly farther away.

Rating Approach

Rate results with an expected status of PERMANENT_CLOSURE as if they were open. This means that in most cases the rating will be either Navigational or Excellent, because even though they are closed, these are the best or only results that could ever be returned.

5.19.1.1.ExpectedPERMANENT_CLOSUREStatus:Navigational Query

The user is looking for a unique business in a specific location.

Result	Rating and Explanation
① 99 Bottles Of Beer On The Wall PERMANENT_CLOSURE 110 Walnut Ave, Santa Cruz, CA	Relevance	Navigational
	Business/POI Closed/Does not exist	Yes
	This permanently closed restaurant is the only result that meets the user intent, which makes it expected. Choose the Business/POI Closed/Does not exist checkbox and rate the business as if it were open.
② 99 Cents Only Stores 260 E 10th St, Gilroy, CA	Relevance	Bad (User Intent)
	The 99 Cents store does not match the user intent and is rated Bad.
⓷ 99 Bottles & Cocktails 8901 Katella Ave, Anaheim, CA	Relevance	Bad (User Intent)
	While also a restaurant with a name that starts with 99 Bottles, this is not in Santa Cruz, does not match the user intent, and is rated Bad.

5.19.1.2. Expected PERMANENT_CLOSURE Status: All chains closed in area

Chico’s has closed all of its locations in Canada. The closest locations to the user are across the border in the United States.

Result	Rating and Explanation
① Chico’s PERMANENT_CLOSURE 100 City Centre Dr, Mississauga, ON	Relevance	Excellent
	Business/POI Closed/ Does not exist	Yes
	The permanently closed location for Chico’s is in the center of the fresh viewport. Since there are no open locations anywhere close to the area, this result is expected. Choose the Business/POI Closed/Does not exist checkbox and rate the business as if it were open.
② Chico’s 5151 Main St, Williamsville, NY United States	Relevance	Excellent
	The open Chico’s closest to the user is just across the border. Because it is the closest open result, it is rated Excellent.

Result

Rating and Explanation

⓷ Chico’s

3349 Monroe Ave Ste 45,

Rochester, NY

United States

Relevance

Good (Distance/Prominence)

As the second-closest open location, the Chico’s in Rochester is demoted to Good.

5.19.2. Unexpected PERMANENT_CLOSURE Status

These are results whose status is shown as PERMANENT_CLOSURE, but are not expected to be shown at all.

Results with the status PERMANENT_CLOSURE are unexpected when there are also open results in the area of location intent that would satisfy the actual user intent fully without changing or altering the user query.

Examples of situations where results with PERMANENT_CLOSURE status are not expected

•Category queries:

•There are always a large number of possible results for category queries like [restaurant] or [hair salon]

•Note: In cases where there is only one match for the category query within a large area and it is marked with the PERMANENT_CLOSURE status, consider it an exception and choose Business/POI is closed or does not exist, then rate it as if it were open. This should be extremely rare.

•Queries with several possible interpretations:

•These are queries could have several possible results and/or result types available, and do not depend on PERMANENT_CLOSURE results.

•Most chain queries:

•There are often multiple locations of a chain available within the area of location intent, and a permanently closed result does not need to be shown in these cases.

Rating Approach

Unexpected permanently closed results should be demoted by 2 for relevance which means the highest rating starts at Acceptable.

5.19.2.1. Unexpected PERMANENT_CLOSURE Status: Open Options are Nearby

There are many options available near the user for this category query, so there is no reason to show a permanently closed business.

Result	Rating and Explanation
① Meeps Vintage 2104 18th St NW, Washington, DC	Relevance	Excellent
	Meeps Vintage matches the user intent and is close to the user within the fresh viewport.
② Miss Pixie’s 1626 14th St NW, Washington, DC	Relevance	Excellent
	Miss Pixie’s sells vintage clothing and is among the closest results for this category query.
⓷ Buffalo Exchange PERMANENT_CLOSURE 1318 14th St NW, Washington, DC	Relevance	Bad or Acceptable (User Intent)
	Business/POI Closed/ Does not exist	Yes

Result	Rating and Explanation
.	The permanently closed Buffalo Exchange did sell vintage clothing, but since there are other open stores that do this in the area of location intent, this result is unexpected. The relevance rating starts at Acceptable. Given the abundance of nearby results, this could be rated Bad, as well.

6. Name and Category Accuracy

The Name and Category Accuracy rating is used to evaluate the accuracy of a Business/POI name. Your final rating will take into consideration both the accuracy of the Business/POI name and the category assigned to it. Inaccuracies in either or both of these elements will result in rating demotions.

You’ll see this rating referred to throughout the guidelines as both “Name Accuracy” and “Name and Category Accuracy.”

Business/POI Name and Category Accuracy can be rated as:

•n/a

•Correct

•Partially Correct

•Incorrect

•Can’t Verify

If you give a final Name and Category Accuracy rating of Partially Correct or Incorrect, you may be presented with two checkboxes:

•Name Issue

•Category issue

If they appear, use one or both them to select the reason(s) for your rating.

6.1. Name Not Applicable (n/a)

The n/a rating should be used for all address type results, including residential addresses, streets, localities, and so on.

Unlike Business/POI type results, address type results do not actually have a result name. Instead, the first line of address type results appears as the result title. This is because this field is meant to present the most relevant part of the address information.

Since address results do not have a name or category, their Name and Category Accuracy cannot be evaluated. This is why the n/a rating should be selected.

Result	Rating and Explanation
1703 NW Taylor St 1703 NW Taylor St Topeka, KS 66608	Name and Category Accuracy	n/a
	This is a residential address, not the name or title of a business or POI.
1600 Pennsylvania Ave 1600 Pennsylvania Ave, Washington, DC, 20500 United States	Name and Category Accuracy	n/a
	Although this is the address of a well-known POI (the White House), it is still an address result and contains no name or title. It cannot be rated for Name Accuracy.
Paris, France Paris, France	Name and Category Accuracy	n/a
	This is the name of a locality and country, not the name of a business or POI.

6.2. Result Names

Your final Name and Category Accuracy rating will take into consideration both the accuracy of the Business/POI name and the category assigned to it.

First, consider the name.

6.2.1. Correct Name

A business/POI result name is accurate when it is used on the POI’s official website or on other official resources. Even if the official website does not use the name, if it is used on other official sources, the name is correct. Note that the name has to be used to refer to the particular POI. The corporate name by itself is not automatically the correct name.

Business Name

...Every name that a specific business location uses to refer to itself is considered correct. These names can be listed in a variety of ways in different sections of the entity’s official website or official social media website(s):

•Chain business locator page on the official website

•About/About Us section on an entity’s official website

•Signs and storefronts seen on official sites or social media pages

•Claimed social media pages

If these resources are not available, other reliable sources that can be used for researching business names include:

•Signs and storefronts seen on recent street or online imagery or crowdsourced review pages

•Recent articles in primary publications (newspapers or similar publications that do their own reporting)

•Official restaurant menus (user photos or scanned images from restaurants)

If the name cannot be confirmed with these resources and there is nothing objectively wrong with it, rate it Can’t Verify.

Corporate name

The corporate name (and sometimes brand or product name) is often used to refer to a particular entity, but might be not the preferred name for a specific location. This depends on the particular company and the specific result.

Some official websites have a local language and an international (English) version. Consider the name given in the international version only if English is the language/script of the test locale, the query, or the result region, or if the official company, chain, or brand name is commonly used and understood in English in this market. If it is not, select the Result name/title is in unexpected language or script checkbox. No further rating is required.

Result Name	Official Business Name	Rating and Explanation
McDonald’s	McDonald’s	Name Accuracy	Correct
		The chain calls all stores McDonald’s and has the name in front of all locations as corporate branding.
Peet’s Coffee & Tea	Peet’s – Polk Street	Name Accuracy	Correct
		The name is considered Correct, because it is used on the sign on the storefront. The store is officially called “Peet’s – Polk Street” on the website, as it refers to the Polk Street location.

Result Name	Official Business Name	Rating and Explanation
Best Buy	Best Buy San Jose	Name Accuracy	Correct
		The chain Best Buy uses a location modifier in their store names, but their store sign (logo) shows only the corporate brand name “Best Buy” and can be rated Correct.
Microsoft	Microsoft Store	Name Accuracy	Correct
		The Microsoft storefront shows only the Microsoft logo without any text. On the webpage, stores are referred to as “Microsoft Store” or named together with a location modifier such as “Microsoft Store – South Coast Plaza”. Because of the close similarities between the result name and the store name, rate Correct.

Location modifier or affiliation

Businesses can include the name of the locality or the larger complex or organization they belong to as additional information. These modifiers, even when not specifically used by the official resources, can be considered correct. A name with missing modifier can also be considered correct as long as the modifier isn’t crucial for understanding the particular result. If a modifier is present, it must be spelled correctly. If it is misspelled, the name is partially correct.

See your Country Specific guidelines for cases where the location modifier is essential for understanding the result.

Result Name	Official Business Name	Rating and Explanation
Apple	Apple Valley Fair	Name Accuracy	Correct
		The result name is missing the location modifier (Valley Fair) given on the official website, but is still considered Correct.
Result Name	.Official Business Name	Rating and Explanation
Peet’s Coffee	Peet’s – Ferry Bldg SF	Name Accuracy	Correct
		The result name is Correct because it is used as the name on the official website, while the specific store has an optional location modifier.
Old Navy – Belle Isle Station	Old Navy	Name Accuracy	Correct
		Old Navy doesn’t officially use the mall name as part of the store name, but the location modifier is correct, so the business name is rated Correct.
Delgado Community College – Charity School of Nursing	Charity School of Nursing	Name Accuracy	Correct
		The result for the Nursing School includes the name of the college that it is affiliated with. Even though this form is not official, it is rated Correct.

Stylized logos and letters

A variation is still correct when a business uses stylized logos and letters (like the backwards “R” in Toys ”R” Us) that are outside the regular set of letters.

Result Name	Official Business Name	Rating and Explanation
Toys ‘R’ Us	Toys Я Us	Name Accuracy	Correct
		This business uses a special character in its name, so the name is still considered Correct.

“The” is missing or added

•Missing: The use of definite articles depends on the individual result. When a company using “the” is often referred to without it, the name is correct both with and without the article.

•Added: A business/POI that does not officially include a definite article in its name but has one included in the result should be considered partially correct.

Result Name

Official Business Name

Rating and Explanation

Home Depot

The Home Depot

Name Accuracy

Correct

The official name of this business is “The Home Depot,” but since it is so often referred to as

“Home Depot” the name is considered Correct.

6.2.2. Partially Correct Name

A partially correct name differs from the official versions but can still be recognized by the user. Partially correct names can include minor and moderate misspellings, service level mismatches, and missing or unnecessary name parts, including holding names/corporate structures. When the business name on the storefront does not include the corporate status (Ltd., Inc., etc.) seen in the result, it is partially correct.

6.2.2.1. Minor and Moderate Misspellings and Missing or Extra Words

Consider names partially correct when they contain issues that don’t prevent the user from identifying the business:

•Name contains repeated, redundant, or missing information or parts

•Incorrect or missing punctuation or special characters

•Unnecessary or missing spaces

•Unexpected use of lower/upper case and ALL CAPS letters

•Acronyms are often expected in ALL CAPS: YMCA, TGIF

•Minor and moderate misspellings that don’t influence the user’s understanding. See your Country-Specific guidelines for more information on misspellings.

•Mix of expected languages in the result title. Expected languages are the languages of the query, test locale, or region, or of an official business name.

Result Name	Official Business Name	Rating and Explanation
Macys	Macy’s	Name Accuracy	Partially Correct
		The result is missing an apostrophe.
HM	H&M	Name Accuracy	Partially Correct
		Missing the “&” in H&M is a misspelling but the name is still recognizable.
uhaul uhaul	U-Haul	Name Accuracy	Partially Correct
		The result name is duplicated and in all lowercase with no “-”.
Seven Eleven	7-Eleven	Name Accuracy	Partially Correct
		The name is in an unexpected form not used by the business itself.

Result Name	.	Official Business Name	Rating and Explanation
GAMESTOP	.	GameStop .	Name Accuracy	Partially Correct
			.The name written in ALL CAPS is not expected for this business and demoted.
Mosjaw	.	Moosejaw	Name Accuracy	Partially Correct
			The name of this outdoor store has two missing letters but is still identifiable.
Ecofuture Buildng Co	.	Ecofutures Building Inc	Name Accuracy	Partially Correct
			The business shows two misspellings and the incorrect business structure (Co. vs. Inc.).
Napoli Coffeehouse & Pastries	.	Napoli Coffee	Name Accuracy	Partially Correct
			The result name has additional parts not used by the business. Because the additional parts are in line with the business and don’t make the result ambiguous, it is considered Partially Correct.
GAP Superstore	.	GAP	Name Accuracy	Partially Correct
			The additional “superstore” is not used officially, but also does not change the user’s ability to identify the well formatted and correct corporate name and is therefore considered Partially Correct.
BerkeleyLaw Univerzitní of California	.	BerkeleyLaw University of California	Name Accuracy	Partially Correct
			The query [law] was issued in a cz_CZ (Czech) locale and the result locale is the US (English). Consider this Partially Correct due to the mix of expected languages: English and Czech.

Result Name	Official Business Name	Rating and Explanation
Aqua Spa 夢幻⽔都	Aqua Spa – Ocean City	Name Accuracy	Partially Correct
		The query [aqua spa] was issued in the zh-Hant_HK (Hong Kong) test locale and the result locale is in the US (English). Consider this Partially Correct due to the mix of expected scripts.
att	AT&T	Name Accuracy	Partially Correct
		URLs can give an indication of a businesses’ official name, but because a URL lacks formatting choices, it should generally not be used to confirm the official business name. Similar restrictions exist for certain social media such as Twitter. The company’s name is confirmed by the URL (att.com), but the name stated in the URL, “att”, is clearly poorly formatted, but still recognizable.
University of California	University of California Irvine	Name Accuracy	Partially Correct
		Irvine is more than a location modifier for this university. The city name differentiates the Irvine location from other universities in the University of California network.
Nike	NIKELAB 21M NYC	Name Accuracy	Partially Correct
		Even though the Nike symbol appears on this store’s official website, the preferred store name for this specific location is not the general brand name but the NIKELAB name seen on the store sign and website.
Result Name	Official Business Name	Rating and Explanation
The Sears	Sears	Name Accuracy	Partially Correct
		Sears does not use “The” in any of their official webpages. Adding it to the company name gives the name a rating of Partially Correct.

6.2.2.2. Service-Level Mismatch

.Chain businesses often indicate the level of service provided at that location in their name. If the result name does not reflect the correct type, it is considered a service-level mismatch and is considered partially correct.

Result Name	Official Business Name	Rating and Explanation
Patagonia Outlet	Patagonia	Name Accuracy	Partially Correct
		The result name suggests that the store is an outlet, which it isn’t.
Delhaize	Delhaize Supermarkt	Name Accuracy	Partially Correct
		Depending on its size and the services available, this chain business comes in five varieties. The result is missing part of the name indicating its service level.
Best Buy	Best Buy Express	Name Accuracy	Partially Correct
		A Best Buy Express is a vending machine and not a store. This is a service-level mismatch and considered Partially Correct.
Best Buy Express	Best Buy	Name Accuracy	Partially Correct
		The user is expecting a vending machine where there is a store. This is a service-level mismatch and considered Partially Correct.

6.2.2.3. Holding Name and Corporate Structure

Holding names or corporate structures are not considered correct if they are not explicitly used by the individual stores. Holding names or names that include corporate structure are considered partially correct or incorrect depending on how recognizable the name is in context with the particular business.

Result Name	Official Business Name	Explanation	.
IT’SUGAR LLC	IT’SUGAR	Name Accuracy	Partially Correct
		The addendum “LLC” refers to the corporate structure and should not be used for the store, However, this name is still recognizable.
Nordstrom, Inc.	Nordstrom	Name Accuracy	Partially Correct
		The addendum “Inc.” refers to the company’s corporate structure and should not be used for the store. However, the store can still be recognized.

6.2.3. Incorrect Name

An incorrect result name is one that can’t be recognized because of severe misspelling or ambiguous or unnecessary/missing parts in the name.

Consider names incorrect when they contain issues including:

•Severe misspellings that prevent the user from identifying the business because of:

•Change in meaning

•Misspelling that results in an entirely different word or gibberish

•Holding names that are completely different than the recognized name

•Unnecessary or missing parts in the name that prevent the user from recognizing the business

•Slang and inappropriate language

•Using the previous name of the same entity that is not a variation of the current name and therefore cannot be identified

Note: If a result name is incorrect, the final Name and Category Accuracy rating will always be Incorrect, even if the category is correct.

Result name	Official Business Name	Rating and Explanation
JAB Holding Company	Peet’s Coffee	Name Accuracy	Incorrect
		The holding company name is completely different than the store name.

Result name	Official Business Name	Rating and Explanation
IEA	IKEA	Name Accuracy	Incorrect
		In short business names, even small errors can prevent the user from recognizing the business.
Zatas Tacos + Tequila	Zacatecas Tacos + Tequila	Name Accuracy	Incorrect
		The misspelling is severe and does not allow the user to identify the business with certainty. The user is likely under the impression that it is a different restaurant.
Taco Bull	Taco Bell	Name Accuracy	Incorrect
		The misspelling changes the meaning, which makes the name ambiguous: It could be a “Taco Bell” chain store or a different business named “Taco Bull”.
Walgreens Pizza	Walgreens	Name Accuracy	Incorrect
		With the unnecessary word “Pizza,” the name becomes ambiguous and the user does not know if this is a pizzeria named Walgreens or a wrongly labeled Walgreens pharmacy.
Cheesecake	The Cheesecake Factory	Name Accuracy	Incorrect
		When missing the word “factory”, the result name “Cheesecake” is not specific enough and the chain business cannot be identified with certainty.

Result name	Official Business Name	Rating and Explanation
UPS	USPS	Name Accuracy	Incorrect
		USPS (United States Postal Service) and UPS (United Parcel Service) are both in the delivery business, but are entirely different companies.
Starbucks & Spencer	Marks & Spencer	Name Accuracy	Incorrect
		By switching Marks with Starbucks, the business can no longer be identified.
Mickey D’s	McDonald’s	Name Accuracy	Incorrect
		Mickey D’s is British slang for McDonald’s brand. However it is not an official variation of the brand name.
Sommet Center	Bridgestone Arena	Name Accuracy	Incorrect
		Showing a former name that is not shown on the arena’s official webpage is not a good experience. Even when the name might still be in use among users, if a result shows a former name, rate it Incorrect.

6.3 Result Category

Once you have determined the correctness of a result’s name, you must evaluate the accuracy of its category in order to determine the final Name and Category Accuracy rating. The category of a result refers to the category of the Business/POI.

Not every result will include a category, but when one appears, you will find it below the address details

If the category is missing or listed as N/A, make your final Name and Category Accuracy rating based on the name alone.

6.3.1. Correct Category

When a business or POI is categorized correctly, the category accurately reflects the business, service, or function of the entity. This includes broad or alternate categorizations that are not the preferred choice but are not misleading.

A correct name paired with a correct category is rated Correct for Name and Category Accuracy.

•Not every result will include a category. If the category is missing or listed as N/A, do not demote.

•Some categories may seem slightly too specific or too general. As long as the categorization is not misleading, consider it correct.

•Categories can vary for similar POIs and vary per market. Localization of categories should be taken into account when deciding on correctness.

Result Name and Category	Category Rating and Explanation
Restaurant Toqué French Cuisine	Category Accuracy	Correct
	Restaurants are commonly classified by their respective cuisines. French Cuisine is an appropriate category for the result.
Jocko’s World Famous Chicken and Seafood Seafood Restaurant	Category Accuracy	Correct
	This categorization may seem a bit too specific but it is not misleading. Although this restaurant sells more than just seafood, categorizing it as a seafood restaurant still properly reflects the nature of this entity.
Walmart Grocery Store	Category Accuracy	Correct
	As long as research shows that the particular Walmart location represented by the result sells groceries, this would be an appropriate category. This is because grocery is a major department in this store and represents a large part of what it sells.
Walmart Discount Store	Category Accuracy	Correct
	This category accurately represents the POI. Walmart is known to offer merchandise at low prices.

6.3.2. Incorrect Category

When the category is wrong, the final Name and Category Accuracy rating is always Incorrect. This is true whether the result name is Correct, Partially Correct, or Incorrect.

Consider the category incorrect when it is:

•Wrong

•Misleading

•Misspelled

•Incomplete (missing parts or has uncommon/odd abbreviations)

•In an unexpected language/script. The category must match the language/script of the query, test locale, or result region

Result Name and Category	Expected Category	Final Rating and Explanation
Denver Zoo Active Life	Zoo	Incorrect (Category Issue)
		Category is wrong since the result is a zoo. This category does not represent the nature of the result. Final Name Accuracy rating is Incorrect.
Macy’s Bar	Department Store	Incorrect (Category Issue)
		The result is a department store and should not be categorized as bar. Final Name Accuracy rating is Incorrect.

Result Name and Category	Expected Category	Final Rating and Explanation
Macy’s Shoe Store	Department Store	Incorrect (Category Issue)
		Category is misleading. The result is a department store and it should be categorized as one. Although this store does sell shoes, the category does not reflect the wide variety of other merchandise it offers. It is misleading to categorize this result as simply a shoe store. Final Name Accuracy rating is Incorrect.
Fry’s Electronics Electrnics	Electronics Store	Incorrect (Category Issue)
		Category is misspelled. Any misspelling in the category name should be considered wrong. Final Name Accuracy rating is Incorrect.
Macy’s Depart Store	Department Store	Incorrect (Category Issue)
		Category is incomplete. The category shown is not a common abbreviation for “Department Store” and appears incomplete. Final Name Accuracy rating is Incorrect.
Result Name and Category	Expected Category	Final Rating and Explanation
Aral Gas Station	Tankstelle	Incorrect (Category Issue)
		Test Locale: Germany Category is in an unexpected language or script. The category “Gas Station” is in English instead of the expected German “Tanskstelle.” English should be considered an unexpected language in this locale (even if it could be understood by a user there). Final Name Accuracy rating is Incorrect.
Macys Bar	Department Store	Incorrect (Name Issue, Category Issue)
		The result name should be considered partially correct because it does not include the proper punctuation: “Macys” should be “Macy’s.” The category of Bar for this department store is wrong. When the name is partially correct and the category is wrong, rate Name Accuracy Incorrect.

6.4. Can’t Verify

When, due to a lack of available data online, the business or POI name cannot be confirmed, rate Can’t Verify. This can be because of a lack of official resources or street imagery, or can be the case for small businesses that don’t maintain their own webpage or social media profile.

6.5 Transit Names

Transit stations and other transit results (like airports and shipping ports), are often named after the locality or street where they are located and can include a transit indication such as “Station” or “Subway.” The transit indication in the result name/title is considered optional and can be missing and still be considered correct. The user query for [san mateo station] can therefore return the name San Mateo Station or simply San Mateo to be Correct. The category of the result will help you correctly interpret the result because categories for transit POIs should be a reference to transit.

Keep in mind that some transit POIs located on street intersections may have a result title that is the name of the street intersection (e.g. “Lincoln Way & 19th Avenue”). This is also considered Correct.

If there are multiple transit systems operating out of the same station, any of the systems can be included in the name to be Correct. If transit stations are close to each other, but users have to walk outside to get from one building to another, consider the results for the two stations as two separate POIs.

...

Query	Result	Rating and Explanation
[Millbrae station] User and viewport in Palo Alto, CA	Millbrae Millbrae, CA Category: Train Station	Name Accuracy	Correct
		“Millbrae” has no transit indication in the result name, but the category indicates it is a train station.
	Millbrae BART Millbrae, CA Category: Train Station	Name Accuracy	Correct
		“Millbrae BART” has a transit indication for the BART system in the result name.
	Millbrae Amtrak Millbrae, CA Category: Train Station	Name Accuracy	Incorrect (Category Issue)
		“Millbrae Amtrak” has an incorrect Amtrak transit indication. Only BART and Caltrain operate from Millbrae station.

6.6. Parking Names

A parking lot may or may not have a name. Confirm this on the lot’s official webpage:

•The generic names “Parking,” “Garage,” or “Lot” (or any other words that indicate parking can take place at the location) are considered Correct when no official name can be found.

•If an official name can be found, rate the generic names “Parking,” “Garage,” or “Lot” (or any other words that indicate parking can take place at the location) as Partially Correct.

•If the parking lot is specialized (for instance, motorcycle-only parking), and the result’s name doesn’t reflect that specialization, the result should be rated Incorrect for name accuracy.

Accept the following combinations of names as Correct if no official name is found:

•(Business name/Plaza name/Shopping area name) + (parking/ garage/lot)

•Without the addition of parking/garage/lot, the name is Incorrect

•Parking/ Garage/Lot (or any other words that indicate parking can take place at the location)

Query	Result	Rating and Explanation
[Parking] User and viewport in San Francisco, CA	Union Square Garage	Name Accuracy	Correct
		This garage’s official website confirms its name.
	Parking (Result is for the Union Square Garage)	Name Accuracy	Partially Correct
		Research shows that the official name for this parking location is Union Square Garage
	Safeway Parking (Result is for the parking lot attached to a Safeway store in San Francisco)	Name Accuracy	Correct
		This parking lot has no official website or name, but is associated with a Safeway store.
Query	Result	Rating and Explanation
.	Parking (Result is for the parking lot attached to a Safeway store in San Francisco)	Name Accuracy	Correct
		This parking lot has no official website or name.
	Safeway (Result is for the parking lot attached to a Safeway store in San Francisco)	Name Accuracy	Incorrect
		Without the words “Parking,” “Garage,” or “Lot,” users will have no way to know they can park at this Safeway.

6.7. Final Name and Category Accuracy Rating (Summary Table)

This table shows the final ratings produced by different combinations of name correctness and category accuracy.

Remember: If you give a final Name and Category Accuracy rating of Partially Correct or Incorrect, you may be presented with two checkboxes:

•Name Issue

•Category Issue

If they appear, use one or both to select the reason(s) for your rating.

7. Address Accuracy: Components and Issues

Address Accuracy is split into two main parts: Components and Issues and Result Expectation. This section will discuss Components and Issues and the next section will discuss Result Expectations.

..7.1. Address Components

Address components are the elements used to form the address details for each result. These components include street number, unit/apt, street name, locality, region/state, postal code, and country. The components are defined in these guidelines with a global approach in mind. There may be variations for specific markets that are not addressed here. See your Country Specific guidelines for more information on the address component requirements for each type of result.

.This section covers the issues that can be associated with the address

components as well as the possible issues that can be associated with the address itself. It refers to the checkboxes presented in the rating interface when Incorrect is selected. Checkboxes should be selected for incorrect or missing information. Issues can occur within parts of the address or the entire address.

Choose all identified incorrect or missing components. If the issue is not tied to individual components but to an issue pattern, choose the appropriate issue instead.

Never select issues and components at the same time to report the same problem. That is, do not rate Incorrect —Language/Script issue for the entire address and also Incorrect—Street Name because street name is in an incorrect language.

When a Language/Script issue is present, make sure to check the address components for accuracy and to select the appropriate checkbox. For instance, if you select language issue for the entire address, you can also mark street number as incorrect, since this is an error that’s not language related.

7.1.1. Street Number

If the result address has an incorrect or missing street number rate Address Accuracy as Incorrect – Street Number.

Result

Official Address

Rating and Explanation

7-Eleven

836 E Fremont Ave,

Sunnyvale, CA 94087

834 E Fremont Ave,

Sunnyvale, CA

94087, USA

Address Accuracy

Incorrect – Street Number

Select the Street number checkbox because the street number in the result address is incorrect.

Result

.Official Address

Rating and Explanation

Defour

Potgieterstraat, 1053 XS Amsterdam,

Netherlands

Potgieterstraat 47,

1053 XS

Amsterdam,

Netherlands

Address Accuracy

Incorrect – Street Number

Select the Street number checkbox because the street number is missing in the result address.

.The way street numbers are assigned varies by country and area. Some patterns you may see:

Street Number Extensions

Some markets use a street number extension system. Street number extensions are part of the street number, not a further breakdown of it like building or unit numbers. If a street number extension is incorrect or missing from a POI address it should be treated as an incorrect/missing street number. Refer to Country Specific Guidelines for more details.

Result	Official Address	Rating and Explanation
Elkjøp Halden Walkersgate 8, 1771 Halden, Norway	Walkersgate 8c, 1771 Halden, Norway	Address Accuracy	Incorrect – Street Number
		In Norway, street number extensions are part of the street number and are not unit numbers. The street number in the result is missing a street number extension ‘c’ which is listed as part of the official street number.
REMA 1000 Tynset Elfengveien 4 A, 2500 Tynset, Norway	Elfengveien 4 B, 2500 Tynset, Norway	Address Accuracy	Incorrect – Street Number
		In Norway, street number extensions are part of the street number and are not unit numbers. The street number in the result has an incorrect number extension ‘A’ when ‘B’ is listed as the official street number extension.

Address Range

Address ranges are used to assign numbers to a stretch of road. Given the nature of these ranges, not all numbers will have a building assigned to them.

When an address range is given by the query or returned as a result, consider it Correct when:

•The result street number is within the queried range

•The result address range exists and contains the queried street number

•The range or number matches the odd/even numbering system for left/right side of street required by the country

The same criteria apply to POI addresses.

Suggestion	Official Address	Rating and Explanation
The Crutched Friar 39 Crutched Friars, London	39-41 Crutched Friars, London EC3N 2AE, United Kingdom	Address Accuracy	Correct
		No need to select the Incorrect – Street Number checkbox since the street number in the suggestion is within the official address range.
.The Crutched Friar 45 Crutched Friars, London		Address Accuracy	Incorrect – Street Number
		Select the Incorrect – Street Number checkbox because the street number in the suggestion is outside of the official address range.
26 Winchester Avenue Burpengary East Qld 4505 Australia	25-27 Winchester Avenue, Burpengary East Qld 4505 Australia	Address Accuracy	Incorrect – Street Number
		Select the Incorrect – Street Number checkbox because the number 26 is even while the official address range is odd.
Apartamentos Oporto Rua de Dom Hugo 1-45 Porto	Rua de Dom Hugo 17 4050 Porto Portugal	Address Accuracy	Incorrect – Street Number
		Select the Incorrect – Street Number checkbox because the street range in the suggestion does not exist. This is a random address range and should be considered incorrect even if it contains the official street number.

7.1.2. Unit/Apt

A street number can be divided into unit, apartment, or building numbers. Commonly seen forms include the use of letters: A, B, C as well as the character # in front of a number: #420. Less common are forms like ½.

A unit number should be correct and present when the official address of a business/POI contains a unit number.

When a business does not list a unit number and the unit number cannot be confirmed, no unit number should be given in the result. Rate Incorrect – Unit/Apt when unit number is present in the rating tool in these cases.

Result	Official Address	Rating and Explanation
Vuori 333 Santana Row Suite #1110 San Jose, CA 95128	333 Santana Row Suite #1110 San Jose, CA 95128	Address Accuracy	Correct
		This is the correct address listed on the official website of the business.
Vuori 333 Santana Row San Jose, CA 95128		Address Accuracy	Incorrect – Unit Number
		.This result address is missing the unit number listed on the official website. Unit number checkbox should be selected.
Vuori 333 Santana Row Suite #2490 San Jose, CA 95128	.	Address Accuracy	Incorrect – Unit Number
		This result address has a different unit number than the one listed on the official website. Unit number checkbox should be selected.

7.1.3. Street Name

If the result address has an incorrect or missing street name rate Address Accuracy as Incorrect – Street Name.

Result	Official Address	Rating and Explanation
Van Gogh Museum Museumplein 6, 1071 DJ Amsterdam, Netherlands	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Correct
		This is a correct address as displayed on the official website of the business.
Van Gogh Museum Van Baerlestraat 6, 1071 DJ Amsterdam, Netherlands		Address Accuracy	Incorrect – Street Name
		This address has the wrong street name. Street name checkbox should be selected.
Van Gogh Museum 6, 1071 DJ Amsterdam, Netherlands		Address Accuracy	Incorrect – Street Name
		This address has a missing street name. The street name checkbox should be selected.

Alternative Street Names

A street can have alternate names or route numbers, as well as multiple names in different languages in bilingual areas. Consider a street name correct when the name used is still valid and in the language expected given the test language, result region, or user query. If the street name has changed and the older name is no longer in use, consider the old name to be Incorrect – Street Name if returned.

Misspelled Street Names

..Misspellings in the street name, including missing diacritics (e instead of é, l instead of ł) may be rated Incorrect – Street Name. See your Country Specific guidelines for more information.

Result	Official Address	Rating and Explanation
Van Gogh Museum Musuemplein 6, 1071 DJ Amsterdam, Netherlands	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Incorrect – Street Name
		The result street name is misspelled (“ue” instead of “eu”), so the street name checkbox should be selected
Bozego Ciala Bozego Ciala, Wrocław, Poland	Bożego Ciała, Wrocław, Poland	Address Accuracy	Incorrect – Street Name
		The result street name is missing diacritics in two letters (ż and ł) which should be treated as a misspelling in the Polish market. The street name checkbox should be selected.

Street Directions and Types

Street directions (North (N), East (E), South (S), West (W)) as well as street types (Street (St), Boulevard (Blvd), Avenue (Ave), etc.) are part of the street name. If they are incorrect or missing, the street name is wrong and rated Incorrect – Street Name. Exceptions are when such elements of a street name are used interchangeably for the same street.

Result	Official Address	Rating and Explanation
Van Gogh Museum Museumkade 6, 1071 DJ Amsterdam, Netherlands	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Incorrect – Street Name
		This address has the wrong type of street: both “plein” (square) and “kade” (quay) are street types. The official street type is listed as ‘plein’, therefore the type of street is considered wrong and the checkbox for street name should be selected.
118 El Camino Real 118 El Camino Real, Sunnyvale, CA 94087	118 E El Camino Real, Sunnyvale, CA 94087	Address Accuracy	Incorrect – Street Name
		This address is missing the direction. Therefore, the street name checkbox should be selected.
Blue Stones 3530 E El Camino Real, Santa Clara, CA 95051	3530 El Camino Real, Santa Clara, CA 95051	Address Accuracy	Incorrect – Street Name
		This result address has additional direction, which is wrong and does not exist in real life.

7.1.4. Sub-Locality and Administrative Subdivisions

Some countries or areas require an additional subdivision, like sub-locality, for correct address information. See your Country Specific guidelines for specific guidance. The general guidance is to rate a wrong sub-locality as Incorrect – Sub-Locality. This is also true if the sub-locality is required but missing.

7.1.5. Locality

Misspellings within the locality name, including missing diacritics (e instead of é, l instead of ł) may be rated Incorrect – Locality. See your Country-Specific Guidelines for more information.

Every result requires the correct locality component within its address details. In cases of local government reforms or other changes in naming, the result has to reflect the official name. Exceptions are made when alternate names are accepted.

Result	Official Address	Rating and Explanation
Van Gogh Museum Museumplein 6, 1071 DJ Utrecht, Netherlands	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Incorrect – Locality
		This address has the wrong locality. Locality checkbox should be selected.
Van Gogh Museum Museumplein 6, 1071 DJ Netherlands	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Incorrect – Locality
		This address has a missing locality therefore locality checkbox should be selected.
Jana Matejki Jana Matejki 90-237 Lódz Polska	Jana Matejki 90-237 Łódź Polska	Address Accuracy	Incorrect – Locality
		The locality within the street result is missing diacritics in two letters (Ł and ź) which should be treated as a misspelling in the Polish market.

7.1.5. Region or State

Each country has its own understanding of what is expected (correct) or unusual (incorrect) for region or state components. See your Country Specific guidelines for more information.

Result	Official Address	Rating and Explanation
1384 Navarro Dr 1384 Navarro Dr, Sunnyvale, 94087, USA	1384 Navarro Dr, Sunnyvale, CA 94087, USA	Address Accuracy	Incorrect – Region/ State
		This address is missing the state, which is a mandatory component in the USA. Region/State checkbox will be selected for missing state.
Van Gogh Museum Museumplein 6, 1071 DJ Amsterdam, North- Holland, Netherlands	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Correct with formatting issue
		Region is not a mandatory component in the Netherlands, but if present has to be correct. Since the region is not necessary, it is a redundant address component and should be flagged as a formatting issue.
Result	Official Address	Rating and Explanation
Van Gogh Museum Museumplein 6, 1071 DJ Amsterdam, North- Brabant, Netherlands	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Incorrect – Region/ State
		Region is not a mandatory component in the Netherlands, but if present has to be correct. The correct region for the city of Amsterdam is North-Holland, therefore the region/state checkbox will be selected.

7.1.6. Postal Code

The postal code accompanies the locality name and provides further detail about where a feature is located. See your Country Specific guidelines for more information on when postal codes are a mandatory address component.

If a postal code is not mandatory but is present in the result address, rate Correct as long as the postal code applies to at least part of the feature (street, locality, large POI without an expected address, etc.).

For many localities, more than one postal code may apply. As long as the returned postal code is correct for at least a part of the feature (street, locality, large POI) rate Correct. Columbus, OH, for example, contains 45 postal codes.

Postal code standards differ between locales so be sure to consult your Country Specific guidelines for details.

Consider any country specific conventions when it comes to the use and correctness of shortened postal codes. For example, the US postal code does not need the four-digit extension that follows the five digit main code. Ignore it if it appears.

Result	Official Address	Rating and Explanation
Van Gogh Museum Museumplein 6, 1071 DM Amsterdam, Netherlands	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Incorrect – Postal Code
		This address has the wrong postal code “DM” instead of the correct “DJ.” Postal code checkbox should be selected.
Van Gogh Museum Museumplein 6 Amsterdam, .Netherlands	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Incorrect – Postal Code
		This address has a missing postal code. Postal code checkbox should be selected.

7.1.7. Country

The country is a “nice to have” piece of information, but should always be included when the result is located in a country other than the test locale. If missing, rate Incorrect and select the Country checkbox.

Test Locale	Address in Result	Address in Reality	Rating and Explanation
en_US	1384 Navarro Dr, Sunnyvale, CA 94087	1384 Navarro Dr, Sunnyvale, CA 94087	Address Accuracy	Correct
			This is a correct address since the test locale is the USA.
de_DE	1384 Navarro Dr, Sunnyvale, CA,94087, USA	1384 Navarro Dr, Sunnyvale, CA 94087, USA	Address Accuracy	Correct
			This address includes the country name since it is outside of Germany, which is the test locale.
en_US	Museumplein 6, 1071 DJ Amsterdam	Museumplein 6, 1071 DJ Amsterdam, Netherlands	Address Accuracy	Incorrect – Country
			The country is mandatory in this example because the address is in the Netherlands, which is outside the user’s US test locale. Country checkbox should have been selected.

7.2. Address Does Not Exist

An address-type result needs to be associated with a building or with a plot of land that has been officially assigned that address. If you find strong evidence that there is no building at the address or that no plot of land has been officially assigned the address, rate Incorrect – Address does not exist and leave a detailed comment including links. If there are not enough resources available to make an informed decision, rate Can’t Verify.

Within markets that use street number extensions, if the result offers an address with a generic street number, but the street number alone does not exist without an extension (e.g. result returns “163

Main St.”, but only “163A Main St.” and “163B Main St.” exist), rate the address as Incorrect – Address does not exist, since the street number without the extension is not a valid, real-life address. Note that street number extensions differ from the building numbers referred to in Unit/Apt section.

In rare cases, an existing street address can be found in a different locality. Always mark such cases as Incorrect – Address does not exist and not as Incorrect – Locality.

Use the Incorrect – Address does not exist checkbox for any address result that doesn’t exist, like a street or sub-locality result. This rating should not be applied to POI addresses.

7.3. Language/Script Issue in Address

The address details must be in a language and script that matches the test language (like en_US), the user query, or the result region.

Use the Address Accuracy Incorrect – Language/Script Issue checkbox if you find issues of this nature in any of the address components in the address details of any result. (For business/POI names and address titles in unexpected languages, see Result title/name in unexpected language.)

These examples show incorrect language and script for the result “Cafe Timber, 1066 Hippy Hollow Rd, Red Boiling Springs, TN 37150” with a user in the US using English.

Any combination of unexpected language or script is considered Incorrect. An exception can be made in cases when only special characters are added that are not used in the expected language.

Results located in a market with different scripts are hard to verify. Use the normal research steps to confirm the address. It is not expected that you understand markets and address formats outside your local knowledge expertise, so do the best you can without doing more research than normal.

Rate mixed languages and scripts found in any address details as Language/script Issue when they use a different language or script than the one used in the query, test, or result location.

7.4. Country-Specific Issue

The variety of address formats and distinctive country specific issues is broad. In general, use the checkbox for Country Specific Issue when there is an address problem that is not covered by the available checkboxes. This includes things like instances when an address also requires a municipality in addition to locality and state, but those components are either missing or incorrect. See your Country Specific guidelines for more information.

7.4.1.Country-Specific Issue vs. Correct with Formatting Issue

Any additional components beyond the regular address format for a country are rated:

•Correct with formatting issue if correct

•Incorrect – Country Specific Issue if very unusual or wrong

7.5. Other Issue

Issues not mentioned in these guidelines are reported using the Incorrect – Other Issue checkbox. If you choose this option, be sure to leave a comment as well as links to the resources you used to identify the issue. The checkbox is also used for:

•Duplicate address components

•Name of POI reappears in the address details

•Natural features that contain street address elements (see Natural Features) • P.O. Box addresses

You may also use this checkbox if you notice formatting issues along with any Incorrect components you’ve already selected. This is because once you mark a component Incorrect, you can no longer choose Correct with Formatting issue.

7.6. Correct with Formatting Issue

When all the result address information is correct and present, but not in the format expected, mark it as Correct with Formatting issue. Formatting issues include the order of address components and other minor issues that are not necessarily considered wrong. They do not include misspellings which are addressed under the relevant individual components. Some reasons to rate Correct with formatting issue:

•Components are in unexpected order

•Non-required but correct address components

•Extra spacing

•Double comma [,,]

•Rate Incorrect if a required component is missing

•Valid but redundant or unnecessary components (See your Country Specific guidelines for more information)

If optional components already discussed in this chapter appear in the address, use the guidance provided in the section that applies to those components, not the Correct with formatting issue rating. If your market has additional address components, consult your Country Specific guidelines before demoting.

7.7. Can’t Verify

An address is rated Can’t Verify when it cannot be confirmed as Correct or Incorrect. This is usually the case when there is a lack of resources or an unexpected address format is given on the official website.

•Lack of resources:

•No official webpage found

•No official address listed

•General lack of official resources

•Lack of street imagery

•Unexpected address formats used by official resources can include:

•Intersection address (Main St and 2nd St)

•Exit address (Exit 5, Hwy 101)

•Descriptive address (Main St between 6th and 7th Streets)

Ask yourself: Is the location given by the result address at least not wrong given all information I have found? Use the following examples to learn how reach a conclusion.

Note: If an address contains missing or incorrect components, it can be confirmed as Incorrect. Mark the individual incorrect components. See your Country Specific guidelines for more information on missing or incorrect components.

7.7.1. Can’t Verify for Street Number

The following example shows address details for a business called “Utopia.” No information online confirms the address and only a few street numbers can be confirmed with official resources.

7.7.2. Can’t Verify for Street Name

The following examples give different address details for a business called Aztec. The official address is “Granite Ave & W Babcock St, Bozeman, MT” and is in a different format than the result addresses.

Rule and Explanation

Example

The result address has to provide access and be within the possible range of street numbers.

W Babcock St provides access to the business and the number appears to be within the range of addresses we can confirm.

Rate the address Can’t Verify.

Granite Ave provides access to the business and the number appears to be within the range of addresses we can confirm.

Rate the address Can’t Verify.

8. Address Accuracy: Result Expectations

You will see three kinds of results:

•Business/POI results

•Address type results

•Features without an expected address

...Different address details are expected for each result type. For example, the Amazon river has no street address but its name is accompanied by the country, while the fast food place Wienerschnitzel shows a complete address. You will need the appropriate expectations in order to rate addresses accuracy.

Wienerschnitzel

75 Saratoga Ave

Santa Clara, CA 95050

United States

Eiffel Tower

5 Avenue Anatole France

75007 Paris

France

1 Church St

1 Church St

Burlington, VT 05401 United States

Grand Central Terminal Manhattan

New York, NY

United States

Amazon

Brazil Natural Feature

Yellowstone National Park Yellowstone National Park

WY 82190

United States

8.1. Business/POI Result

The address details of a business or POI generally consist of: street number, street name, locality, region, postal code, and country.

The address given is confirmed Correct when it points to the same physical location listed on the official webpage or other official resources and all mandatory components for your market are present. See your Country Specific guidelines for details on mandatory address components.

8.1.1. Business/POI Address Research and Resources

Use official resources to confirm an address when a business or POI’s official website does not provide a full address or if there is a conflict with the address in the result. These resources can include:

•Social media sites claimed by the business/POI and updated within the last 6 months

•Street imagery (any recent online imagery showing streets and buildings)

•Postal authorities (for postal codes and locality names)

•Government property registries

If a business or POI has more than one official address for the same location, accept any of them as Correct.

If no official resources can be found, rate according to consensus from multiple reliable sources. Reliable resources include:

•Articles in primary publications (newspapers or similar publications that do their own reporting)

•Crowdsourced user review sites

When no official resources are available and address components are missing or clearly incorrect, check to see if those components are required in the market or region of the result and select the appropriate checkboxes under Incorrect. See your Country Specific guidelines for more information.

Some resources, like data aggregators, spammy directory sources, and so on, are unreliable, and consensus between unreliable resources should be disregarded.

•Valid alternate street and locality names When valid alternate names for streets or localities are used interchangeably, consider all versions Correct.

•Valid alternate street addresses: When valid alternate addresses are returned, consider all versions found on the official website or confirmed by official sources as Correct.

When the address listed by the official website or resources is in a different format than the one seen in the result, confirm if the address given is Incorrect or Can’t Verify. See Can’t Verify for more information.

If there is any obvious error in an official source, like a misspelled country or city name, and the same error appears in the result, rate the erroneous component Incorrect.

8.1.2. Alternative Official Addresses

Some official addresses may contain a vanity street name or have no official street number. In rare cases they may even lack a complete street address. Refer to the official webpage as well as to common address formats in your country to determine the accuracy of each business or POI address.

8.1.2.1. Department Addresses

When the official webpage does not give a complete address with street number and name for a large entity (mall, university, etc.):

•Rate the result address Correct when it matches the address of any department or entity at the same location complex or campus.

•Rate the result address Correct when it lists the correct locality for the entity. If the exact locality is unclear because the entity borders multiple localities, rate Can’t Verify.

•The locality is always expected, even when no complete address is provided. If missing, rate Incorrect – Locality.

•Region and postal code must also appear in countries where they are mandatory components. See your Country Specific guidelines for more information.

Result Address	Official Address	Rating and Explanation
Dartmouth College 10 N Main St, Hanover, NH 03755	. Dartmouth College Hanover, NH 03755 USA	Address Accuracy	Correct
		Dartmouth College has no official street address listed on their website. The address in the result belongs to the Office of Admissions, which is part of the larger university, therefore the address is Correct.
Dartmouth College Hanover, NH 03755		Address Accuracy	Correct
		The college lists only the locality as its official address and the result matches that address and is therefore Correct (even though it is missing a street number and name).
Dartmouth College NH 03755		Address Accuracy	Incorrect – Locality
		A locality is expected in the address details for most POIs. Rate Incorrect – Locality when the locality is given by the official webpage but missing in the result.

In addition to a street address, a store or department can have the name of the complex or campus that contains it and be rated Correct.

Result Address	Official Address	Rating and Explanation
Stanford School of Law Stanford University, Stanford, CA 94305	559 Nathan Abbott Way, Stanford, CA 94305	Address Accuracy	Correct
		This result is a department within Stanford University. The address is Correct because the larger entity name and correct locality is listed as the address.
The Cheesecake Factory Westfield Valley Fair, Santa Clara, CA 95050	3041 Stevens Creek Blvd Santa Clara, CA 95050	Address Accuracy	Correct
		This result is a business within Westfield Valley Fair shopping mall. The address is Correct because the larger entity name and correct locality are listed as the address.

Result Address	Official Address	Rating and Explanation
Stanford School of Law 450 Serra Mall, Stanford, CA 94305	559 Nathan Abbott Way, Stanford, CA 94305	Address Accuracy	Incorrect – Street Number & Street Name
		The result address belongs to the larger entity (Stanford University). The department has an official address that is different from the university’s address, so the result address is rated Incorrect.
Stanford University 559 Nathan Abbott Way, Stanford, CA 94305	450 Serra Mall, Stanford, CA 94305	Address Accuracy	Incorrect – Street Number & Street Name
		The result address belongs to a smaller entity within the university (Stanford School of Law). The university has an official address that is different, so this address is rated Incorrect.
The Cheesecake Factory 2855 Stevens Creek Blvd Santa Clara, CA 95050	3041 Stevens Creek Blvd Santa Clara, CA 95050	Address Accuracy	Incorrect – Street Number
		The result address belongs to the larger entity (Westfield Valley Fair mall) that contains the restaurant. The Cheesecake Factory’s official website lists a different address, so the address Incorrect.
Westfield Valley Fair 3041 Stevens Creek Blvd Santa Clara, CA 95050	2855 Stevens Creek Blvd Santa Clara, CA 95050	Address Accuracy	Incorrect – Street Number
		The result address belongs a business (The Cheesecake Factory) within the mall. the shopping mall has an official address that is different, so the address is rated Incorrect.

.If the entity has an official address and the result address belongs to another entity in the same complex or on the same campus, rate Incorrect.

8.1.2.2. Result Missing the Street Address

When both the result and the official webpage show identical but incomplete street addresses (missing street name, street number, or both) and the locality is correct, consider the address Correct.

Result Address	Official Address	Rating and Explanation
Pilot Thomas Cardlock State Hwy 115 & Mustang Drive, Andrews, TX 79714	State Hwy 115 & Mustang Drive, Andrews, TX 79714	Address Accuracy	Correct
		This result is for a self-serve gas station for commercial vehicles. While we would normally expect a full address for this type of result, the official website lists only an intersection. The result listing matches the official website, so the result is rated Correct.
Strand Central Park Kiosk 789 5th Ave Manhattan, NY 10065	E. 60th St & 5th Ave, Manhattan, NY 10065	Address Accuracy	Can’t Verify
		This result is for a automated book kiosk. While we would normally expect a full address for this type of result, the official website lists only an intersection. The result listing is a full street address, so the result is rated Can’t Verify.

When the street address is missing from the result but the official webpage has a street address, consider the address Incorrect.

Result Address	Official Address	Rating and Explanation
Pilot Thomas Cardlock Andrews, TX 79714	State Hwy 115 & Mustang Drive, Andrews, TX 79714	Address Accuracy	Incorrect – Street Name
		This result is for a self-serve gas station for commercial vehicles. The official website lists an intersection as the address. The result address is missing the two streets (State Hwy 115 & Mustang Dr). Mark the result Incorrect – Street Name.
Stanford University Serra Mall, Stanford, CA 94305	450 Serra Mall, Stanford, CA 94305	Address Accuracy	Incorrect – Street Number
		The result address is missing the street number given on the official website, therefore the address accuracy is Incorrect – Street Number.

8.1.2.3. P.O.Box, Mailing Addresses, and Management Offices

Addresses that point to a different location or building, such as management offices, P.O. Boxes, mailing addresses, or shared office spaces, are not valid and are rated Incorrect – Other Issue.

Result Address

Official Address

Rating and Explanation

John F. Kennedy

International Airport

World Trade Center,

Greenwich St.,

New York, NY 10007

World Trade

Center, 150

Greenwich St.,

New York, NY 10007

.Address Accuracy

Incorrect – Other Issue

.The address on the official airport site is for The Port Authority of NY & NJ corporate office, which is not the location of the actual airport. Even though the result address matches the address listed on the official site, it is a different location and does not refer to the location of the result POI.

.8.1.3. Businesses/POI with Moving Locations

Moving entities can either be on a fixed schedule or move at random. In some cases, entities appear to be moving because they are located in a truck or similar vehicle, but are actually permanently based at the same location.

Random location

The location of randomly moving entities cannot be predicted and should not be shown on the map. Rate such businesses as Result is closed or does not exist.

Fixed schedule

Some entities, such as food trucks, move on fixed schedules. Their stops and schedule are usually posted on their official webpage. The most accurate location for these entities is the address of their longest stay over a full cycle of the given schedule (i.e. if a business has a weekly cycle and stays in one spot for four days, that spot will be the only acceptable address for it). Mark any other address as Incorrect – Other Issue. In cases of a tie, consider any of those locations Correct.

Fixed location

Businesses that operate out of a truck or bus that is stationary are evaluated like a standard business. The vehicle does not need to be parked at the location after closing hours to be considered at a fixed location.

8.1.4. Events

Some events can be very prominent but happen only a few days a year, others are regularly recurring.

Regularly recurring (weekly/monthly)

Rate events like weekly farmer’s markets like any other business.

Annual Events

Events that happen once a year, such as Burning Man or the Sziget Festival, only occur for a short time each year, but attract massive amounts of people and media attention. Users can be interested in these kinds of events outside the time they are held.

If the query and location intent are clear, assign an appropriate relevance rating, otherwise rate Bad. Treat seasonal stores (like Spirit Halloween store) and seasonally open POIs (like California’s Great America theme park) the same way as annual events.

Depending on its location, an event may not have an official address. This kind of event should be treated like a landmark or natural feature.

Singular or Non-Established Events

Non-recurring events from the past don’t carry any map significance and are rated Business/POI closed or does not exist, unless the location of the event became a POI (due to its historical importance) that carries the name of the event further.

8.2. Address Type Results

Address type results generally include:

• Street number, street name, postal code, locality and country (when outside of the test locale).

8.2.1. Address Result

An address usually includes a street number, street name, locality, postal code and state/region. It can be the same as the mailing address or other addresses provided by the official resources. Use the chapter Address Formats, below, as well as your local knowledge, to identify the necessary components and format.

•The address given has to be associated with a building or a plot of land officially assigned that address.

•P.O. boxes are not valid addresses in the map context. Rate such addresses as Incorrect – Other Issue.

If your research shows that the address does not exist, see Address Does Not Exist.

Address

Mandatory components

Explanation

13 Navarro Dr,

Sunnyvale, CA 94087

Street Number, Street Name,

Street Type, Locality, State, Postal code

Mandatory components may vary by country. Check your country specific guidelines for a complete list.

Museumplein 5,

1071DJ Amsterdam

Street Name, Street Type,

Street Number, Postal code, Locality

8.2.2. Address Formats

Various address formats are used in different countries. The sequence and presentation of the components is adapted to match the local needs. For example, the street number comes before the street name in the United States but after it in Germany. See your Country Specific Guidelines for more information.

It might be impossible to confirm an address on official resources when another format is used. Rate Can’t Verify if the address points to the same location using an alternate format and there are no missing or incorrect components.

8.2.3. Street Result

Depending on the length of the street, different address components are expected.

Mandatory components may vary per country. See your Country Specific guidelines for a complete list.

•Local roads: Local roads that stay within a certain locality or county should carry the appropriate locality and state name.

•Highways/Interstates: Longer roads that travel between cities or even states don’t require a locality or state component. If such components are present, they must be accurate.

U.S. Route 395 covers almost the entire distance from Canada to Mexico. It passes through four states and connects countless towns, such as Big Pine, CA. Within that locality, it carries for a short distance the name N Main St. The two results have different expectations regarding components.

There are several forms that are appropriate for the result U.S. Route 395 because of its length:

•U.S. Route 395 or US 395

•U.S. Route 395, California

•US 395, Big Pine, California, United States

For the result N Main St, there is only one expected form because the road is short and stays within the locality:

•N Main St, Big Pine, CA United States

8.2.4. Locality Result

A locality result can be a city, village, municipality or neighborhood. In the U.S. the result usually includes the state it is part of.

Address	Mandatory components	Explanation
Sunnyvale, CA	Locality, State	Mandatory components may vary per country. Check your country specific guidelines for a complete list.
Amsterdam, Netherlands	Locality

8.2.5.Postal Code Result

Query

Address

Mandatory components

Explanation

[94087]

Sunnyvale, CA 94087

Locality, State, Postal

code

Mandatory components may vary per country. Check your Country Specific guidelines for a complete list.

[1071 DJ]

Museumplein, 1071

DJ Amsterdam,

Netherlands

Postal code, Locality

.Postal codes are returned when the user query matches the specific numbers or letters.

8.2.6.State/Region/Territory Result

If the user makes the query from a country outside the state or region result, that result must be accompanied by the country name.

Address	Mandatory components	Explanation
California	State name	If the result is outside of the test locale, country name is a mandatory component.
North-Holland	Region name
Puerto Rico	Territory name

8.2.7. Country Result

The country is the highest administrative level and has no other address component.

Address	Mandatory components	Explanation
Netherlands	Country	The country is the highest administrative level and has no other address components.
Romania	Country

8.3. Features Without an Expected Address

There are several kinds of features where no address is expected by the user. This could mean no street name or number or, in some cases, no address components of any kind, are expected to shown. These types of results have different rating criteria when it comes to Address Accuracy.

8.3.1. Natural Features

Natural features are specific landforms or ecosystems like rivers, mountains, jungles, and other geological features. They have names that can be rated, but typically do not have a street address, so natural feature results should be returned without any street address components. If a street address

is present, it is considered Incorrect – Other Issue, even when pointing to a building that is associated with the feature (like a ranger station or visitor center).

•A locality may be an acceptable address if it is appropriate for the size of the feature, as it can help locate a small feature, like a hot spring, within the locality. For larger features, such as a mountain, a locality may not make sense and would be rated Incorrect – Other Issue.

•The state the feature is in can also be provided as the feature’s address and rated Correct.

•If a postal code is included, it must be correct

•If a very large or expansive natural feature, like a river or mountain, spans more than one country, any one of these countries can be a Correct address. If no country at all appears, this is also Correct.

Parks and other protected areas are artificial boundaries created to help protect the natural feature. Those POIs are often named after the natural feature they contain. In cases where a feature can be considered a POI, such as Mount Rushmore or a state park beach, the official address can be returned. There might be other natural features within a park that reference the park name as part of their address details.

Result Details	Official Address	Rating and Explanation
Wreck Beach Vancouver	No official address	The beach can be confirmed as located within Vancouver and is therefore rated Correct.
Wreck Beach NW Marine Dr, Vancouver	No official address	The result contains the name of a street that provides access to the beach. But the street is not expected and therefore rated Incorrect – Other Issue.
Wreck Beach W King Edward Ave, Vancouver	No official address	The result contains the name of a street that is nowhere near the beach. The street is not expected and therefore rated Incorrect – Other Issue.
Mendenhall Glacier Juneau, AK 99801	No official address	This glacier is located in the borough of Juneau. The postal code in the address is associated with that locality, therefore both components should be considered Correct.
Mendenhall Glacier 8510 Mendenhall Loop Rd, Juneau	No official address	.The given address belongs to the overseeing Forest Office. The glacier should not have this address and is rated Incorrect – Other Issue.
Mount Rushmore 13000 HWY 244 #31-1, Keystone, SD	13000 Highway 244 Building 31, Suite 1 Keystone, SD 57751	Mount Rushmore is both a mountain and a monument within its own national park. The address given on the official National Parks Service website confirms the result address as Correct.

Treat landmasses like islands and continents as natural features with no expectation of a street address. If one is returned, rate it Incorrect – Other Issue.

8.3.. POIs Without an Expected Address

POIs without an expected address differ from regular POIs and businesses in that the user would not expect a street name and number to be shown.

These types of POIs include, but are not limited to:

Parks (Golf courses and theme parks not included)

•Monuments

•National Landmarks

•Park-like heritage sites

•Bridges

•Squares

•Parking lots

•Transit POIs – these can include airports (including airport terminals), ferry ports, subways, bike share stations, train and bus stops

The rating criteria described in the sections below apply to all features without an expected address.

See Official Address Present if this kind of POI has an official address

8.3.2.1. Minimum Address Component

Whether there is an official street address or not, POIs without an expected address do not need to show it. The locality must be returned whenever a POI is small enough to fit entirely within it. A POI that covers an area overlapping several localities does not require a locality component.

See your Country Specific guidelines for more information on mandatory address components.

Result	Official Address	Rating and Explanation
Quincy Station Chicago, IL	220 S. Wells St, Chicago, IL 60606	Address Accuracy	Correct
		This is a transit POI so no street address is expected. The result address is just the locality. The POI is physically located within the given locality and it matches the one listed within the official address.

Result	Official Address	Rating and Explanation
Stonehenge Amesbury, Salisbury	None	Address Accuracy	Correct
		.This is a heritage site so no street address is expected. The result address is just the locality. There is no official street address but since the POI is physically located only within given locality it can be considered correct.
Tongariro National Park New Zealand	None	Address Accuracy	Correct
		This is a park and heritage site POI so no street address is expected. No locality is listed for this result. Since this POI covers an area that overlaps several localities no locality component is required.
Founders Park Virginia	351 N Union St, Alexandria, VA 22314	Address Accuracy	Incorrect – Locality
		This is a park POI so no street address is expected. No locality is listed for this result. Since this park fits into only one locality, Alexandria, the missing locality component must be considered incorrect.

8.3.2.2. Official Address Present

Official address is a full street address:

Some of these features have an official street address listed on their website or found on other official sources, but users wouldn’t necessarily expect the address to be shown. Since a full address is not expected, the result does not need to contain a full address even if there is one found via official sources. Do not automatically consider any missing street address components as incorrect.

Full Address: When the result contains a full address it should be rated Correct if:

•The result and official addresses are identical

Street: When the result address is just a street it should be rated Correct if:

•The result address street is the same as the street within the official address

Consider the following examples for Embarcadero Station in San Francisco:

Result	Official Address	Rating and Explanation
Embarcadero Station 298 Market St, San Francisco, CA 94111	.	Address Accuracy	Correct
		The result has a full address that is the same as the address listed on the official website.
Result	Official Address	Rating and Explanation
Embarcadero Station Market St, San Francisco, CA	298 Market St, San Francisco, CA 94111	Address Accuracy	Correct
		The result address is just a street. Since the street matches the street listed within the official address it should be considered correct.
Embarcadero Station 388 Market St, San Francisco, CA 94111		Address Accuracy	Incorrect – Street Number
		The result has a full address. The street number does not match the number listed within the official address and should be considered incorrect.
Embarcadero Station Davis St, San Francisco, CA		Address Accuracy	Incorrect – Street Name
		The result address is just a street. The street is not the same as the street listed within the official address and should be considered incorrect.
Embarcadero Station		Address Accuracy	Incorrect – Locality
		The result has no address details at all. Locality is the minimum component required for this kind of result and should be rated Incorrect – Locality if missing.

Official address is a partial street address

When the result address is more specific than the official address it can be rated as Can’t Verify or Incorrect, unless another official source can be found to confirm the returned address.

Full Address: When the result contains a full address it should be rated Can’t Verify if all of the following apply:

•It does not belong to another POI/entity

•The street within the result address matches the street listed as the official address

Result	..Official Address	.Rating and Explanation
St. Nicholas’ Church Nikolaikirchplatz 3, 10178 Berlin, Germany	Nikolaikirchplatz 10178 Berlin, Germany	Address Accuracy	Can’t Verify
		The result POI is a landmark church and museum. The result has a full address. The street number is not associated with any other POI or entity and the street within the address matches the official street address.
St. Nicholas’ Church Nikolaikirchplatz 5, 10178 Berlin, Germany		Address Accuracy	Incorrect – Street Number
		The result has a full address. The street number belongs to a building not associated with the POI and should be considered incorrect. The street within the address matches the official street address.
St. Nicholas’ Church Poststraße 1, 10178 Berlin, Germany		Address Accuracy	Incorrect – Street Number, Street Name
		The result has a full address. The street name does not match the official street name, so both address components should be considered incorrect.

Consider St Nicholas’ Church in Berlin, Germany:

8.3.2.3. No Official Address

If no official street address is found, the result street address can be rated as Can’t Verify or Incorrect unless another official source, like street imagery, can be found to confirm the returned address. When there is no official address the returned street address cannot be confirmed and therefore cannot be considered correct. (Please see your Country-Specific guideline for more details.)

Full Address: If the full address belongs to another building not associated with the POI (even if they are on the same street) it should be rated Incorrect. When the result contains a full address it should be rated Can’t Verify if all of the following apply:

•It belongs to a building associated with the intended POI/entity

•The street provides access to the intended POI

Street: When the result address is just a street it should be rated Can’t Verify if:

•The street provides access to the intended POI

Determining Access:

A person must be able to reach the POI directly from the given street by walking or driving. This includes:

•Streets providing access by following the boundaries of the feature or leading into it.

•T-intersections:

•A street ending at an intersection does not provide access to the feature on the other side of it.

•A street provides access to a feature when it continues through the feature (even as a different road type with or without a different name) as long as it keeps its original name on the other side of the feature, and by doing so provides physical access.

•Ignore access restrictions when considering whether a street provides access or not.

Result	Rating and Explanation
Grosvenor Square Garden Grosvenor Square, London	Address Accuracy	Can’t Verify
	The result address is the street in front of the result POI. Like the example illustrated above, this street provides direct access to the result POI.
Grosvenor Square Garden N Audley St. London	Address Accuracy	Incorrect – Street Name
	The street ends in a T-intersection with the park on the other side. The street does not continue on the other side as it becomes S Audley St. and is therefore rated Incorrect.
Result	Rating and Explanation
Charles de Gaulle Airport Rue A. Parreux, Mauregard	Address Accuracy	Incorrect – Street Name
	The result address is just a street. The street does not provide direct access to the result POI and should be considered Incorrect.
Charles de Gaulle Airport 4 Rue de la Fossette, Mauregard	Address Accuracy	Incorrect – Street Number
	The result has a full address. The street provides direct access however, the street number belongs to a building not associated with the result POI and should be considered Incorrect.
Charles de Gaulle Airport 64 Rue de Claye, Mauregard	Address Accuracy	Incorrect – Street Number, Street Name
	The result has a full address. The street does not provide direct access to the suggestion POI and the street number is associated with another entity. Both address components should be considered Incorrect.
Zurich Airport	Address Accuracy	Incorrect – Locality
	Since the airport is associated with a distinct locality “Kloten” it requires at least a locality name and is rated Incorrect – Locality if this is missing.

9. Pin Accuracy

For each result returned, you will find a pin on the map. Rate the accuracy of each pin’s placement using this scale:

•Perfect

•Approximate

•Next Door

•Wrong

•Can’t Verify

The pin should be a reflection of the result: The location represented by the pin should reflect the result. Pin ratings should be evaluated individually and not influenced by address or other data. A pin can be correct when other rating components are rated wrong.

Missing Pins

If a pin for a result does not appear on a map, rate the pin Wrong. If this happens five or more times in a row, release the task for technical reasons and explain why in the comments box.

9.1. General Considerations

The following chapters will provide more detail on how to evaluate pin placement based on the feature presented. They describe issues to consider when evaluating the pin.

9.1.1. Pin Resources

Rate pin placement according to consensus from multiple reliable online resources. These resources should include both aerial and street-level views whenever possible. Reliable resources include:

•Reliable online map resources, including vector and hybrid views

•Hybrid views may provide additional points of reference including the locations of streets

•Street imagery

•Accurate street-level imagery can also be found on claimed social media sites, crowdsourced user review sites, and articles in primary publications (newspapers or similar publications that do their own reporting)

•Official venue maps and directories, which can contain information not available in street imagery

•Government property registries

Note: After consulting these research resources, always verify pin placement using the layers provided in the TryRating tool. This means that even if you have gathered coordinates elsewhere, you must rate pin placement based on what you can actually see in the TryRating layers.

Leave comments including coordinates and links to resources if the pin rating was difficult to determine or if you could not verify the pin’s placement.

9.1.2. Satellite Imagery vs. Vector Map

When there is a difference between where the pin drops in satellite imagery and the vector map, always use the map layer in the TryRating tool that is more generous toward the pin. For example, if the pin drops on the rooftop in satellite imagery, but in the parking lot in vector map, rate using satellite imagery because a pin on a rooftop will get a better rating than a pin placed in a parking lot. Even if there is a major shift between satellite imagery and the vector map in a larger area, such as a complete city or region, rate according to the most generous layer.

Note: When deciding between the generosity of the vector and satellite views for pin rating, use only the vector/satellite views provided in the TryRating tool.

In cases where the vector map is more generous toward the pin but does not show any building contours, use satellite imagery as a reference to locate buildings on the vector map to allow for rooftop ratings. If satellite imagery is unavailable, use your best judgment to determine the correct location of the pin.

In the screenshot below, the vector map does not provide any contours, so it is difficult to determine where each building is. Satellite imagery should be used instead.

9.1.3. Boundaries of the Feature

To rate a pin for a feature, you must first understand where the feature begins and ends. Some features have rooftops (houses, businesses, malls, and so on) and sit on a parcel of land that belongs to or is associated with them. Other features do not have rooftops (mountains, parks, waterfalls), but there is still a place where they begin and where they end.

A boundary is a real or imaginary line that separates one feature, or the parcel it sits on, from another. The boundaries of a feature always include half of the road when a road is present (see the Half ‘n Half rule below).

Boundaries can include:

•Fences

•Walls

•Garden plants and bushes

•Bodies of water (lakes, riverseans)

•Other dividers

•Property boundaries confirmed by official sources

If there is no divider, draw an imaginary 90-degree line to the road or use the Half ‘n Half rule below.

9.1.3.1. Half’nHalf and Tennis Rules

In dense areas it may be difficult to find boundaries between parcels. In these cases, divide the space between the street or next building in half to create a boundary. Use this “Half ‘n Half” rule to determine the Approximate and Next Door pin locations.

Use natural boundaries as much as possible to draw the line between a Perfect, Approximate, Next Door, and Wrong pin locations.

To address close calls, apply the Tennis Rule:

^•If the tip of the pin is still touching the line, the pin is considered inside the boundary.

^•If the tip of the pin points outside, it is outside of the boundary.

9.1.3.2. Boundaries in Shared Spaces

When a feature shares a parcel or parking lot with other entities and the same space is used to access many entities, it can be difficult to determine a feature’s boundaries. In cases like this:

•Use the guidance provided in Boundaries of the Feature

•If the shared parcel contains any through roads, apply the Half ’n Half rule to them:

•Through roads are part of the public road network that can be used to go to places other than the specific parcel

•The Half n‘ Half Rule may also be applied to the parcel’s internal access roads if they meet all three of these conditions:

•They must continue all the way through the parcel and

•They must have an exit and entrance to public roads at both ends and

•They must be continuously and clearly separated from the shared parking areas by curbs, barriers, road markings, do not cross lines, or similar features

Note: If all three of the conditions above do not apply, then an internal access road cannot be considered a boundary.

9.1.4. Next Door

A pin is considered Next Door if it drops on the immediate property next to the intended one. A Next Door pin must be:

•On the same street as the intended property

•The Next Door property must share the same street name as the intended property

•On the same side of the street as the intended property

•The first property to any side of the intended property

•On the same block as the intended property

“Immediate property” refers to the boundaries of the property as a whole, even if there is no rooftop. Determine the boundaries as explained in Boundaries of the Feature.

Note that the Half ’n Half rule explained in Section 9.1.3.1 also applies to Next Door properties

No Next Door for Shared Spaces

A feature cannot be Next Door to another feature within the same property boundaries. This means that two buildings in the same shared parking lot or parcel can never be rated as Next Door to one another.

There will also be no Next Door ratings made outside the parcel or shared space. Any pin falling outside of the Approximate area will be marked Wrong.

9.1.5. Can’t Verify (Smallest Identified Area)

When a specific rooftop or boundary cannot be identified, Perfect cannot be given. In these cases, Can’t Verify can be used if the pin is within the smallest area that can be identified as a potentially correct location. A pin outside of this area is considered Wrong.

The Can’t Verify rating will also be used when an address is rated Incorrect (Address does not exist).

Rating	Explanation
Perfect	Pin falls on the rooftop of the intended property.
Approximate	Pin falls within the boundaries of the property, including on any rooftop that’s not the intended one.
Next Door	Pin falls on the next door property.
Wrong	Pin falls outside of the property boundaries or next door property. Any pin falling outside of the Approximate area in a shared space will be rated Wrong.
Can’t Verify	See Can’t Verify

9.2. Single Rooftop

Best Available Evidence

Pin rating for most features found under rooftops is based on the best available evidence for that feature’s location. That is, the more evidence that can be found to verify a feature’s location, the more precise the pin’s location must be in order to be rated Perfect. This is true whether or not the feature’s address includes a unit number, because the goal is to find the location of the feature.

If street imagery or other strong evidence can be found to confirm a feature’s specific location under a shared rooftop, only that location will be considered Perfect.

If a shared rooftop can be identified but there’s no strong evidence to confirm the feature’s specific location under it, the entire rooftop will be considered Perfect.

If several rooftops share an address and there’s no strong evidence to confirm which rooftop the feature is under, all rooftops will be considered Can’t Verify.

Note: This general pin rating rule does not apply to transit POIs, which have their own pin rating rules.

9.2.1. Leaning Buildings

When a building or buildings appear to be leaning in satellite view, the satellite photos do not always give accurate information about their shape or location. This is why even when a pin appears to land on a leaning building, you cannot rely on satellite photos alone to rate pin placement.

Research the leaning building’s true location using consensus from multiple reliable resources, including:

•Vector views

•Hybrid views, which may provide additional points of reference, including the locations of streets the buildings lean over

•Street imagery

•Reliable online map resources

Always rate the pin based on the building’s actual location as confirmed by reliable resources, not on the appearance of leaning caused by satellite photography.

If no reliable resources are available and the pin is not objectively wrong, rate Can’t Verify and leave a comment.

Rating	Explanation
Perfect	Pin falls on the building’s actual location as confirmed by consensus among multiple reliable sources.
Approximate	Pin falls within the boundaries of the property including on any rooftop that’s not the intended one.
Next Door	See Next Door

9.2.2. Residential Property with Multiple Buildings

Some residential properties consist of more than one building associated with a single address. A pin falling on any house(s) on the property will be rated Perfect. If the evidence shows that any other structure on the property could also be a dwelling, it can also be rated Perfect.

In cases where the property also contains support or auxiliary buildings, including, but not limited to, sheds, garages, greenhouses, playhouses, pergolas, and doghouses, these structures should be considered part of the Approximate area.

9.3. Multiple Rooftops

Some POI results consist of multiple buildings associated with/owned by a single business or POI. To rate the pins for these results correctly, you must research and understand the relationship of the structures within the boundaries of the property (or parcel of land) to the POI.

Two kinds of ratings are possible for multiple rooftop POI results:

1.Multiple-Rooftop: When the result is a POI that contains no other POIs and consists of more than one building with multiple rooftops on the property. In cases like this, primary rooftops will be rated Perfect. The parcel of land inside the boundaries of the POI will be rated Approximate. Properties next door to the intended one will be rated Next Door.

2.Campus: When a multiple-rooftop result is a higher education institution like a university or a medical, retail, or other type of business complex, most often containing other public-facing POIs. In cases like this, the entire parcel or property inside the boundaries of the POI, including all rooftops, will be rated Perfect. There will be no Approximate or Next Door ratings.

9.3.1. Multiple-Rooftop Rating

Single entities with multiple rooftops are single businesses or POIs that operate out of more than one building. This category includes, but is not limited to, entities like factories, gas stations, apartment complexes, company headquarters, lumberyards, and breweries, and other types of POIs that generally do not contain additional businesses or POIs within them.

Every primary rooftop on the property will be rated Perfect. The parcel of land inside the boundaries of the POI will be rated Approximate. Properties next door to the intended one will be rated Next Door.

In cases where the property also contains support or auxiliary buildings, including, but not limited to, guardhouses, dumpsters, storage sheds, donation bins, utility buildings, and toilets, these buildings should be considered part of the Approximate area.

Rating	Explanation
Perfect	Pin falls on rooftops within the boundaries of the property.
Approximate	Pin falls within the boundaries of the property (or on support or auxiliary buildings within it)
Next Door	The pin falls on the Next Door property.
Wrong	Pin falls outside of the property boundaries or outside of the Next Door property.
Can’t Verify	See Can’t Verify

9.3.2. Campus Rating

Some POIs will encompass entire business complexes, meaning that there are multiple buildings associated with/owned by the POI. These POIs include (but are not limited to) institutional complexes like universities, hospitals, and airports. Other POI types that can be rated as campuses include shopping and strip malls with multiple rooftops, amusement parks, resorts, and zoos.

In cases like this, the entire parcel or property inside the boundaries of the POI, including all rooftops, will be rated Perfect. There will be no Approximate or Next Door ratings.

The property of a campus complex includes the buildings themselves, any shared parking lots, and half ’n half as described in the guidelines.

How to Identify Campus Results

Wondering if a multiple-rooftop result is a campus? Many, though not all, campuses can be identified by the presence of POIs within the POI. That is, individual stores within multiple-rooftop malls, departments within hospitals, libraries and colleges within universities, spas and restaurants within multiple-rooftop resorts, terminals within airports, and so on.

Rating	Explanation
Perfect	Pin falls within the campus/complex boundaries, as determined by research.
Approximate	N/A – Campuses and business complexes do not have Approximate ratings.
Next Door	N/A – Campuses and business complexes do not have Next Door ratings.
Wrong	Pin falls outside of the campus/business complex boundaries.
Can’t Verify	See Can’t Verify

Note: The result may be for an address that should have the campus rule applied.

Some of these types of POIs may have polygons outlining the boundaries of the POI on the map. These polygons may not always be present or perfect, so rate against the true boundaries of the campus or complex as determined by research. A few examples of campuses are below, including one that is missing its polygon.

9.3.2.1. Single-Complex Campus

Entities made up of a single cluster of buildings are the easiest to identify. On the map, these entities can be represented by a single, continuous polygon. Remember, the polygons on the map may not always be perfect or present, so rate against the true boundaries of the campus or complex as determined through research.

The property of a complex includes the buildings themselves, any shared parking lot and half n half as described in the guidelines.

Note: Parking facilities may be included in the polygon if they are owned by the POI and contiguous to the campus (contiguous means that they are not separated from the campus by a street).

Tip: Start by looking at the pin’s location, then determine if the property surrounding the pin belongs to the POI you are looking at.

...

9.3.2.2. Dispersed Campus

In some cases, a campus will not have a single, continuous location that a polygon can be drawn over, but rather be spread out across a large area like a city. Sometimes, the pin will fall on this type of campus and there may not be a polygon for the campus. If this occurs, research to determine where the polygon should be. Pins that fall on campus-associated, non-parking facilities are valid pins. Apply the single rooftop rules to any standalone, non-parking facility, campus building.

If two or more campus buildings are next to each other, apply the single complex campus logic.

9.4. Features Without a Rooftop

This chapter will describe how to rate pin accuracy for features that typically do not have a rooftop.

.

Rating	Explanation
Perfect	If the natural feature has a defining feature (water for a river/ocean, cliffs/peak of a mountain, the sand for a beach), pin should fall on that feature or if the natural feature is defined by an arbitrary boundary (parks, national forests), pin should fall in their polygon or where it would be if we had that information.
Approximate	The pn falls outside of the defining features of a natural feature but still on it (slope of a mountain rather than the peak, the shore of a river or lake). Consider each situation individually. Not all natural features will have an Approximate. If the natual feature exists within an urban or suburban area, you may apply the
.	Half ‘n’ Half rules..
Next Door	N/A – Natural features do not have Next Door ratings.
Wrong	The pin falls in an area that does not meet the criteria for Perfect or Approximate.
Can’t Verify	See Can’t Verify

. .

9.4.2. Administrative Divisions

Administrative divisions can refer to, but are not limited to:

•Countries

•States

•Counties

•Provinces

•Cities

•Neighborhoods

9.4.3. Streets

In this chapter, rating pins for streets will be explained.

9.5. Parking Lots and Structures

Shared parking lots are considered Approximate even though they do not belong entirely to the feature. Assume that parking lots belong to the feature and do not split them. When Sourdough & Co (green building) is the result, the approximate area (yellow) extends up to the public road to the north.

In cases where the parking structure can be verified in satellite imagery (by either parked cars or parking lot striping), consider the entire structure a parking lot. The assumption is that there will be no businesses within the structure other than parking-related businesses.

....

split them by applying the Half n’ Half rule.

In the example below, the result is a shopping mall with both a parking lot and parking structures. Pins that fall on the building (rooftop) of the mall are Perfect. Apply the Half ‘n Half rule to the street and internal access roads, if any, to determine the area for Approximate, as parking lots and parking structures are also Approximate.

9.6. Transit POIs

Transit POIs include, but are not limited to, POIs that are transit stops, stations, toll booths, and terminals. A transit stop is a small, transit location typically designated by a bench or covered bench area, a posted sign, or street markings. Transit stations, on the other hand, are typically larger stops that have a major structure associated with them, which include, but are not limited to, platforms, ticketing offices, enclosed waiting/seating areas, or terminal buildings/structures encompassing the transit area. The term “station” is primarily used to refer to transit locations that are larger and typically a terminal housing multiple routes. All stations are stops, but not all stops are stations.

If a transit POI does not meet the criteria to be rated under the Single Rooftop or Campus/Complex guidelines, please use the following criteria to rate.

Rating	Explanation
Perfect	The pin falls on the polygon for the transit POI or in the area where you would wait for transit or within the entrance polygon for an underground transit station
Approximate	The pin falls within 50 meters of where you would wait for transit or within station parking lots and surrounding property up to where the Half ‘n’ Half rule allows.
Next Door	Transit POIs do not have Next Door ratings.
Wrong	The pin is wrong if it falls: •Farther than 50 meters from the ideal location •Within 50 meters on a non-associated rooftop •Outside of the boundaries as allowed by the Half ‘n’ Half rule •Beyond half a city block (applies to bus stops only)
Can’t Verify	See Can’t Verify

9.6.1. Bus, Tram, and Streetcar Stops

Bus, tram and streetcar stops are often, but not always, on the side of the street and have one spot where users can wait to board the bus. Perfect pins should be placed on the spot where a user would wait while Approximate pins can be placed within 50m of the waiting spot and within the Half ‘n’ Half rule.

9.6.2. Transit POI with Multiple Rooftops/Platforms

If the POI has multiple rooftops/platforms, apply similar ideas from the campus/complex pin guidelines when determining the valid area for the pin to land on.

For campuses/complexes, a polygon was created using the boundaries of the campus. Use the same concept with transit POIs and create a polygon for the transit POI using the POI’s multiple rooftop/ platforms.

9.6.3. Transit POIs with a Single Rooftop

If the POI has a single rooftop, apply the single rooftop guidelines.

9.6.4. Underground Transit Stations

Some transit stations are underground. For stops that do not have polygons available on the tool map, apply similar rules to the satellite campus and use your imagination to draw a polygon using the entrances to the underground station as polygon coordinates. A pin dropped within 50 meters of this polygon should be rated Approximate.

If a pin is placed on a building within the polygon, the building must contain an entrance to the transit stop for the pin to be Perfect. If the pin falls on any other building within the polygon, rate it Approximate.

Some entrances can be seen via satellite view of the station, while other entrances can only be found by doing more research, including using street imagery of the station area to find entrances or using a station map from the transit provider that shows entrances to the station.

9.6.4.1. How to Establish an Entrance Polygon

1.Find at least three entrances to the underground transit station, if possible.

•Use street imagery or an official source to determine where the transit station entrances are located.

•Tip: A good place to start looking for transit entrances is near the pin’s location. This will make it easier to determine whether or not the pin falls within the polygon.

2.Connect the entrance points to create an enclosed polygon, where each entrance is a point of the polygon.

3.Determine whether or not our pin falls within 50 meters of the polygon or within the property covered by the Half ‘n’ Half rule.

If the underground transit station only has two entrances, use the following steps to create an entrance polygon.

9.6.4.2. Examples

10. How to Rate Results

This chapter shows how to rate a variety of examples.

For each result, you will need to answer questions about:

•Relevance

•Name accuracy

•Address accuracy

•Pin accuracy

10.1. Specific Address

When the query is an address, and the user explicitly states their location intent by including the name of a locality, this is an explicit location query and the user’s viewport and location are irrelevant.

The query refers to a unique location, so a result for this exact location should be rated Navigational.

Address queries can also be just the name of a street, neighborhood, intersection, or city. Results for streets, neighborhoods, intersection, and city queries are eligible for Navigational.

The expectation is that we return the requested entity with the same level of broadness that the user requested, which may not include all of the elements of a full address.

10.2. Non-Specific Address

For queries where the location intent is not explicitly stated in the query, you must infer the location intent from the user’s viewport and the user’s location. One difficulty in evaluating address queries has to do with partial addresses, which could refer to multiple locations.

Generally, for partial addresses, users are looking for the location closest to their position or within/ near the fresh viewport. Some general considerations:

•Exact location that exists returned very close to the user is eligible for Navigational, unless other addresses that satisfy the user intent are also very close.

•Exact location returned within a fresh viewport is eligible for Navigational, unless other addresses that satisfy the user intent are also very close.

•Results that are farther away from an ideal result but can potentially satisfy the user’s intent can be rated as high as Good (Distance/Prominence)

•Results are demoted based on distance.

•Results are demoted based on the density of potential results in a given area

•Results that are too far away will be rated Bad (Distance/Prominence) Consider the query [154 orchard st] in different scenarios:

Scenario 1: Fresh viewport over Midtown Manhattan in New York City.

Scenario 2: Viewport/user’s location is in Maine. There is no address in the vicinity that contains [154 Orchard St]. All locations with this address are far from the location intent of the query..

Here is another example of an address query with implicit location intent. Although this is a common address in US cities, it overwhelmingly refers to the location of the White House and should be considered the primary intent of the query unless an additional location modifier is provided (i.e.

.state, city, or postal code) or a different 1600 Pennsylvania Ave that’s not the White House is already in or very near the location intent.

Scenario 1: Fresh viewport on Los Angeles, CA (not near Richmond, CA):

Scenario 2: The fresh viewport is in San Francisco, CA (near Richmond, CA). The proximity of the user viewport to the Richmond location makes this location just as relevant as the prominent location in Washington DC.

10.3. Query Address Does Not Exist

To be useful to the user, address results need to point to either a building with that address or a plot of land that has been officially assigned the address. Sometimes your research will show that a full query address does not exist (or no major search engine will be able to determine its exact location).

In cases like this, you will see one of three types of results:

•The closest verified address (on the same street and in the same city and state): Rate relevance as Excellent.

•The same address as the query address: Rate relevance as Excellent and address as Incorrect – Address does not exist. The pin will be rated Can’t Verify.

•The queried street without a street number: Rate relevance as Acceptable.

When a queried address does not exist, the answer to the question “Is there a navigational result for this query” will always be No.

10.4. Point of Interest

A correct location result should be rated Navigational for queries referring to a point of interest. Consider the query [mount rushmore], the famous US monument. Although the query terms do not contain an explicit location, the Mount Rushmore monument is so prominent and unambiguous that its location should be considered the primary location intent. The user viewport and user location are irrelevant for this query because the user is looking for the place of interest no matter where they currently are.

10.5. Understanding Multiple Query Interpretations: Beyond Viewport and User Location

Many address/POI queries have multiple interpretations. Understanding which interpretation is most likely often requires extra research in addition to considering the viewport and user’s location. It is critical to fully understand the query through research rather than guessing.

10.6. Business Queries

Before you rate a business query, consider:

•Multiple interpretations: Are there ways to interpret the query terms other than as a business name? French Laundry is both a famous restaurant in Napa and a type of laundromat/cleaner.

•Number of locations: How many locations of this business exist? If there are many of them (like Starbucks) then the user expects to find locations in or very close to the their location intent.

•Population density: Is the area of the business results urban, suburban, or rural? City dwellers are less willing to travel longer distances than suburban and rural users.

10.6.1. Non-Chain Businesses

Some businesses have only one location, so treat and rate them as navigational queries. The correct location should be rated Navigational and any other results should receive Bad (User Intent).

Query	.	Results	Ratings and Explanation
[Gary Danko 800 North Point St, San Francisco, California 94109]	.	.Is there a Navigational Result for this Query?		Yes
		Restaurant Gary Danko ...800 North Point St, San Francisco, California 94109 Category: American (New)	Relevance	Navigational
			Name Accuracy	Correct
			Address Accuracy	Correct
			Pin Accuracy	Perfect
			This is a nationally famous restaurant in San Francisco with no other possible results.
		800 North Point 800 North Point St, San Francisco, California 94109	Relevance	Bad (User Intent)
			Name Accuracy	n/a
			Address Accuracy	Correct
			Pin Accuracy	Perfect
			When a user types in the name of a business or a POI along with its address, they expect to see the business mentioned in the query as part of the result. If the result is the address alone and does not mention the business, the user cannot be certain that the query and the result refer to the same thing. This is why the result’s relevance is Bad.
		Garry Lee, DDS 1259 Mason St, San Francisco, California 94108 Category: Dentist	Relevance	Bad (User Intent)
			Name Accuracy	Correct
			Address Accuracy	Correct
			Pin Accuracy	Perfect
			Result is unrelated to the queried restaurant.

Sometimes there are multiple unrelated businesses with the same name in the same area. In this case, rate each business independently considering its relevance and distance from the location intent of the query.................

10.6.2. Chain Businesses

A query for a chain business is generally not considered an explicit query and is therefore not eligible a for Navigational rating unless the query contains a location modifier that points to a single unique location (see Chain Business with Location Modifier).

Chain businesses are businesses that have more than one location, and include everything from national chains like Starbucks, Target, Auchan, Albert Heijn, and Boots down to small local chains.

Distance evaluations should be based on chain business locations in the real world, not only the returned results.

•The more locations the chain business has in the area, the closer the results should be in order to be considered relevant.

•The fewer locations the chain business has in the area, the more you can widen the distance for relevant results.

Note: Go to the chain business’ official website and check the address of the specific branch you’re rating. For each branch of a chain business, research to verify that it actually exists at the address listed in the result.

•User and viewport location should always be ignored when there is a location modifier

•Ratings must always be based on possible real-world results

There are two types of location modifiers:

•General location modifier (locality or postal code)

•Specific location modifier (street or full address)

10.6.3.1. General Location Modifier

Location modifiers in the form of localities or areas are considered to be general location modifiers.

Results inside the requested location

When the location modifier is general, results inside the specified location are not demoted for distance.

Results outside the requested location

Location of results

..Quantity of results inside the requested location

Rating and Explanation

Inside the requested location

One

If there is only one result present in the requested location, it should receive a rating of Navigational.

It is promoted because it is the only result that satisfies the user intent

More than one

.When there are multiple results for the chain business in the requested location, they should all receive a rating of Excellent.

Ignore user and viewport location. The user has already told you exactly where they are looking for results.

When the location modifier is general, results outside the requested location are demoted based on how many results there are inside it.

Location of results

Results inside the requested location

Rating and Explanation

Outside the requested location

None

If there are no results for the chain business inside the requested location, results outside the area may be demoted less severely depending on how close they are to the named location and how many branches of the chain exist in the real world. Ratings may range from Excellent to Bad.

This is because when there are no locations of a chain business inside the requested area, results that are outside it can still be relevant. However, it’s also possible for results to be so far away from the location that they have no relevance at all.

Location of results

Results inside the requested location

Rating and Explanation

Few

...If there are few results for the chain business inside the requested location, results outside the area may be demoted less severely depending on how close they are to the named location and how many branches of the chain exist in the real world. Ratings may range from Good to Bad.

This is because when there are fewer locations of a chain business inside the requested area, results that are outside it can still be relevant. However, it’s also possible for results to be so far away from the location that they have no relevance at all.

Many

When there are many results for the chain business inside the requested location, results outside the area are demoted to Bad. This is because there are multiple

results that would satisfy user intent inside the requested area, so there’s no need to look outside it.

.....................

10.6.3.2. Specific Location Modifier

Location modifiers in the form of streets or full addresses are referred to as specific location modifiers.

A specific location modifier might also be the name of a POI, as seen here: [university of kentucky starbucks] in a query about the location of the coffee chain at a particular university.

When the a query includes a specific location modifier, results that satisfy the location intent should not be demoted for distance:

Location of results

Number of results inside the requested location

Rating and Explanation

Inside the requested location

None

Consider the location modifier a general one and demote results based on distance to the specified location.

One

If there is a only a single result present in the specified location it should receive a rating of Navigational.

It is promoted because it is the only result that satisfies the user intent

When a query is for a POI on a named street, results for this POI that are on the named street but have an official address on another street can also be rated Excellent.

10.6.4. Back Office and Businesses with No Physical Location

These type of results are most often rated Bad because they are either not useful to the user or have no maps intent at all.

Business Type	Example	Rating and Explanation
No Physical location	Mobile locksmith	Businesses with no physical location, like mobile locksmiths or mobile dog groomers, have no relevance for maps and are rated Bad.
Back Office	Administrative offices of a company	Back offices are usually not open to the public. Rate them with a lower prominence. In most cases the relevance rating will be Bad unless these offices match the specific user intent.
Home Business	Home based calligrapher	Unless your Country Specific guidelines state otherwise, give businesses running out of a home very low prominence. These businesses see customers by appointment only and have no regular opening hours or brick and mortar businesses (like a store with regular hours where customers can drop by and choose from a large selection of merchandise). The most likely relevance rating is Bad, unless there is a likely user intent or the suggestion is similar to a bed and breakfast or other business that a user may want to visit.

For these types of results, rate the address accuracy according to the official website or consensus of three sources, otherwise rate Can’t Verify. If the entire address is missing, rate Can’t Verify. If an address is present, rate the pin according to the address in the result. If no address is present, rate the pin Can’t Verify.

10.6.5. Business/POI Does Not Exist

There are situations where a result returned is for a business that does not exist. You must still rate relevance as if the business were open.

10.7. Category Queries

Category queries are the broadest query type because they can return the greatest variety of results. Give prominence more weight than distance when rating them.

The highest relevance rating most category query results will receive is Excellent.

Category queries with a location modifier are the only category queries eligible for a Navigational rating (see Navigational Results for Category Queries).

You’ll need to determine the user’s intent based on the query:

•Primary Intent refers to the businesses or POI the user is expecting to see for the given category.

•Secondary Intent refers to businesses that fit the category intent but would not be the user’s first choice. Secondary intent results will receive a demotion of -1 or an initial rating of Good.

•Unlikely Intent refers to results that technically match the query terms or category intent, but woud not be an obvious, or even secondary, intent for most users. They receive a demotion of -2 or an initial rating of Acceptable.

Significant Increase in Distance

..Given how many results are often within close proximity to the location intent, results for category queries are not demoted for distance simply because there is something closer (in the result set or in real life). Instead, use significant increases in distance to group possible results for the appropriate demotions.

10.7.1.Demotion for Distance with User Inside Viewport

Results	Ratings and Explanation
① Domino’s Pizza 930 W Hamilton Ave Campbell, CA Category: Pizza	Relevance	Excellent
	.At about 1 km away, this pizza place is among the closest results to the user, who is in an area with many pizza options. Other results in the same general range would also receive this rating.
② Luigi’s Pizza and Pasta 2495 S Winchester Blvd, Campbell Category: Pizza	Relevance	...Good (Significant Distance)
	Luigi’s is significantly further away from the user is and is demoted for distance, as are all other results in this general range.
③ Pizza My Dear 2590 S Bascom Ave, San Jose, CA Category: Pizza	Relevance	Acceptable (Significant Distance)
	Pizza My Dear is over 4km away from the user, which is significantly further than many other results. It is demoted -2 for distance.

10.7.2. Demotion for Distance with User Outside Viewport

Results	Ratings and Explanation
① EDC TX 964 N Walnut Ave, New Braunfels, Tx Category: Gun Store	Relevance	Excellent
	This result is inside the very specific viewport and carries weapons and other accessories.
② Freddy’s Pawn & Jewelry 671 S Seguin Ave, New Braunfels, TX Category: Pawn Store	Relevance	Excellent
	The pawn shop is very close to the fresh Viewport and sells guns and related items.

Results	.	Ratings and Explanation
③ Guntrap 850 Schneider, Cibolo, TX Category: Gun Store	.	Relevance	Good (Significant Distance)
		This weapons store satisfies the user intent but is significantly further away from the fresh viewport in a more rural area, with fewer such stores close by.
.④ River City Pawn & Jewelry 417 Main St, Scherz, TX Category: Pawn Store	.	Relevance	...Good (Significant Distance)
		The pawn shop sells guns and other related equipment but is also significantly further away from the fresh viewport.

10.7.3. Demotion for Distance with Fewer Possible Results

Results	Ratings and Explanation
① Deerfield Park 2401 Deerfield Dr, Kennesaw, GA Category: Park	Relevance	Excellent
	Deerfield Park is among the closest results to the user. There are other results that are within an acceptable range given the number of other possible results that are within 13km.
② Allatoona Creek Pitner Rd NW, Acworth, GA 30101 Category: MTB Trails	Relevance	Good (Significant Distance)
	These mountain bike trails are about 25km away from the user and significantly more distant than the closest group of results. The result, and others in the same general range, is demoted by -1 for distance.

Results

Ratings and Explanation

③ Iron Hill Trail Head Cartersville, GA 30121

Category: Hiking Area

Relevance

Acceptable (Significant Distance)

Iron Hill Trail Head is about 30km away. It is still within the Acceptable group in terms of distance to the location intent given the smaller number of possible results in the area, and gets a demotion of -2 for distance.

10.7.4. Category Queries and User Intent

10.7.5. Category Queries and Prominence

10.7.6. Category Query with Location Modifier

If the query contains a location modifier, all results within the specified location should receive the highest possible initial rating of Excellent. Results outside the expected location should be demoted based on their distance from the location modifier and the number of possible real-world results within it. User and viewport location should be disregarded. Further demotions can be applied based on how well the result fits the intent of the category.

Note that location modifiers for category queries can take many forms, from the names of cities, neighborhoods, and streets to the names of POIs, like [Aeropuerto barcelona gasolinera], a query about gas stations at or near Barcelona’s airport.

10.7.7. Navigational Results for Category Queries

Sometimes it’s not clear whether a query is for a category or a specific business/POI. What first appears to be a category quy with a location modifier may lead to a single prominent result which can be rated Navigational.

Due to the ambiguity of these queries, even if there is a Navigational result, the query should still be treated as a potential category query and other results considered relevant.

When deciding if a category query could have a Navigational result, consider:

•The actual wording of the query

•Possible results in the real world

•Local knowledge

•Prominence

10.7.8. Clear Categories

When a query is clearly for a straightforward category, results must also belong to that category

10.7.9. Soft Categories

Some category queries are straightforward and may have results that satisfy an intent exactly, like [italian restaurant]. However, sometimes categories are very small or have multiple interpretations, so several different types of results could satisfy the user’s intent. These are called “soft categories.”

These category queries will produce a variety of results, each of which could satisfy the query intent to a greater or lesser degree. How well a result satisfies a category query will vary depending on the services typically offered by the result entities in that market and the customs of the specific market.

It’s often difficult to define which businesses or chains belong in any one soft category, so you’ll have to research the individual results provided to make your rating decisions. For instance, the query [ski shop] is very specific and is associated with the more general Sporting Goods category.

10.7.10. Category Query: Parking

“Parking” generally refers to:

•Parking lots

•Parking garages

•Parking decks

10.8. Rating Results with PERMANENT_CLOSURE Status

When rating results whose status is PERMANENT_CLOSURE, be sure to understand whether the results are expected or unexpected. For more information on the differences between expected and

unexpected results, see section 5.19. Rating Relevance When Result Status isPERMANENT_CLOSURE.

10.8.1. Expected PERMANENT_CLOSURE Status

When a result with PERMANENT_CLOSURE status is expected, choose the Business/POI is closed or does not exist checkbox and rate as if the location were open.

10.8.2. Unexpected PERMANENT_CLOSURE Status

When many branches of a chain are available in the area of intent, we would not expect to see any locations with a status of PERMANENT_CLOSURE, whether that status is correct or not.

Suggestion	Rating and Explanation
① McDonald’s PERMANENT_CLOSURE 615 N Mathilda Ave, Sunnyvale, CA 94085	Relevance	Acceptable
	Business/POI closed/ does not exist	Yes
	This permanently closed restaurant (confirmed by research) has its status set to PERMANENT_CLOSURE. It is considered unexpected because there are many other locations of this chain in the area. This means the highest rating it can get is Acceptable. Name, address, and pin will not be rated.
② McDonald’s 550 Lawrence Expy, Sunnyvale, CA 94086	Relevance	Excellent
	Business/POI closed/ does not exist	No
	Name Accuracy	Correct
	Address Accuracy	Correct
	Pin Accuracy	Perfect
	This open chain location is among the closest to the user and rated Excellent.
⓷ McDonald’s PERMANENT_CLOSURE 556 E El Camino Real, Sunnyvale, CA 94086	Relevance	Acceptable
	Business/POI closed/ does not exist	No
	Name Accuracy	Correct
	Address Accuracy	Correct
	Pin Accuracy	Perfect
	This chain location is showing a status of PERMANENT_CLOSURE. Research shows that it is, in fact, open. Its relevance is rated independent of whether it is really open or closed in the real world. The result is considered unexpected and rated Acceptable. Do not mark this result as Business/POI closed/does not exist, and continue rating the result for relevance plus name, address, and pin details.

Suggestion	Rating and Explanation
④ McDonald’s 1082 E El Camino Real, Sunnyvale, CA 94087	Relevance	Good (Distance)
	Business/POI closed/ does not exist	Yes .
	.The chain location does not exist according to official resources and should be rated as Business is closed or does not exist. The result does not show the status as PERMANENT_CLOSURE, so it should be rated as if it were open, which is Good given the distance.

10.9.Other Query Types

Queries can sometimes be less obvious or even ambiguous.

10.9.1. Routing Queries

When a query mentions two distinct locations that are not near each other, the user was likely looking for driving directions, so returning either of the two distinct locations used for the route is the best experience Search can offer. Each location should be rated Excellent for Relevance.

10.9.2.Coordinate and “My Location” Queries

Note: Results for coordinate and ‘’my location” queries (or any variation of these, like “current location,” “where I am,” and so on) are rated differently from the other results described in these guidelines. Please read this section carefully.

Pin and Relevance are rated separately using a 50m radius around the queried coordinates or the user location (for “my location” queries):

If no result can be found within 50m, then the closest address or street up to 100m away will be accepted. If no address or street can be found within 100m, depending on the market, a locality or sub-locality will also be accepted.

Note that the pin and result title do not need to match. For example, if a result for United Furniture

Club is returned, but the pin is placed at the queried coordinates or user location, the Relevance Rating will still be Bad, but the pin rating will be Perfect even though it is far from United Furniture’s location.

10.10. Unclear Results

There may be times when it is difficult to determine exactly what the result is because the business name can’t be identified with certainty or the business does not correspond with the given location and it cannot be confirmed as closed. When this happens, use all provided information (including information that is not rated, such as URL and phone number – never call a business) to reach a conclusion.

If there is a tie in the evidence or you are rating Search Relevance only, rate Business/POI is closed/ does not exist, then rate relevance as if the entity did exist.

11.Top Rating Tips

Solutions to the most common rating dilemmas.

11.1.What’s the relevance when query is business/POI name and address and the result is only the address?

When a user types in the name of a business or a POI along with its address, they expect to see the business mentioned in the query as part of the result. If the result is the address alone and does not mention the business, the user cannot be certain that the query and the result refer to the same thing. This is why the result’s relevance is Bad.

Address and pin accuracy can be rated as usual.

Query	Results	Ratings and Explanation
[Gary Danko 800 North Point St, San Francisco, California 94109]	800 North Point 800 North Point St, San Francisco, California 94109 Category: New American Cuisine	Relevance	Bad (User Intent)
		Name Accuracy	n/a
		Address Accuracy	Correct
		Pin Accuracy	Perfect
		Result is for the address and not the business stated in the query, so the user cannot know whether or not the query and result refer to the same thing. This is why the result’s relevance is Bad.

11.2.What’s the relevance when the query is a street name and the result is a single business on that street?

A query for an entire street is very broad. When the result is just one business or one address on that street, it is too specific to satisfy the broad intent of the query.

Name, address, and pin accuracy can be rated as usual.

11.3. How do I know what the location intent is when user is outside/inside a fresh viewport?

When a viewport is fresh, the location intent for results depends on whether the user is inside or outside the viewport:

•When a user is inside the viewport, that user’s location is the location intent

•When a user is outside the viewport, results are expected in or near the viewport, no matter how near or far the user is from that viewport.

Viewport	User	Location Intent
Fresh	Inside Viewport	When the user is within a fresh viewport, take the user location as location intent. Results are generally expected in or near the viewport, and results inside the area cannot be rated Bad because of distance alone.
	Outside Viewport	Results are expected in or near the viewport area. If no results can be found in or near the viewport, use the user location as a secondary location intent.
	Missing	When the user is missing, the viewport remains the location intent.

11.4. How do I know what the location intent is when user is outside/inside a stale viewport?

When a viewport is stale, the user’s location is considered the location intent for the results whether the user is inside or outside the viewport

Viewport	User	Location Intent
Stale	Inside Viewport	When the viewport is stale, consider only the user location as location intent.
	Outside Viewport
	Missing	Use the stale viewport as location intent when the user location is missing.

11.5. This full address result does not exist. How do I rate it?

An address type result needs to be associated with a building or officially assigned to a plot of land in order to be useful to a user. Sometimes your research will show that a queried address simply does not exist. In cases like, this you’ll see one of three types of results:

•The closest verified address (on the same street, in the same city and state): Rate relevance as Excellent.

•The same address as the query address: Rate relevance as Excellent and address as Incorrect – Address does not exist. The pin will be rated Can’t Verify.

•The queried street without a street number: Rate relevance as Acceptable.

When a queried address does not exist, the answer to the question “Is there a navigational result for this query” will always be No.

Appendix: Release Survey

You may occasionally be unable to rate a survey. To skip it, click Release Survey at the top right of the tool.

A pop-up window with a list of reasons for release will appear.You must choose one before you can release the survey.

Note: If you change your mind and decide to rate the survey, click Cancel at the bottom left of the pop-up window.

Reasons to Release

1. Adult Content

You are not comfortable rating queries and/or results that include businesses offering adult services or entertainment, like strip clubs.

2. Technical Issue

Technical issues that prevent rating include:

•Map or map features not loading.

•Query field is empty.

•Tool is not interactive. You cannot zoom in and out or move the map at all.

•Cannot submit ratings.

•More than 5 pins in a row are missing or pins do not stay in place.

•Receiving 5 consecutive surveys with all results in China, if China, or Hong Kong, are not the markets you are rating.

If you choose this option, leave a comment describing the issue before releasing the survey.

3. Not Enough Time Allocated

Select this option if the Estimated Rating Time is not sufficient to complete the task. Remember, the Estimated Rating Time is the average time over multiple tasks.

4. Other

Issues not mentioned above that prevent you from rating.

If you choose this option, you must leave a comment before you can release the survey.

Do Not Release

These issues will not prevent rating and are not reasons to release surveys:

•Poor satellite image, including clouds over target area or lack of image detail

•You can zoom in and out on map, but not as much as you’d prefer

•Rating is too difficult

•Query does not appear to be related to maps

•Query or results are from outside your market

•A query has no results

•If you see a query with no results, answer the query-level “Navigational Result” question and submit.

•Query or results in unexpected languages and scripts:

•If you see a query in an unexpected language or script, research it or use an online translation tool to find out what it means, then rate it as usual.

•For results in an unexpected language or script see Result Name/Title in UnexpectedLanguage or Script and Language/Script Issues in Address

Name	Screenshot	Description
Task Bar	.	The task bar shows: •Task Type: The name of the task. •Task ID: A number that identifies the specific task item. Use this number whenever you communicate about a task or rating. •Request ID: A reference number for internal use only. Do not use this number to communicate about a task or rating. •Estimated Rating Time: Time to rate all results
Rating Guidelines		The Rating Guidelines shortcut button opens the guidelines for this task.
Release Survey		Allows you to skip a survey in rare cases where a technical or other issue prevents you from rating. See Appendix for complete instructions.
Show User		Viewport shortcut buttons provide convenient controls to pan or zoom to common map viewport settings. Show User moves the result viewport to the user’s location without changing the zoom level (map scale).
Show Viewport		Show Viewport moves the result viewport to the center of the user viewport and zooms in on the viewport area.
Show All		Use Show All to return from any zoom level to the initial display showing the user, the viewport, and all the results.

Name	Screenshot	Description
Status		.A result may also display its status. If the status appears and is set to ....PERMANENT_CLOSURE, the result requires special treatment when rating relevance.
Result with no Rating Choices	`	You may occasionally see one or more normal looking or grayed-out results with no rating fields beneath them. No rating is required for these results. This is expected and is not a technical issue or a reason to release the survey.
User Viewport		User viewport: A transparent purple box centered over a device icon shows the user’s viewport at the time the query was issued (if known). The user viewport represents the map area the user was looking at when making the query. The shape of the viewport may vary depending on the size and orientation of the device being used. Note: In some cases, the user viewport may be so small that it cannot be seen, but (if the viewport information is available) the device icon will always be visible, regardless of the map scale. Note: The viewports you see in the examples in these guidelines may be grey.

Name	Screenshot	Description
Pop-up Information Box		Clicking any individual result or pin causes a pop-up box to appear over the pin. The text within the box may additional information and should show only be used for research purposes to help understand the result: •Phone number (for research only, never call a business) •URL •Coordinates (latitude/longitude) Note: Do not rate the information in the pop-up box. Rate only the information in the result.
Map Layer Selector		Allows you to select which map layer is displayed.
Zoom Controller		Allows zooming in and out on the map. Change the zoom level by clicking the + (zoom in) or – (zoom out) buttons.
Copy to Clipboard		Click to copy information to a clipboard
Drop Location Pin .	.	Click the pin icon at the top right of the map to place a pin anywhere on the map. A purple pin and a pop-up window showing the pin’s address and latitude/ longitude will appear.

Name	Screenshot	Description
Measure Distance	.	To measure the distance, click the ruler icon at the top right of the map. You can now drop a start pin (green) and an end pin (red). The tool will display the distance between them.
Draw Tool		Click to draw custom polygons.
Submit		Click Submit to save your ratings and display the next available task.

.Result Type	Explanation	Example
. Business/POI	Businesses and POI type results will show a name in the top field, followed by an address and a category. The same rating rules will apply to both Business and POI results.	. .
Result Type	.Explanation	Example
. Address	..These results can be one of multiple address types, including residential address or locality. Instead of a business or POI name, this kind of result will show the first line of the address in the top field, followed by the entire address below. This type of result will not include a category. Note: If a result does not include a category, don’t assume the result type is an address. You must still do research to verify this, as not all POIs will display a category.
Features without an expected address	The results for POIs or features without an expected address, like bus stops, bridges, or mountains, may sometimes look like address type results. Check to see if the result includes a category. If it does, the result refers to a feature or POI, not an address. At first glance, the result on the right might look like it’s for an intersection. However, since it includes a category, you can see that it actually refers to a POI — in this case, a bus stop.
Features without an expected address	This result for a landmark square might be mistaken for a result referring to a street with the same name. Since it contains a category, in this case “Park,” you can see that it actually refers to a POI.	.

...Query and Screenshot	Result	.Rating and Explanation
. [starbucks] User and fresh viewport in San Francisco, CA	① Starbucks 865 Market Street	Relevance	Excellent
		Rate Excellent for Starbucks locations that are in close proximity to the user.
	② Starbucks 170 O’Farrell St	Relevance	Good (Distance/Prominence)
		Rate Good for locations that are a bit farther away from the closest relevant locations to the user.
	③ Starbucks 264 Kearny St	Relevance	Acceptable (Distance/ Prominence)
		Rate Acceptable for relevant locations that are even farther away from the user but still inside the viewport.
	④ Starbucks 580 California St	Relevance	Bad (Distance/Prominence)
		When there are many locations close to the user, rate Bad for locations that are significantly farther away and outside the viewport.
	⑤ Starbucks 140 Mason Street	Relevance	Excellent
		Rate Excellent for Starbucks locations that are in close proximity to the user, even if they are outside the viewport.

.Query and Screenshot	Result	....Rating and Explanation
[starbucks] User in Alameda with fresh viewport in San Francisco, CA .	① Starbucks 865 Market Street	Relevance	Excellent
		Rate any Starbucks within the fresh viewport Excellent when the user is outside the viewport.
	② Starbucks 170 O’Farrell St	Relevance	Excellent
		Rate any Starbucks within the fresh viewport Excellent when the user is outside the viewport.
	③ Starbucks 264 Kearny St	Relevance	Excellent
		Rate any Starbucks within the fresh viewport Excellent when the user is outside the viewport.
	④ Starbucks 580 California St	Relevance	Bad (Distance/Prominence)
		When there are many possible results within the viewport, including those not returned for rating (purple dots), rate Bad for results outside it.
	⑤ Starbucks 140 Mason Street	Relevance	Bad (Distance/Prominence)
		When there are many possible results within the viewport, including those not returned for rating (purple dots), rate Bad for results outside it.

Query and Screenshot	Result	Rating and Explanation
[zara] User and fresh viewport in North Miami Beach, FL .	① Zara 19501 Biscayne Blvd, Aventura	Relevance	Excellent
		Rate Excellent for the closest Zara location, even when outside the fresh viewport.
	② Zara 420 Lincoln Rd, Miami Beach	Relevance	Good (Distance/Prominence)
		When there aren’t any possible results in the fresh viewport, rate Good for locations that are a bit farther away from the user/viewport.
	③ Zara 590 Collins Ave, Miami Beach	Relevance	Good (Distance/Prominence)
		When there aren’t any possible results in the fresh viewport, rate Good for locations that are a bit farther away from the user/viewport.
	④ Zara 701 S Miami Ave, Miami	Relevance	Good (Distance/Prominence)
		When there aren’t any possible results in the fresh viewport, rate Good for locations that are a bit farther away from the user/viewport.
	⑤ Zara 7535 N Kendall Dr, Miami	Relevance	Acceptable (Distance/ Prominence)
		The Zara store is located significantly further away than the second closest stores and is therefore rated Acceptable.

......Query and Screenshot	Result	....Rating and Explanation
. [great wolf lodge] User in Savannah inside large, fresh viewport	① Great Wolf Lodge Atlanta	Relevance	Excellent
		This result is amongst the closest possible to the user and is inside the fresh viewport.
	② Great Wolf Lodge Charlotte	Relevance	Excellent
		This result is amongst the closest possible to the user and is inside the fresh viewport.
	③ Great Wolf Lodge Williamsburg	Relevance	Good (Distance/ Prominence)
		Though still inside the fresh viewport, this result is significantly farther away from the user location with closer options available.
	④ Great Wolf Lodge Pocono Mountains	Relevance	Bad (Distance/ Prominence)
		This result not only is very far away from the user, it is also outside the fresh viewport.
	⑤ Great Wolf Lodge Niagara Falls	Relevance	Acceptable (Distance/Prominence)
		While very far away from the user, this suggestion is still within the fresh viewport and therefore rated Acceptable.
	⑥ Great Wolf Lodge Traverse City	Relevance	Bad (Distance/ Prominence)
		This result is not only very far away from the user, it is also outside the fresh viewport.

Example	Explanation
EXAMPLE TYPE: SINGLE RESIDENTIAL PROPERTY	.
1556 Tobias Dr San Jose, CA 95118	..
	The property boundary is defined by half of the road on one side and physical dividers like brick, concrete, and wooden fences on the others.
EXAMPLE TYPE: SHOPPING CENTER	.
2086 NewPark Mall Newark, CA 94560	.
	The property boundary is defined by the roads surrounding the shared parking lot.

Example	Explanation
EXAMPLE TYPE: HALF ’N HALF TO STREET	.
. ..	Extend the feature boundaries to the middle of the road(s). The example shows how Will Rogers Dr (to the south) and Saratoga Ave (to the east) are divided in half to define the outer limits of the Approximate area.
EXAMPLE TYPE: HALF ’N HALF TO NEXT BUILDING	.
	The space between two buildings can be divided in half to create a boundary for an Approximate pin placement.

Example	Explanation
EXAMPLE TYPE: BOUNDARIES OF AN ENTIRE CAMPUS
Stanford Shopping Center	.
	This is the Stanford Shopping Center, a campus containing multiple buildings and POIs. When this shopping center is the result, the entire parcel is Perfect. This includes all buildings, parking lots, and parking structures on both the north and south sections of the mall on either of side Arboretum Rd, the public road that cuts through the parcel.

Example	Explanation
EXAMPLE TYPE: BOUNDARIES OF A POI IN A SHARED SPACE WITH THROUGH ROADS
..Nordstrom @ Stanford Shopping Center
	The Nordstrom department store is located in the south section of the shopping center’s parcel. When Nordstrom (shown in green) is the result, only the south part of the mall up to 1/2 of Arboretum Road is rated Approximate (shown in yellow). This is because Arboretum Road is a public road that can be used to access places outside the parcel. The north side of the parcel (shown in red) is rated Wrong since the public road acts as boundary to Nordstrom’s Approximate area.
Example	Explanation
EXAMPLE TYPE: BOUNDARIES OF A POI IN A SHAR	ED SPACE WITH NO THROUGH ROADS
Pho 2 Love 6593 W Colfax Ave Lakewood, CO 80214	.
	Pho 2 Love is a Vietnamese restaurant in a strip mall with shared parking that has no public or internal access roads passing through it. The entire shared parcel is rated Approximate (yellow area) and only the section of the rooftop over the restaurant is rated Perfect (green area). This is because the exact location of the business can be confirmed with street imagery or other strong evidence.

Example	Explanation
EXAMPLE TYPE: INDIVIDUAL HOME	.
5507 Dent Ave San Jose, CA 95118	.
	Perfect -> The pin is on the rooftop specified in the query address (green area) Approximate -> The pin is within the boundaries of the feature (yellow area) Next Door -> The pin is on the immediate property next to the intended feature (purple area) Wrong -> The pin falls outside of the property boundaries or Next Door property (everything else) Can’t Verify -> See Can’t Verify

Example	Explanation
EXAMPLE TYPE: MULTIPLE STREET NUMBERS UNDER ONE ROOFTOP WITHOUT SHARED PARKING
Breitensteinstrasse 77, 8037 Zurich	.
.Available resources can have a big impact on rating when a pin is dropped on a rooftop with multiple street numbers under it. This result is a residential townhouse without shared parking Note that the same rating principles apply to buildings with patchwork rooftops.
With evidence	Using street imagery or other strong evidence, four entrances for the street numbers can be identified: 75, 77, 79 and 81. Once the rooftop area where the pin should be dropped is determined, draw imaginary lines to delimitate that area. A pin dropped within that area will be rated Perfect. A pin dropped within the property boundaries of the perfect rooftop will be rated Approximate. The rest of the connected rooftop with the appropriate property boundaries will be rated Next Door (purple area) or Wrong (red area) if the corresponding rooftop portion is farther than the Next Door property. Perfect -> The pin is on correct portion of the rooftop (green area) Approximate -> The pin is within the boundaries of the feature (yellow area) Next Door -> The pin is on the immediate property next to the intended feature (purple area) Wrong -> The pin falls outside of the property boundaries or Next Door property. (everything else) Can’t Verify -> See Can’t Verify

Example	Explanation
Without evidence	Without street imagery or other strong evidence, research may indicate that the address range under the rooftop is 75 to 81, but the locations where the middle numbers 77 and 79 start and end cannot be determined. In this case, a pin falling anywhere on the correct rooftop will be rated as Perfect, even if the precise location on the rooftop for #77 cannot be determined. A pin falling within the property boundaries will be considered Approximate and the buildings to the right and to the left will be considered Next Door. Perfect -> The pin is on the rooftop (green area) Approximate -> The pin is within the boundaries of the feature (yellow area) Next Door -> The pin is on the immediate property next to the intended feature (purple area) Wrong -> The pin falls outside of the property boundaries or Next Door property (everything else) Can’t Verify -> See Can’t Verify
Example	Explanation
EXAMPLE TYPE: ON BUILDING WITH MULTIPLE STREET NUMBERS IN A SHARED PARKING LOT
Supercuts 1054 E El Camino Real Sunnyvale, CA 94087	.
The result is a business (or an address) under a connected rooftop with multiple street numbers under it. This result shares a parking lot with ot her features. Note: If there are any other buildings within the shared parking lot that are not connected to the Perfect location, they will also be rated Approximate. gs with shared rooftops, like townhouses. This guidance also applies to residential buildin
With evidence	With street imagery or other strong evidence, the exact location of the business (or address) can be confirmed. Draw imaginary lines on that part of the shared rooftop to create the area for Perfect. The rest of the connected rooftop and the shared parking lot are rated Approximate. Remember: Make no Next Door ratings outside the parcel or shared space. Any pin falling outside of the Approximate area in a shared space will be rated Wrong.
Without evidence	Without street imagery or other strong evidence, only the location of the entire rooftop under which the business (or address) is located can be determined. The business’ exact location under the rooftop cannot be determined. In this situation, the whole rooftop becomes Perfect. The shared parking lot will be Approximate.

Example	Explanation
EXAMPLE TYPE: ONE BUILDING WITH A SINGLE STREET NUMBER AND MULTIPLE UNIT NUMBERS IN ASHARED PARKING LOT
Subway 1923 59th Ave #155 Greeley, CO 80634	.
The result is a business (or an address) under a unit numbers, and shares a parking lot with other features. It will be rated exactly the same way as a result whose address does not include a unit number. Whether a result contains a unit number or not, the goal is to use the best available evidence to find the reult’s location Note: If there are any other buildings within the Perfect location, they will also be rated Approximate. This guidance also applies to residential buldings with shared rooftops, like townhouses.
With evidence	With street imagery or other strong evidence, the exact location of the business (or address) can be confirmed. Draw imaginary lines on that part of the shared rooftop to create the area for Perfect. The rest of the connected rooftop and the shared parking lot are rated Approximate.
Without evidence	Without street imagery or other strong evidence, only the location of the entire rooftop under which the business (or address) is located can be determined. The business’ exact location under the rooftop cannot be determined. In this situation, the whole rooftop becomes Perfect. The shared parking lot will be Approximate.

Example	Explanation
Without evidence for rooftop or unit locations, but with evidence for parcel	When there is no strong evidence for the location of the rooftop or the specific unit under it, but there is evidence for the location of the parcel, all rooftops within the parcel will be rated Can’t Verify and the parcel will be rated Approximate.
EXAMPLE TYPE: ONE BUILDING WITH SINGLE STREET NUMBER AND MULTIPLE BUSINESSES UNDER THE ROOFTOP
Circle K 925 Wonderland Rd South London, Ontario, Canada N6K 2V8	.
The result is a business under a single rooftop that has one street number and contains multiple business. A nail salon, a pharmacy, a dental office, and other businesses all share this address.
With evidence	With street imagery or other strong evidence, the exact location of the result under the shared rooftop can be confirmed. Draw imaginary lines on that part of the shared rooftop to create the area for Perfect. The rest of the connected rooftop and parcel, including all other rooftops that are not the intended one on the parcel, if they exist, are rated Approximate.
Example	Explanation
Without evidence	Without street imagery or other strong evidence, only the location of the entire rooftop under which the business is located can be determined. The business’ exact location under the rooftop cannot be determined. In this situation, the whole rooftop becomes Perfect. The parcel will be Approximate.

Example	Explanation
EXAMPLE TYPE: SHOPPING MALL WITH A SINGLE ROOFTOP
Westfield Valley Fair 2855 Stevens Creek Blvd Santa Clara, CA 95050
This large shopping mall contains many stores under one rooftop at the same address.
	Perfect -> The pin is on the rooftop (green area) Approximate -> Parking structures (entire structure included). Approximate boundaries are found when Half ‘n Half rule is applied to the street and parking lots are included (yellow area)

Example	Explanation
EXAMPLE TYPE: STORE IN A MULTI–STORY MALL .
H&M 1004 Stoneridge Mall Rd Suite A 105 Pleasanton, CA 94588
With evidence	The exact location of this POI cannot be determined by using street imagery, but the official mall directory provides strong evidence for the H&M’s location. Using this evidence, the exact location of the result under the shared rooftop can be confirmed. Draw imaginary lines on that part of the shared rooftop to create the area for Perfect. The rest of the connected rooftop and parcel are rated Approximate.

Example	Explanation
EXAMPLE TYPE: EMPTY PLOT	.
86 Liberty St. Binghamton, NY	.
.All features that do not have a rooftop will be rated as Perfect if the pin is dropped within the boundaries of the feature.
	Perfect -> The pin is within the boundaries of the feature (green area) Approximate -> There is no Approximate Next Door -> The pin is on the immediate property next to the intended feature (purple area) Wrong -> The pin falls outside of the property boundaries or Next Door property (everything else) Can’t Verify -> See Can’t Verify
EXAMPLE TYPE: HIGH–DENSITY URBAN AREA
57 Neal Street London, England
..In this example, the rooftop behind the intended feature will not be considered Next Door. Next Door will be considered to the left or right of the intended feature. The screenshot shows that the boundaries of the Next Door property contain half of the street. The building behind the Perfect building is not considered Next Door because it has a different street address.

Example	Explanation
EXAMPLE TYPE: RURAL AREA	.
100 Mulligan Rd Laytonville CA	.
.In rural areas, the Next Door property will often be quite large. This is expected.
	Perfect -> The pin is on the rooftop specified in the address (green area) Approximate -> The pin is within the boundaries of the feature (yellow area) Next Door -> The pin is on the immediate property next to the intended feature (purple area) Wrong -> The pin falls outside of the property boundaries or Next Door property (everything else) Can’t Verify -> See Can’t Verify
EXAMPLE TYPE: PIN ON THE SAME BLOCK	.
1491 Kooser Rd San Jose, CA 95118	.
This property has only one next door neighbor because the property to the left, in red, is not on the same block as the intended property.
	Perfect -> The pin is on the rooftop specified in the address (green area) Approximate -> The pin is within the boundaries of the feature (yellow area) Next Door -> The pin is on the immediate property next to the intended feature (purple area) Wrong -> The pin falls outside of the property boundaries or Next Door property Can’t Verify -> See Can’t Verify

Example	Explanation
EXAMPLE TYPE: INDIVIDUAL HOME WITH SEPARATE GARAGE
Navarro Dr Sunnyvale CA 94087	.
This property contains two buildings: the main house and a garage. Both buildings are on the same
parcel, belong to the same entity, and share an address.
	Perfect -> The pin is on the rooftop of the house (green area) Approximate -> The pin falls on the garage or within the property boundaries (yellow area) Next Door -> The pin falls on the next door property (purple area) Wrong -> The pin is outside the boundaries (everything else) Can’t Verify -> See Can’t Verify
.	.

Example	Explanation
EXAMPLE TYPE: BUSINESS WITH MULTIPLE ROOFTOPS AT SAME ADDRESS (GAS STATION)
Super Gaz 700 Chambly Road Longueuil, QC J4H 3M1 Canada	.
This business (or address) is a gas station with a main building and a canopy over the pumps.
All structures are on the same property, belong to the same business, and share an address.
Figure 1: Satellite view of a gas station Figure 2: Annotated satellite view of gas station	Perfect (green area in Figure 2) -> If the pin falls on any rooftop. Approximate (yellow area in Figure 2) -> If the pin falls within the property boundaries. Next Door (purple area in Figure 2) -> If the pin falls on the immediate property next to the intended one.

Example	Explanation
Figure 1: Satellite view of multiple-rooftop POI Figure 2: Annotated satellite view of multiple-rooftop POI	Perfect (green area in Figure 2) -> If the pin falls on any of the buildings. Approximate (yellow area in Figure 2) -> If the pin falls within the property boundaries. This particular POI has no Next Door area.
EXAMPLE TYPE: MULTIPLE BUILDINGS ON ONE PARCEL SHARING ONE ADDRESS (APARTMENT COMPLEX)
Merkuriuksentie 13 00750 Helsinki Finland
This query is for an address that is shared by several apartment buildings on the same parcel.

Example	Explanation
. ....	.There are five buildings with the same address on the parcel. A pin on any of them is rated Perfect while a pin on the parcel or any of the support structures is rated Approximate. ....
EXAMPLE TYPE: BUSINESS WITH A SHED IN	THE BACK
33 Mile Roadhouse 33 Mile Haines Hwy Haines AK 99827 United States	.
This business is a restaurant with a suppor	t building — a storage shed — in the back.
Annotated satellite view shows restaurant and shed (arrow points to street imagery of shed).	This restaurant has a detached storage shed on its property. Perfect (green area in top image) -> If the pin falls on the main restaurant building. Approximate (yellow area in top image) -> If the pin falls within the property boundaries, including the small support shed behind the restaurant. Next Door (purple areas in top image) -> If the pin drops on either of the immediate properties next to the intended one.

POI type	Screenshot	.
Hospital		.
Park		.
Airport		.
Outlet Mall		.

POI	Screenshot	Rating Explanations
El Camino Hospital	. Fig. 1: Vector view of El Camino Hospital Campus Fig. 2: Annotated vector view of El Camino Hospital campus	Perfect (green area in Fig. 2) -> If the pin falls within the hospital boundaries. Wrong (everything else outside of green area in Fig. 2)-> If the pin falls outside of hospital boundaries, as determined by your research.
POI	Screenshot	Rating Explanations
Disneyland Park	. . Fig. 1: Vector view of Disneyland Park Fig. 2: Annotated satellite view	There is no polygon available for this POI, so an imaginary polygon outline must be created according to the actual boundaries of the POI. To determine where the valid pin area is, look at the location of the attractions within the amusement park and visualize the boundary of the park to determine the POI’s polygon. The green area highlighted in a satellite view of the POI (Fig. 2) represents the valid area for the pin to land in. A pin landing anywhere outside of this green area would be a Wrong pin. Note: Parking lots separated by a road should not be included in the polygon because they outside the boundary. Utility buildings located outside of the public park premises are not included in the polygon.

Rating	Explanation
Perfect .	Pin can fall anywhere on the street, including medians and intersections the street passes through completely (but not on sidewalks).
Approximate	N/A.
Next Door	N/A
Wrong	Pin falls in an area that does not meet the criteria for Perfect.
Can’t Verify	See Can’t Verify
Example		Explanation
		Perfect – The pin can fall anywhere on the street, including medians and intersections the street passes through completely, whether seen in satellite view or on the vector map. If there is a discrepancy between the satellite view and vector map of the street boundaries, use the most favorable layer. Note that sidewalks are not part of the Perfect area. Approximate – not applicable Next Door – not applicable Wrong – everything outside of Perfect, including intersections the street does not pass through Here, only the Jersey St. & Lafayette St. intersection is a valid intersection for the pin to fall on, since Jersey St. completely passes through
Jersey St. in New York City		Lafayette St. The intersections where Jersey St. meets Mulberry St. and Crosby St. are not included because Jersey St. does not pass through these streets.

Query	Results	Ratings & Explanation
[stevens creek blvd cupertino]	Is there a Navigational Result for this Query?		Yes
	Stevens Creek Blvd Stevens Creek Blvd Cupertino, CA Category: n/a	Relevance	Navigational
		Name Accuracy	n/a
		Address Accuracy	Correct
		Pin Accuracy	Perfect
		The query is for a street in Cupertino, CA. The postal code is optional for such results, but if the postal code is returned, any portion of the street has to belong to that postal code area. The pin appears to fall outside the street boundaries in the vector view. However, in satellite it is still within the boundaries. Whenever there is a discrepancy the most favorable layer should be used to determine pin accuracy.

Results	Ratings and Explanation
① 154 Orchard St 154 Orchard St, New York, NY 10002 Category: n/a	Relevance	Navigational
	Name Accuracy
	Address Accuracy
	Pin Accuracy
	The result is in Lower Manhattan, very close to the user location.
② 154 Orchard St 154 Orchard St, Newark, NJ 07102 Category: n/a	Relevance	Good (Distance/Prominence)
	Name Accuracy
	Address Accuracy
	Pin Accuracy
	The result is farther away from the viewport, but still matches the query.

Results	Ratings and Explanation
③ 154 Orchard St 154 Orchard St, Garfield, NJ 07026 Category: n/a	Relevance	Good (Distance/Prominence)
	Name Accuracy	n/a
	Address Accuracy	Correct
	Pin Accuracy	Perfect
	The result is farther away from the viewport, but still matches the query.
④ 154 Orchard Dr 154 Orchard Dr, Levittown, Pennsylvania 19054 Category: n/a	Relevance	Bad (Distance/Prominence)
	Name Accuracy	n/a
	Address Accuracy	Correct
	Pin Accuracy	Wrong
	The result is in a different state and very far away from the user’s viewport. The street type does also not match the query. The pin lands on the house with the street number 114, which can be verified via street imagery. Since this is several houses down from the intended address, it is incorrect.

Results	Ratings and Explanation
④ 154 Orchard Dr 154 Orchard Dr, Levittown, Pennsylvania 19054 Category: n/a	Relevance	Bad (User Intent)
	Name Accuracy	n/a
	Address Accuracy	Correct
	Pin Accuracy	Wrong .
	.This location is farther from the viewport/user location than the result in New York and therefore demoted for distance. The user is also looking for a Street, but has been presented with a Drive. The pin lands on the house with the street number 114 which can be verified via street imagery. Since this is several houses down from the intended address, it is Wrong.

Query	.Results	Ratings and Explanation
[Machu Picchu] Fresh viewport over Massachusetts, some distance away from Somerville, MA	Is there a Navigational Result for this Query?		Yes
	Machu Picchu Aguas Calientes, Peru. ....	Relevance	Navigational
		Name Accuracy	Correct
		Address Accuracy	Correct
		Pin Accuracy	Perfect
		The result is an internationally known historical site and the most relevant result, making it the primary user intent.
	Machu Picchu Restaurant 307 Somerville Ave, Somerville, MA 0143	Relevance	Good (User Intent)
		Name Accuracy	Correct
		Address Accuracy	Correct
		Pin Accuracy	Perfect
		Since the result is reasonably close to the fresh user viewport, it will be considered secondary intent for this query.

Query	Results	.Ratings and Explanation
[new york] User and viewport somewhere in France	Is there a Navigational Result for this Query?		Yes
	New York City New York City, NY, US Category: n/a	Relevance	Navigational
		Name Accuracy	n/a
		Address Accuracy	Correct
		Pin Accuracy	Perfect
		This is a very prominent result that satisfies the user intent.
	New York State New York, US Category: n/a	Relevance	Good (User Intent)
		Name Accuracy	n/a
		Address Accuracy	Correct
		Pin Accuracy	Perfect
		This result refers to the state of New York and satisfies a secondary intent of the query.
	Avenue de New York Avenue de New York, Paris, France Category: n/a	Relevance	Good (User Intent)
		Name Accuracy	n/a
		Address Accuracy	Correct
		Pin Accuracy	Perfect