a. Research the “user’s query” with location intent- Utilize search engines and map applications. These tools will allow you to research possible suggestions, businesses, or POIs that are near the location intent.
Common Mistake 1: Demoting Addresses, when there are no real- world closer results.
Guideline Section: 8.1.1 Address Does not Exist
Query: [2373 chestnut] user located within Fresh Viewport
Correct Relevance Rating: Excellent
Explanation: Query [2373 chestnut] The first step in Autocomplete evaluation tasks, is to determine if the result matches the suggestion. The following suggestion of “2373 Chestnut Street, San Francisco, CA” matches the query of [2373 Chestnut], in addition it is located near the user’s location in the fresh viewport. At first glance, we may be inclined to demote based on distance, as it appears that there are closer results. In this case, all other pins point to addresses that do not exist in the real world. Per section 8.1.1, “When assessing the distance for valid, existing address suggestions, they should be evaluated only in relation to other existing addressing, including any potential unreturned valid suggestion. Relevance is EXCELLENT.
Common Mistake 2: Utilizing Business/Closed for Addresses that donot exist.
Guideline Section: 6.2.1 Address Does Not Exist
Example: Per the official address verification tool, the given address does not exist. When checking addresses always use the official tool and web research before rating. The address accuracy is INCORRECT- ADDRESS DOES NOT EXIST. Many analysts select Business Closed/Does Not Exist, this should only be used for POI/Business Type Suggestions.
Please rate the below address
Common Mistake 3: Not Matching the Query String
Guideline Section: 4.1. Matching the Query String
Correct Relevance: Bad
Explanation: Per section 4.1. Matching the Query String, when the suggestion does not match the query string and would be considered useless to the user, it is to be rated Bad. In this case the query is for ‘i.c.’ The resulting suggestion does not match the string and should be rated Bad.
Common Mistake 4: Omitting all possible suggestions for Query String
Guideline Section: Section 2.1 User query string.
Example: [wa] in this general query we want to research the location intent, to ensure that they are valid real-world suggestions. For example, if you find there exists Wa Municipal Hospital, Wa Airport, Wa New Market, and Wa Technical Institute within a short distance of the location intent, these all would be valid due to matching the general query [wa]. Please keep in mind, when approaching autocomplete tasks, we are asked how well the suggestion matches the query terms. Please note, there are times, we may need to consider spelling mistakes, please see below.
Common Mistake 5: Before rating the Relevancy of a result, two high level questions must be considered:
Explanation: The Autocomplete guidelines, section 1.2.1. Suggestion-level Rating Checkboxes, state that the checkbox should be checked, if ‘Suggestion is not in the language of the query, the test locale, or the suggestion region (the expected languages) or is in a mix of expected and unexpected languages (unless it is an official company, chain, or brand name).’ This also includes the language of Query Suggestion results as covered by the Autocomplete guidelines, section 3.1.1. Query Suggestion Language.