AC Reminders:
- Goal of Task:
a. Determine how well each suggestion satisfies the user’s potential intent.
b. Check the data accuracy of the suggestion (name and address) * No pin evaluation is done in AC Task. - No pin evaluation
- Relevance: This is established on the following: matching suggestions, intent, and distance/prominence.
- Determining how well suggestion match the user’s potential intent:
a. Research the “user’s query” with location intent- Utilize search engines and map applications. These tools will allow you to research
possible suggestions, businesses, or POIs that are near the location intent. - Query and Category Suggestions: (Research steps same as above)
a. Per section 2.2.2 of the Guidelines, we do not evaluate address for these suggestions, the address accuracy will be prepopulated with N/A. In addition, no Pins will be shown.
b. Research immediate area for the suggestions- is this a valid suggestion based on the location intent?
Five AC Common Mistakes
Common Mistake 1: Demoting Addresses, when there are no real-world closer
results.
Guideline Section: 8.1.1 Address Does not Exists
Query: [2373 chestnut] user located within Fresh Viewport
Correct Relevance Rating: Excellent
Explanation: Query [2373 chestnut] The first step in AC evaluation, is to determine if the result matches the suggestion. The following suggestion of “2373 Chestnut Street, San Francisco, CA” matches the query of [2373 Chestnut], in addition it is located near the user’s location in the fresh viewport. At first glance, we may be inclined to demote based on distance, as it appears that there are closer results. In this, particular case all other pins point to addresses that do not exist in the real world. Per section 8.1.1, “When assessing the distance for valid, existing address suggestions, they should be evaluated only in relation to other existing addressing, including any potential unreturned valid suggestion. Relevance is EXCELLENT.
Common Mistake 2: Utilizing Business/Closed for Addresses that do not exist.
Guideline Section: 6.2.1 Address Does Not Exist
Example: Per the official postal service address verification tool, the given address does not exist. When checking addresses always use the official postal tool and web research before rating. The address accuracy is INCORRECT- ADDRESS DOES NOT EXIST. Many analysts select Business Closed/Does Not Exist, this should only be used for POI/Business Type Suggestions.
Common Mistake 3: Not Matching the Query String
Guideline Section: 4.1.Matching the Query String
Correct Relevance: Bad
Example: Per section 4.1.Matching the Query String, when the suggestion does not match the query string and would be considered useless to the user, it is to be rated Bad. In this case the query is for 41-11 “na”. The resulting suggestion does not match the final portion of the string and is to be rated Bad. In addition, there is no indication this may have been misspelled.
Common Mistake 4: Omitting all possible suggestions for Query String
Guideline Section: Section 2.1 User query string.
Example: [wa] in this general query we want to research the location intent, to ensure that they are valid real world suggestions. For example, if you find there exists there Walgreens, Walmart, Walmart Pharmacy, and Wawa within a short distance to the location intent, these all would be valid due to matching the general query [wa]. Please keep in mind, when approaching autocomplete tasks, we are asked how well the suggestion matches the query terms. Please note, there are times, we may need to take into account spelling mistakes, please see below.
Common Mistake 5: Evaluating a suggestion near location intent, and not taking into account user may have misspelled word.
Guideline Section: 4.1.5 Spell Correction
Correct Relevance: Good
Example: Per section 4.1.5. Spell Correction, when the user query contains a possible typo and the result address contains a string that is one-letter different and in the same vicinity as the key typed, i.e. L and O on the keyboard. The result has possible relevance. In this case, the address is valid, with a building associated with it. Since there is a direct match nearby, this spell correct result is considered secondary intent. Relevance for the result is GOOD.
