Goals of Autocomplete task:

.

Determine relevance by evaluating how well each suggestion satisfies the user’s potential intent.
Check the data accuracy (name and address) of the suggestion.

  General Reminders:

Relevance is established based on how well the result matching the query string, how well the result satisfies the user intent, and distance/prominence of the result.
It is necessary to identify the significant part of the query string and factor in all possible intentions as the user may not have finished typing.
Always rate each suggestion individually.
Rate against the real world.
Rate relevance without considering the accuracy of name and address.
When judging distance/prominence disregard closed and non-existent suggestions.
For Query and Category suggestions, research immediate area for suggestions to help determine if this is a valid suggestion based on the query and location intent.
The mandatory address components that you use to verify addresses in Search tasks can be different to the mandatory address components for Autocomplete tasks. Check the Country Specific Guidelines of your market before rating.
In Search tasks, we presume to know the user intent and return only the results that would satisfy the user intent. However, in Autocomplete task, you need to consider the possibility of the query being incomplete and provide a more broader search base for the user. The relevance rating of Autocomplete tasks could be more lenient than the relevance rating of Search tasks.

Some Common Mistakes in Autocomplete Tasks

1.The result can still match user intent even though it does not match the start of the query string.

Relevance: Business/POI Type Results:

Query

Result

Rating and Explanation

台北市松山區 健康服務中心

Excellent

It is common to see users drop certain parts of a POI’s name when performing a search. This can include non-essential terms that indicate things such as the POI’s location or operating nature. Even though the string of the result starts with “”, we will consider this a match because “台北市松山區” is a location modifier and the query “” matches the start of “健康服務中心”.

國立東華大學

Excellent

Even though the string of the result starts with “”, we will consider this a match because “國立” indicates their operating nature and the query “” matches the start of “東華大學

Relevance: Address Type Results:

Query

Result

Rating and Explanation

市府路

台北市信義區 市府路

Excellent

It is common to see users drop certain address components when performing a search. Even though the string of the result starts with “”, we will consider this a match because we managed to return the correct street queried for, “市府路”.

2.The query string may be misspelled or contains parts that may have been entered incorrectly.

If the result listing matches the query string or matches by auto correct, then the starting rate of Excellent to Acceptable may be applied. If the result listing does not match the query string or spell correction is not considerable, then this is a No Match that would lead to a rating of Bad.

Query

Result

Explanation

ㄈㄥ

風神廟

” is spelt as “ㄈㄥ”, so it can be considered as a match

屈臣ㄕ

屈臣氏

It is likely an incompletely query string and user is likely looking for the chain business 屈臣氏 (is spelt as “ˋ”). The result is a good match considering the given string.

23 Clair blv

23 Clair St

When there is no “Clair Blvd” within reasonable distance, we consider “Clair Street” to be a spell correction and a match to the incorrect user query.

covfefe

Coffee

This query string has no obvious spell corrections that would clearly satisfy the user intent. The returned spell- corrected suggestion should not be considered a match since user intent is obscure.

  3.Suggestions with high prominence may still be relevant even when far away from the user. .

Suggestions with lower prominence, on the other hand, may not be as relevant even if they are close to the user.

Query

Result

Explanation

Sydney

Sydney Opera House

The Opera House has overwhelming international prominence and should be considered a primary intent suggestion for this query string.

Sydney Town Hall

This suggestion directly matches the query string but is not as prominent as internationally known Opera House.

101

台北101

台北101 has overwhelming international prominence and should be considered a primary intent suggestion for this query string.

1010

This suggestion matches the query string but is not as prominent as internationally known 台北101.

Once you’ve decided how relevant the suggestion is based on user intent, consider the suggestion in the context of all possible suggestions in the real world. Each suggestion is given a rating based on prominence and distance to the user or viewport.

Distance Prominence

Close

Medium*

Far“

High

Excellent

Good

Acceptable

Medium

Excellent to Good

Good to Acceptable

Acceptable to Bad

Low

Excellent to Acceptable

Good to Bad

Bad

Medium distance away compared to similar prominent suggesticrn

** Far distance away compared to similar prominent suggestion

4.Addresses in Taiwan are written from largest to smallest components, making it difficult to guess user intent when the string is incomplete. Please consider the following on top of the usual rules when rating address type results:
                   Width/length of the street
Nature (commercial/residential/arterial) of the street
Location of the street (population density, amount of traffic)

Query

Result

Rating and Explanation

木柵路三

木柵路三段

Excellent

A commercial road with 2-way traffic in the area.

木柵路三段102

Good – Distance / Prominence issue

A relatively longer residential road in the area.

木柵路三段1026

Acceptable – Distance / Prominence issue

Smaller residential road which should be demoted as less prominent.

木柵路二段

Bad – User Intent issue

The result does not match the query string.

 

5.When a query address and an address suggestion are not exactly the same, the kind of connection they have depends on their relationship:

Street number is the same in both query and suggestion but the unit number is different or missing:

If both the query and suggestion addresses exist in the real world, and neither address is a street extension, rate suggestion relevance Good when:

The query contains a unit number and the suggestion does not.
The suggestion contains a unit number and the query does not.
The query contains one unit number and the suggestion contains another.

The query is a full address including street name and street number, but the suggestion is the street name only:

Since this suggestion is an unlikely secondary intent, rate relevance as Acceptable.

Query is for a street [Main Street, Pleasanton, CA] and the suggestion is just the locality (Pleasanton, CA).

Rate the suggestion relevance Bad as it does not satisfy the user intent

.

 

end

 

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Post