Autocomplete and Search tasks share many characteristics, but it is important to know the differences between the two task types when rating them. In this learning material, we will go through the differences and some typical rating scenarios in the Finnish market.
Here is a list of the main differences between Autocomplete and Search tasks:
Please review the Autocomplete Evaluation Guidelines section 2.2 Suggestion Types for a complete explanation.
The Search results, on the other hand, are divided into three types:
See the Maps Search Evaluation Guidelines section 2.2 Result Types for a complete explanation.
Please study the Autocomplete Evaluation Guidelines thoroughly to understand all aspects of the Autocomplete task https://geoworkerz.telusinternational.ai/mapping/StartWorking/Docume nts/AC%20Guidelines.pdf
Common Rating Scenarios Finland
Example 1. Query suggestion – name is missing a required sign/symbol
Query: smarke
User location: 62.61513622308853, 24.639234195741594
Fresh viewport: 63.42364257735526, 25.012073799163293
Query suggestion: smarket
Correct Relevance rating: Excellent.
Correct Name rating: Partially Correct.
Query suggestions are rated for Relevance and Name Accuracy. This query suggestion is a very likely match to the query but the business name is missing the character “-“. According to the official business web site, correct name would be s-market or S-market. https://tinyurl.com/2bp2qszc The accuracy of query and category suggestions is rated using the guidelines for business/POI name accuracy. One difference is that if a query suggestion appears in all lower case, this is not a reason for demotion. See Guidelines sections 5.2. Suggestion Name and 5.7. Query and Category Suggestions
Example 2. Suggestion in Unexpected Language/Script
Query: Gr
User location: 61.505697686875855, 23.69699248025236
Fresh viewport: 61.49828329163012, 23.732353158524248
Category suggestion: Groceries
Correct rating: Result name/title is in unexpected language or script – TRUE.
In this case the query string is only two letters but a very likely match does exist: Grilli or grillikioski. The category suggestion ”Groceries” is in English and it is not an expected language in the Finnish locale.
A suggestion’s name or title is in an expected language or script when it is:
Example 3. Relevance: Good – Distance
Query: ab
User location: 62.737315427576654, 28.374890656140686
Fresh viewport: 62.71766780854674, 29.011196522670737
Suggestion 1:
ABC
Name: ABC
Address: Polvijärventie 1, Polvijärvi
Red dot = user location. Red square = Fresh viewport. Blue dot = Suggestion 1.
Correct Relevance rating: Good – Distance. User location is outside fresh viewport, so fresh viewport is the location intent. The most likely user intent is the gas station ABC so we need to research where the closest branches to the viewport are located. On the official web page https://www.abcasemat.fi/asemat we can find a map tool that shows all the branches.
We can see that there is one branch located inside the viewport and our result is the second closest real world result. Therefore, it is demoted once for distance. See Guidelines sections
8.3. Chain Business Query and 8.4. Category Query.
Example 4. Relevance: Bad – User Intent
Query: Kajaani
User location: 64.20306371855598, 27.339753854495417
Stale viewport: 63.94284750015068, 26.759347529416157
Suggestion 1:
Prisma Kajaani
Name: Prisma Kajaani
Category: Ruokakauppa
Address: Veturitie 1, Kajaani
Correct Relevance rating: Bad – User Intent. The location intent is the user location as the viewport is stale. There are many matching real world results for the query ”Kajaani”. We can see all existing address matches on https://asiointi.maanmittauslaitos.fi/karttapaikka/ The most likely user intent is the city of Kajaani located very close to the user. There are some secondary matches like ”Kajaanintie” nearby as well.
Our result is Prisma Kajaani, a prominent supermarket in the locality. https://www.prisma.fi/myymalat/590772240. However, the query matches only with the location modifier part of the suggestion. The use of “Kajaani” in the business name is extra information and not a part of the official branding for this business chain. This is not considered a match according to the Guidelines section 4.1.1. General Match.
Example 5. Address: Incorrect – Street Number
Query: Scandic Vaas
User location: 63.38076365784365, 26.052384321026295
Stale viewport: 64.24901787924853, 25.275191613640356
Suggestion 1:
Scandic Vaasa
Name: Scandic Vaasa
Category: Hotelli
Address: Rosteninkatu 3, Vaasa
Correct Relevance rating: Excellent
Correct Name rating: Correct
Correct Address rating: Incorrect – Street Number.
The suggestion is the most likely user intent but there is an error in the address details. We can confirm on the official web page that the street number should be 6. https://www.scandichotels.fi/hotellit/suomi/vaasa/scandic-vaasa
