Rating for Relevance
Relevance ratings can be given starting from Navigational (if applicable) to Bad.
Prominence issue. And please don’t forget that leaving a comment is mandatory.
Steps for Rating Relevance
2.3.2. Implicit Location
If the user’s location intent is not expressly stated in the query, for example [chinese] or [479 margarita ave], use the user location, viewport location, and viewport age to determine the area of expected results.
Viewport | User | Location Intent |
Fresh | Inside Viewport | When the user is within a fresh viewport, take the user location as location intent. Results are generally expected in or near the viewport, and results inside the area cannot be rated Bad because of distanee alone. |
Outside Viewport | Results are expected in or near the viewport area. All relevant results inside the viewport are eligible for; a rating of Excellent. If no results can be found in or near the viewport, consider the user location a secondary location intent. | |
Missing | When the user is missing, the viewport is the location intent. | |
Stale | Inside Viewport | When the viewport is stale, consider only the user location as location intent. |
Outside Viewport | ||
Missing | Use the stale viewport as location intent wtlen the user location is missing. | |
Viewport Age Missing | Present or Missing | Consider the viewport fresh when tfle viewport age is missing. |
Missing entirely | Present | The user location sets the location intent when the viewport is not present. |
Missing | When the user location and viewport are missing, the test locale becomes the location intent with a strong focus on prominent results. |
5.1.8. Lack of Connection
Previously the guidelines contained the concept of ‘Associated Entities’. The concept of Associated Entities has been removed and no longer applies. This means that a result cannot satisfy user intent only because of its containment relationship with the POI/business in the query. Therefore, according to the guideline section 5.1.8. Lack of Connection, If the result does not satisfy the user intent either because there is no relationship between the query and the result or because the connection between the query and the result will not be immediately obvious to the user, rate the relevance Bad. The relevancy rating for this result is therefore Bad.
User Query | Result | Type of Connection | Description |
[airport] | Santa Cruz Boardwalk Santa Cruz | None | Does not satisfy user intent: Research shows th at there is no connection between the query and the re:suit. Rate Bad. |
[Raging Waters 2333 South White Rd San Jose] | 2333 South White Rd San Jose | None | Does not satisfy userintent: Result is for the correct address, 1:JUtdoe; not include the business named in the query, sc) users will have no way of know ing if this is truly connected to the business or not. Rate Bad. |
[Raging Waters] | 2333 South White Rd San Jose | None | Does not satisfy user intent: Query is the name of a business and result is for the correct address, but does not include the business name. Users will have no way of knowing whether or not this address is connected to the business they asked for. Rate Bad. |
[valley fair mall] | Macy’s 2801 Stevens Creek Blvd, Santa Clara, | None | Does not satisfy user intent: Result is for a store, not a mall. This store is insidethe requested mal1, but users Willi nave no way of knowing this simply frc>m looking at the result. Rate Bad |
[macy’s] | Westfield Valley Fair 2855 Stevens Creek Blvd, Santa Clara, | None | Does n<Jt satisfy user intent: Result is for a mall, not a store. This mall contains the requested store, but users will have no way of knowing this. Rate Bad. |
Example 1 –
To verify the Pin for this task, firstly we can see in TryRating that the pin appears to fall on a rooftop on Hanbury Street. Copying the pin co-ordinates to a map application and using street imagery, we can see that the pin falls on ‘Ace & Tate’ opticians on Hanbury Street which is clearly incorrect.
Viewport Center: 51.518544, -0.077651
User Location: 51.518326, -0.077591
Location and Real-World Results
Checking the results address on Brick Lane, we can see the actual location of Brick Lane Vintage Market is on a nearby rooftop at 51.520739, -0.071972. So the Pin accuracy should be rated Wrong as per MS GLs 9.1.3, as the pin falls outside the boundaries of the correct location. Again, street imagery can be used to verify the exact location.
Example 2 –
Viewport Center: 51.736062, -0.465585 User Location: 51.731881, -0.448390
For this task, the query is a chain business query but has no location modifier, so we use implicit location to determine location intent. As the user is inside the fresh viewport, we use the user’s location as location intent as per MS GLs 2.3.2.
Copying the user co-ordinates to a map application and searching for the query ‘Currys PC World’ shows that this result (red arrow) is a matching chain business (the chain has recently been renamed Currys) but is further away than multiple real-world results.
Location and Real-World Results
We can also use the store finder on the official website https://www.currys.co.uk/ and enter either the nearest postal code or locality to the user location – in this case Hemel Hempstead. We can see that our result in Stevenage is the 5th nearest and clearly further away than several other options, so Relevance should be demoted -2 to Acceptable.
Example 3 –
Viewport Center: 50.816969, -0.359221 User Location: 50.814514, -0.357056
To verify the Pin for this task, we can see in TryRating that the pin falls on a rooftop on Harlech Close and should be rated Wrong. For street results the pin should fall on the street itself and not on a building or rooftop, as per MS GLs 9.4.3
Example 4 –
Viewport Center: 50.826002, -0.164136
User Location: 50.825500, -0.159302
For this task, the user is located inside the fresh viewport, so results are expected near the user location. Copying the user co- ordinates to a map application and searching for the query ‘Pub’ shows that this result is one of the nearest to user location.
Location and Real-World Research
Now we know that this result is located nearby but we also need to check if it satisfies the category query. As per MS GL 10.7, category queries are the broadest query type and return the greatest variety of results, which may satisfy primary, secondary or unlikely intent. The official Facebook page for the business shows that this business is cleary a pub and would satisfy primary intent for this category query, and so Relevance should be rated Excellent.
Example 5 –
Viewport Center: 52.411505, -0.163494
User Location: 52.536621, -0.241699
The query is for a specific address (a Postal Code), so the user location and viewport are irrelevant and can be ignored as per MS GLs 10.1. We can use the Royal Mail address finder to check that the Postal Code is valid and exists, as shown below. ,
As the result is an exact match for the query and refers to a unique location, Relevance should be rated Navigational.
Example 6 –
Viewport Center: 51.479499, -0.198962 User Location: 51.479187, -0.203247
The query is a chain business query but has no location modifier, so we use implicit location to determine location intent. As the user is inside the fresh viewport, we use the user’s location as location intent as per MS GLs 2.3.2. Copying the user co-ordinates to a map application and searching for the query shows that this result (red arrow) is a matching chain business but is further away than some real-world options not returned in TryRating (blue arrows).
Location and Real-World Research
We can also check the official store locator https://stores.sainsburys.co.uk/ and enter the nearest postal code or street to the user location, which shows us that our result is the 9th nearest store. So, Relevance should be demoted to either Good or Acceptable as it’s relatively nearby (just over 1 mile) but there are closer results that better satisfy the query.
Example 7 – Incorrect Postal Code
Viewport Center: 51.346933, -2.265263
User Location: 51.350098, -2.263184
Again, the query is a chain business query but has no location modifier so we use implicit location to determine location intent. As the user is inside the fresh viewport, we use the user’s location as location intent as per MS GLs 2.3.2.
Location and Real-World Research
Copying the user co-ordinates to a map application and searching for the query ‘Santander’ shows that there are some results nearer to user location, including a real-world result nearer to user location (blue arrow). Always rate against the real-world options, and not just those returned in TryRating.
Example 8 –
Viewport Center: 56.108375, -3.268639 User Location: 55.964355, -3.317871
To rate the pin for this task, we look at where the pin falls in Tryrating, and view this location using street imagery. We find that the pin is on another business, Room & Rumours Coffee. Therefore, we need to search for the correct location in satellite and street imagery. In this case, we can easily see that the correct rooftop for the result Chop House, is two addresses away, and the pin should be at 55.951374, -3.184916. As per the MS GLs 9.2 the pin should be rated as Wrong, and we leave a comment explaining why the pin is not correct and where it should be.
Location and Real-World Research
Example 9 – 
Viewport Center: 53.071851, -1.112540
User Location: 52.995300, -1.170044
In this task, we can see that the query is for a street address. As this is a partial address, the user hasn’t specified the location intent in the query. As the user is not within the viewport, we can tell that the viewport is the intended location. We therefore need to research the query, to find locations where this address exists. Royal Mail Postcode Finder confirms that 5 Harlow Street is a non-unique address, and it exists in three locations.
The result we are rating matches the query exactly, and our research shows us that it is the only location close to the fresh viewport. As we can be sure that this result is the query intent, and the address is confirmed to exist in this location, it is rated Navigational, according to MS GLs 10.2.
Note – that in cases where a result address does not exist, it would not be eligible for a rating of Navigational.
Example 10 –
Viewport Center: 51.479499, -0.198962 User Location: 51.479187, -0.203247
To rate the data for this store, we use the chain business official URL https://tinyurl.com/4yrrn5jr to search for the address of the result. In this case, we find that the official URL shows that a branch doesn’t exist at this location. We can confirm this by looking at street imagery, which shows that the result location is a residential address.
As there is no evidence this POI exists, it is rated Business/POI is Closed or Does not Exist, as per MS GLs 4.2, and we leave a comment explaining our research.
Note – that we do still rate the Relevance as if the result does exist.
Location and Real-World Research
Example 11 – 
Viewport Center: 51.346933, -2.265263 User Location: 51.350098, -2.263184 In this example, we can see that the user has queried a chain business. As the user is in the fresh viewport, we know that the user location is the location intent. So, we have determined that the query is for a branch of the chain business near the user.
The chain business official URL has a branch finder https://tinyurl.com/2p9cum6t which shows the branches nearest the user location. There is a real world result possible closer to the user than the result we are rating. Even though this branch is not returned as a result in TryRating, MS GLs 10.6.2 explain that we must consider all possible results when rating relevance. Our result meets the chain brand intent of the query, but should be demoted to Good for distance, as there is another option closer to the user.
Location and Real-World Research
