403 when accessing ExplainRank.aspx in SharePoint

October 8, 2013

One of the most recent projects I have been working on is a SharePoint 2013 public facing web site. The site has been live for a while but the client recently reported the search results were not returning in the order they expected.

I checked the search results page and it was sorting by rank so I decided I should invest some time looking at how the SharePoint ranking model works and what I can do to impact this. While there is a lot of information out there I found it difficult to grasp how all these factors were combining in my environment, especially when things like number of page clicks can influence the ranking.

After some further research I found a link by Waldek Mastykarz Understanding item ranking in SharePoint 2013 Search. This is an excellent article which documents this very useful feature, however when I tried to access this on my environment I got an error, see figure 1.


Figure 1

As with all SharePoint problems I jumped on the server and started up ULS Viewer and hit the page again. I have a quick look through the logs and the item displayed in figure 2


Figure 2

I had a check on the site and I was a site collection admin so next I opened central admin and navigated to the service application list. Once there I selected my search service application by clicking next to the name, see figure 3.


Figure 3

Once selected I then clicked on administrators in the ribbon, see figure 4


Figure 4

In the popup I added my account as an admin and when I refreshed my explainrank.aspx page it now worked. While its not difficult to work out what the error is on the page I thought I would quickly blog this for others and myself, oh and if anyone from Microsoft reads this you could use a better error message on the page 🙂

SharePoint 2013 Search Web Part Missing Results

April 26, 2013

While working on a few SharePoint 2013 projects I have been utilising the SharePoint search in a lot of places to provide dynamic content and there has been quite a few occassions when things don’t seem to work as expected such as CBS Server Side Date Error or Results web part ordering. I suppose this is to be expected when the search engine has been completely replaced with a mix of the old SharePoint search engine and FAST search.

When working on one aspect of a recent project I was using the results script web part to query some events stored in a list. As per you would expect these events had the typical details such as title, date, description, etc. At first everything seemed to be working as expected until the client reported there was a bunch of events missing from the search generated list. My first thought was these were new events and weren’t in the search index so I kicked off a full crawl. Once this was completed I checked the site and the events were still missing.

Next I looked at my query to see if there was something wrong with it. For example I was filtering out all past events and restricting the search results to the events content type but everything seemed ok.

I spent some time looking around and eventually I noticed that the missing items were very similar to other items which were getting displayed in the search list. There would be 4/5 events in the list with the same title and description but with different dates or locations. I had a look at the web part properties and I noticed there was an option to show a link to display duplicate results, see figure 1.


Figure 1

I selected this option but it never done anything. From previous experience with some web parts I knew there are sometimes web part properties which are not exposed via the UI which can still be set if you export the web part, a good example of this is to get the Content By Search web part to run server side. I exported the web part and opened this with notepad and searched for “Duplicate”. Wouldn’t you know it there was a boolean flag in the DataProviderJSON property called TrimDuplicates and this is defaulted to true, see snippet below in figure 2.

<property name="DataProviderJSON" type="string">{"QueryGroupName":"Default","QueryPropertiesTemplateUrl":"sitesearch://webroot","IgnoreQueryPropertiesTemplateUrl":false,"SourceID":"8413cd39-2156-4e00-b54d-11efd9abdb89","SourceName":"Local SharePoint Results","SourceLevel":"Ssa","CollapseSpecification":"","QueryTemplate":"ContentTypeId:0x0100440D27D8CDA949078CD3C29F3F3D7012*{searchboxquery}","FallbackSort":null,"FallbackSortJson":"null","RankRules":null,"RankRulesJson":"null","AsynchronousResultRetrieval":false,"SendContentBeforeQuery":true,"BatchClientQuery":true,"FallbackLanguage":-1,"FallbackRankingModelID":"","EnableStemming":true,"EnablePhonetic":false,"EnableNicknames":false,"EnableInterleaving":true,"EnableQueryRules":true,"EnableOrderingHitHighlightedProperty":false,"HitHighlightedMultivaluePropertyLimit":-1,"IgnoreContextualScope":false,"ScopeResultsToCurrentSite":false,"TrimDuplicates":true,"Properties":{},"PropertiesJson":"{}","ClientType":"","UpdateAjaxNavigate":true,"SummaryLength":180,"DesiredSnippetLength":90,"PersonalizedQuery":false,"FallbackRefinementFilters":null,"IgnoreStaleServerQuery":false,"RenderTemplateId":"DefaultDataProvider","AlternateErrorMessage":null,"Title":""}</property>

Figure 2 – Please note this has the QueryTemplate element remove so the exact query being executed isn’t visible to others 🙂

Most of the snippet above is default settings which you can ignore the import one is TrimDuplicates. In my case as soon as I set the this to false, saved the web part, imported it into the web part gallery and added it to a page all my results started to display correctly.

The key element here is how SharePoint determines if items are duplicates and in my case I suppose it’s fair enough as most of the key information such as the title and description were pretty much exactly the same.

This is another classic example of SharePoint and how its working as expected but because there are 101 configuration options you are not getting the results expected. It’s important to fully understand the product and all the options and what it can offer. This is one I will not forget and hopefully it provides a prompt for others in a similar situation.

Export User Profile Information Using PowerShell

April 5, 2013

While working on my most recent project I have spent a lot of time using the user profile service and while it offers some real benefits I have found it can be difficult to troubleshoot, especially when it comes to synchronizing with external sources like active directory (AD).

After configuring the user profile service on the client’s live environment I noticed that the email and manager fields were missing for some users. I wanted to try and get an idea of the extent of the issue so I went to the manage user profiles screen in central admin (CA) and while its useful for finding individual users its doesn’t really help view certain properties for all users.

Thinking about the options available I knew from previous experience I could write some C# code to get this information but I have been making an effort to brush up on my PowerShell (PS) skills so I decided this would be a perfect opportunity.

As with all PS scripts its essential these are first tested in a development environment before run on live. I began by creating my PS script on my development environment using PowerGUI as I think this is an excellent tool which makes creating PS scripts much easier.

Before starting I worked out what the rough steps would be to get the user profile information and i came up with the following:

  1. Get the site the user profile service is associated with
  2. Get the user profile service
  3. Get all user profiles
  4. Loop through all user profiles outputting the required information
  5. Collate steps and parameterise into a function

I then started working on each step and finally put it all together. Each of the steps is detailed below.

Step 1

From previous experience I knew the easiest way to get the user profile service was to get the service context for the site the user profile service is associated with. The first line gets the site passing a variable holding the site URL, next I check we have the site. I then got the service context associated with the site by calling the Get-SPServiceContext method passing the site

  1. $site = Get-SPSite $SiteURL -ErrorAction SilentlyContinue
  2.     if($site -eq $null)
  3.     {
  4.         Write-Host "Unable to access site " $SiteURL " please ensure the URL is correct and you have access to this"
  5.         return
  6.     }
  8.     $serviceContext = Get-SPServiceContext($site) -ErrorAction SilentlyContinue        
  9.     if($serviceContext -eq $null)
  10.     {
  11.         Write-Host "Unable to get the service context for site " $SiteURL
  12.         return
  13.     }


Step 2

Once I had the service context the next step is to get the user profile manager which is done by creating a new UserProfileManager instance passing in the associated service context.

  1. $userProfileManager = new-object Microsoft.Office.Server.UserProfiles.UserProfileManager($serviceContext)    
  2.     if($userProfileManager -eq $null)
  3.     {
  4.         Write-Host "User Profile Manager does NOT exist"
  5.         return
  6.     }

Step 3

With the instance of the user profile manager I then got all users by calling the GetEnumerator method.

  1. $users = $userProfileManager.GetEnumerator()

Step 4

In my situation there were hundreds of users so I decided outputting the information to the PS console would be difficult to read of I decided to output the details to a text file instead.

I looped though each user profile outputting the users preferred name. In addition so I could get additional user profile property values I looped through an array containing the names of user profile properties I wanted to get and output a string containing the property name and the property value for the current user profile

  1. foreach ($user in $users)
  2.     {        
  3.         $userName = $user.Item("PreferredName")
  4.         Write-Output "User Profile $userName" | Out-File $FileLocation -Append
  5.         if($UserProperties.Count -gt 0)
  6.         {        
  7.             for ($i = 0; $i -lt $UserProperties.Count;$i++)
  8.             {
  9.                 $UserProperty = $UserProperties[$i]                
  10.                 if($UserProperty -ne "" -and $UserProperty -ne $null)
  11.                 {
  12.                     try
  13.                     {
  14.                         $userPropertyValue = $user.Item($UserProperty)
  15.                         Write-Output "User Profile Property $UserProperty value is $userPropertyValue" | Out-File $FileLocation -Append
  16.                     }
  17.                     catch
  18.                     {
  19.                         Write-Host "Property $UserProperty doesn't exist"
  20.                     }    
  21.                 }
  22.             }
  23.         }
  24.     }

Step 5

Putting all the above steps together adding additional logging, variable checks and ensuring all settings can be passed in I ended up with the function below.

  1. function GetAllUserProfiles([string]$SiteURL, [string]$FileLocation, [array]$UserProperties)
  2. {
  3.     if($siteURL -eq $null -or $FileLocation -eq $null)
  4.     {
  5.         Write-Host "Please ensure all parameters are supplied"
  6.         return
  7.     }        
  9.     Write-Host "Starting to get all user profiles"
  11.     if($SiteURL -eq $null)
  12.     {
  13.         Write-Host "Please supply the site URL the User Profile service application is assocaited with"
  14.         return
  15.     }
  17.     $site = Get-SPSite $SiteURL -ErrorAction SilentlyContinue
  18.     if($site -eq $null)
  19.     {
  20.         Write-Host "Unable to access site " $SiteURL " please ensure the URL is correct and you have access to this"
  21.         return
  22.     }
  24.     Write-Host "Got site " $SiteURL
  26.     $serviceContext = Get-SPServiceContext($site) -ErrorAction SilentlyContinue
  28.     if($serviceContext -eq $null)
  29.     {
  30.         Write-Host "Unable to get the service context for site " $SiteURL
  31.         return
  32.     }
  34.     $userProfileManager = new-object Microsoft.Office.Server.UserProfiles.UserProfileManager($serviceContext)
  36.     if($userProfileManager -eq $null)
  37.     {
  38.         Write-Host "User Profile Manager does NOT exist"
  39.         return
  40.     }
  42.     Write-Host "Got User Profile Manager "
  44.     Write-Host "There are " $userProfileManager.Count " user profiles starting to loop through them. The results will be output to specified text file"
  46.     $users = $userProfileManager.GetEnumerator()
  47.     foreach ($user in $users)
  48.     {        
  49.         $userName = $user.Item("PreferredName")
  50.         Write-Output "User Profile $userName" | Out-File $FileLocation -Append
  51.         if($UserProperties.Count -gt 0)
  52.         {        
  53.             for ($i = 0; $i -lt $UserProperties.Count;$i++)
  54.             {
  55.                 $UserProperty = $UserProperties[$i]                
  56.                 if($UserProperty -ne "" -and $UserProperty -ne $null)
  57.                 {
  58.                     try
  59.                     {
  60.                         $userPropertyValue = $user.Item($UserProperty)
  61.                         Write-Output "User Profile Property $UserProperty value is $userPropertyValue" | Out-File $FileLocation -Append
  62.                     }
  63.                     catch
  64.                     {
  65.                         Write-Host "Property $UserProperty doesn't exist"
  66.                     }    
  67.                 }
  68.             }
  69.         }
  70.     }
  72.     Write-Host "Function completed"
  73. }


I have already used this function several times without any issues and I have found it a very easy and flexible way to pull information from the user profile store.

I hope this is helpful for others too.

Tagging Service is not available when using hashtags in Microblog entry

February 8, 2013

One of the most interesting new features in SharePoint 2013 is the changes to My Sites and the improvements to the social features such as the newsfeed, communities, hashtags, etc. In order to get up to speed I have spent some time looking at what’s new and how we can use this to help our clients.

As part of this a test environment was configured as a playground but when I came to test the hashtag functionality, see figure 1, it gave and an error saying the tagging service was unavailable, see figure 2.


Figure 1


Figure 2

I checked all the services and everything seemed fine, so I checked all the event logs and ULS logs. I came across one error in the ULS logs, see below.

ULS Log Error

SPMicrofeedStore.UpdateFromListItem: Expected field ‘HashTags’ not found in list item, verify fieldList paramter

From my research into the new social features I knew there was a MicroFeed list which stored some of the content so my initial impression was the HashTags column may have been missing from this list, however when I checked the list I found the column was there.

After some further research I found an article which explained that the hashtags and keywords were partially stored in the Managed Metadata Service so I decided to focus my attention on this.

I had already checked the service had been created and it was associated with the web application. In addition I knew I could access it from Central Admin so I checked accessing the term store from within the site collection. While I was able to access the term store I could see the hashtags and keywords term sets were empty, see figure 3.


Figure 3

From previous experience with using the term store to generate the navigation in another project I knew that even if the Managed Metadata Service is associated with the web application and term store is accessible from within the site collection it still needs to be set as the default for the web application. I checked this by logging into Central Administration going to manage service applications and locating my Managed Metadata Service in the list. To set the service as the default you need to click on the proxy service, see figure 4, and click properties in the ribbon. This will launch a popup and the top two check boxes need to be selected, see figure 5.


Figure 4


Figure 5

As soon as I changed this the hashtag functionality started to work, see figure 6. One thing to note is you have to be careful when setting the Managed Metadata Service to be the default as if you have multiple Managed Metadata Services associated with the one web application it can cause some issues.


Figure 6

Hopefully this will help out any other people who encounter the same issue. Happy SharePointing

SharePoint 2013 sever unable to save form error

January 18, 2013

I have been working with SharePoint 2013 a lot over the past few months going back to when it was still in Beta. There has been a few occasions when I have encountered an error when trying to add or edit list items on my development environment, see figure 1.


Figure 1

When I first encountered this error it was on a list which was created from a custom list definition so I double checked all the settings in Visual Studio but I couldn’t see any obvious errors. To establish if it was my custom list definition which was causing the error I created another list using the OOTB custom list definition and I noticed the same thing happened.

I had only ever encountered this issue on my development environment not on integration or production so I started thinking what was different across the environments and the most obvious thing was the number of servers. As with most cases my development environment was a standalone box whereas the integration and production environments were small to medium farms. I checked the performance of my development server and I noticed the physical memory was extremely high, see figure 2.


Figure 2

I checked the running process and I could see there were numerous noderunner process running each taking a between 100,000 and 500,000K. Anyone who has worked with SharePoint 2013 or done reading on it will know one of the main changes has been around search and the integration of Fast Search into the OOTB SharePoint search . As part of this the search now launches multiple instances of the noderunner process, see figure 3.


Figure 3

Since this was a development box I restarted the SharePoint Search Host Controller and SharePoint Server Search 15 services and after a minute or so the physical memory drop down, see figure 4. I then tested the form and it worked as expected.


Figure 4


After further testing the magic figure is the physical memory has to be under 95%. I hope this helps others who encounter an similar issue

Content by search web part server side version displays dates out by an hour

December 20, 2012

Since the release of SharePoint 2013 I have been lucky enough to use it in an actual project rather than playing around with it and I must admit there are several things which can be very frustrating. I came across a good example of one earlier today when doing some testing.

The project I am working on uses search to display news articles but since most of the search web parts all use JavaScript we also used the content by search web part set to run server side. Everything seemed to be working as expected until I noticed the dates on the JavaScript, figure 1, and non JavaScript version, figure 2, were different


Figure 1


Figure 2

The data being displayed is held in a list behind the screens and the field in question is a date only field. I double checked the date through the UI, figure 3, and I also wrote a PowerShell script, figures 4 & 5, but this confirmed the data was correct.


Figure 3


Figure 4


Figure 5

It just seemed like on the server side version it always removed an hour from the actual time so 20 June 2012 00:00:00 becomes 19 June 2012 23:00:00. I had a look around but I couldn’t see why it might be this and I looked on Google but I couldn’t see anything else on this.

In the implemented solution the list items being crawled via the search service were getting added via code so this allowed me to update the date field to set it to be 9am on the morning of the date entered. Even though the field type is date only it still stores the content in the DB with a time parameter so I could safely update this without fear of overwriting users content. This meant when the server side version removed an hour the value being returned is always 8am on the morning of the date entered, thus solving my problem.

I fully recognise there will be others who don’t have control over the content in the same manner so this solution will only work in certain circumstances. I would be very interested to see if anyone else has come across this and if so how they managed to resolve the issue.

SharePoint 2013 search results web part ignores sort order entered via Query Builder

November 22, 2012

I have been working with SharePoint 2013 since it was released a few weeks ago and even before that when it was still in Beta and if I’m honest I have been slightly disappointed. There are some great new features which have been added to the product but I have found there are certain elements which just don’t work as I would expect and this blog post is about one of these issues.

I am currently working on a project for a public facing website which is built on SharePoint 2013 and one of the components is to display the latest news. The news content is stored in a list and I decided to use the ResultScriptWebPart rather than writing some custom code or using the ContentBySearchWebPart or ContentByQueryWebPart to display the news items.

I added the search results web part to the news page and opened the ‘Change ‘Query’ popup in the web part properties, see figure 1.


Figure 1

I then configured my search criteria using the advanced mode settings, see figure 2.


Figure 2

Next I clicked on the ‘Sorting’ tab and entered the sort criteria I wanted the results to use, see figure 3.


Figure 3

Once I was happy with the query and the sorting options I tested my query by clicking on the test tab and checking the results I expected were return and they were in the correct order, see figure 4 for screen without any results for privacy reasons


Figure 4

At this point the query and sort order had been set and it was returning the correct results in the preview. When I saved the settings and returned to the page I noticed that while there were results they weren’t in the order I specified.

I spent ages checking settings and looking around but nothing I changed seemed to make a difference. Eventually just for testing I enabled the sort dropdown and noticed when this was on the page I was able to select different options from the drop down and this altered the sorting. From looking at the options in the drop down, see figure 5, it looked like these options were being pulled from the ‘Available sort orders (JSON)’ web part property under settings.


Figure 5

As soon as I looked at the options in the ‘Available sort orders (JSON)’ web part property it occurred to me that for some reason the web part ignores whatever you set in the actual query builder and only sorts based on the values in this property. I changed this to sort on the fields I wanted, see figure 6, and the results came back in the correct order.


Figure 6

There may be some setting I am missing but to me this seems backwards in that you can set the sort order in the ‘Change Query’ settings and they have a nice UI for it and it even uses these settings in the preview but as soon as you save the settings it sorts on the ‘Available sort orders (JSON)’ web part property. If anyone has any experience of this and there is another way around it please let me know

%d bloggers like this: