While working on a few SharePoint 2013 projects I have been utilising the SharePoint search in a lot of places to provide dynamic content and there has been quite a few occassions when things don’t seem to work as expected such as CBS Server Side Date Error or Results web part ordering. I suppose this is to be expected when the search engine has been completely replaced with a mix of the old SharePoint search engine and FAST search.
When working on one aspect of a recent project I was using the results script web part to query some events stored in a list. As per you would expect these events had the typical details such as title, date, description, etc. At first everything seemed to be working as expected until the client reported there was a bunch of events missing from the search generated list. My first thought was these were new events and weren’t in the search index so I kicked off a full crawl. Once this was completed I checked the site and the events were still missing.
Next I looked at my query to see if there was something wrong with it. For example I was filtering out all past events and restricting the search results to the events content type but everything seemed ok.
I spent some time looking around and eventually I noticed that the missing items were very similar to other items which were getting displayed in the search list. There would be 4/5 events in the list with the same title and description but with different dates or locations. I had a look at the web part properties and I noticed there was an option to show a link to display duplicate results, see figure 1.
I selected this option but it never done anything. From previous experience with some web parts I knew there are sometimes web part properties which are not exposed via the UI which can still be set if you export the web part, a good example of this is to get the Content By Search web part to run server side. I exported the web part and opened this with notepad and searched for “Duplicate”. Wouldn’t you know it there was a boolean flag in the DataProviderJSON property called TrimDuplicates and this is defaulted to true, see snippet below in figure 2.
Figure 2 – Please note this has the QueryTemplate element remove so the exact query being executed isn’t visible to others :)
Most of the snippet above is default settings which you can ignore the import one is TrimDuplicates. In my case as soon as I set the this to false, saved the web part, imported it into the web part gallery and added it to a page all my results started to display correctly.
The key element here is how SharePoint determines if items are duplicates and in my case I suppose it’s fair enough as most of the key information such as the title and description were pretty much exactly the same.
This is another classic example of SharePoint and how its working as expected but because there are 101 configuration options you are not getting the results expected. It’s important to fully understand the product and all the options and what it can offer. This is one I will not forget and hopefully it provides a prompt for others in a similar situation.