SharePoint Lookup Field Throttling Causes Missing Fields in CSOM Query

A very annoying bug appeared few weeks ago in one of our production environments with SharePoint 2013.

SYMPTOMS

You have a custom list with some lookup columns that refer to other lists. In our case the main list contained news and the lookup columns contained the classification of the news.

You add a new lookup column to the list, due to customer feedback.

Suddenly, you can’t get the list results any more by code. When you do a CSOM query, the lookup fields are lost. Only the non-lookup fields are retrieved.

CAUSE

SharePoint throttling also includes list item maximum lookup references. It is set to eight (8) lookup fields per list, by default.

Resource Throttling

This particular configuration is set on the web application throttling page in Central Administration, under the heading “List View Lookup Threshold”.

SOLUTION

If you don’t need all of the lookups at the same time, you can still make the query by choosing the fields you want to retrieve. However, in our case we needed all of the classification columns.

In this case you have two choices:

  • Increase the list lookup threshold limit to more than 8 (that’s what we did)
  • Establish a large query window, an interval during the day during which you can perform the queries

UpdatePanel Troubles with SharePoint 2013

Few weeks ago I had a nasty bug on customer premises where a combination of SharePoint 2013 and ASP.NET UpdatePanel control resulted in some very weird behaviour.

THE SYMPTOMS

Our project consisted of several full-trust code ASPX pages, dutifully placed in _layouts folder. Two of these pages had ASP.NET AJAX UpdatePanel controls to give “a kind of” smooth user interaction experience. The reason to use it was the simplicity of wrapping the existing SharePoint server controls such as grids and buttons in the update panel to prevent page reloads on postbacks. (Yes, I know it’s legacy technology but still, sometimes it’s the best and simplest solution for a given problem).

However, these pages gave random errors as if their event was somehow lost between the UpdatePanel client-side code and the server-side postback. Randomly, we had errors in our pages, with no discernible pattern. The only clue was in the ULS logs, where there were no user claims associated with the request, as if it were a non-authenticated call. What’s worse, as it had no clear cause, our reproduction procedure was to mindlessly click the buttons in the hope of a random error. Needless to say, it was way too inefficient.

Googling around, I found a way to reproduce the errors using Fiddler HTTP debugging tool. The sequence of steps is to make the first page load with Fiddler running in the background. Then, I deleted all the captured traces in Fiddler and clicked the button inside UpdatePanel. Immediately, the UpdatePanel threw the dreaded error.

THE CAUSE

According to the Microsoft KB3062825 article, the cause is the mismatch between the HTTP protocol implementation in UpdatePanel and SharePoint 2013, where instead of returning a 401 code in the handshake, SharePoint returns 200 with the login redirect and the UpdatePanel loses its changes. It only applied to a certain combination of SharePoint 2013 and .NET Framework, and even in these combination it wasn’t certain that the error would happen.

Unfortunately, we were in that situation, with the right combination of SharePoint and .NET versions.

THE SOLUTION

The solution is simple: applying a specific .NET Framework hotfix (for Windows Server 2012 R2, for Windows Server 2012) that’s meant to patch the HTTP protocol handling causing the error. The patch is available via Microsoft support, but it’s not publicly available. I think that it’s because the error isn’t deterministic enough to be included in a regular cumulative update or service pack.

You also have to install SharePoint 2013 SP1, together with a Cumulative Update from December 2014 (or later).

Galileo and SharePoint (via Azure) at SPS London 2015

Yesterday (July 11th) I was presenting at the very first SharePoint Saturday London. The SPS event was very well organized by Peter Baddeley and Seb Matthews.

Galileo with PIR sensor sending data to Azure
Galileo with PIR sensor sending data to Azure

My session was about connecting IoT to Office 365 (via Azure). I used an Intel Galileo prototyping board with a Passive Infrared sensor (PIR). The sensor data was used to determine if a meeting room was empty or occupied. The raw data is uploaded by Galileo to an Azure Notification Hub. A continuously-running Stream Analytics job is then used to translate the raw data to 1-minute resolution of the room availability and to insert this data into Azure Table Storage. Finally, a provider-hosted Office 365 SharePoint application is used to visualize the room availability.

The slides for my talk are available at SlideShare and the code is now live at GitHub.

SharePoint App Catalog And Missing Apps

Another weird SharePoint app bug happened yesterday. The solution was fairly easy once you know what’s going on, but it’s just weird altogether.

SYMPTOMS

You have a custom app in your SharePoint 2013 App Catalog.

A custom app inside app catalog under Apps for SharePoint
A Custom App Inside App Catalog

You want to add this app to a SharePoint site.  You can’t find your app in the “From Your Organization” section when you click at “Add an app” in a site.

The App Is Missing From "Your Apps"
The App Is Missing From “Your Apps”

CAUSE

I first suspected that the current user doesn’t have permissions to add an app. However, the user is the site collection administration and thus has the permission to install an app.

Yet…a slight detail. The App Catalog site is, well, a SharePoint site. With its own permissions. And, by default, containing only the user who created the catalog in the first place (the system admin).

So, the current user, although a site collection admin, doesn’t have permissions to read from the app catalog. (This is the weird part, as I expected SharePoint to do the reading using a system account behind the scenes.)

SOLUTION

Add the users that should be able to install your custom apps to the site permissions of the App Catalog site, with Read permission level. In my case it was “Dani Alves” (yes, I’m a Barcelona fan).

Adding Read Permissions to the App Catalog

Now, the app is visible in “Your Apps” when you try to add it to a site. Yeah!Custom App Is Now Visible