SharePoint Lookup Field Throttling Causes Missing Fields in CSOM Query

A very annoying bug appeared few weeks ago in one of our production environments with SharePoint 2013.

SYMPTOMS

You have a custom list with some lookup columns that refer to other lists. In our case the main list contained news and the lookup columns contained the classification of the news.

You add a new lookup column to the list, due to customer feedback.

Suddenly, you can’t get the list results any more by code. When you do a CSOM query, the lookup fields are lost. Only the non-lookup fields are retrieved.

CAUSE

SharePoint throttling also includes list item maximum lookup references. It is set to eight (8) lookup fields per list, by default.

Resource Throttling

This particular configuration is set on the web application throttling page in Central Administration, under the heading “List View Lookup Threshold”.

SOLUTION

If you don’t need all of the lookups at the same time, you can still make the query by choosing the fields you want to retrieve. However, in our case we needed all of the classification columns.

In this case you have two choices:

  • Increase the list lookup threshold limit to more than 8 (that’s what we did)
  • Establish a large query window, an interval during the day during which you can perform the queries

UpdatePanel Troubles with SharePoint 2013

Few weeks ago I had a nasty bug on customer premises where a combination of SharePoint 2013 and ASP.NET UpdatePanel control resulted in some very weird behaviour.

THE SYMPTOMS

Our project consisted of several full-trust code ASPX pages, dutifully placed in _layouts folder. Two of these pages had ASP.NET AJAX UpdatePanel controls to give “a kind of” smooth user interaction experience. The reason to use it was the simplicity of wrapping the existing SharePoint server controls such as grids and buttons in the update panel to prevent page reloads on postbacks. (Yes, I know it’s legacy technology but still, sometimes it’s the best and simplest solution for a given problem).

However, these pages gave random errors as if their event was somehow lost between the UpdatePanel client-side code and the server-side postback. Randomly, we had errors in our pages, with no discernible pattern. The only clue was in the ULS logs, where there were no user claims associated with the request, as if it were a non-authenticated call. What’s worse, as it had no clear cause, our reproduction procedure was to mindlessly click the buttons in the hope of a random error. Needless to say, it was way too inefficient.

Googling around, I found a way to reproduce the errors using Fiddler HTTP debugging tool. The sequence of steps is to make the first page load with Fiddler running in the background. Then, I deleted all the captured traces in Fiddler and clicked the button inside UpdatePanel. Immediately, the UpdatePanel threw the dreaded error.

THE CAUSE

According to the Microsoft KB3062825 article, the cause is the mismatch between the HTTP protocol implementation in UpdatePanel and SharePoint 2013, where instead of returning a 401 code in the handshake, SharePoint returns 200 with the login redirect and the UpdatePanel loses its changes. It only applied to a certain combination of SharePoint 2013 and .NET Framework, and even in these combination it wasn’t certain that the error would happen.

Unfortunately, we were in that situation, with the right combination of SharePoint and .NET versions.

THE SOLUTION

The solution is simple: applying a specific .NET Framework hotfix (for Windows Server 2012 R2, for Windows Server 2012) that’s meant to patch the HTTP protocol handling causing the error. The patch is available via Microsoft support, but it’s not publicly available. I think that it’s because the error isn’t deterministic enough to be included in a regular cumulative update or service pack.

You also have to install SharePoint 2013 SP1, together with a Cumulative Update from December 2014 (or later).

SharePoint App Catalog And Missing Apps

Another weird SharePoint app bug happened yesterday. The solution was fairly easy once you know what’s going on, but it’s just weird altogether.

SYMPTOMS

You have a custom app in your SharePoint 2013 App Catalog.

A custom app inside app catalog under Apps for SharePoint
A Custom App Inside App Catalog

You want to add this app to a SharePoint site.  You can’t find your app in the “From Your Organization” section when you click at “Add an app” in a site.

The App Is Missing From "Your Apps"
The App Is Missing From “Your Apps”

CAUSE

I first suspected that the current user doesn’t have permissions to add an app. However, the user is the site collection administration and thus has the permission to install an app.

Yet…a slight detail. The App Catalog site is, well, a SharePoint site. With its own permissions. And, by default, containing only the user who created the catalog in the first place (the system admin).

So, the current user, although a site collection admin, doesn’t have permissions to read from the app catalog. (This is the weird part, as I expected SharePoint to do the reading using a system account behind the scenes.)

SOLUTION

Add the users that should be able to install your custom apps to the site permissions of the App Catalog site, with Read permission level. In my case it was “Dani Alves” (yes, I’m a Barcelona fan).

Adding Read Permissions to the App Catalog

Now, the app is visible in “Your Apps” when you try to add it to a site. Yeah!Custom App Is Now Visible

Lightswitch Custom Validation Refresh

I found an odd problem this week in a SharePoint-enabled Lightswitch HTML Client application. I am sharing the solution here in case there is somebody out there with a similar situation.

SYMPTOMS

You have a Lightswitch HTML application (banner definition in my case). You use custom validation function in beforeApplyChanges function.

Your validation works! However, when you try to correct the field that is being validated, the Save button doesn’t work anymore and it keeps the custom validation message on the screen (it says in Spanish that the end date has to be later than the start date).

image

Furthermore, you get an error message that says “Cannot continue. There are validation errors on the page”.

image

CAUSE

Apparently, Lightswitch code generator attaches the validation refresh on control events such as focus, blur, keyup etc, but only on default LS validations (data type, length, required fields and similar).

Custom validations on beforeApplyChanges are not refreshed on data changes in the controls and due to the page being in invalid state, the Save button doesn’t trigger the beforeApplyChanges function again. It’s a kind of a subtle bug in LS code generation that causes deadlock: we can’t save because the validation isn’t firing and it isn’t firing because we can’t save.

SOLUTION

Well, the best solution would be for Lightswitch to expose an additional hookup point for custom validation, one that should be refreshed automatically with any control change. However, it is improbable that it would ever be included in Lightswitch code.

The available solution for now is to register a change listener for those properties in our screen that trigger custom validation. Once the value is changed, we remove the validation errors for that field if there are any. This ensures that the Save button will work, triggering the custom validation logic again. We can hook up the change listener in the screen created event.

However, we have to be careful to unregister the change listener when the screen is closed. This technique is explained in this MSDN Forum post by Huy Nguyen.

Checking for User Permissions and Getting UnauthorizedAccessException

In a recent project I have been writing code to check if an arbitrary user can create new documents in certain document libraries. In order to do the check, I used the good old DoesUserHavePermissions method, which is present in SPWeb, SPList and SPListItem objects (securable objects).

SYMPTOMS

When using DoesUserHavePermissions() method on a securable object, you get UnauthorizedAccessException.

CAUSES

There are multiple causes for this behavior.

FIrst, the current user context is such that the current user has no rights to enumerate permissions on the SPWeb/SPList/SPListItem object. If so, the exception will be raised.

So, your first inclination is to use RunWithElevatedPrivileges to check the permissions. However, it also throws the same exception. The cause is a token check that the DoesUserHavePermissions method includes in its code (as explained by Phil Harding). The user token is compared against the current user. Somehow, the user token for elevated object is not the same as the current user in the context and the exception is being thrown.

SOLUTION

I managed to solve this issue by explicitly opening the securable object with a System Account token, instead of using RunWithElevatedPrivileges.

Access Denied with RunWithElevatedPrivileges

A strange situation happened to me few days ago, when checking a portion of SharePoint 2013 server-side code on a custom form. Basically, it uses RunWithElevatedPrivileges to check that the current user has access to a certain site and certain libraries, before uploading the file to a content organizer enabled library.

The Symptoms

The code that runs with elevated privileges on a POST event triggered "Access Denied" errors when trying to access SPWeb and SPList objects. The objects were declared under the elevated privilege code block but the ULS logs still show the "access denied" errors.

The Cause

According to MSDN blog the code running with elevated permissions has to validate the form digest before entering the elevated permissions code block. If not, it might give "Access Denied" errors.

The Solution

Just add SPUtility.ValidateFormDigest(); before the elevated permissions block and the "Access Denied" errors dissappear.

Failed to create a custom control ‘PublishingSiteActionsMenuCustomizer’

A very weird and hard to pinpoint SharePoint error has haunted me these last days.

The Symptoms

You have a SharePoint site collection that uses Publishing features. Suddenly, the users can’t access your site. All user accounts, including site collection administrators, get the dreaded "Access Denied" error. In my case, it was SharePoint 2010 with a custom site template with publishing features included in it.

The SharePoint log files mention this:

The Cause

It is really strange that SharePoint can’t load it’s own components. But, the real cause it that the web application that a culprit site collection is running on is missing its "superuser" settings. The SuperUsers are the users configured for Publishing infrastructure to read and write publishing cache. It seems that if the users are not correctly configured, the publishing infrastructure fails badly and SharePoint interprets it as "Access Denied".

Two blog posts were of great help: Khashish Sukhija and Nico Marten’s. Thank you guys! I checked the web application properties from PowerShell and the super user entries were empty for the web application that was behaving strangely.

The Fix

Execute the script found on Nico’s post (reproduced here for convenience, all credit is his) and IISRESET.

How to Enable Custom JavaScript on MDS Pages in SharePoint 2013

If you have custom JavaScript file loaded in your master page, as we usually do in SharePoint, you might have stumbled upon the problems with custom JS and SharePoint 2013 new Minimum Download Strategy (MDS).

By default, MDS is enabled on Team Sites in SharePoint 2013 and allows for refreshing data on SharePoint pages without causing reloads. But, in order to do so, all the content and JavaScript in MDS pages must play along nicely. If not, the symptoms include:

  • blank page on loading custom JS in a MDS-enabled page
  • custom JS script not loading

The solution

The first part is to use ScriptLink control in the master page, instead of using script tags directly. Specify "LoadAfterUI" attribute in order for the script to be loaded after the page is loaded in MDS.

<SharePoint:ScriptLink language="javascript" ID="Whatever" name="~sitecollection/Style Library/js/yourcustom.js" OnDemand="false" LoadAfterUI="true" runat="server" Localizable="false" />

The second part is to encapsulate all your custom JS in a single function and call it from your custom code. Your yourcustom.js file should look like this:

function $_global_customjs(){
    _spBodyOnLoadFunctionNames.push(‘DoSomething’);
}
    var DoSomething = function ()
    {
        // — Your custom JS here
    }
$_global_customjs();

BDC Visual Studio Project and Missing Assembly Trouble

I just had a strange error the other day, deploying Business Connectivity Services (BCS) model arranged around a NET assembly. When accessing the external list data, I found the following error:

Assembly was requested for LobSystem with Name 'Namespace.LobSystem', but this assembly was not returned. SystemUtility of Type 'Microsoft.SharePoint.BusinessData.SystemSpecific.DotNetAssembly.DotNetAssemblySystemUtility' requires the assembly to be uploaded.

Of course, I checked the assembly and it was loaded in the GAC. So, where’s the error coming from?

Well, our friend BCS registers the assemblies for your external content type when you activate the feature containing your BCS Model and Assembly. This feature is made automatically when you create a new BCS project in Visual Studio. The feature has a custom feature receiver and also has a custom entry in the feature.xml declaration.

<Properties>
  <Property Key="GloballyAvailable" Value="true" />
  <Property Key="IncrementalUpdate" Value="false" />
  <Property Key="ModelFileName" Value="YourModelYourModel.bdcm" />
  <Property Key="BdcModel1" Value="BdcAssembliesYourAssembly.dll" />
</Properties>

It has to match the name of the LOB System in the BDCM file (the entity model):

<LobSystem Name="LobSystemName" Type="DotNetAssembly">
  <LobSystemInstances>
    <LobSystemInstance Name="LobSystemInstance" />
  </LobSystemInstances>

My error was renaming the model in some point of time. It went well for the model XML, but the old name ("BdcModel1") was still remaining in the feature.xml. After manually editing the feature.xml and pointing it to the new name of the LOB System, the error was gone:

<Properties>
  <Property Key="GloballyAvailable" Value="true" />
  <Property Key="IncrementalUpdate" Value="false" />
  <Property Key="ModelFileName" Value="YourModelYourModel.bdcm" />
  <Property Key="LobSystemName" Value="BdcAssembliesYourAssembly.dll" />
</Properties>

Activate Windows 8 Enterprise License without Volume Activation

I was trying to activate my company’s Windows 8 Enterprise license but I was met repeatedly with the “No DNS Servers Configured: Code 0x8007267C” error during activation. It is caused by the missing Volume Activation Management Tool deployed in the company Active Directory. However, we still haven’t deployed this setup but I wanted to activate my license.

In the previous version of Windows, you could change the key into a MAK (Multiple Activation Key) and then activate it without hassle. In Windows 8 this possibility is still available, but it is somewhat hidden.

You should run the following tool to change the key and activate Windows online:

  • slui.exe 3 (for online activation)
  • slui.exe 4 (for phone activation)

I hope that it helps.