Error 404 after publishing in MSCRM 2011
Some days ago, I encountered an interesting issue.
A developper had a just upgraded CRM 2011 environment and right after publishing a change; the organisation stopped working and all he got was a 404 error.
The other organizations worked ok, but all urls pointing to the failed organization returned a 404.
I enabled the Trace on the server, and here's what I found :
>A failure occurred during the execution of the AuthenticationPipeline: System.ArgumentException: An item with the same key has already been added.
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
at Microsoft.Crm.Metadata.PreloadedMetadataCacheDataProvider.AddEntityRelationshipsToCollections(IPreloadedMetadataInitializationContext context)
at Microsoft.Crm.Metadata.PreloadedMetadataCacheDataProvider.Initialize(IOrganizationContext organizationContext, MetadataContainer container, LoadMasks masks)
at Microsoft.Crm.Metadata.MultiOrgSharableMetadataCacheLoader.CreateMetadataCacheDataProvider(MetadataContainer container, IOrganizationContext context, LoadMasks masks, CounterList counter)
at Microsoft.Crm.Metadata.DynamicMetadataCacheLoader.BuildMetadataCacheFromMetadataContainer(MetadataContainer container, LoadMasks masks, IOrganizationContext context, CounterList counter)
at Microsoft.Crm.Metadata.DynamicMetadataCacheLoader.LoadCacheFromDatabaseInternal(LoadMasks masks, CrmDbConnection connection, CrmTransaction transaction, IOrganizationContext context, CounterList counter)
at Microsoft.Crm.Metadata.DynamicMetadataCacheLoader.LoadCacheFromDatabase(LoadMasks masks, IOrganizationContext context, CounterList counter)
at Microsoft.Crm.Metadata.DynamicMetadataCacheFactory.LoadMetadataCache(LoadMethod method, CacheType type, IOrganizationContext context)
at Microsoft.Crm.Metadata.MetadataCache.LoadCache(IOrganizationContext context, Boolean fileOnlyIfExists)
at Microsoft.Crm.Metadata.MetadataCache.GetInstance(IOrganizationContext context)
at Microsoft.Crm.BusinessEntities.BusinessEntityMoniker..ctor(Guid id, String entityName, IOrganizationContext orgContext)
at Microsoft.Crm.Caching.UserDataCacheLoader.LoadCacheData(Guid key, ExecutionContext context)
at Microsoft.Crm.Caching.ObjectModelCacheLoader`2.LoadCacheData(TKey key, IOrganizationContext context)
at Microsoft.Crm.Caching.CrmMultiOrgCache`2.CreateEntry(TKey key, IOrganizationContext context)
at Microsoft.Crm.Caching.CrmMultiOrgCache`2.LookupEntry(TKey key, IOrganizationContext context)
at Microsoft.Crm.BusinessEntities.SecurityLibrary.GetUserInfoCommon(IOrganizationContext context, Guid userGuid, LocatorServiceContext locatorServiceContext)
at Microsoft.Crm.BusinessEntities.SecurityLibrary.GetCallerAndBusinessGuidsFromThread(WindowsIdentity identity, Guid organizationId, LocatorServiceContext locatorServiceContext)
at Microsoft.Crm.Authentication.CrmWindowsIdentity..ctor(WindowsIdentity innerIdentity, Boolean publishCrmUser, Guid organizationId)
at Microsoft.Crm.Authentication.WindowsAuthenticationProviderBase.Authenticate(HttpApplication application, WindowsIdentity userIdentity)
at Microsoft.Crm.Authentication.AuthenticationStep.Authenticate(HttpApplication application)
at Microsoft.Crm.Authentication.AuthenticationPipeline.Authenticate(HttpApplication application)
at Microsoft.Crm.Authentication.AuthenticationEngine.Execute(Object sender, EventArgs e)
So, it was failing while building the metadata cache for that organization with some kind of duplicate key "An item with the same key has already been added".
However, there wasn't an easy way to find the culprit; all I got from the developper was that he was handling connections in an entity when the organization failed on him; so I reverse read the verbose log to got which data could contain the duplicate and here're the requests made by the platform while building the cache :
select * from ManagedProperty where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
select * from OptionSet where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
select * from Entity where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
select * from Attribute where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
select * from AttributeLookupValue where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
select * from AttributePicklistValue where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState, DisplayOrder asc
select * from EntityRelationship where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
select * from Relationship where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
select * from RelationshipExtraCondition where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
select * from EntityRelationshipRole where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
select * from EntityRelationshipRelationships where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
select * from ViewAttribute where SolutionId <> 'fd140aad-4df4-11dd-bd17-0019b9312238' and OverwriteTime = 0 and ComponentState in (0, 2) order by ComponentState asc
I made a guess that "name" was the key field and started checking each table with queries as such:
select schemaname,count(*) from EntityRelationship
group by schemaname
having count(*)>1
After a bit of drilldrown, two records were duplicated in RelationShip and EntityRelationShip :
xxx_agreement_connections1 C7866019-D3D1-40D7-B483-381FCDCCFFB2 D6CE8EBC-86DC-49CB-839A-
xxx_agreement_connections1 4E4D2C0B-BD41-E111-9A50-005056A877F9 256FA569-E732-4675-9519-
xxx_agreement_connections2 4E4D2C0B-BD41-E111-9A50-005056A877F9 05F6B370-01FE-4F3E-BF56-
xxx_agreement_connections2 C7866019-D3D1-40D7-B483-381FCDCCFFB2 48C02973-80A2-431B-8805-
In CRM 4.0, a custom entity was created called "connection" with links to other entities, in order to mimic some of the CRM2011 "connection" entity functions. The upgrade process went without issues and the organization was stable afterwards; however when the developper created a new 'CRM2011' connection, the platform created a new relationship with the exact same name as the one named with the 'CRM4' connection. As you cannot change the relationship name in connection, it was quite unavoidable and the platform crashed.
In order to fix the issue and get the organization back up, I manually changed the relationship name with the following statements:
UPDATE Relationship set name='xxx_agreement_connections1C' where RelationshipId='20E35F70-5965-4C6E-98EE-CAAA38ABCB5E'
UPDATE Relationship set name='xxx_agreement_connections2C' where RelationshipId='BE921610-4237-4A11-8883-00A4016D1DA0'
update EntityRelationship set schemaname = 'xxx_agreement_connections1C' where EntityRelationshipId = '296A89B6-DA41-E111-9A50-005056A877F9'
update EntityRelationship set schemaname = 'xxx_agreement_connections2C' where EntityRelationshipId = '70335CCE-DA41-E111-9A50-005056A877F9'
Warning : this put back the organization up but the two lasts statements change the schemaname, you would also need to change the actual columns and views in the custom entity manually; in that particular case we wanted to get rid of the old connection, so we dropped them. Also, making direct changes to the system tables is quite dangerous so if you're unsure, it may be worth just restoring a backup of the organization.
In order to avoid those kind of issues; the easiest is to change the custom prefix to something different than the one in CRM 4.0; for example X11_ instead of XXX_ after the migration process has complete; that will force the platform to use a different name than what you could have in CRM 4.0
MSCRM 2011 UR11 in MS Update
On October 23rd, the update Rollup 11 will be included in the Microsoft Update, so be sure to inform your customers beforehand.
This is especially important for the Clients update as "client packages installed via the Windows Update system, will not require administrative privileges."
For information, here's the CRM Blog link, the Knowledge Base link and the direct download link.
Using Forefront UAG for publishing MSCRM in IFD mode
In looking for solutions for some exotic security architecture for MSCRM deployment, I found out a not well known fact is that Forefront Universal Application Gateway (UAG) has a template to publish MSCRM 2011 and 4.0; the MSCRM 2011 publishing was released in Forefront UAG Service Pack 1 Update 1.
This offer quite a few advantages and a big drawback (see here below)
The advantages of using this mode are the following :
Control downloads and uploads─For example, you can prevent file downloads for unmanaged client endpoints, or for endpoints that do not comply with corporate access policy.
Control data export─You can control who exports CRM data to Excel, and from where.
Provide session clean-up capabilities─You can clean an endpoint cache and temporary files after a session ends.
Control session access─Forefront UAG adds timeout and logoff functionality to reduce the risk of session hijacking.
Provide frontend authentication─You can authenticate clients on the Forefront UAG server, to ensure that only authenticated traffic reaches backend CRM servers. Forefront UAG provides a variety of frontend authentication mechanisms, including strong authentication using smartcards and one-time passwords.
Provide backend authentication─Forefront UAG provides single sign-on so that clients need to authenticate once only. Credentials provided for session access to Forefront UAG sites can be delegated to backend CRM servers that require authentication. Forefront UAG also supports Active Directory Federation Services (ADFS).
Verify endpoint health─You can configure access policies using Forefront UAG inbuilt policies, or using Network Access Protection (NAP) policies downloaded from a Network Policy Server (NPS). Only clients complying with the policies can access Forefront UAG, and backend CRM servers
The big drawbacks is the following :
"Publishing Dynamics CRM 2011 via Forefront UAG does not support the Dynamics CRM client for Outlook."
Obviously, this is a big drawback but in some scenarii, like the one I was assessing, Outlook integration is out of scope so it's quite a valid possibility.
Furthermore, using UAG, you can activate IFD mode in MSCRM 2011 without using claims based authentication (see http://technet.microsoft.com/en-us/library/hh490315.aspx), so no need for ADFS.
Of course, you can also use it with claims based authentication (http://technet.microsoft.com/en-us/library/hh490310.aspx)
MSCRM 2011 R10 Has been released
The rollup 10 has just been released, bypassing the rollup 9 (aka the old Q2 Service Update which will be released with the Q4 2012 Service Update)
Here's some key features :
• More Windows 8 / Internet Explorer 10 compatibility
• Many performance and stability-related fixes, including fixes to the CRM Client for Microsoft Outlook
• Contains all fixes originally scheduled for release via Update Rollup 9
• Quick Find performance optimizations and EnableRetrieveMultipleOptimization SQL query performance optimizations:
For more information, here's the KB Article : 2710577 and also the relevant CRM In the Field article.
Windows Identity Foundation and Windows Authentication on MSCRM 2011
I had a query the other day to validate if WIF (windows indentity foundation) was actively used in a MSCRM 2011 implementation configured with windows authentication (ie: Kerberos).
Now, WIF is a required component of the server installation, of the Outlook addin installation and of the SDK (see this article), but that does not means it is actively used in a non claims based mode.
Looking in the Claims authentication white paper, I found this little note :
Microsoft Dynamics CRM Server 2011 replaces forms authentication with claims-based authentication, an identity access solution designed to provide simplified user access and single sign-on access to Microsoft Dynamics CRM data.
Interoperability is provided through reliance on industry standard protocols such as WS-Federation, WS-Trust, and Security Assertion Markup Language 1.1 (SAML)."
Actually, the data flow for Windows authentication in the white paper is exactly the same as for MSCRM 4.0 (or any standard asp.net windows authentication by the way), so no need to involve another technology