【Azure 微服務】Service Fabric中微服務在升級時,遇見Warning – System.Collections.Generic.KeyNotFoundException 服務無法正常運行

問題描述

使用.Net Framework 4.5.2為架構的Service Fabric微服務應用,在升級後發佈到Azure Fabric中,服務無法運行。通過Service Fabric Explorer查看到服務出現Warning。全部的錯誤消息為:

SF Explorer中查看狀態
SF副本節點中的全部狀態錯誤

‘System.RA’ reported Warning for property ‘ReplicaOpenStatus’. Replica had multiple failures during open on _ggamenode_0.

API call: IStatelessServiceInstance.Open(); Error = System.Collections.Generic.KeyNotFoundException (-2146232969)

The given key was not present in the dictionary. at System.ThrowHelper.ThrowKeyNotFoundException()

at System.Collections.Generic.Dictionary`2.get_Item(TKey key)

at Microsoft.AspNetCore.Mvc.Internal.DefaultAssemblyPartDiscoveryProvider.CandidateResolver.ComputeClassification(String dependency)

at Microsoft.AspNetCore.Mvc.Internal.DefaultAssemblyPartDiscoveryProvider.CandidateResolver.ComputeClassification(String dependency)

at Microsoft.AspNetCore.Mvc.Internal.DefaultAssemblyPartDiscoveryProvider.CandidateResolver.ComputeClassification(String dependency)

at Microsoft.AspNetCore.Mvc.Internal.DefaultAssemblyPartDiscoveryProvider.CandidateResolver.d__4.MoveNext()

at System.Linq.Enumerable.d__17`2.MoveNext() at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()

at Microsoft.Extensions.DependencyInjection.MvcCoreServiceCollectionExtensions.GetApplicationPartManager(IServiceCollection services)

at Microsoft.Extensions.DependencyInjection.MvcCoreServiceCollectionExtensions.AddMvcCore(IServiceCollection services)

at Microsoft.Extensions.DependencyInjection.MvcServiceCollectionExtensions.AddMvc(IServiceCollection services)

— End of stack trace from previous location where exception was thrown —

at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

at Microsoft.AspNetCore.Hosting.ConventionBasedStartup.ConfigureServices(IServiceCollection services)

at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureApplicationServices()

at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()

at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()

at Microsoft.ServiceFabric.Services.Communication.AspNetCore.AspNetCoreCommunicationListener.OpenAsync(CancellationToken cancellationToken)

at Microsoft.ServiceFabric.Services.Runtime.StatelessServiceInstanceAdapter.d__13.

 

問題分析及解決

在錯誤消息中,Microsoft.AspNetCore.Mvc組件拋出了「System.Collections.Generic.KeyNotFoundException (-2146232969)The given key was not present in the dictionary.」 異常。在Stack Overflow中,查到KeyNotFoundException是當前ASP.NET Core 2.1的一個已知Issue。

 

是因為在安裝 .Net Core 2.1後,與舊版本之間存在環境變量的依賴衝突問題。可以通過修改版本(如2.0)來避免這個問題。 

 

如在項目文件中修改Microsoft.AspNetCore.Mvc版本(PS: 最便捷的方式是在Visual Studio 2019 IDE中通過NuGet修改版本,它會同步更新相關依賴),

<PackageReference Include="Microsoft.AspNetCore.Mvc" Version="2.0.4" />
<PackageReference Include="Microsoft.AspNetCore.Server.Kestrel" Version="2.0.3" />

Microsoft.AspNetCore 2.0.4

Dependencies

而在升級AspNetCore的版本後,在項目中有些依賴也會同步升級(//www.nuget.org/packages/Microsoft.AspNetCore/2.0.4),所以想要注意以下幾點:

1) Microsoft.ServiceFabric.AspNetCore.WebListener升級後UseWebListener 已經被替換成UseHttpSys。(Link: //github.com/aspnet/Hosting/issues/1128)

 

2)實際使用中,發現UseHttpSys後需要的依賴包,與Microsoft.AspNetCore包含相同的依賴,而引起包衝突。應使用UseKestrel方法

        /// <summary>
        /// Optional override to create listeners (like tcp, http) for this service instance.
        /// </summary>
        /// <returns>The collection of listeners.</returns>
        protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
        {
            return new ServiceInstanceListener[]
            {
                new ServiceInstanceListener(serviceContext =>
                    new KestrelCommunicationListener(serviceContext, "ServiceEndpoint", (url, listener) =>
                    {
                        ServiceEventSource.Current.ServiceMessage(serviceContext, $"Starting Kestrel on {url}");

                        return new WebHostBuilder()
                                    .UseKestrel()
                                    .ConfigureServices(
                                        services => services
                                            .AddSingleton<StatelessServiceContext>(serviceContext))
                                    .UseContentRoot(Directory.GetCurrentDirectory())
                                    .UseStartup<Startup>()
                                    .UseServiceFabricIntegration(listener, ServiceFabricIntegrationOptions.None)
                                    .UseUrls(url)
                                    .Build();
                    }))
            };
        }

 

3) 部署包中依賴混亂,拋出加載Abstractions 1.1.1.0舊版本的Dll。正確的版本應該為2.0及以上

'System.RA' reported Warning for property 'ReplicaOpenStatus'. Replica had multiple failures during open on _ggamenode_0. 
API call: IStatelessServiceInstance.Open(); Error = System.IO.FileLoadException (-2146234304)
Could not load file or assembly
'Microsoft.AspNetCore.Hosting.Abstractions, Version=1.1.1.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
System.IO.FileLoadException (-2146234304) Could not load file or assembly
'
Microsoft.AspNetCore.Hosting.Abstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=adb9793829
at SmsServiceApi.SmsServiceApi.<>c.b__1_0(StatelessServiceContext serviceContext)
at Microsoft.ServiceFabric.Services.Runtime.StatelessServiceInstanceAdapter.d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown
--- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.ServiceFabric.Services.Runtime.StatelessServiceInstanceAdapter.d__0.MoveNext()
For more information see: https://aka.ms/sfhealth

在修改版本問題方面,除了在project的config文件中修改為正確的版本,還需要檢查打包後的部署包中Dll的版本。

 

參考資料

AspNet Core WebApi fails at startup with error System.Collections.Generic.KeyNotFoundException ://stackoverflow.com/questions/51446570/aspnet-core-webapi-fails-at-startup-with-error-system-collections-generic-keynot

Service fabric API cannot run in azure SF cluster : //github.com/microsoft/service-fabric-issues/issues/1190

Service Fabric & ASP.NET Core 2.0 fails to run/start site: //github.com/aspnet/Hosting/issues/1128

Application services not starting – KeyNotFoundException (AspNetCore 1.1): //github.com/microsoft/service-fabric-issues/issues/1086

Microsoft.AspNetCore: //www.nuget.org/packages/Microsoft.AspNetCore/2.0.4