Discussion

Problems with Internet Connection Sharing after 2017-11 Windows 10 Update [Temporary fix found]

I have a Windows 10 machine set up with wired Ethernet  to the Internet and using its wireless card to act as an access point using a Windows Hosted WiFi Network

Last night (early hours of 16 November 2017) a Windows 10 update was installed which prevented this working.

Rolling back to an automatic restore point from 13 November fixed the problem. However, the next evening the update was automatically reinstalled and connection sharing broke again.

Some investigation showed that the Internet Connection Sharing (ICS) service could not complete its startup. Services shows that it was stuck in the starting state. Attempting to stop the service had no effect. Killing the process stopped the service but attempting to restart the service put it into the same state.

Looking in Task Manager showed that two svchost processes were, between them, consuming one CPU's worth of activity. One was ICS the other hosted three services, most notably Windows Firewall. One was at 15% CPU and one was at 10% CPU (this system has four logical processors). It looks like they've got into a fight where one service is asking the other service to do something then going idle waiting for a response. The other processes the request and returns, presumably, a failure. At which point the first service tries again.

I've discovered that restarting the Windows Firewall service while the system is in this state clears the problem and the internet connection appears to be shared correctly. This seems to work as a temporary fix but I suspect (but have not yet tried) that on reboot the problem will reappear. I say this as stopping and restarting the ICS service makes the problem come back (and then stopping and starting Windows Firewall again allows ICS to restart but then the network connection has to be unshared and reshared to recover).

Task Manager wait chain analysis shows that the ICS service is sometimes waiting for the other service's process.

A lot of updates were installed overnight. However, looking at the list of updates that were installed most were Office related so I would guess that one of KB4049011 or KB4048954 was responsible, probably the latter.

My system is Windows 10 Home 64 bit Version 1703 (OS Build 15063.726)

-----

Some technical information in case it gives someone a clue (and I've got nowhere else to write it down where it might to anyone any use):

Process Explorer says that the ICS thread with the start address of sechost.dll!BuildSecurityDescriptorForSharingAccessEx+0x6c0 is consuming 10% CPU and one of the Windows Firewall (MpsSvc) threads with the start address of ntdll.dll!TpReleaseWork+0x2c0 is consuming 15% CPU.

Attempting to catch the threads in the act of doing something interesting has failed. I've always just caught them in the act of marshalling RPC records or waiting for synchronisation.

Process Monitor says that the last event it caught ICS doing before the CPU moved to MpsSvc was reading HKLM\SOFTWARE\Microsoft\Rpc\SecurityService\DefaultAuthLevel. The first thing it catches MpsSvc doing when it gets control is reading HKLM\System\CurrentControlSet\Control\Lsa\FipsAlgorithmPolicy.

Process Monitor grabs stacks when tracks what's going on. So, although it's not exactly at the point of failures, it might be close:

Frame  Module           Location                                         Address             Path
0      ntoskrnl.exe     SeLockSubjectContext+0x3355                      0xfffff80352f32bc5  C:\WINDOWS\system32\ntoskrnl.exe
1      ntoskrnl.exe     ObOpenObjectByNameEx+0x1966                      0xfffff80352f3ab96  C:\WINDOWS\system32\ntoskrnl.exe
2      ntoskrnl.exe     setjmpex+0x3b63                                  0xfffff80352c0b413  C:\WINDOWS\system32\ntoskrnl.exe
3      ntdll.dll        NtQueryValueKey+0x14                             0x7ffc679d5684      C:\WINDOWS\SYSTEM32\ntdll.dll
4      KERNELBASE.dll   MapPredefinedHandleInternal+0xa6e                0x7ffc6497d60e      C:\WINDOWS\System32\KERNELBASE.dll
5      KERNELBASE.dll   RegQueryValueExA+0x120                           0x7ffc6497a8b0      C:\WINDOWS\System32\KERNELBASE.dll
6      KERNELBASE.dll   SHRegCloseUSKey+0xff                             0x7ffc6494e70f      C:\WINDOWS\System32\KERNELBASE.dll
7      KERNELBASE.dll   RegGetValueA+0x79                                0x7ffc6494e809      C:\WINDOWS\System32\KERNELBASE.dll
8      RPCRT4.dll       RpcSsContextLockExclusive+0xef8                  0x7ffc656c1438      C:\WINDOWS\System32\RPCRT4.dll
9      RPCRT4.dll       RpcSsContextLockExclusive+0xe6a                  0x7ffc656c13aa      C:\WINDOWS\System32\RPCRT4.dll
10     RPCRT4.dll       RpcBindingSetAuthInfoExW+0x32c                   0x7ffc656bb1dc      C:\WINDOWS\System32\RPCRT4.dll
11     FirewallAPI.dll  FWOpenPolicyStore+0xc89                          0x7ffc62f05539      C:\Windows\System32\FirewallAPI.dll
12     FirewallAPI.dll  FWOpenPolicyStore+0xb1b                          0x7ffc62f053cb      C:\Windows\System32\FirewallAPI.dll
13     FirewallAPI.dll  FWOpenPolicyStore+0x4ff                          0x7ffc62f04daf      C:\Windows\System32\FirewallAPI.dll
14     FirewallAPI.dll  FWGetGlobalConfig2+0x101                         0x7ffc62f0c701      C:\Windows\System32\FirewallAPI.dll
15     FirewallAPI.dll  FWOpenPolicyStore+0x1b4                          0x7ffc62f04a64      C:\Windows\System32\FirewallAPI.dll
16     FirewallAPI.dll  FWEnumFirewallRules+0x231                        0x7ffc62f09651      C:\Windows\System32\FirewallAPI.dll
17     FirewallAPI.dll  IcfIsPortAllowed+0xb7c5                          0x7ffc62f2d365      C:\Windows\System32\FirewallAPI.dll
18     FirewallAPI.dll  IcfIsPortAllowed+0xba5f                          0x7ffc62f2d5ff      C:\Windows\System32\FirewallAPI.dll
19     ipnathlp.dll     ServiceMain+0x1c2dc                              0x7ffc52ee552c      c:\windows\system32\ipnathlp.dll
20     ipnathlp.dll     ServiceMain+0x111ff                              0x7ffc52eda44f      c:\windows\system32\ipnathlp.dll
21     ipnathlp.dll     ServiceMain+0x10b32                              0x7ffc52ed9d82      c:\windows\system32\ipnathlp.dll
22     ipnathlp.dll     NatShutdownTranslator+0x2c916                    0x7ffc52ec61a6      c:\windows\system32\ipnathlp.dll
23     ipnathlp.dll     ServiceMain+0x42b                                0x7ffc52ec967b      c:\windows\system32\ipnathlp.dll
24     ipnathlp.dll     ServiceMain+0x114d                               0x7ffc52eca39d      c:\windows\system32\ipnathlp.dll
25     ipnathlp.dll     ServiceMain+0x145                                0x7ffc52ec9395      c:\windows\system32\ipnathlp.dll
26     svchost.exe      svchost.exe+0x284b                               0x7ff663d1284b      C:\WINDOWS\System32\svchost.exe
27     sechost.dll      BuildSecurityDescriptorForSharingAccessEx+0x6e2  0x7ffc678d76f2      C:\WINDOWS\System32\sechost.dll
28     KERNEL32.DLL     BaseThreadInitThunk+0x14                         0x7ffc655c2774      C:\WINDOWS\System32\KERNEL32.DLL
29     ntdll.dll        RtlUserThreadStart+0x21                          0x7ffc679a0d51      C:\WINDOWS\SYSTEM32\ntdll.dll

If I look at a range of stacks captured by Process Monitor, it appears that the frame numbered 19 in that trace (the last one in ipnathlp.dll:ServiceMain) is the important one. The system never seems to get back to that frame whereas the next frame (the first one in FirewallAPI.dll:IcfIsPortAllowed) shows different addresses in different captures.

The version of ipnathlp.dll is 10.0.15063.726

The version of FirewallAPI.dll is 10.0.15063.674



* Please try a lower page number.

* Please enter only numbers.

* Please try a lower page number.

* Please enter only numbers.

After much more searching around, I've found other people reporting the same problem including one person who had found a fix that worked for them:

Another person had the problem and reports the same fix worked:

And two other people with the same problem after the same update:


I've now been brave enough to reset my firewall policy and it seems to have fixed the problem.

I did the reset with:

Control Panel->System and Security->Windows Firewall->Advanced Settings->Actions->Restore Default Policy

Before I did this I exported the current policy d (Actions->Export Policy...) so I could restore it if anything went wrong and I also exported the rules (Inbound Rules->Actions->Export List..., Oubound Rules->Actions->Export List ... and Monitoring->Firewall->Actions-Export List...) so I had a record I could use as a basis for reinstalling rules if saw problems later.

I now have two .wfw files, one from before the policy reset (1.5 MB) and one from after (0.2 MB) and if I install the one I saved I can make the problem come back. So this is definitely the issue.

Other statistics:

Before the reset I had 1623 inbound firewall rules, 964 outbound firewall rules and 1440 monitoring rules, afterwards just 205, 144 and 48 respectively. Note that the latter counts are immediately after policy reset and before ICS had been re-established. When ICS gets set up again the counts change.

There are a lot of duplicate rules, nearly all associated with Internet Connection Sharing. I had over 160 repeats of some ICS rules (although, interestingly one is repeated just 138 times). In contrast, after reset there's just one duplicated rule (one AllJoyn rule in each of the three sets).

After deduplication, my pre-reset counts are 384, 346 and 201 respectively.

Since ICS is a service so is not associated with a user, I can strip out all rules that are associated with an individual user account (after reset there are none of these).

After deduplication and user rule stripping my counts are 297, 175 and 120 respectively.


I've narrowed it down to the inbound firewall rules for the Internet Connection Sharing service.

If I delete all 1243 inbound rules (I said there were a lot of duplicates) then the problem goes away.

There are eight distinct inbound rules listed in the exported .txt file (two appear 138 times, five 161 times and one 162 times).

[For reference, there are four distinct outbound rules, one appears 138 times, three 161 times for a total of 621 rules.]

If, after deleting all the ICS inbound rules, I then unshare and reshare the network connection the eight rules reappear each one with a duplicate (for a total of 16 rules).

It now looks like it's the duplicates themselves that were the problem, rather than what they contained. It's possible some of the rules were damaged in some way that doesn't show in the .txt rule export but, if so, I can't tell.

Anyway, it looks like for me a minimal fix is to delete all the ICS rules (Inbound and Outbound to be safe; this causes all the ones under monitoring to disappear too). I need to ignore a message that it couldn't delete one copy of each rule because it did actually manage to delete them. This will leave the rest of my firewall rules intact. It also allows the ICS service to complete its startup. Then unshare and reshare the network connection to get the rules back.

If I unshare my network connection and then reshare it I get an extra copy of all the rules. It looks like there's a rule leak somewhere. So, I suspect this problem will come back eventually.

I think I've taken this as far as I can.

It'd be interesting to know if other people with the same problem also have large numbers of duplicate firewall rules. I'm also going to keep an eye on my firewall rules to see if duplicates gradually creep in.


More recently created threads with people with the same problem:

The following are old threads so may have been different issues but people have been following up to them recently so the follow-ups may be related to the more recent problems:




And another couple of recent ones:


> I think I've taken this as far as I can.

OK, I've been looking for an excuse to practice my Powershell.

I've been running the following sequence:

  • Ensure the ICS service is stopped
  • Reload the saved Windows Firewall policy (.wfw file) I exported
  • Delete a semi-random set of the ICS rules
    • The rules all have Name properties that are GUIDs that appear to be randomly distributed so I used the first character of the GUID to select by sixteenths so I can select repeatably.
    • I'm deleting only rules with a Profile property of "Any" as opposed to the more modern "Domain, Private, Public" because for historical reasons I initially thought those might be the problem and never updated the script not to filter on this.
    • I've tried two sets of GUID match regexps one is {[0-n] and one was {[n-F] where "n" is the changing range limit.
  • Time how long it takes the service to start.

The results are interesting. Starting the ICS service is close to order N-squared in the number of ICS firewall rules. If the ICS service is then stopped and restarted then the restart is always fast indicating that this is a one off processing cost. You have to reload the the rules (.wfw file) to get consistent measurements.

If a small enough number of rules are deleting (or, perhaps more pertinently, if a large enough number of rules are left) then the service will never start.

The highest startup time I can measure before the service fails to start is about 43 seconds. Perhaps there's a 45 second timeout somewhere. Having said that, if a service takes 45 seconds to start, that's a bug in itself so turning up the timeout isn't the answer.

Total Rules ICS Rules Startup time (seconds)
1005 282 1.54
1104 381 1.79
1188 465 4.30
1284 561 5.56
1397 674 8.60
1500 777 11.58
1583 860 13.39
1683 960 17.17
1759 1036 19.86
1866 1143 23.21
1959 1236 26.90
2070 1347 30.01
2172 1449 33.67
2272 1549 37.69
2387 1664 43.00
2485 1762 Gave up after twenty minutes

In passing, I'll note that most ICS rules have a Group property of "@ipnathlp.dll,-140" but a small number (presumably of older rules) have "@hnetcfg.dll,-140". As already noted, there's a difference in Profile properties. Apart from those differences, which are insignificant as far as I can tell, all the apparently duplicate rules are genuinely duplicates (apart from the GUID). I've checked using the exported rule .txt files, Powershell (including checking Filters) and Registry settings under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SharedAccess (the latter also shows a version difference aligned with the Profile difference).


youre a life saver, I did the restore on windows firewall policy and it finally worked. even as a temporary solution. this is way better than nothing. I don't know if it will still be working once I reboot my laptop.. hopefully it will still work. btw thanks a lot keep on sharing


Thank you for the research and the solution to this problem.  I have spent several days fighting this problem.  Hyperv works great since I followed your solution.  thanks again. 

* Please try a lower page number.

* Please enter only numbers.

* Please try a lower page number.

* Please enter only numbers.

 
Discussion Info

Views: 9,017 Last updated: June 17, 2018 Applies to: