Enabling network service

[deepak1556]

This is a meta bug to track all the work required to use the network service from chromium, which is being experimented in canary and stable channels at the moment. To use the service, pass --enable-features=NetworkService

What is network service ?

As a part of chrome modularization project, network code is being moved from browser process into a separate process.

High level Design doc: https://docs.google.com/document/d/1wAHLw9h7gGuqJNCgG1mP1BmLtCGfZ2pys-PdZQ1vg7M/edit?pref=2&pli=1#heading=h.w406cmqiibno

Upstream meta bug: https://crbug.com/598073

Changes required in Electron

As we move to consuming the service, we will start to remove all usages from //net and use interfaces from //services/network. So existing terminologies like URLRequest, URLRequestContext, URLRequestJob, URLFetcher will now be URLLoaderRequest, NetworkContext, URLLoaderJob, URLLoader

• Link network service code
  • Build //services/network
• Create legacy URLRequestContext with network service glue code
  • Use URLRequestContextBuilderMojo to populate request context params
  • Upstream related changes: https://crbug.com/825242, https://crbug.com/715695
  • Changes made in PR: refactor: create request context from network context #14656
• Use Cookie Manager to track cookie changes - P1
  • Upstream related changes: https://crbug.com/721395
  • Update atom::api::Cookies to use cookie store from network context
  • Changes made in PR: TODO
• Update proxy configuration with network service
  • Upstream related changes: https://crbug.com/715697, https://crbug.com/717253, https://crbug.com/754007
  • Changes made in PR: refactor: create request context from network context #14656
• Update legacy net::URLRequestContext consumers in atom::api::Session module - P2
  • Upstream related changes: https://crbug.com/837333
  • Changes made in PR: TODO
• Update atom::api::WebRequest module - P2
  • Upstream related changes: https://crbug.com/721414
  • Changes made in PR: TODO
• Update atom::api::net module - P3
  • Convert URLRequest variations into URLLoaderRequest
  • Upstream example changes: https://crbug.com/715632
  • Remove patch: expose_net_observer_api.patch
  • Changes made in PR: TODO
• Update net::URLFetcher to use URLLoader - P1
  - Upstream related changes: https://crbug.com/746977, https://crbug.com/773295
  - In-Progress PR: [WIP] refactor: Migrate net::URLFetcher callsites over SimpleURLLoader #15705
• Verify devtools - P2
  • Verify devtools network throttling
  • Upstream related changes: https://crbug.com/721408
  • Changes made in PR: TODO
• Ensure file://, filesystem:// and blob:// work as expected - P2
  • Upstream related changes: https://crbug.com/715677, https://crbug.com/759230
  • Changes made in PR: TODO
• Customize CORB handling with URLLoadFactoryParams - P2
  • Upstream related changes: https://crbug.com/792546
  • Remove patch: fix: bypass CORB when web security is disabled #15737
  • Changes made in PR: TODO
• Ensure graceful shutdown of network process
  • Verify all network context are destroyed before primary network context
  • Upstream related changes: https://crbug.com/820996
• Ensure sandbox of network process
  • Upstream related changes: https://crbug.com/715679
• Enable encrypted cookies
• Update AtomNavigationThrottle - P3
  • Upstream related changes: https://crbug.com/756642
  • Changes made in PR: TODO
• Verify tests pass on all platforms

[deepak1556]

Chromium 74 has this service enabled by default on all desktop platforms, will work on enabling it following the upgrade.

[deepak1556]

The next phase is to remove usage of net::URLRequestContext and net::URLRequestContextGetter in the codebase. The following user facing modules are the primary consumers

• Obtain NetworkContext from StoragePartition, since we have 1:1 relationship with BrowserContext, prefer the following

content::BrowserContext::GetDefaultStoragePartition(browser_context)->GetNetworkContext()

or we have a new class atom::NetworkContextService that has 1:1 relationship with
BrowserContext , add a getter there and use it. Consider this the new version of
URLRequestContextGetter

1. Session Module (Assigned: @nornagon)
   • Session::DoCacheAction
     • Clear Cache - Refactor with NetworkContext::ClearHttpCache
     • Get cache size - Refactor with NetworkContext::ComputeHttpCacheSize
   • Session::ClearHostResolverCache - Refactor with NetworkContext::ClearHostCache
   • Session::ClearAuthCache - Refactor with NetworkContext::ClearHttpAuthCache
   • Session::SetUserAgent - No interface available, this is worth patching in upstream
   • Session::AllowNTLMCredentialsForDomains - No interface available in network_context.mojom to modify this dynamically, however we do provide a chance to customize it during network context creation as part of NetworkContextParams, investigate benefit of this dynamic api to patch the interface and proposing a fix in upstream.
   • Session::SetCertVerifyProc - No interface available for the same reasons mentioned above.

• The following apis that manipulate cookies must first retrieve the cookie manager using NetworkContext::GetCookieManager and use the interfaces defined in cookie_manager.mojom

2. Cookies Module (Assigned: @nornagon)

   • Cookies::Get - Refactor with CookieManager::GetCookieList
   • Cookies::Remove - Refactor with CookieManager::DeleteCookies
   • Cookies::Set - Refactor with CookieManager::SetCanonicalCookie
   • Cookies::FlushStore - Refactor with CookieManager::FlushCookieStore
   • allow custom cookie schemes this doesn't work properly in the pre-network-service code, so no urgent need to support it in the new code.

3. WebRequest Module (Assigned: @zcbenz )

   • Refactor NetworkDelegate https://bugs.chromium.org/p/chromium/issues/detail?id=721414

4. Protocol Module (Assigned: @zcbenz) - Preferred migration route

   • Register/Intercept APIs
   • protocol.registerSchemesAsPrivileged

5. Net Module (Assigned: @zcbenz)

   • Use URLLoader instead of net::URLRequest

6. net::URLRequestFetchJob special case

   • Port to use SimpleURLLoader https://bugs.chromium.org/p/chromium/issues/detail?id=773295
   • @zcbenz: No need to work on it as it is no longer used in the new implementation of protocol module.

THIS WILL BE PART II , remaining refactors will followup.

[deepak1556]

Couple more tasks at the end of Part II

• verify graceful shutdown of the network service process. At the moment it isn't

[86722:0320/233317.511490:ERROR:message_pump_kqueue.cc(345)] StopWatchingFileDescriptor kevent64: No such file or directory (2)
[86719:0320/233317.541776:ERROR:browser_process_sub_thread.cc(210)] Waited 24 ms for network service
[86719:0320/233317.542885:WARNING:discardable_shared_memory_manager.cc(410)] Some MojoDiscardableSharedMemoryManagerImpls are still alive. They will be leaked.
[86719:0320/233317.554350:ERROR:process_posix.cc(330)] Unable to terminate process 86722: No such process (3)
[86719:0320/233317.556644:ERROR:message_pump_kqueue.cc(345)] StopWatchingFileDescriptor kevent64: No such file or directory (2)

Related bug: https://bugs.chromium.org/p/chromium/issues/detail?id=820996

• Allow sandboxing the network service process
  • MacOS - https://bugs.chromium.org/p/chromium/issues/detail?id=915910
    • The seatbelt sandbox profile declaration takes the user data directory from the //content layer through content::ContentBrowserClient::GetNetworkContextsParentDirectory to allow write properties. We should override this.
  • Windows - https://bugs.chromium.org/p/chromium/issues/detail?id=841001
  • Linux - https://bugs.chromium.org/p/chromium/issues/detail?id=715679

[deepak1556]

Now that network service is enabled by default on all platforms in chrome, the old code path is being deleted. https://bugs.chromium.org/p/chromium/issues/detail?id=934009 is the tracking bug for it.

[deepak1556]

Service is on by default on master, #19602 has followup tasks.