[🐛 Bug]: Intercepting network response produces exception

[dokisha]

What happened?

Intercepting network response crashes the browser.

@titusfortner said:
the code is assuming responseHeaders will be there, but that's optional according to chromedevtools.github.io/devtools-protocol/tot/Fetch

How can we reproduce the issue?

#issue is reproduced by typing `irb` and pasting the code below

require "watir"
require "selenium/devtools"

Selenium::WebDriver::Chrome::Service.driver_path = '/chromedriver99.0.4844.51'
Selenium::WebDriver::Chrome.path = '/chrome99.0.4844.84/google-chrome'
options = Selenium::WebDriver::Chrome::Options.new
options.add_option("args", ["--no-sandbox"])

browser = Watir::Browser.new :chrome, options: options

log_file = File.open("/tmp/network.log", 'w')
browser.wd.intercept do |request, &continue|
    continue.call(request) do |response|
      log_file << "#{request.id} \t
                   #{request.method} \t
                   #{response.code} \t
                   #{request.url} \n"
    end
end

browser.goto "youtube.com" #this crashes the browser

Relevant log output

#<Thread:0x00005590f1bcda90@/selenium-webdriver-4.1.0/lib/selenium/webdriver/devtools.rb:145 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
        3: from /selenium-webdriver-4.1.0/lib/selenium/webdriver/devtools.rb:155:in `block in callback_thread'
        2: from /selenium-webdriver-4.1.0/lib/selenium/webdriver/common/driver_extensions/has_network_interception.rb:68:in `block in intercept'
        1: from /selenium-webdriver-4.1.0/lib/selenium/webdriver/common/driver_extensions/has_network_interception.rb:109:in `intercept_response'
/selenium-webdriver-4.1.0/lib/selenium/webdriver/devtools/response.rb:38:in `from': undefined method `each_with_object' for nil:NilClass (NoMethodError)

Operating System

Ubuntu

Selenium version

ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]

What are the browser(s) and version(s) where you see this issue?

Chrome 99.0.4844.84

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver99.0.4844.51

Are you using Selenium Grid?

No

[github-actions]

@dokisha, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[meetme2meat]

@titusfortner I'm facing the similar. If it works for you we can safeguard that (using a PR). replacing this

https://github.com/SeleniumHQ/selenium/blob/trunk/rb/lib/selenium/webdriver/devtools/response.rb#L38

params.fetch('responseHeaders', []).each_with_ob...

[titusfortner]

PRs very appreciated, ideally with a spec. Do we know when the response header won't be present?

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.