tools.web: We should deprecate the world

[dgw]

Filing this after terribot's encouragement earlier in the evening:

<+dgw> !how much deprecation?
<+terribot> dgw: 816 deprecations

On a more serious note, we had a look at what's actually useful in the tools.web module. The answer? Not much.

In my fact-based-but-still-opinionated view, the only possibly-useful utilities left are search_urls(), trim_url(), iri_to_uri() (maybe), USER_AGENT, and DEFAULT_HEADERS.

• search_urls() is used in core code (but should probably be called find_urls() or extract_urls() instead)
• trim_url() is used in search_urls()
• iri_to_uri() seems helpful vs. just using encodings.idna because it understands the whole URL and IDN-encodes components of the domain name + urlencodes non-ASCII components of the path
• urlencode_non_ascii is probably replaceable with something in stdlib, but I haven't found anything yet
• USER_AGENT and DEFAULT_HEADERS are for plugin authors' convenience
  • DEFAULT_HEADERS should also be better insulated against being modified by plugins (i.e. some module-level equivalent to a property would be great, something that looks like a module attribute but returns a copy of its default-header dictionary)
  • Or we could go the simple way and replace DEFAULT_HEADERS with a function like get_default_headers() that returns a copy plugins can tamper with without affecting other code

Unless I've missed some other important stuff, that's going to leave a pretty barebones module behind. We might consider moving the remaining bits around (again; sopel.web moved to sopel.tools.web in #1616 / 7.0) but I'd feel weird about having the few bits and bobs left live in the root sopel.tools module.

Plenty of time to think about this stuff, though.