RFC: Custom Fetch Handler

[cometkim]

Preview

[cometkim]

Considering: should it handle XMLHttpRequest too?

[cometkim]

https://stackoverflow.com/a/64359741/5734400

It seems handling AbortSignal in the Service Worker is still UB... :/

---

Since signal is passed through the request interface as-is, its behavior can be determined at the user level anyway.

[cometkim]

Considering: should it handle XMLHttpRequest too?

Having support for XMLHttpRequest is necessary for better compatibilities, due to the popular HTTP client libraries like superagent (8M downloads/week) and axios (44M downloads/week) still use/support it.

It may not be that difficult, see https://github.com/kitsonk/xhr for example.

However, the async=false should throw exception since it cannot be implemented in a single tread context. Or at least it should be a seperated handler.

[cometkim]

GPT-4 response:

Looking at the proposed design strategy, it provides a way for the user to delegate fetch requests to different contexts. This bears similarities with the fetch handler of a Service Worker but differs in that it should be able to customize origin constraints.

The proposed FetchObject interface provides a fetch request and various methods necessary for handling the request.

A new FetchObject instance can be created using the register method of the FetchHandlerModule. And the created FetchObject includes a fetch method for making and handling fetch requests and an onfetch method for listening to events.

The FetchObject is essentially a ponyfill to the ServiceWorkerGlobalScope. The methods of the FetchObject basically perform network requests similar to the Fetch API, Beacon API, and XMLHttpRequest API. However, these methods can be intercepted by the fetch event listeners.

From the usage examples, one can confirm that a FetchObject is created using the register method of FetchHandler, and the fetch method of this FetchObject is used to replace the original Fetch API.

Overall, this design is well organized and has a clear purpose and implementation strategy. However, in order for this implementation to actually work, the following points need to be considered:

1. Whether the FetchObject can actually replace the ServiceWorkerGlobalScope: While the FetchObject has a similar interface to the ServiceWorkerGlobalScope, it needs careful consideration as to whether it can implement or replace all the features and behaviors of the ServiceWorkerGlobalScope.

2. Whether they can perfectly mimic the behavior of the Fetch API, Beacon API, and XMLHttpRequest API: These three APIs have complex and subtle behaviors, and mimicking these behaviors accurately can be challenging.

3. Issues related to security: It should carefully handle issues related to origin policy, CORS, and other security-related matters. This implementation should adequately handle requests from different origins.

4. Whether this design is compatible with all web browsers: It needs to be verified if this design is compatible with all web browsers. The behavior of the Fetch API or other APIs may operate differently on specific web browsers, and unexpected problems may occur if these differences are not considered.

5. Error handling: The current design is missing parts regarding error handling. Network requests can fail, and an error handling mechanism is necessary to prepare for these cases.

[cometkim]

Prompt: For your point 1, I should say the FetchObject is designed to be "subset" of the ServiceWorkerGlobalScope, and no need to support full interface of it. I think it's enough to get interop with existing fetch handlers for service workers. Is that makes sense?

GPT-4 response:

Absolutely, that does make sense. If your primary goal is to support interoperability with existing fetch handlers for service workers, creating a FetchObject that supports a subset of the ServiceWorkerGlobalScope should be sufficient. This approach can offer the key functionalities you need, while potentially simplifying the implementation and reducing overhead.

Just keep in mind to clearly document the supported subset of the ServiceWorkerGlobalScope functionalities in the FetchObject. This ensures users of the FetchObject understand its capabilities and limitations, and use it correctly. Additionally, even if it's a subset, the subset functionalities should still mimic the behavior of the ServiceWorkerGlobalScope as accurately as possible to ensure expected behavior.

[cometkim]

Prompt: for your point 2, indeed it can be tricky at some points, but we don't have to implement its full behavior because custom fetch handler is bit more like proxy to original stub. If there still are any difficult or complex points to be expected, please let me know.

GPT-4 response:

That's correct. As a proxy to the original stub, the custom fetch handler doesn't need to replicate every aspect of the Fetch API, Beacon API, and XMLHttpRequest API behavior. Rather, its main role is to intercept requests and allow for custom processing, before potentially passing requests onto the original implementation.

However, a few complexities could still arise in the design:

1. Asynchronous processing: Fetch API and XMLHttpRequest are inherently asynchronous operations. It's important to ensure that your custom fetch handler handles asynchronous tasks properly and doesn't inadvertently introduce blocking behavior.

2. Error and exception handling: Network requests can fail for a variety of reasons. Your custom fetch handler should be able to handle these errors gracefully and propagate them in a way that's consistent with the behavior of the original APIs.

3. Intercepting all types of requests: Fetch API can be used to make various types of requests, including GET, POST, PUT, DELETE, etc. Your custom fetch handler should be able to handle all these types of requests if you want it to be a full-fledged fetch handler.

4. Support for advanced Fetch API features: Fetch API supports a number of advanced features such as streaming responses, aborting requests, etc. Depending on your use case, you may also need to support these in your custom fetch handler.

5. CORS and security considerations: When implementing any form of request interception, it's crucial to consider security implications. Make sure your custom fetch handler respects the same-origin policy and handles CORS properly.

[cometkim]

Prompt: About to CORS you mentioned, this is intentionally designed to work in a cross-origin context, so the user can use it to override the relevant policy if desired. At least the custom fetch handler is treated as a "trusted code" by the first party.

GPT-4 response:

Understood, that makes sense. If your FetchObject is designed to work in a cross-origin context and is considered "trusted code" by the first party, then it can indeed have the capability to override default policies as needed.

However, even in trusted environments, it's still important to maintain good security practices. Even if CORS restrictions are loosened, it's vital to ensure that proper authentication, validation, and error-handling mechanisms are in place. This helps prevent potential security issues, such as data leaks, cross-site request forgery (CSRF), or injection attacks.

Always ensure that the code handling cross-origin requests is secure, tested, and monitored for potential abuse. It might be useful to provide some form of access control or restriction to make sure only authorized parties are allowed to use the custom fetch handler, and provide robust logging and alerting mechanisms to detect any misuse or anomalies.

[cometkim]

Prompt: In other words, a custom fetch handler can never be implemented in a way that overrides the default security policy, as it is still on top of the same browser environment. One of the possible scenarios is using a controlled MessagePort to communicate with another thread, which is not a security policy violation.

GPT-4 response:

Indeed, you're correct. Custom fetch handlers, even if they are designed to work in a cross-origin context, still operate within the confines of the web browser environment and its security model. The browser's inbuilt security measures, such as same-origin policy and CORS, cannot be overridden at a fundamental level by JavaScript code running within the browser.

Using a controlled MessagePort for cross-thread communication is a common and accepted practice in web development. This is a mechanism provided by the browser's Web Workers API and is designed to safely enable communication between different execution contexts (like the main thread and worker threads) without violating the browser's security policies. This is achieved through structured cloning of messages, which ensures that only safe, serializable data is transferred between threads, and direct object references are not shared.

In the context of a custom fetch handler, it means that while you can intercept and modify requests within the constraints of the Fetch API, any modifications or actions that would violate the browser's security policies would still be blocked. Therefore, the responsibility for implementing and managing additional security measures lies primarily with the developer, ensuring that the custom fetch handler is used in a secure and responsible manner.

[cometkim]

EventSource which make a HTTP request for SSE(Server-Sent Events) should also be intercepted by the service worker