URLFormEncoder does not escape all reserved characters

[davedelong]

Describe the issue

URLFormEncoder should be encoding characters like $, but does not

Vapor version

4.92.5

Operating system and version

macOS 14.3

Swift version

5.9.0

Steps to reproduce

let form = ["hello": "world$"]
let encoder = try ContentConfiguration.global.requireEncoder(for: .urlEncodedForm)
var buffer = ByteBuffer()
var headers = HTTPHeaders()
try encoder.encode(form, to: &buffer, headers: &headers)

print(String(bytes: ByteBufferView(buffer), encoding: .utf8)!)

Outcome

This should print hello=world%24. Instead, it prints hello=world$.

Additional notes

According to Wikipedia, the reserved characters include $ and should be percent-escaped.

[davedelong]

I believe that this CharacterSet is not excluding all the characters it should be.

[0xTim]

Ok me and @gwynne did some diving into this and it's definitely a bug - I have no idea how it's been undiscovered this long. There doesn't appear to be a good actual RFC spec to refer to, but the best we've found is https://url.spec.whatwg.org/#application-x-www-form-urlencoded-percent-encode-set

The application/x-www-form-urlencoded percent-encode set contains all code points, except the ASCII alphanumeric, U+002A (*), U+002D (-), U+002E (.), and U+005F (_).

[0xTim]

Will schedule a fix for this asap