Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

URLEncodedForm date encoding strategy #2273

Merged
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
4c6301f
Added ability to configure date coding/decoding for UrlEncodedForm
thecheatah Mar 29, 2020
a318cac
Added documentation to the `DateFormat` enum
thecheatah Mar 29, 2020
569205e
renamed internetDateTime to iso8601
thecheatah Mar 29, 2020
47d275c
Added custom date formatter
thecheatah Mar 29, 2020
ad50caa
Added comment about `ISO8601DateFormatter` performance
thecheatah Mar 29, 2020
4f8165f
Added ISO8601DateFormatter.threadSpecific so a new ISO8601DateFormatt…
thecheatah Mar 29, 2020
f212661
Use typealias instead of declaring DateFormat 2x
thecheatah Mar 29, 2020
78908d1
Added `ThreadSpecificDateFormatter` to ensure thread safety when usin…
thecheatah Mar 29, 2020
b9bdedb
Fixed comment
thecheatah Mar 29, 2020
287dff9
Changed custom interface to mimic `JSONDecoder.DateDecodingStrategy.c…
thecheatah Mar 29, 2020
bbaf3e8
Removed unused ThreadSpecificDateFormatter
thecheatah Mar 29, 2020
73617bf
`ISO8601DateFormatter` seems to be thread safe. This bug was filed: h…
thecheatah Mar 30, 2020
2cdccb4
Default the date format for URLEncodedFrom coding/decoding to `timeIn…
thecheatah Mar 30, 2020
ab98e1e
Default to `timeIntervalSince1970`
thecheatah Mar 30, 2020
8b96414
Removed `timeIntervalSinceReferenceDate` option
thecheatah Mar 31, 2020
860d81a
Made `Date: URLQueryFragmentConvertible`
thecheatah Mar 31, 2020
70d35a6
Date decoding uses an array of DateFormats and tries in order
thecheatah Mar 31, 2020
be68616
Comments
thecheatah Mar 31, 2020
76ef4fe
Removed unnecessary import
thecheatah Mar 31, 2020
4750b51
Comments
thecheatah Mar 31, 2020
8c99199
Merged with Vapor 4.0.1
thecheatah Apr 11, 2020
d5cade4
Reintroduced threadSpecific `ISO8601DateFormatter`
thecheatah Apr 17, 2020
0138d63
Renamed unixTimestamp to secondsSince1970 to match Apple's naming con…
thecheatah Apr 17, 2020
d0fcd9a
Throw last error when decodingDate using multiple decoding approaches
thecheatah Apr 17, 2020
b62257b
Use `DateDecodingStrategy` and `DateEncodingStrategy` naming convention
thecheatah Apr 17, 2020
567f686
Updated test cases
thecheatah Apr 17, 2020
57144c5
Merge branch 'master' into Feature/2272-URL-Coding-Internet-Dates
thecheatah Apr 17, 2020
2a80df9
Updated documentation
thecheatah Apr 17, 2020
fe655cb
Added additional default test and updated documentation
thecheatah Apr 19, 2020
914b8b7
Improved error handling
thecheatah Apr 20, 2020
0c173b3
Removed unnecessary comment
thecheatah Apr 20, 2020
bca712a
Replaced dateDecodingStrategies with dateDecodingStrategy
thecheatah Apr 21, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
24 changes: 24 additions & 0 deletions Sources/Vapor/URLEncodedForm/DateFormatter+threadSpecific.swift
@@ -0,0 +1,24 @@
//
// File.swift
//
//
// Created by Ravneet Singh on 3/29/20.
thecheatah marked this conversation as resolved.
Show resolved Hide resolved
//

import NIO

fileprivate final class ISO8601 {
fileprivate static let thread: ThreadSpecificVariable<ISO8601DateFormatter> = .init()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't ISO8601DateFormatter thread safe itself? Making sure

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure. I found some discussion online that indicated otherwise. I can take this change back if you know or if we can confirm that ISO8601DateFormatter in Linux is thread safe.

I had found the following issue, but it seems like it's no longer reproducible (I didn't test myself): https://bugs.swift.org/browse/SR-7745?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aall-tabpanel

I didn't want to introduce something last minute with a threading issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Joannis I tried the attached project with 10000000 iterations on vapor/swift:5.2 docker image and didn't run into an issue. I will undo the multithreading.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}

extension ISO8601DateFormatter {
static var threadSpecific: ISO8601DateFormatter {
if let existing = ISO8601.thread.currentValue {
return existing
} else {
let new = ISO8601DateFormatter()
ISO8601.thread.currentValue = new
return new
}
}
}
38 changes: 35 additions & 3 deletions Sources/Vapor/URLEncodedForm/URLEncodedFormDecoder.swift
Expand Up @@ -13,9 +13,21 @@
public struct URLEncodedFormDecoder: ContentDecoder, URLQueryDecoder {
/// Used to capture URLForm Coding Configuration used for decoding
public struct Configuration {
/// Supported date formats
public enum DateFormat {
/// Seconds since 00:00:00 UTC on 1 January 2001
case timeIntervalSinceReferenceDate
/// Seconds since 00:00:00 UTC on 1 January 1970
case timeIntervalSince1970
/// ISO 8601 formatted date
case iso8601
/// Using custom callback
case custom((Decoder) throws -> Date)
}

let boolFlags: Bool
let arraySeparators: [Character]

let dateFormat: DateFormat
/// Creates a new `URLEncodedFormCodingConfiguration`.
/// - parameters:
/// - boolFlags: Set to `true` allows you to parse `flag1&flag2` as boolean variables
Expand All @@ -26,10 +38,12 @@ public struct URLEncodedFormDecoder: ContentDecoder, URLQueryDecoder {
/// populate a key named `arr` of type `Array` to be decoded as `["v1", "v2"]`
public init(
boolFlags: Bool = true,
arraySeparators: [Character] = [",", "|"]
arraySeparators: [Character] = [",", "|"],
dateFormat: DateFormat = .iso8601
) {
self.boolFlags = boolFlags
self.arraySeparators = arraySeparators
self.dateFormat = dateFormat
}
}

Expand Down Expand Up @@ -175,7 +189,25 @@ private struct _Decoder: Decoder {
}
} else {
let decoder = _Decoder(data: child, codingPath: self.codingPath + [key], configuration: configuration)
return try T(from: decoder)
if type == Date.self {
switch configuration.dateFormat {
case .timeIntervalSince1970:
return Date(timeIntervalSince1970: try TimeInterval(from: decoder)) as! T
case .timeIntervalSinceReferenceDate:
return Date(timeIntervalSinceReferenceDate: try TimeInterval(from: decoder)) as! T
case .iso8601:
//Creating a new `ISO8601DateFormatter` everytime is probably not performant
if let date = ISO8601DateFormatter.threadSpecific.date(from: try String(from: decoder)) {
return date as! T
} else {
throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: self.codingPath, debugDescription: "Unable to decode date. Expecting ISO8601 formatted date"))
}
case .custom(let callback):
return try callback(decoder) as! T
}
} else {
return try T(from: decoder)
}
}
}

Expand Down
38 changes: 35 additions & 3 deletions Sources/Vapor/URLEncodedForm/URLEncodedFormEncoder.swift
@@ -1,3 +1,5 @@
import NIO

/// Encodes `Encodable` instances to `application/x-www-form-urlencoded` data.
///
/// print(user) /// User
Expand Down Expand Up @@ -27,15 +29,31 @@ public struct URLEncodedFormEncoder: ContentEncoder, URLQueryEncoder {
case values
}

/// Supported date formats
public enum DateFormat {
/// Seconds since 00:00:00 UTC on 1 January 2001
case timeIntervalSinceReferenceDate
/// Seconds since 00:00:00 UTC on 1 January 1970
case timeIntervalSince1970
/// ISO 8601 formatted date
case iso8601
/// Using custom callback
case custom((Date, Encoder) throws -> Void)
}
/// Specified array encoding.
public var arrayEncoding: ArrayEncoding
public var dateFormat: DateFormat

/// Creates a new `Configuration`.
///
/// - parameters:
/// - arrayEncoding: Specified array encoding. Defaults to `.bracket`.
public init(arrayEncoding: ArrayEncoding = .bracket) {
public init(
arrayEncoding: ArrayEncoding = .bracket,
dateFormat: DateFormat = .iso8601
) {
self.arrayEncoding = arrayEncoding
self.dateFormat = dateFormat
}
}

Expand Down Expand Up @@ -167,11 +185,25 @@ private class _Encoder: Encoder {
func encode<T>(_ value: T, forKey key: Key) throws
where T : Encodable
{
if let convertible = value as? URLQueryFragmentConvertible {
if let convertible = value as? URLQueryFragmentConvertible {
internalData.children[key.stringValue] = URLEncodedFormData(values: [convertible.urlQueryFragmentValue])
} else {
let encoder = _Encoder(codingPath: self.codingPath + [key], configuration: self.configuration)
try value.encode(to: encoder)
if let date = value as? Date {
switch configuration.dateFormat {
case .timeIntervalSince1970:
try date.timeIntervalSince1970.encode(to: encoder)
case .timeIntervalSinceReferenceDate:
try date.timeIntervalSinceReferenceDate.encode(to: encoder)
case .iso8601:
//Creating a new `ISO8601DateFormatter` everytime is probably not performant
try ISO8601DateFormatter.threadSpecific.string(from: date).encode(to: encoder)
case .custom(let callback):
try callback(date, encoder)
}
} else {
try value.encode(to: encoder)
}
self.internalData.children[key.stringValue] = try encoder.getData()
}
}
Expand Down
84 changes: 84 additions & 0 deletions Tests/VaporTests/URLEncodedFormTests.swift
@@ -1,5 +1,6 @@
@testable import Vapor
import XCTest
import NIO

final class URLEncodedFormTests: XCTestCase {
// MARK: Codable
Expand Down Expand Up @@ -114,6 +115,85 @@ final class URLEncodedFormTests: XCTestCase {
XCTAssert(result.contains("isCool=true"))
}

func testDateCoding() throws {
let toEncode = DateCoding(date: Date(timeIntervalSince1970: 0))
let resultForTimeIntervalSince1970 = try URLEncodedFormEncoder(
configuration: .init(dateFormat: .timeIntervalSince1970)
).encode(toEncode)
XCTAssertEqual("date=0.0", resultForTimeIntervalSince1970)

let decodedTimeIntervalSince1970 = try URLEncodedFormDecoder(
configuration: .init(dateFormat: .timeIntervalSince1970)
).decode(DateCoding.self, from: resultForTimeIntervalSince1970)
XCTAssertEqual(decodedTimeIntervalSince1970, toEncode)

let resultForTimeIntervalSinceReferenceDate = try URLEncodedFormEncoder(
configuration: .init(dateFormat: .timeIntervalSinceReferenceDate)
).encode(toEncode)
XCTAssertEqual("date=-978307200.0", resultForTimeIntervalSinceReferenceDate)

let decodedTimeIntervalSinceReferenceDate = try URLEncodedFormDecoder(
configuration: .init(dateFormat: .timeIntervalSinceReferenceDate)
).decode(DateCoding.self, from: resultForTimeIntervalSinceReferenceDate)
XCTAssertEqual(decodedTimeIntervalSinceReferenceDate, toEncode)

let resultForInternetDateTime = try URLEncodedFormEncoder(
configuration: .init(dateFormat: .iso8601)
).encode(toEncode)
XCTAssertEqual("date=1970-01-01T00:00:00Z", resultForInternetDateTime)

let decodedInternetDateTime = try URLEncodedFormDecoder(
configuration: .init(dateFormat: .iso8601)
).decode(DateCoding.self, from: resultForInternetDateTime)
XCTAssertEqual(decodedInternetDateTime, toEncode)

XCTAssertThrowsError(try URLEncodedFormDecoder(
configuration: .init(dateFormat: .iso8601)
).decode(DateCoding.self, from: "date=bad-date"))

class DateFormatterFactory {
private var threadSpecificValue = ThreadSpecificVariable<DateFormatter>()
var currentValue: DateFormatter {
get {
guard let dateFormatter = threadSpecificValue.currentValue else {
let threadSpecificDateFormatter = self.newDateFormatter
threadSpecificValue.currentValue = threadSpecificDateFormatter
return threadSpecificDateFormatter
}
return dateFormatter
}
}

private var newDateFormatter: DateFormatter {
let dateFormatter = DateFormatter()
dateFormatter.locale = Locale(identifier: "en_US_POSIX")
dateFormatter.dateFormat = "'Date:' yyyy-MM-dd 'Time:' HH:mm:ss 'Timezone:' ZZZZZ"
dateFormatter.timeZone = TimeZone(secondsFromGMT: 0)
return dateFormatter
}
}
let factory = DateFormatterFactory()
let resultCustom = try URLEncodedFormEncoder(
configuration: .init(dateFormat: .custom({ (date, encoder) in
var container = encoder.singleValueContainer()
try container.encode(factory.currentValue.string(from: date))
}))
).encode(toEncode)
XCTAssertEqual("date=Date:%201970-01-01%20Time:%2000:00:00%20Timezone:%20Z", resultCustom)

let decodedCustom = try URLEncodedFormDecoder(
configuration: .init(dateFormat: .custom({ (decoder) -> Date in
let container = try decoder.singleValueContainer()
let string = try container.decode(String.self)
guard let date = factory.currentValue.date(from: string) else {
throw DecodingError.dataCorruptedError(in: container, debugDescription: "Unable to decode date from string '\(string)'")
}
return date
}))
).decode(DateCoding.self, from: resultCustom)
XCTAssertEqual(decodedCustom, toEncode)
}

func testEncodedArrayValues() throws {
let user = User(name: "Tanner", age: 23, pets: ["Zizek", "Foo"], dict: ["a": 1, "b": 2], foos: [.baz], nums: [3.14], isCool: true)
let result = try URLEncodedFormEncoder(
Expand Down Expand Up @@ -469,3 +549,7 @@ private struct Users: Codable, Equatable {
private enum Foo: String, Codable {
case foo, bar, baz
}

struct DateCoding: Codable, Equatable {
let date: Date
}