Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arrays of enums #91

Closed
flowbe opened this issue Apr 25, 2019 · 5 comments · Fixed by #119
Closed

Arrays of enums #91

flowbe opened this issue Apr 25, 2019 · 5 comments · Fixed by #119
Assignees

Comments

@flowbe
Copy link
Contributor

flowbe commented Apr 25, 2019

I am trying to decode this XML:

<?xml version="1.0" encoding="UTF-8"?>
<container>
    <p>
        <run>
            <id>1518</id>
            <text>I am answering it again.</text>
        </run>
        <properties>
            <id>431</id>
            <title>A Word About Wake Times&#xD;</title>
        </properties>
        <br />
    </p>
    <p>
        <run>
            <id>1519</id>
            <text>I am answering it again.</text>
        </run>
    </p>
</container>

This is my implementation:

struct Run: Decodable {
    let id: Int
    let text: String
}

struct Properties: Decodable {
    let id: Int
    let title: String
}

struct Break: Decodable {
    
}

enum Entry: Decodable {
    case run(Run)
    case properties(Properties)
    case br(Break)
    
    private enum CodingKeys: String, CodingKey {
        case run, properties, br
    }
    
    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        do {
            self = .run(try container.decode(Run.self, forKey: .run))
        } catch DecodingError.keyNotFound {
            do {
                self = .properties(try container.decode(Properties.self, forKey: .properties))
            } catch DecodingError.keyNotFound {
                self = .br(try container.decode(Break.self, forKey: .br))
            }
        }
    }
}

struct Paragraph: Decodable {
    let entries: [Entry]
    
    public init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        entries = try container.decode([Entry].self)
    }
}

do {
    let result = try XMLDecoder().decode([Paragraph].self, from: data)
    for paragraph in result {
        print("Paragraph :")
        for entry in paragraph.entries {
            switch entry {
            case .run(let run):
                print("Run : \(run)")
            case .properties(let properties):
                print("Properties : \(properties)")
            case .br(let br):
                print("Break : \(br)")
            }
        }
        print("End paragraph")
    }
}
catch {
    print(error)
}

But I am getting an odd DecodingError.keyNotFound

I know that there is already an issue about enums but I think this one is different because I am trying to do an array of array of enums and I don't get the same error.

I am using an enum because I want to be able to decode a <p> with different optional elements and keep the order in which they are.

Can you tell me if this is a bug from the library or from my own code? Can achieve what I want to do by some other way?

Thank you!

@MaxDesiatov MaxDesiatov self-assigned this Apr 28, 2019
@MaxDesiatov
Copy link
Collaborator

Hi @TheFlow95, thank you for providing the snippets. There's an additional issue here caused by a missing struct that reflects the XML:

struct Container: Decodable {
    let paragraphs: [Paragraph]

    enum CodingKeys: String, CodingKey {
        case paragraphs = "p"
    }
}

After adding it and decoding into Container.self instead of [Paragraph].self there are no more errors thrown. The other problem though is that Entry.init(from:) uses a keyed container, but reads only a single key from it. This means that only a single Entry is decoded into a property of Paragraph. I'm currently looking for a best way to solve this, will keep you updated. Thanks!

@flowbe
Copy link
Contributor Author

flowbe commented Apr 29, 2019

Thanks for your answer, I'm looking forward to have a fix for this!

@flowbe
Copy link
Contributor Author

flowbe commented Apr 29, 2019

By playing with the library, I ran into another bug related to this one. When trying to decoding the following XML with the same code as before:

<?xml version="1.0" encoding="UTF-8"?>
 <container>
    <p>
        <run>
            <id>1518</id>
            <text>I am answering it again.</text>
        </run>
        <properties>
            <id>431</id>
            <title>A Word About Wake Times&#xD;</title>
        </properties>
        <br/>
        <run>
            <id>1519</id>
            <text>Hello!</text>
        </run>
    </p>
    <p>
        <run>
            <id>1520</id>
            <text>I am answering it again.</text>
        </run>
    </p>
</container>

I get the following error:

typeMismatch(Swift.Dictionary<Swift.String, Any>, Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "p", intValue: nil), XMLKey(stringValue: "0", intValue: 0), XMLKey(stringValue: "0", intValue: 0), XMLKey(stringValue: "0", intValue: 0), XMLKey(stringValue: "0", intValue: 0), CodingKeys(stringValue: "run", intValue: nil)], debugDescription: "Expected to decode Dictionary<String, Any> but found SharedBox instead.", underlyingError: nil))

I think that the error is because I have two run elements into the first p but I thought that it would handle it. Can you help?

@MaxDesiatov
Copy link
Collaborator

MaxDesiatov commented Apr 30, 2019

After spending some time on this I think that using singleValueContainer for Paragraph here wouldn't work. We need to preserve the order and duplicate keys in the container, so the best fit would be to rely on the standard unkeyed container in the compiler-generated init(from: Decoder) for Paragraph. On the other hand, Entry should be able to read the name of its element to determine the actual enum value in its own init(from: Decoder).

This means we need to pass through the element name into the keyed container somehow and we can't get it from the key in the parent Paragraph.init(from: Decoder) as we're required to use an unkeyed container there. Seems like Decodable/Encodable is a good fit for simpler formats like JSON, but can't express all these nuances of XML directly. This is similar to a problem we had previously with element values, which was resolved by introducing the value intrinsic.

Based on this, I'm thinking of adding a new intrinsic, potentially called xmlElementName. According to the XML spec, element and attribute names can't start with xml string, so users are very unlikely to need that name used for a real element or an attribute. During decoding you'd be able to get the element name via that key and initialize a proper enum case in Entry.init(from: Decoder). I'm open to considering other options and names for the intrinsic. Thanks!

@flowbe
Copy link
Contributor Author

flowbe commented May 2, 2019

Sounds good to me!

MaxDesiatov added a commit that referenced this issue May 22, 2019
`UnkeyedBox` is a wrapper for `[Box]`, the assumption is that adding direct `Box` conformance to `[Box]` would simplify things a bit. Also, `KeyedStorage` now always stores `[Box]` per key without any special handling for single items per key to simplify this even more. In addition, order of values is preserved for all keys as required for #91 and #25.

This should also unblock #101 providing unified handling for elements without any content and attributes.

By pure serendipity this also fixes tests introduced in #38.

* Replace UnkeyedBox with Array, refine KeyedStorage
* Fix more of the broken tests
* One unfixed test left in benchmarks
* Single out failing benchmark in a separate test
* Fix all tests 🎉
* Fix compiler warning
* Fix Xcode 10.1 compilation error
* Remove unused AnyArray protocol
* Remove unused elementType function
* Simplify code to improve test coverage
MaxDesiatov pushed a commit that referenced this issue Jul 30, 2019
## Introduction

This PR introduces the `XMLChoiceCodingKey` protocol, which enables the encoding and decoding of union-type–like enums with associated values to and from `XML` choice elements.

Resolves #25.
Resolves #91.

## Motivation

XML schemas support [choice](https://www.w3schools.com/xml/el_choice.asp) elements, which constrain their contents to a single instance of a member of a known set of types. Choice elements exhibit the properties of [union types](https://en.wikipedia.org/wiki/Union_type) and can be represented in Swift as enums with associated values, wherein each case of the enum carries with it a single associated value that is one of the types representable.

An example of how such a type is implemented in Swift:
   
```Swift
enum IntOrString {
    case int(Int)
    case string(String)
}
```

There is currently no automatic synthesis of the `Codable` protocol requirements for enums with assocated types in today's Swift. As such, it is required to provide custom implementations of the `init(from: Decoder)` initializer and the `encode(to: Encoder)` method to conform to the `Encodable` and `Decodable` protocols, respectively.

When encoding to and decoding from `JSON`, a single-element keyed container is created that uses the enum case name as the single key.

An example of adding Codable conformance to such a type when working with JSON
   
```Swift
extension IntOrString: Codable {
    enum CodingKeys: String, CodingKey { case int, string }
    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        do {
            self = .int(try container.decode(Int.self, forKey: .int))
        } catch {
            self = .string(try container.decode(String.self, forKey: .string))
        }
    }
    func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        switch self {
        case let .int(value):
            try container.encode(value, forKey: .int)
        case let .string(value):
            try container.encode(value, forKey: .string)
        }
    }
}
```

This may not be the most handsome approach, but it does the job without imprinting any format-specfic logic onto otherwise format-agnostic types.

This pattern works out of the box with the `JSONEncoder` and `JSONDecoder` provided by the `Foundation` framework. However, due to syntactic characteristics of the `XML` format, this pattern will **_not_** work automatically for encoding and decoding `XML`-formatted data, regardless of the tool used.

## Proposed solution

The proposed solution is to define a new `XMLChoiceCodingKey` protocol:

```Swift
/// An empty marker protocol that can be used in place of `CodingKey`.
/// It must be used when conforming a union-type–like enum with associated values to `Codable`
/// when the encoded format is `XML`.
public protocol XMLChoiceCodingKey: CodingKey {}
```

The `XMLChoiceCodingKey` protocol inherits from `CodingKey` and adds no new requirements. This conformance can be made retroactively, without additional implementation.

An example usage:
    
```Swift
extension IntOrString.CodingKeys: XMLChoiceCodingKey {}
``` 

## Detailed design

This proposal adds a single `public` `protocol` `XMLChoiceCodingKey`, as well as several `internal` types.

Under the hood, the `XMLChoiceEncodingContainer` and `XMLChoiceDecodingContainer` are used to provide `encode` and `decode` methods tuned for `XML` choice elements.

Because of the characteristics of the `XML` format, there are some ambiguities (from an encoding and decoding perpsective) between unkeyed container elements that contain choice elements and those that contain nested unkeyed container elements.

In order to untangle these ambiguities, the new container types utilize a couple of new `Box` types to redirect elements along the encoding and decoding process.

## Source compatibility

This is purely an additive change.
MaxDesiatov pushed a commit that referenced this issue Jul 31, 2019
## Introduction

In merging in #119, we fixed most but not quite all of #91! Decoding of _null_ choice elements (represented as enums with empty struct associated values on the Swift side) still results in errors. This PR adds a test to demonstrate and fixes this issue by wrapping each `NullBox` inside of a `SingleKeyedBox` at the `merge` phase (called by `transformToBoxTree`).

## Motivation

One of the main lessons from #119 was that we have to wrap choice elements in the decoding phase to hold onto their keys. The keys are needed for directing us to the correct branch of the do-catch pyramid used for decoding. 

```swift
private enum Entry: Equatable {
    case run(Run)
    case properties(Properties)
    case br(Break)
}

extension Entry: Decodable {
    private enum CodingKeys: String, XMLChoiceCodingKey {
        case run, properties, br
    }

    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        do {
            self = .run(try container.decode(Run.self, forKey: .run))
        } catch {
            do {
                self = .properties(try container.decode(Properties.self, forKey: .properties))
            } catch {
                self = .br(try container.decode(Break.self, forKey: .br))
            }
        }
    }
}
```
where one of the associated values could be an empty struct (represented by null):

```swift
private struct Break: Decodable {}
```

Although we _can_ throw out keys for non-choice null elements, a mechanism is needed for holding onto the keys while transforming from the `XMLCoderElement` tree to the `boxTree`. Only later will we know if the key is needed (if this wrapped element is transformed to a `ChoiceBox`); if not, we will be able to throw out the key. 

## Proposed solution

The Public API is unchanged. On the implementation side, we catch `NullBox` values in `merge` and wrap them in `SingleKeyedBox` instances. 

## Detailed Design

In `merge`, we wrap each `NullBox` in a `SingleKeyedBox` with the appropriate key bundled in. An `XMLChoiceDecodingContainer` can be constructed from the `SingleKeyedBox` by converting it to a `ChoiceBox` (just transferring over the contents) - as normal. In `XMLKeyedDecodingContainer`, when preparing the `elements` for concrete decoding, we unwrap all `SingleKeyedBox` values that may be contained therein, as any choice elements contained would have already been transformed to a `ChoiceBox` by this point in decoding: any stray `SingleKeyedBox` wrappers can thus be thrown out. 

## Source compatibility

This is purely an additive change.
arjungupta0107 pushed a commit to salido/XMLCoder that referenced this issue Jun 26, 2020
`UnkeyedBox` is a wrapper for `[Box]`, the assumption is that adding direct `Box` conformance to `[Box]` would simplify things a bit. Also, `KeyedStorage` now always stores `[Box]` per key without any special handling for single items per key to simplify this even more. In addition, order of values is preserved for all keys as required for CoreOffice#91 and CoreOffice#25.

This should also unblock CoreOffice#101 providing unified handling for elements without any content and attributes.

By pure serendipity this also fixes tests introduced in CoreOffice#38.

* Replace UnkeyedBox with Array, refine KeyedStorage
* Fix more of the broken tests
* One unfixed test left in benchmarks
* Single out failing benchmark in a separate test
* Fix all tests 🎉
* Fix compiler warning
* Fix Xcode 10.1 compilation error
* Remove unused AnyArray protocol
* Remove unused elementType function
* Simplify code to improve test coverage
arjungupta0107 pushed a commit to salido/XMLCoder that referenced this issue Jun 26, 2020
## Introduction

This PR introduces the `XMLChoiceCodingKey` protocol, which enables the encoding and decoding of union-type–like enums with associated values to and from `XML` choice elements.

Resolves CoreOffice#25.
Resolves CoreOffice#91.

## Motivation

XML schemas support [choice](https://www.w3schools.com/xml/el_choice.asp) elements, which constrain their contents to a single instance of a member of a known set of types. Choice elements exhibit the properties of [union types](https://en.wikipedia.org/wiki/Union_type) and can be represented in Swift as enums with associated values, wherein each case of the enum carries with it a single associated value that is one of the types representable.

An example of how such a type is implemented in Swift:
   
```Swift
enum IntOrString {
    case int(Int)
    case string(String)
}
```

There is currently no automatic synthesis of the `Codable` protocol requirements for enums with assocated types in today's Swift. As such, it is required to provide custom implementations of the `init(from: Decoder)` initializer and the `encode(to: Encoder)` method to conform to the `Encodable` and `Decodable` protocols, respectively.

When encoding to and decoding from `JSON`, a single-element keyed container is created that uses the enum case name as the single key.

An example of adding Codable conformance to such a type when working with JSON
   
```Swift
extension IntOrString: Codable {
    enum CodingKeys: String, CodingKey { case int, string }
    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        do {
            self = .int(try container.decode(Int.self, forKey: .int))
        } catch {
            self = .string(try container.decode(String.self, forKey: .string))
        }
    }
    func encode(to encoder: Encoder) throws {
        var container = encoder.container(keyedBy: CodingKeys.self)
        switch self {
        case let .int(value):
            try container.encode(value, forKey: .int)
        case let .string(value):
            try container.encode(value, forKey: .string)
        }
    }
}
```

This may not be the most handsome approach, but it does the job without imprinting any format-specfic logic onto otherwise format-agnostic types.

This pattern works out of the box with the `JSONEncoder` and `JSONDecoder` provided by the `Foundation` framework. However, due to syntactic characteristics of the `XML` format, this pattern will **_not_** work automatically for encoding and decoding `XML`-formatted data, regardless of the tool used.

## Proposed solution

The proposed solution is to define a new `XMLChoiceCodingKey` protocol:

```Swift
/// An empty marker protocol that can be used in place of `CodingKey`.
/// It must be used when conforming a union-type–like enum with associated values to `Codable`
/// when the encoded format is `XML`.
public protocol XMLChoiceCodingKey: CodingKey {}
```

The `XMLChoiceCodingKey` protocol inherits from `CodingKey` and adds no new requirements. This conformance can be made retroactively, without additional implementation.

An example usage:
    
```Swift
extension IntOrString.CodingKeys: XMLChoiceCodingKey {}
``` 

## Detailed design

This proposal adds a single `public` `protocol` `XMLChoiceCodingKey`, as well as several `internal` types.

Under the hood, the `XMLChoiceEncodingContainer` and `XMLChoiceDecodingContainer` are used to provide `encode` and `decode` methods tuned for `XML` choice elements.

Because of the characteristics of the `XML` format, there are some ambiguities (from an encoding and decoding perpsective) between unkeyed container elements that contain choice elements and those that contain nested unkeyed container elements.

In order to untangle these ambiguities, the new container types utilize a couple of new `Box` types to redirect elements along the encoding and decoding process.

## Source compatibility

This is purely an additive change.
arjungupta0107 pushed a commit to salido/XMLCoder that referenced this issue Jun 26, 2020
## Introduction

In merging in CoreOffice#119, we fixed most but not quite all of CoreOffice#91! Decoding of _null_ choice elements (represented as enums with empty struct associated values on the Swift side) still results in errors. This PR adds a test to demonstrate and fixes this issue by wrapping each `NullBox` inside of a `SingleKeyedBox` at the `merge` phase (called by `transformToBoxTree`).

## Motivation

One of the main lessons from CoreOffice#119 was that we have to wrap choice elements in the decoding phase to hold onto their keys. The keys are needed for directing us to the correct branch of the do-catch pyramid used for decoding. 

```swift
private enum Entry: Equatable {
    case run(Run)
    case properties(Properties)
    case br(Break)
}

extension Entry: Decodable {
    private enum CodingKeys: String, XMLChoiceCodingKey {
        case run, properties, br
    }

    public init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        do {
            self = .run(try container.decode(Run.self, forKey: .run))
        } catch {
            do {
                self = .properties(try container.decode(Properties.self, forKey: .properties))
            } catch {
                self = .br(try container.decode(Break.self, forKey: .br))
            }
        }
    }
}
```
where one of the associated values could be an empty struct (represented by null):

```swift
private struct Break: Decodable {}
```

Although we _can_ throw out keys for non-choice null elements, a mechanism is needed for holding onto the keys while transforming from the `XMLCoderElement` tree to the `boxTree`. Only later will we know if the key is needed (if this wrapped element is transformed to a `ChoiceBox`); if not, we will be able to throw out the key. 

## Proposed solution

The Public API is unchanged. On the implementation side, we catch `NullBox` values in `merge` and wrap them in `SingleKeyedBox` instances. 

## Detailed Design

In `merge`, we wrap each `NullBox` in a `SingleKeyedBox` with the appropriate key bundled in. An `XMLChoiceDecodingContainer` can be constructed from the `SingleKeyedBox` by converting it to a `ChoiceBox` (just transferring over the contents) - as normal. In `XMLKeyedDecodingContainer`, when preparing the `elements` for concrete decoding, we unwrap all `SingleKeyedBox` values that may be contained therein, as any choice elements contained would have already been transformed to a `ChoiceBox` by this point in decoding: any stray `SingleKeyedBox` wrappers can thus be thrown out. 

## Source compatibility

This is purely an additive change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants