Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #1610 unexpected field-error related to a well-formatted value example of a schema's boolean field using frictionless validate #1615

Open
wants to merge 24 commits into
base: main
Choose a base branch
from

Conversation

amelie-rondot
Copy link
Contributor

@amelie-rondot amelie-rondot commented Jan 5, 2024

This PR fixes the field-error which occured doing frictionless validate data.csv --schema schema.json with a TableSchema schema containing a BooleanField customised with 'trueValues' or 'falseValues' and 'example' value , for example:

schema.json
----
{
  "$schema": "https://frictionlessdata.io/schemas/table-schema.json",
  "fields": [
    {
      "name": "IsTrue",
      "type": "boolean",
      "trueValues": ["yes"],
      "falseValues": ["no"],
      "example": "yes"
    }
  ]
}
data.csv
----
isTrue
yes
no

This PR add tests cases to ensure that example value is correctly evaluated according to customized 'trueValues' or 'falseValues', as expected.

@roll
Copy link
Member

roll commented Jan 24, 2024

Thanks @amelie-rondot! Can you please fix the lining error?

@amelie-rondot
Copy link
Contributor Author

@roll thanks a lot for reviewing!
I committed linting fixes.

…on-boolean-field-with-true-or-false-values-customised
Copy link
Member

@roll roll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

I added a few comments

@@ -263,6 +280,12 @@ def metadata_validate(cls, descriptor: IDescriptor): # type: ignore
field = Class(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just try field = Class.from_descriptor(descriptor) to make it work in general

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not work with from_descriptor, it leads to a RecursionError.

if isinstance(descriptor, dict):
if "trueValues" in descriptor.keys():
assert descriptor["type"] == "boolean"
descriptor["trueValues"] = descriptor["trueValues"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand this block of code =)

BTW as Table Schema is an open spec regarding custom properties we can't disable true/falseValues for other fields

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, you're right. This block is now useless. I've just committed to remove it.

@amelie-rondot
Copy link
Contributor Author

Thanks @roll for reviewing! I've committed updates.

amelie-rondot and others added 2 commits January 30, 2024 08:20
@pierrecamilleri
Copy link

@roll

The change requested have been taken into account and the PR is ready for review !

I noticed some linting errors that seem unrelated to this review, and that I corrected in the other PR (should I have ?). I haven't corrected them in this one as it would mostly lead to merge conflicts. Tell me if you prefer me to do something about it.

…d-error-on-boolean-field-with-true-or-false-values-customised
`from_descriptor` leads to a `RecursionError`
@pierrecamilleri
Copy link

I merged the main branch, this PR is ready as well !

@pierrecamilleri
Copy link

@pdelboca, would it please be possible to prioritize the merges of this bug fix as well as PR #1633 ?
Validata currently needs to rely on a fork with these fixes to use frictionless v5, but we wouldn't want this situation to last...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants