Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLN: Enforce read_csv(keep_date_col, parse_dates) deprecations #58622

Merged
merged 18 commits into from May 10, 2024

Conversation

mroeschke
Copy link
Member

xref #56569

@mroeschke mroeschke added IO CSV read_csv, to_csv Clean labels May 7, 2024
@mroeschke mroeschke added this to the 3.0 milestone May 8, 2024
pandas/io/parsers/base_parser.py Show resolved Hide resolved
cols_needed = []

cols_needed = list(cols_needed)
if not isinstance(self.parse_dates, list):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch feels a bit odd in this method - this isn't handled by the caller at all?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah appears not. I suppose since this is called by the c parser and python parser maybe this was here to consolidate logic?

list(columns), self.index_col
)
self._name_processed = True
date_index = self._get_complex_date_index(data, columns)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can _get_complex_date_index be removed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Removed

warnings.warn(
"The 'keep_date_col' keyword in pd.read_csv is deprecated and "
"will be removed in a future version. Explicitly remove unwanted "
"columns after parsing instead.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think i screwed up this deprecation. the message here suggests that the post-deprecation behavior will retain the un-parsed column, which i think is not the case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I don't think the workaround messaging was quite right, but I think it's OK since we didn't get any issues AFAICT where people were confused by this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plus this keyword only really made sense when parse_dates specified multiple columns which was clearly deprecated

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good, thanks for taking a look

depr = True
if depr:
warnings.warn(
"Support for nested sequences for 'parse_dates' in pd.read_csv "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you keep any tests that current get here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah there are tests where parse_dates is a list and valid and a list and not valid

@mroeschke
Copy link
Member Author

Going to merge this in. Happy to follow up if needed

@mroeschke mroeschke merged commit a17d449 into pandas-dev:main May 10, 2024
47 checks passed
@mroeschke mroeschke deleted the cln/csv_date_parsing1 branch May 10, 2024 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Clean IO CSV read_csv, to_csv
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants