Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: convert StreamedResultSet to Pandas Dataframe #226
base: main
Are you sure you want to change the base?
feat: convert StreamedResultSet to Pandas Dataframe #226
Changes from 12 commits
483e9fb
362c143
fc5ab60
cb5733a
0aee1d8
e7a4d4c
537d498
b4d715e
f219b0d
021b3c2
a584fa0
a83a7ce
be5c241
4057e76
ad1d039
f460174
7906b14
f919276
0028b7b
0104348
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we please use a more specific exception?
Exception
is way too broad.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, please add some comments about why we have these two different methods of converting to datetime. I don't understand it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also also, please add back in the "localize" logic to the
TIMESTAMP
type. TIMESTAMP is intended to represent UTC datetimes.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also also also, we need tests for the
datetime64[ns, UTC]
data type. I want to make sure the data type is the one we expect in the test (not just that we got a row and it didn't error)For example, in pandas-gbq we test that the dataframe we got matches one that we manually created and localized in the test.
https://github.com/pydata/pandas-gbq/blob/46c579ac21879b431c8568b49e68624f4a5ea25e/tests/unit/test_timestamp.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tests are failing because this import is incorrect.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it. Changed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reusing the
contacts
table would be more appropriate than creating our own.python-spanner/tests/_fixtures.py
Line 19 in e990ff7
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like the name from the BigQuery Ibis tests. Please use the existing
all_types
table frompython-spanner/tests/_fixtures.py
Line 31 in e990ff7
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Optional] In fact, we may be able to import test data from here:
python-spanner/tests/system/test_system.py
Line 1096 in e990ff7
For example, the DB-API tests import some useful test fixtures.
python-spanner/tests/system/test_system_dbapi.py
Lines 32 to 39 in e990ff7
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indentation looks off. Can you run
nox -s blacken
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done