Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force minimum of one header row; Add to docs #596

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

brynpickering
Copy link
Member

Fixes #573

Based on the decision in #573, I have forced the need for a header row by setting header=[0] if a user does not provide columns data in their data source config.

It can lead to a difficult to understand failure. When a user forgets a header row, data ends up as the column / index level names. This gets picked up later when Calliope tries to match index/column levels to what has been defined in the config. That seems to be the only place we can catch it, but it leads to an error like this:

calliope.exceptions.ModelError: (data_sources, ds_name) | Trying to set names for index but names in the file do no match names provided | in file: ['bar1', 'bar2'] | defined: ['test_row1', 'test_row2'].

Maybe just adding to that error with something like "Make sure you have a header row in your CSV file"?

Summary of changes in this pull request

  • Force header=[0] as a minimum
  • Added to docs with a bit about how to structure the CSV
  • Updated tests

Reviewer checklist

  • Test(s) added to cover contribution
  • Documentation updated
  • Changelog updated
  • Coverage maintained or improved

Copy link

codecov bot commented May 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.96%. Comparing base (872978d) to head (65badad).
Report is 13 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #596      +/-   ##
==========================================
+ Coverage   95.85%   95.96%   +0.10%     
==========================================
  Files          24       24              
  Lines        3619     3666      +47     
  Branches      788      735      -53     
==========================================
+ Hits         3469     3518      +49     
+ Misses         86       84       -2     
  Partials       64       64              
Files Coverage Δ
src/calliope/preprocess/data_sources.py 100.00% <100.00%> (ø)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Multiple rows and columns introduce false index items
1 participant