Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data_* Cleanup #2336

Open
10 tasks
bvanessen opened this issue Oct 2, 2023 · 0 comments
Open
10 tasks

Data_* Cleanup #2336

bvanessen opened this issue Oct 2, 2023 · 0 comments
Assignees
@bvanessen @tbennun @benson31

Comments

@bvanessen
Copy link
Collaborator

bvanessen commented Oct 2, 2023 •
edited by tbennun

  • Input layer output size is grabbed directly from data coordinator
  • remove access to the global mini-batch size - all mini-batch buffer sizes are derived
  • identify, correct, or deprecate JAG data reader issues that are encumbering process
  • multi-key-value input data reader
  • remove data_* from everywhere
  • callbacks???
  • input layer no longer modifies data reader / coordinator state. Move to outside of model
  • Get sample indices per mb
  • get num iterations per epoch - move to SGB
  • The location of shuffle_required should be moved out of data reader metadata
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bvanessen bvanessen

@tbennun tbennun

@benson31 benson31

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bvanessen @tbennun @benson31