Improve way "homerooms" are created and assigned to educators

[kevinrobinson]

This particularly impacts New Bedford. This is related to #1879 as well.

There's a separate case where this happens, for H-TUT. So the assumptions in the EducatorsImporter about setting the Homeroom seem to be wrong for NB (and maybe for Somerville too, we'd have to check).

  /home/newbedford$ cat istaff.csv | grep '"H-TUT"' | wc -l
  8

This is true for at least 86 other values for NB...

  /home/newbedford$ cat istaff.csv | rev |  cut -d',' -f4 | rev | sort | uniq -c | sort -rn
   1254 ""
     24 "0-000"
      8 "H-TUT"
      5 "K-004"
      ...

For New Bedford, they appear to scope these by school, and re-use values like "K-004" across schools. So the homeroom name is a not a valid primary key across the district, which our code assumes it is (https://github.com/studentinsights/studentinsights/blob/master/app/importers/file_importers/educators_importer.rb#L96).

For Somerville most of the homeroom names are prefixed with the school initials so it's only for two values, which I'd guess is a data bug.

So overall, this needs more work. It impacts which students teachers are authorized to see, so is probably critical before September. We could look at inferring our own primary key, or I wonder if we can just export Aspen primary keys to avoid a lot of this matching work.

[alexsoble]

@kevinrobinson Looking into this further — I ran the following script against production New Bedford data to see if there are any homeroom teachers currently assigned to incorrect homerooms:

Educator.all.each do |educator|
  if educator.homeroom.present? && educator.school.present?
    raise 'School mismatch!!' if educator.school != educator.homeroom.school
  end
end ; nil

The script didn't raise, and I did some manual inspection too to verify the same thing. So for the two schools currently imported, no New Bedford educator account is assigned to an incorrect homeroom.

[kevinrobinson]

@alexsoble nice, awesome to check this! I fixed all this up manually when I removed all the other students and educators and homerooms (I think! this is great to check :))

One tiny clarification - I think it might be risky to consider these homerooms correct. They are still identified with names that won't let us discriminate between homerooms across schools. And the way that homerooms get created or updated is in the student importer, where it's based on the student's school. So I think to check this all the way around it involves looking at whether the students in that homeroom also match. There's that circular dependency in the importers between students, educators and homerooms. EducatorsImporter runs first, and that expects homerooms to exist. Then StudentsImporter runs and that creates the student records and homeroom records.

[kevinrobinson]

Possibly related, about the school filter: #1894, #1891.

[kevinrobinson]

#2023 fixed some things, but there are a few other edge cases impacting different educators/students/homerooms:

• because of past configurations, there are stale Homeroom records (eg, created from a Student import row, that wouldn't be created now)
• in Somerville the SIS has semantically confusing data (eg, student in the Brown school but the CAP-141 homeroom). This gets created at CAP-141 within the Brown (from Validate educators assigned to correct homerooms #1994) which is isolated and good, but misleading in Insights as it is in the SIS. Probably nothing to do here, might be better to mirror bad SIS data and surface it than hide it.
• handle stale Student records (eg, NB#443 who moved to a school that is not imported, and so is never updated or removed - maybe mark them as archived?)
• handle stale Educator records (mark them as archived?)

And these need doing, but probably as separate issues:

• multiple educators with homerooms are valid, would need to change model
• remove homeroom#grade since it doesn't make sense (or migrate to homeroom#grades if necessary)