Route_schedule with calendars issue for services starting on the first day of a time_change

[eturck]

Hello!

We've encountered a problem on our navitia instance for timetable. Each line in this kraken is a version of a classic commercial line, so we have some things out of the ordinary, like a line starting on March 20th.
On this instance we're using the grid_rel_calendar_trip NTFS file to associate services to grid calendar and requesting route_schedules with each calendar to make this (each block is a calendar)

The line is starting on March 20th, and each block will have separate services (even if it's the same departure times, a service running on Saturdays will be different than the one running on week days, only because we're taking data from HASTUS doing it this way). So there is a service on Saturday 25th, starting at 25:00:00 on its first stop. The NTFS is :

stop	departure	arrival
stop1	25:00	25:00
stop2	25:05	25:05
stop3	25:10	25:10

When we parse the NTFS :

• The calendars are parsed, and multiple validity patterns are created because of DST
  • There is one for March 25th only (dst_1),
  • And another one for the future (dst_2).
• The stop_times are parsed, and we shift them to UTC. For this we get the current utc offset of the validity_pattern. For dst_1 it's 3600, dst2 7200.

We have something like :

vj	stop	departure	arrival	vp
vj_dst1	stop1	24:00	24:00	...000000000000000100000
vj_dst1	stop2	24:05	24:05	...000000000000000100000
vj_dst1	stop3	24:10	24:10	...000000000000000100000
vj_dst2	stop1	23:00	23:00	...001000000100000000000
vj_dst2	stop2	23:05	23:05	...001000000100000000000
vj_dst2	stop3	23:10	23:10	...001000000100000000000

After this, in Data::shift_stop_times we make sure that the first stop_time is in [0:00; 24:00[, so vj_dst1 needs to be shifted one day. We then have in ed (vp is shifted to the left for vj_dst1):

vj	stop	departure	arrival	vp
vj_dst1	stop1	0:00	0:00	...000000000000001000000
vj_dst1	stop2	0:05	0:05	...000000000000001000000
vj_dst1	stop3	0:10	0:10	...000000000000001000000
vj_dst2	stop1	23:00	23:00	...001000000100000000000
vj_dst2	stop2	23:05	23:05	...001000000100000000000
vj_dst2	stop3	23:10	23:10	...001000000100000000000

And the problem now is, in route_schedules, when we getting all the stop_times by calendar, we're doing something like

result = st.departure_time + tz_handler->get_first_utc_offset(vj->vp)

And since the first active day of the validity_pattern is Sunday March 26th, the UTC offset returned is 7200, and we're displaying a departure at 2:00.

I made a failing unit test Tisseo/navitia@2c55d39 in a branch : https://github.com/Tisseo/navitia/tree/issue_route_schedules_calendar_dst, it's may be easier to understand the problem.

I don't know the right way to solve this, and maybe there is other usecases where the problem arises. We discussed this with @stifoon and @TeXitoi yesterday, they had some idea. The main one was to use a library doing the job for us, but a comment made by @antoine-de three years made us think that was maybe not a good idea : https://github.com/CanalTP/navitia/blob/cf2a6067fe159bd39921f12e5b6a3656b84e4522/source/ed/connectors/tz_db_wrapper.h
It's really a specific case and not that important, but if there is some ideas I'm open to fix this when I have a bit of time.

[pbougue]

🕑 👍 👎 ❓
My guess is you don't want to mess with TZ unless you have something really important to win...
Maybe @antoine-de (who struggled a lot with TZ) will have some ideas about that, but I'm not even sure there's a lib knowing hours of change and using a lib might break some other things.

[eturck]

I don't have something huge to win on this, but there is still a bug I had to fix by modifying some data in ed directly, and I'm sure it will happen again.
I know implementing a lib would probably break some things, that's why I made the issue to have some feedback of everybody. Maybe the best way is to do it with the current timezone manager, in tz_db_wrapper we seem to have the hours of change : https://github.com/CanalTP/navitia/blob/dev/source/ed/connectors/tz_db_wrapper.cpp ?