leader election: a records's observedTime is checked instead of an actual timestamp

[kdomanski]

in https://github.com/kubernetes/client-go/blob/master/tools/leaderelection/leaderelection.go#L269 the client makes sure that at least LeaseDuration has expired since the election record is observed and the leader doesn't renew the lease in this time.

However this means that the client will always wait at least LeaseDuration on startup, since observedTime is only ever set to time.Now() and the actual age of the record is irrelevant.

The client should instead take the record's AcquireTime and RenewTime under consideration when waiting for hearbeat, and thus instantly acquire the lease if the last record is already expired.

[liggitt]

I don't think this is correct. The recorded times are more for debugging and observability. Using them instead of observed time would make us vulnerable to clock skew.

[liggitt]

cc @wojtek-t @mtaufen

[liggitt]

see discussion in the formalization of the Lease API - kubernetes/kubernetes#64246 (comment)

/close