Validate timestamps when creating windowed or timestamped values

[kennknowles]

In debugging a problem with out-of-bounds timestamps, some bad timestamps are caught when a DoFn outputs, when it would be better to catch it early as possible.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

ValidatesRunner compliance status (on master branch)

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---					---
Java
Python	---					---
XLang						---

Examples testing status on various runners

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---	---	---	---	---	---	---
Java	---		---	---	---	---	---
Python	---	---	---	---	---	---	---
XLang	---	---	---	---	---	---	---

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Go	Java	Python

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---			---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[kennknowles]

R: @laraschmidt

[kennknowles]

Run Java PreCommit

[kennknowles]

green!

[y1chi]

R: @laraschmidt

[laraschmidt]

This is going to break some internal tests as is because of the bug we found on the portable runner. We may have to wait for that to be fixed and rolled out first.

[kennknowles]

On the other hand, we know from those tests that the portable runner is broken, so would sickbaying the tests work?

[laraschmidt]

That issue is only affecting a very small subset of jobs because of where the check is located (e.g. it requires allowed skew be infinite). I'm pretty sure this would affect all jobs with impulse.

[kennknowles]

run dataflow validatesrunner

[kennknowles]

Run Java Dataflow V2 ValidatesRunner

[kennknowles]

Run Java Dataflow V2 ValidatesRunner Streaming

[aaltay]

What is the next step on this PR?

[laraschmidt]

Needs to wait for an internal dataflow fix to roll out, otherwise it will break Dataflow. :)

[github-actions]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

[kennknowles]

I'm thinking this might be ready now.

[kennknowles]

Run Dataflow ValidatesRunner

[kennknowles]

known pulsar & fnapi flakes

[kennknowles]

Run Java PreCommit

[kennknowles]

Run Java Dataflow V2 ValidatesRunner

[kennknowles]

Run Java Dataflow V2 ValidatesRunner Streaming

[kennknowles]

Yup, looks good now. PTAL.

[kennknowles]

Run Java Dataflow V2 ValidatesRunner Streaming

[kennknowles]

@dpcollins-google

[aaltay]

@dpcollins-google - could you please review this?

[aaltay]

@dpcollins-google - ping?

[dpcollins-google]

This seems to be a different issue than the one I'm running into. In particular, its validating that the timestamp is in bounds [MIN, MAX] while my issue is that impulse is not returning MIN. LGTM though

[kennknowles]

Taking a look at whether runner v2 is still producing invalid timestamps.

[kennknowles]

Seeing Provided timestamp -290308-12-21T19:59:05.224Z must be within bounds [-290308-12-21T19:59:05.225Z, 294247-01-10T04:00:54.775Z]. Unless I am reading it wrong, that timestamp is within bounds, so the check added here is buggy.

[aaltay]

Last ping from me on this PR :)

What is the next step on this PR? Should this be closed?

[kennknowles]

No need to ping. Just my limited bandwidth. I will get to it.

[github-actions]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

[github-actions]

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

[github-actions]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

[github-actions]

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

[hpvd]

added this to [Parent issue] Support for Apache Pulsar #31078