Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: exactly once support v3 #1022

Merged
merged 89 commits into from Mar 4, 2022
Merged
Show file tree
Hide file tree
Changes from 80 commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
439ae44
feat: add exactly once delivery flag
gcf-owl-bot[bot] Feb 4, 2022
b03e3e8
🦉 Updates from OwlBot
gcf-owl-bot[bot] Feb 4, 2022
bd0aed0
chore: enable auto release in java (#1349) (#997)
gcf-owl-bot[bot] Feb 7, 2022
02f3da7
chore: update auto-release script to fix breaking changes in v5 (#135…
gcf-owl-bot[bot] Feb 8, 2022
7bf2598
build(deps): update dependency com.google.cloud:google-cloud-shared-c…
renovate-bot Feb 8, 2022
faec051
build(deps): update dependency org.apache.maven.plugins:maven-project…
renovate-bot Feb 8, 2022
62a6081
chore(deps): update dependency com.google.cloud:libraries-bom to v24.…
renovate-bot Feb 8, 2022
4c317ae
deps: update dependency com.google.cloud:google-cloud-core to v2.5.0 …
renovate-bot Feb 8, 2022
4391501
chore(main): release 1.115.3 (#1002)
release-please[bot] Feb 8, 2022
3e28da2
chore(main): release 1.115.4-SNAPSHOT (#1003)
release-please[bot] Feb 9, 2022
3a99fb8
Merge branch 'upstream_exactly_once_proto_changes' into exactly_once_…
mmicatka Feb 9, 2022
ff95cc9
pulling in work from other branch
mmicatka Feb 9, 2022
4c9d656
forgot a file
mmicatka Feb 9, 2022
bf2dc68
wip
mmicatka Feb 9, 2022
7ea3430
wip
mmicatka Feb 9, 2022
69b6c55
fixed a misssing flow controller
mmicatka Feb 10, 2022
0634281
typo
mmicatka Feb 10, 2022
955a845
working on integration tests
mmicatka Feb 10, 2022
5e0b040
wip
mmicatka Feb 10, 2022
bef2023
wip
mmicatka Feb 10, 2022
27e27a9
wip
mmicatka Feb 10, 2022
60546fd
wip
mmicatka Feb 11, 2022
e4a1af7
wip
mmicatka Feb 11, 2022
5fb59c1
possibly partially working
mmicatka Feb 11, 2022
e3c5efb
I think it's working?
mmicatka Feb 11, 2022
b7424a5
should work. need to add tests
mmicatka Feb 11, 2022
12a3a48
wip, cleaning things up
mmicatka Feb 14, 2022
371d220
wip
mmicatka Feb 14, 2022
91f5580
wip
mmicatka Feb 15, 2022
bb59dd1
owlbot changes
mmicatka Feb 15, 2022
bda7673
wip
mmicatka Feb 16, 2022
48cdd7a
partially working but not quite
mmicatka Feb 16, 2022
c891320
wip
mmicatka Feb 17, 2022
39b6322
wip
mmicatka Feb 17, 2022
364601b
nearly working with tests
mmicatka Feb 18, 2022
1f386c7
some unit tests are done
mmicatka Feb 18, 2022
d915e22
wip
mmicatka Feb 18, 2022
0b0ee12
unit tests are starting to work
mmicatka Feb 18, 2022
12d242d
nearly done
mmicatka Feb 18, 2022
e226557
fixed a file/class name
mmicatka Feb 18, 2022
ec0e43a
cleanup
mmicatka Feb 18, 2022
c251998
cleanup
mmicatka Feb 18, 2022
be1d945
need to fix some tests still but is working.
mmicatka Feb 19, 2022
5f03298
added more unit tests + cleanup
mmicatka Feb 20, 2022
d3fd437
unit tests for streamingsubscriberconnection done
mmicatka Feb 21, 2022
9a2dff7
wip unit tests for messageDispatcher
mmicatka Feb 22, 2022
cf022c2
finished unit tests + cleanup
mmicatka Feb 23, 2022
56c5a78
updating ack duration changes
mmicatka Feb 23, 2022
8371307
code review feedback
mmicatka Feb 23, 2022
2c782ab
using computeIfAbsent
mmicatka Feb 23, 2022
1362c98
added test coverage for setup and teardown
mmicatka Feb 23, 2022
f796347
added sample
mmicatka Feb 24, 2022
02e8f20
fixed a file name, added exponential backoff for retries
mmicatka Feb 24, 2022
02dafd4
refactored modackdeadlines + tests
mmicatka Feb 24, 2022
e534d5c
removed sample
mmicatka Feb 24, 2022
8f37e8a
removed dummy test
mmicatka Feb 24, 2022
2fa6555
updated min/max duration ack extension logic w/ tests
mmicatka Feb 25, 2022
2a60cd0
added response for grpc errors. need to fix a test
mmicatka Feb 25, 2022
042b6de
small fix
mmicatka Feb 25, 2022
e9d4344
fixed a flakey test
mmicatka Feb 25, 2022
e2d5460
removed a try-catch in a test
mmicatka Feb 25, 2022
0c1f705
minor test fix
mmicatka Feb 26, 2022
dd7a217
fixing a test
mmicatka Feb 26, 2022
bab3e8f
added try-catch-finally blocks in test
mmicatka Feb 26, 2022
2b2a6b6
trying to fix some flakey tests
mmicatka Feb 26, 2022
9e3ca1c
code review changes
mmicatka Feb 26, 2022
c43f677
code review feedback + test fixes
mmicatka Feb 26, 2022
4bb3604
refactored retry logic to prevent modack failures from blocking
mmicatka Feb 26, 2022
ea10826
removed unused field
mmicatka Feb 28, 2022
dc0640e
refactored/simplified a number of things. Updated tests
mmicatka Mar 1, 2022
2a13c8a
Merge branch 'main' into exactly_once_support_v3
mmicatka Mar 1, 2022
1607374
made some changes in propagated failures
mmicatka Mar 1, 2022
cc6ae97
fixed tests
mmicatka Mar 1, 2022
5afa56a
code review changes
mmicatka Mar 1, 2022
cb1a33e
added optional message future
mmicatka Mar 2, 2022
6a54061
updated a constant name
mmicatka Mar 2, 2022
8a6019b
removed a isModack from ackRequestData
mmicatka Mar 2, 2022
48e89d0
small naming change
mmicatka Mar 2, 2022
91e1730
code review changes
mmicatka Mar 2, 2022
1697486
more changes
mmicatka Mar 2, 2022
1e5858a
changed a default
mmicatka Mar 2, 2022
6816e45
typo
mmicatka Mar 2, 2022
e96901e
minor change
mmicatka Mar 3, 2022
b3646d7
added logic to change future responses for not exactly once
mmicatka Mar 3, 2022
16b1b0d
forgot a return
mmicatka Mar 3, 2022
6209577
whitespace change to retrigger build
mmicatka Mar 4, 2022
ed7d7f7
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 4, 2022
9f42311
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Mar 4, 2022
a5fc445
Merge branch 'exactly_once_support_v3' of https://github.com/mmicatka…
gcf-owl-bot[bot] Mar 4, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions google-cloud-pubsub/pom.xml
Expand Up @@ -95,6 +95,11 @@
</dependency>

<!-- Test dependencies -->
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
Expand Down
@@ -0,0 +1,25 @@
/*
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.google.cloud.pubsub.v1;

import java.util.concurrent.Future;

public interface AckReplyConsumerWithResponse {
Future<AckResponse> ack();

Future<AckResponse> nack();
}
@@ -0,0 +1,85 @@
/*
* Copyright 2022 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.google.cloud.pubsub.v1;

import com.google.api.core.SettableApiFuture;
import java.util.Optional;

public class AckRequestData {
private final String ackId;
private final Optional<SettableApiFuture<AckResponse>> messageFuture;

protected AckRequestData(Builder builder) {
this.ackId = builder.ackId;
this.messageFuture = builder.messageFuture;
}

public String getAckId() {
return ackId;
}

public SettableApiFuture<AckResponse> getMessageFutureIfExists() {
return this.messageFuture.orElse(null);
}

public AckRequestData setResponse(AckResponse ackResponse, boolean setResponseOnSuccess) {
if (this.messageFuture.isPresent() && !this.messageFuture.get().isDone()) {
switch (ackResponse) {
case SUCCESSFUL:
if (setResponseOnSuccess) {
this.messageFuture.get().set(ackResponse);
}
break;
case INVALID:
case OTHER:
case PERMISSION_DENIED:
case FAILED_PRECONDITION:
// Non-succesful messages will get set for both acks, nacks, and modacks
this.messageFuture.get().set(ackResponse);
break;
}
}
return this;
}

public boolean hasMessageFuture() {
return this.messageFuture.isPresent();
}

public static Builder newBuilder(String ackId) {
return new Builder(ackId);
}

/** Builder of {@link AckRequestData AckRequestData}. */
protected static final class Builder {
private final String ackId;
private Optional<SettableApiFuture<AckResponse>> messageFuture = Optional.empty();

protected Builder(String ackId) {
this.ackId = ackId;
}

public Builder setMessageFuture(SettableApiFuture<AckResponse> messageFuture) {
this.messageFuture = Optional.of(messageFuture);
return this;
}

public AckRequestData build() {
return new AckRequestData(this);
}
}
}
@@ -0,0 +1,25 @@
/*
* Copyright 2021 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package com.google.cloud.pubsub.v1;

public enum AckResponse {
PERMISSION_DENIED,
FAILED_PRECONDITION,
SUCCESSFUL,
INVALID,
OTHER
}