Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

Commit

Permalink
Feature/merge master into federation (#829)
Browse files Browse the repository at this point in the history 
* Revert "Usage of postgres in test environment for database consistency (#686)" (#688)

This reverts commit 932a61f.

* Mitigate CVE-2020-13935 (#690)

* Bump spring parent version

* Liveness health indicator according to. spring-projects/spring-boot#22107

* Revert to test postgresql docker image using test-containers. (#694)

* Test cases and debug support.for submission (#698)

* Add integration test for future submission payload debugging

* Added more test cases around invalid temp key parameter scenarios

* Add support for printing the submission payload during tests

* Enhance debug test with sql statements generation

* Update services/submission/src/test/java/app/coronawarn/server/services/submission/integration/SubmissionPersistenceIT.java

Co-authored-by: Michael Frey <michael.frey@gmx.ch>

Co-authored-by: Hilmar Falkenberg <hilmar.falkenberg@sap.com>
Co-authored-by: Michael Frey <michael.frey@gmx.ch>

* Update maven.config with current version 1.4.0-SNAPSHOT (#697)

Set current version to 1.3.0-SNAPSHOT

Co-authored-by: Hilmar Falkenberg <hilmar.falkenberg@sap.com>

* -fix invalid test and name refactoring (#703)

* Add app-features as external configuration (#704)

* Add app features in configuration file
Adjust tests

* Rename env variable for plausible deniability active.
Remove env name for app feature label.

* Adjust test application.yaml

* Change active type from boolean to integer

* Change app feature 'active' property to 'value'

* use same version for spring-boot-starter-parent accross all components (#709)

* Fix/700 retention policy (#711)

* fix retention policy of stored diagnosis keys

* fix typo

* fix failing test case due to previously injected time

* refactor method names

* Submission Service - updated documentation (#713)

* Updated the validation for the submission service in SUMBISSION.md to match the new rules for TEK's

* markdownlint fixes

* Update docs/SUBMISSION.md

Co-authored-by: Sorin Stefan Iovita <sorin.stefan.iovita@sap.com>

Co-authored-by: Ioan Gut <ioan.gut@sap.com>
Co-authored-by: ioangut <67064882+ioangut@users.noreply.github.com>
Co-authored-by: Sorin Stefan Iovita <sorin.stefan.iovita@sap.com>

* Submission Service - Enhance TEK's  (#712)

* -initial draft commit

* updated checkDuplicateStartIntervalNumberLimit wip

* updated checkDuplicateStartIntervalNumberLimit

* -fix test broke after solved conflict

* changed checkDuplicateStartIntervalNumberLimit to use HashMap

* -fix checklist reports

* add env variables for submission service

* review update add more tests

* solved code smells

* rename attribute maxRollingPeriod

* renamed getter getMaxRollingPeriod

* Update ValidSubmissionPayload.java

* refactor implementation for consistency

* fix checkstyle errors

modify the Violation text accordingly

* fix code smell

* Update ValidSubmissionPayload.java

* Change validation for payloads based on rolling period

* Capture more test scenarios

* add checkstyle indentation

* create test data generation class

* add header to the new class

* resolved review conversation

* fixed checkstyle errors

* resolved code smell

* fix compile error

* create valid TEK, fix test failing

Co-authored-by: Emmet <emmet.power@sap.com>
Co-authored-by: Eugen M <eugen.madean@sap.com>
Co-authored-by: Pit Humke <pithumke@users.noreply.github.com>

* Feature/enrich app config (#726)

* Added supported countries to app config with application.yaml parameter

* Fixed Test failures

* Remove unused code

* Added app version parameters to application.yaml

* Added tests for app version parameters

* Refacted country list implementation

Co-authored-by: Hilmar Falkenberg <hilmar.falkenberg@sap.com>

* Fix Sonar issues (#754)

* Bugfix/submission tek rolling period (#752)

* Replaced possible variable rolling period with constant value

* Add tests for flexible rolling period

* apply sugested review changes

* Add javaDoc comments for changed getExpiryDateTime

* Fix midnight check in SubmissionPayload Validator

Co-authored-by: Michael Burwig <michael.burwig@sap.com>

* Feature/update risk score master (#758)

* Update risk-score-classification.yaml

Problem in iOS 13.7 --> Unbekanntes Risiko bei Risikobegegnung

* Adjust tests

Co-authored-by: Hilmar Falkenberg <hilmar.falkenberg@sap.com>
Co-authored-by: Michael Burwig <michael.burwig@sap.com>

* Update exposure-config.yaml (#765)

patch transmission as requested by Thomas Augsten

Co-authored-by: Hilmar Falkenberg <hilmar.falkenberg@sap.com>

* Update attenuation-duration.yaml (#775) (#776)

* Update attenuation-duration.yaml

* Update exposure-config.yaml

attenuation array = 2

Co-authored-by: Hilmar Falkenberg <hilmar.falkenberg@sap.com>

* prepare release 1.5.0 (#793)

* Fix sping web guava (#808)

* RC 1.4.0

* mitigate CVE-2020-5421 by switching to newer spring-boot-starter-parent (#796)

* Release/1.4 log submission keys (#797)

* Implement the changes from PR #783

* Add check and logs for missing key with TRL 6 or having a key TRL 6 from today midnight

* Fix code smell

* ensure usage of latest guava version 29.0-jre (#806)

Co-authored-by: Michael <66735191+mibrasap@users.noreply.github.com>

* Update CODEOWNERS file (#817)

* Admin/update codeowners (#818)

* Update CODEOWNERS file

* Add additional team members to codeowners file

Co-authored-by: Michael Burwig <michael.burwig@sap.com>

* Add hot fix for submission payload validation (#822) (#825)

* Add hot fix for submission payload validation (#822)

* Fixed code smell

* fix branch after merge conflicts

* Adjustments after code review

* removed unused getExpiryDateTime method

* removed unused import code smell

* Adjust submission payload validation checks in Master(#832) f (#834)

* Adjust submission payload validation checks (#832)

* remove unused import code smell

* Removed unused duplicated methods

Co-authored-by: Frederico <fred.rbittencourt@gmail.com>
Co-authored-by: Sorin Stefan Iovita <sorin.stefan.iovita@sap.com>
Co-authored-by: EugenM-SAP <67458405+EugenM-SAP@users.noreply.github.com>
Co-authored-by: Hilmar Falkenberg <hilmar.falkenberg@sap.com>
Co-authored-by: Michael Frey <michael.frey@gmx.ch>
Co-authored-by: Evgenii Skrebtcov <67064767+EvgeniiSkrebtcov@users.noreply.github.com>
Co-authored-by: emmetsap <emmet.power@sap.com>
Co-authored-by: Eugen M <eugen.madean@sap.com>
Co-authored-by: Pit Humke <pithumke@users.noreply.github.com>
Co-authored-by: KevponSAP <66735382+KevponSAP@users.noreply.github.com>
Co-authored-by: Michael Burwig <michael.burwig@sap.com>
Co-authored-by: Michael <66735191+mibrasap@users.noreply.github.com>
Co-authored-by: Michael Burwig <64439292+michael-burwig@users.noreply.github.com>
  • Loading branch information
@ioangut @fredrb
@EugenM-SAP @hilmarf @MichaelPFrey @EvgeniiSkrebtcov @emmetsap @pithumke @KevponSAP @michael-burwig @mibrasap
14 people committed Oct 1, 2020
1 parent 5d6493d commit 3356f9d
Show file tree
Hide file tree
Showing 4 changed files with 40 additions and 84 deletions.
7 changes: 7 additions & 0 deletions ...ronawarn/server/services/distribution/assembly/diagnosiskeys/ProdDiagnosisKeyBundler.java
View file
Expand Up @@ -133,6 +133,13 @@ private static Optional<LocalDateTime> getEarliestDistributableTimestamp(
return distributableDiagnosisKeys.keySet().stream().min(LocalDateTime::compareTo);
}

/**
* Calculates the earliest point in time at which the specified {@link DiagnosisKey} can be distributed, while
* respecting the expiry policy and the submission timestamp. Before keys are allowed to be distributed, they must be
* expired for a configured amount of time.
*
* @return {@link LocalDateTime} at which the specified {@link DiagnosisKey} can be distributed.
*/
private LocalDateTime getDistributionDateTimeByExpiryPolicy(DiagnosisKey diagnosisKey) {
return sharingPoliciesChecker.getEarliestTimeForSharingKey(diagnosisKey,
ExpirationPolicy.of(expiryPolicyMinutes, ChronoUnit.MINUTES));
Expand Down
37 changes: 0 additions & 37 deletions ...ain/java/app/coronawarn/server/services/submission/validation/ValidSubmissionPayload.java
View file
Expand Up @@ -3,8 +3,6 @@
package app.coronawarn.server.services.submission.validation;

import static java.util.function.Predicate.not;
import static java.util.stream.Collectors.groupingBy;
import static java.util.stream.Collectors.summingInt;
import static java.util.stream.Collectors.toList;

import app.coronawarn.server.common.protocols.external.exposurenotification.TemporaryExposureKey;
Expand All @@ -15,7 +13,6 @@
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.concurrent.atomic.AtomicBoolean;
Expand Down Expand Up @@ -80,7 +77,6 @@ public boolean isValid(SubmissionPayload submissionPayload, ConstraintValidatorC

if (keysHaveFlexibleRollingPeriod(exposureKeys)) {
return checkStartIntervalNumberIsAtMidNight(exposureKeys, validatorContext)
&& checkKeysCumulateEqualOrLessThanMaxRollingPeriodPerDay(exposureKeys, validatorContext)
&& checkOriginCountryIsValid(submissionPayload, validatorContext)
&& checkVisitedCountriesAreValid(submissionPayload, validatorContext)
&& checkRequiredFieldsNotMissing(exposureKeys, validatorContext)
Expand All @@ -89,7 +85,6 @@ && checkTransmissionRiskLevelIsAcceptable(exposureKeys, validatorContext)
} else {
return checkStartIntervalNumberIsAtMidNight(exposureKeys, validatorContext)
&& checkKeyCollectionSize(exposureKeys, validatorContext)
&& checkUniqueStartIntervalNumbers(exposureKeys, validatorContext)
&& checkOriginCountryIsValid(submissionPayload, validatorContext)
&& checkVisitedCountriesAreValid(submissionPayload, validatorContext)
&& checkRequiredFieldsNotMissing(exposureKeys, validatorContext)
Expand All @@ -112,38 +107,6 @@ private boolean checkKeyCollectionSize(List<TemporaryExposureKey> exposureKeys,
return true;
}

private boolean checkUniqueStartIntervalNumbers(List<TemporaryExposureKey> exposureKeys,
ConstraintValidatorContext validatorContext) {
Integer[] startIntervalNumbers = exposureKeys.stream()
.mapToInt(TemporaryExposureKey::getRollingStartIntervalNumber).boxed().toArray(Integer[]::new);
long distinctSize = Arrays.stream(startIntervalNumbers)
.distinct()
.count();

if (distinctSize < exposureKeys.size()) {
addViolation(validatorContext, String.format(
"Duplicate StartIntervalNumber found. StartIntervalNumbers: %s", startIntervalNumbers));
return false;
}
return true;
}

private boolean checkKeysCumulateEqualOrLessThanMaxRollingPeriodPerDay(List<TemporaryExposureKey> exposureKeys,
ConstraintValidatorContext validatorContext) {

boolean isValidRollingPeriod = exposureKeys.stream()
.collect(groupingBy(TemporaryExposureKey::getRollingStartIntervalNumber,
summingInt(TemporaryExposureKey::getRollingPeriod)))
.values().stream()
.anyMatch(sum -> sum <= maxRollingPeriod);

if (!isValidRollingPeriod) {
addViolation(validatorContext, "The sum of the rolling periods exceeds 144 per day");
return false;
}
return true;
}

private boolean keysHaveFlexibleRollingPeriod(List<TemporaryExposureKey> exposureKeys) {
return exposureKeys.stream()
.anyMatch(temporaryExposureKey -> temporaryExposureKey.getRollingPeriod() < maxRollingPeriod);
Expand Down
29 changes: 3 additions & 26 deletions ...test/java/app/coronawarn/server/services/submission/controller/PayloadValidationTest.java
View file
Expand Up @@ -37,7 +37,7 @@ class PayloadValidationTest {

@BeforeEach
public void setUpMocks() {
when(this.tanVerifier.verifyTan(anyString())).thenReturn(true);
when(tanVerifier.verifyTan(anyString())).thenReturn(true);
}

@Autowired
Expand Down Expand Up @@ -127,7 +127,7 @@ void check400ResponseStatusForMissingTrlAndDsos() {
}

@Test
void check400ResponseStatusForKeysWithFixedRollingPeriodAndDuplicateStartIntervals() {
void check200ResponseStatusForKeysWithFixedRollingPeriodAndDuplicateStartIntervals() {
int rollingStartIntervalNumber = createRollingStartIntervalNumber(2);
var keysWithDuplicateStartIntervalNumber = Lists.list(
buildTemporaryExposureKey(VALID_KEY_DATA_1, rollingStartIntervalNumber, 1,
Expand All @@ -136,7 +136,7 @@ void check400ResponseStatusForKeysWithFixedRollingPeriodAndDuplicateStartInterva

ResponseEntity<Void> actResponse = executor.executePost(keysWithDuplicateStartIntervalNumber);

assertThat(actResponse.getStatusCode()).isEqualTo(BAD_REQUEST);
assertThat(actResponse.getStatusCode()).isEqualTo(OK);
}

@Test
Expand Down Expand Up @@ -210,22 +210,6 @@ private Collection<TemporaryExposureKey> buildPayloadWithMoreThan14KeysAndFlexib
return flexibleRollingPeriodKeys;
}

@Test
void check400ResponseStatusWhenTwoKeysCumulateMoreThanMaxRollingPeriodInSameDay() {
ResponseEntity<Void> actResponse = executor.executePost(buildPayloadWithKeysThatCumulateMoreThanMaxRollingPeriodPerDay());
assertThat(actResponse.getStatusCode()).isEqualTo(BAD_REQUEST);
}

private Collection<TemporaryExposureKey> buildPayloadWithKeysThatCumulateMoreThanMaxRollingPeriodPerDay() {
ArrayList<TemporaryExposureKey> temporaryExposureKeys = new ArrayList<>();
temporaryExposureKeys.add(buildTemporaryExposureKeyWithFlexibleRollingPeriod(VALID_KEY_DATA_1,
createRollingStartIntervalNumber(2), 3, 100));
temporaryExposureKeys.add(buildTemporaryExposureKeyWithFlexibleRollingPeriod(VALID_KEY_DATA_1,
createRollingStartIntervalNumber(2), 3, 144));

return temporaryExposureKeys;
}

@Test
void check200ResponseStatusWithTwoKeysOneFlexibleAndOneDefaultOnDifferentDays() {
ResponseEntity<Void> actResponse = executor.executePost(buildPayloadWithTwoKeysOneFlexibleAndOneDefaultOnDifferentDays());
Expand All @@ -245,13 +229,6 @@ private Collection<TemporaryExposureKey> buildPayloadWithTwoKeysOneFlexibleAndOn
return flexibleRollingPeriodKeys;
}

@Test
void check200ResponseStatusWhenKeysCumulateToMaxRollingPeriodInSameDay() {
ResponseEntity<Void> actResponse = executor.executePost(buildPayloadWithTwoKeysWithFlexibleRollingPeriod());

assertThat(actResponse.getStatusCode()).isEqualTo(OK);
}

private Collection<TemporaryExposureKey> buildPayloadWithTwoKeysWithFlexibleRollingPeriod() {
ArrayList<TemporaryExposureKey> flexibleRollingPeriodKeys = new ArrayList<>();

Expand Down
51 changes: 30 additions & 21 deletions ...t/java/app/coronawarn/server/services/submission/controller/SubmissionControllerTest.java
View file
Expand Up @@ -2,19 +2,20 @@

package app.coronawarn.server.services.submission.controller;

import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.VALID_KEY_DATA_1;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.VALID_KEY_DATA_2;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildPayload;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildMultipleKeys;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildMultipleKeysWithoutDSOS;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildMultipleKeysWithoutDSOSAndTRL;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildMultipleKeysWithoutTRL;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildPayload;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildPayloadWithInvalidKey;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildPayloadWithInvalidOriginCountry;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildPayloadWithOneKey;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildPayloadWithPadding;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildPayloadWithTooLargePadding;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildPayloadWithVisitedCountries;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildPayloadWithTooLargePadding;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.createRollingStartIntervalNumber;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildPayloadWithInvalidOriginCountry;
import static app.coronawarn.server.services.submission.controller.SubmissionPayloadMockData.buildTemporaryExposureKey;
import static org.apache.commons.lang3.StringUtils.defaultIfBlank;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
Expand All @@ -33,6 +34,7 @@

import app.coronawarn.server.common.persistence.domain.DiagnosisKey;
import app.coronawarn.server.common.persistence.service.DiagnosisKeyService;
import app.coronawarn.server.common.protocols.external.exposurenotification.ReportType;
import app.coronawarn.server.common.protocols.external.exposurenotification.TemporaryExposureKey;
import app.coronawarn.server.common.protocols.internal.SubmissionPayload;
import app.coronawarn.server.services.submission.config.SubmissionServiceConfig;
Expand Down Expand Up @@ -88,21 +90,6 @@ class SubmissionControllerTest {
@Autowired
private SubmissionServiceConfig config;

private static Stream<Arguments> createIncompleteHeaders() {
return Stream.of(
Arguments.of(HttpHeaderBuilder.builder().build()),
Arguments.of(HttpHeaderBuilder.builder().contentTypeProtoBuf().build()),
Arguments.of(HttpHeaderBuilder.builder().contentTypeProtoBuf().withoutCwaFake().build()),
Arguments.of(HttpHeaderBuilder.builder().contentTypeProtoBuf().cwaAuth().build()));
}

private static Stream<Arguments> createDeniedHttpMethods() {
return Arrays.stream(HttpMethod.values())
.filter(method -> method != HttpMethod.POST)
.filter(method -> method != HttpMethod.PATCH) /* not supported by Rest Template */
.map(Arguments::of);
}

@BeforeEach
public void setUpMocks() {
when(tanVerifier.verifyTan(anyString())).thenReturn(true);
Expand Down Expand Up @@ -214,8 +201,8 @@ void checkDSOSIsPersistedForKeysWithTRLOnly() {
}

/**
* The test verifies that even if the payload does not provide keys with TRL, the information
* is still derived from the DSOS field and correctly persisted.
* The test verifies that even if the payload does not provide keys with TRL, the information is still derived from
* the DSOS field and correctly persisted.
*
* <li>DSOS - days since onset of symptoms
* <li>TRL - transmission risk level
Expand Down Expand Up @@ -441,4 +428,26 @@ private DiagnosisKey findDiagnosisKeyMatch(TemporaryExposureKey temporaryExposur
diagnosisKey -> temporaryExposureKey.getKeyData().equals(ByteString.copyFrom(diagnosisKey.getKeyData())))
.findFirst().orElseThrow();
}

public static SubmissionPayload buildPayloadWithInvalidKey() {
TemporaryExposureKey invalidKey =
buildTemporaryExposureKey(VALID_KEY_DATA_1, createRollingStartIntervalNumber(2), 999,
ReportType.CONFIRMED_CLINICAL_DIAGNOSIS, 1);
return buildPayload(invalidKey);
}

private static Stream<Arguments> createIncompleteHeaders() {
return Stream.of(
Arguments.of(HttpHeaderBuilder.builder().build()),
Arguments.of(HttpHeaderBuilder.builder().contentTypeProtoBuf().build()),
Arguments.of(HttpHeaderBuilder.builder().contentTypeProtoBuf().withoutCwaFake().build()),
Arguments.of(HttpHeaderBuilder.builder().contentTypeProtoBuf().cwaAuth().build()));
}

private static Stream<Arguments> createDeniedHttpMethods() {
return Arrays.stream(HttpMethod.values())
.filter(method -> method != HttpMethod.POST)
.filter(method -> method != HttpMethod.PATCH) /* not supported by Rest Template */
.map(Arguments::of);
}
}

0 comments on commit 3356f9d

Please sign in to comment.