created name resolution local component #1340

jigargandhi · 2021-11-26T15:02:55Z

Description

Created a local name resolution component that does parsing of command line arguments to daprd/daprd.exe to identify different dapr instances.

Issue reference

We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.

Please reference the issue this PR will close: dapr/dapr#3256

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

Code compiles correctly
Created/updated tests
Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

…ts-contrib into local-dns-component

beiwei30

This change looks good to me. @jigargandhi would you mind to run go mod tidy for both pubsub.kafka certification test and pubsub.rabbitmq certification test?

jigargandhi · 2021-12-02T09:39:10Z

@beiwei30 ran go mod tidy and fixed issues

beiwei30

LGTM.

artursouza · 2022-01-07T00:15:29Z

nameresolution/local/local.go

+}
+
+func (resolver *resolver) ResolveID(req nameresolution.ResolveRequest) (string, error) {
+	processes, err := ps.Processes()


Can this be cached? This seem to be a super expensive operation per Invoke() call. Doing it for a cache miss is kind of OK but once cached, we should be able to reuse. Also, invalidating the cache is super important. Maybe we change this logic to be running in the background updating a lookup table and then the ResolveID() method just does a lookup.

The value can be cached and we can run a go routine which invalidates the cache every 30 seconds.
We will then need to store process id also in cache, and validate that the process id is actually for dapr process only (it may be that the process id is reassigned)
If the process id does not match then it will start the cache invalidation process again.
Also, the cache invalidation process will also run only once 30 seconds.

Correct. We need to keep track of the PIDs too.

Taction · 2022-01-08T08:18:40Z

@jigargandhi Thanks for this PR! Would you please fix DCO issue? You can refer to here about how to fix it.

jigargandhi · 2022-01-08T13:38:43Z

@Taction Sure, I will be adding one more commit as per a review comment from @artursouza , will sign off that commit.

beiwei30 · 2022-01-12T06:49:34Z

@jigargandhi Jigar, at the same time, pls. fix go mod tidy check failure.

Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* cosmosdb output binding certification tests Signed-off-by: GitHub <noreply@github.com> * linter touchup Signed-off-by: GitHub <noreply@github.com> * Rename some remnants from copy paste Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com> Co-authored-by: Looong Dai <long.dai@intel.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@gmail.com> Co-authored-by: Looong Dai <long.dai@intel.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* not modifying policy if disableEntity is set Signed-off-by: Amit Mor <amit.mor@hotmail.com> * not modifying policy if disableEntity is set Signed-off-by: Amit Mor <amit.mor@hotmail.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* feature/pubsub: add delay queue params for pulsar * feature/pubsub: add delay queue params for pulsar * feature/pubsub: add delay queue params for pulsar * hotfix: upgrade dgrijalva/jwt-go * hotfix: upgrade dgrijalva/jwt-go Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* add S3ForcePathStyle support for s3 binding Signed-off-by: rainfd <rainfd@live.cn> * add S3ForcePathStyle unit test Signed-off-by: rainfd <rainfd@live.cn> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

…able names" - Introduce `fmt.Sprintf` where applicable (dapr#1388) * - Introduce `fmt.Sprintf` where applicable - Add conformance test for cassandra component. Signed-off-by: mbimbij <joseph.mbimbi@gmail.com> * Start cassandra via docker-compose for its conformance test Signed-off-by: mbimbij <joseph.mbimbi@gmail.com> * Apply `go fmt` on `tests/conformance/common.go` as part of PR correction Signed-off-by: mbimbij <joseph.mbimbi@gmail.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* remove duplicated err handle Signed-off-by: Taction <zchao9100@gmail.com> * fix test go mod tidy Signed-off-by: Taction <zchao9100@gmail.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* parent dbb18e4 author Scott Hussey <sthussey@gmail.com> 1632277042 -0500 committer Scott Hussey <sthussey@gmail.com> 1639111222 -0600 Support Oauth2 authentication for Kafka - Utilize the SASL OAUTHBEARER mechanism to support the Oauth2 client_credentials flow for Kafka authentication - Deprecate `authRequired` field and introduce `authType` to support varied authentication mechanisms - Add a metadata upgrade mechanism to support backwards compatability - Recommend broker specific scopes to guard against a compromised broker replaying a token Signed-off-by: Scott Hussey <sthussey@gmail.com> * kafka test - use volumes instead of root CI test fails due to losing data written to container root Signed-off-by: Scott Hussey <sthussey@gmail.com> * Clean up volumes - When bringing down the docker-compose context, cleanup volumes Signed-off-by: Scott Hussey <sthussey@gmail.com> * Clean up stale comment Signed-off-by: Scott Hussey <sthussey@gmail.com> * Resume config defaults - Return some Kafka config to default values to lower config footprint and stabilize flaky CI runs Signed-off-by: Scott Hussey <sthussey@gmail.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* pub/sub rocketmq:upgrade with client v2 Signed-off-by: zach <zachchou016@gmail.com> * pubsub/rocketmq:remove cache and add start with setup Signed-off-by: zach <zachchou016@gmail.com> * pubsub/rocketmq:fix variable golint Signed-off-by: zach <zachchou016@gmail.com> * pubsub/rocketmq:change license and fix goimports Signed-off-by: zach <zachchou016@gmail.com> Co-authored-by: Looong Dai <long.dai@intel.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Signed-off-by: Mukundan Sundararajan <msundar.ms@outlook.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Issue reference: dapr/docs#2039 Signed-off-by: Will Tsai <william.wl.tsai@gmail.com> Co-authored-by: Artur Souza <artursouza.ms@outlook.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* Implement State Store for OCI ObjectStorage service Signed-off-by: lucasjellema <lucasjellema@gmail.com> * add go.mod and go.sum with dependencies on OCI SDK Signed-off-by: lucasjellema <lucasjellema@gmail.com> * Removed dependency in unit test on OCI account Signed-off-by: lucasjellema <lucasjellema@gmail.com> * Adding extensive unit test (coverage) using mock-OCI client Signed-off-by: lucasjellema <lucasjellema@gmail.com> * Correcting linting issues and review requests Signed-off-by: lucasjellema <lucasjellema@gmail.com> Co-authored-by: Looong Dai <long.dai@intel.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* Kafka PubSub: Setting metadata as message headers. Signed-off-by: Phil Kedy <phil.kedy@gmail.com> * running go mod tidy on certification tests Signed-off-by: Phil Kedy <phil.kedy@gmail.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Signed-off-by: Mukundan Sundararajan <msundar.ms@outlook.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* MQTT Certification Test Signed-off-by: shivam <shivamkm07@gmail.com> * Using paho.mqtt.golang fork with AutoAck fix Signed-off-by: shivam <shivamkm07@gmail.com> * Adding MQTT component in certification.yml Signed-off-by: shivam <shivamkm07@gmail.com> Co-authored-by: Artur Souza <artursouza.ms@outlook.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* binding kafka: check if topics are not empty before reading Signed-off-by: Tim Burkert <burkert.tim@gmail.com> * binding kafka: log warning instead of returning error Signed-off-by: Tim Burkert <burkert.tim@gmail.com> Co-authored-by: Yaron Schneider <schneider.yaron@live.com> Co-authored-by: Artur Souza <artursouza.ms@outlook.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

* conformance tests Signed-off-by: Amit Mor <amit.mor@hotmail.com> * latest localstack. snssqs to conformance list Signed-off-by: Amit Mor <amit.mor@hotmail.com> Co-authored-by: Mukundan Sundararajan <musundar@microsoft.com> Co-authored-by: Artur Souza <artursouza.ms@outlook.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

…eue (dapr#1418) Signed-off-by: 1046102779 <seachen@tencent.com> Co-authored-by: Yaron Schneider <schneider.yaron@live.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

codecov · 2022-01-21T13:19:47Z

Codecov Report

Merging #1340 (26e5911) into master (09fb60c) will decrease coverage by 0.03%.
The diff coverage is 22.53%.

❗ Current head 26e5911 differs from pull request most recent head d44a4dc. Consider uploading reports for the commit d44a4dc to get more accurate results

@@            Coverage Diff             @@
##           master    #1340      +/-   ##
==========================================
- Coverage   35.02%   34.98%   -0.04%     
==========================================
  Files         148      151       +3     
  Lines       12825    13059     +234     
==========================================
+ Hits         4492     4569      +77     
- Misses       7853     8004     +151     
- Partials      480      486       +6

Impacted Files	Coverage Δ
authentication/azure/auth_amqp.go	`0.00% <0.00%> (ø)`
bindings/aws/kinesis/kinesis.go	`2.61% <0.00%> (ø)`
bindings/aws/s3/s3.go	`15.06% <0.00%> (-0.11%)`	⬇️
bindings/azure/eventgrid/eventgrid.go	`3.84% <0.00%> (ø)`
bindings/mqtt/mqtt.go	`25.00% <0.00%> (-1.42%)`	⬇️
configuration/redis/redis.go	`37.22% <0.00%> (-1.12%)`	⬇️
pubsub/aws/snssqs/policy.go	`0.00% <0.00%> (ø)`
pubsub/gcp/pubsub/pubsub.go	`24.60% <0.00%> (+0.38%)`	⬆️
pubsub/kafka/sarama_log_bridge.go	`0.00% <0.00%> (ø)`
pubsub/kafka/sasl_oauthbearer.go	`0.00% <0.00%> (ø)`
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4918900...d44a4dc. Read the comment docs.

jigargandhi · 2022-01-21T13:41:42Z

please review this #1455 PR, closing this as messed up sign-off

jigargandhi added 9 commits November 26, 2021 20:30

created name resolution local component

8b8aa77

Merge branch 'master' into local-dns-component

399bc4e

added go-ps for local component resolution

eb86ef3

Merge branch 'local-dns-component' of github.com:jigargandhi/componen…

80892c0

…ts-contrib into local-dns-component

Merge branch 'master' into local-dns-component

3c75f77

removed test file to try if lint succeeds

6758b18

fix lint

285b303

fix go fumpt error on local.go

15d4312

moved github.com/dapr to latest

b796c6f

jigargandhi marked this pull request as ready for review November 29, 2021 12:27

jigargandhi requested review from a team as code owners November 29, 2021 12:27

Updated go sum files for pubsub kafka

3ca7344

yaron2 mentioned this pull request Nov 29, 2021

File based naming resolver #1342

Closed

3 tasks

dapr-bot added 7 commits November 29, 2021 09:49

Merge branch 'master' into local-dns-component

b78f024

Merge branch 'master' into local-dns-component

7a74b2d

Merge branch 'master' into local-dns-component

d7aaf0b

Merge branch 'master' into local-dns-component

79cb759

Merge branch 'master' into local-dns-component

1510fff

Merge branch 'master' into local-dns-component

a57443b

Merge branch 'master' into local-dns-component

848bca6

beiwei30 suggested changes Dec 1, 2021

View reviewed changes

ran go mod tidy on kafka and rabbitmq cert test

840f1e1

beiwei30 previously approved these changes Dec 2, 2021

View reviewed changes

daixiang0 previously approved these changes Dec 3, 2021

View reviewed changes

artursouza requested changes Jan 7, 2022

View reviewed changes

rainfd and others added 25 commits January 21, 2022 18:48

fix binding list test (dapr#1317)

928d0bb

Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

pub/sub rabbitmq: add publishing retries (dapr#1382)

9fb5050

Signed-off-by: Dmitry Shmulevich <dmitry.shmulevich@gmail.com> Co-authored-by: Looong Dai <long.dai@intel.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

1385 disable policy creation (dapr#1386)

0a59db2

* not modifying policy if disableEntity is set Signed-off-by: Amit Mor <amit.mor@hotmail.com> * not modifying policy if disableEntity is set Signed-off-by: Amit Mor <amit.mor@hotmail.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

add S3ForcePathStyle support for s3 binding (dapr#1360)

11dfb12

* add S3ForcePathStyle support for s3 binding Signed-off-by: rainfd <rainfd@live.cn> * add S3ForcePathStyle unit test Signed-off-by: rainfd <rainfd@live.cn> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

fix nits: duplicated err handle (dapr#1341)

1c6b54e

* remove duplicated err handle Signed-off-by: Taction <zchao9100@gmail.com> * fix test go mod tidy Signed-off-by: Taction <zchao9100@gmail.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

rewrite eventhubs component (dapr#1292)

b8d2885

Signed-off-by: Mukundan Sundararajan <msundar.ms@outlook.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

update license to Apache v2.0 (dapr#1406)

1da7642

Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Fix Blobstorage Bindings Conformance Test (dapr#1410)

2f8285f

Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

Ensure all go.mods are tidied with no diff (dapr#1415)

cebfb41

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

use the latest dapr/dapr in pubsub cert tests (dapr#1416)

2bf8afb

Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

update certification tests to latest daprd master (dapr#1417)

8466f22

Signed-off-by: Bernd Verst <4535280+berndverst@users.noreply.github.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

updated test and made it scan every 30sec

06366b8

Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

fix eventhubs conformance tests (dapr#1421)

0fd86e1

Signed-off-by: Mukundan Sundararajan <msundar.ms@outlook.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

hotfix: bindings.cron Input & Output communicates through a signal qu…

d44a4dc

…eue (dapr#1418) Signed-off-by: 1046102779 <seachen@tencent.com> Co-authored-by: Yaron Schneider <schneider.yaron@live.com> Signed-off-by: jigargandhi <jigarr.gandhi@gmail.com>

jigargandhi force-pushed the local-dns-component branch from 26e68e8 to d44a4dc Compare January 21, 2022 13:19

jigargandhi mentioned this pull request Jan 21, 2022

Added local component which scans processes every 30 seconds and cach… #1455

Closed

3 tasks

jigargandhi closed this Jan 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

created name resolution local component #1340

created name resolution local component #1340

jigargandhi commented Nov 26, 2021 •

edited

beiwei30 left a comment

jigargandhi commented Dec 2, 2021

beiwei30 left a comment

artursouza Jan 7, 2022

jigargandhi Jan 7, 2022

artursouza Jan 10, 2022

Taction commented Jan 8, 2022

jigargandhi commented Jan 8, 2022

beiwei30 commented Jan 12, 2022

codecov bot commented Jan 21, 2022

jigargandhi commented Jan 21, 2022

created name resolution local component #1340

created name resolution local component #1340

Conversation

jigargandhi commented Nov 26, 2021 • edited

Description

Issue reference

Checklist

beiwei30 left a comment

Choose a reason for hiding this comment

jigargandhi commented Dec 2, 2021

beiwei30 left a comment

Choose a reason for hiding this comment

artursouza Jan 7, 2022

Choose a reason for hiding this comment

jigargandhi Jan 7, 2022

Choose a reason for hiding this comment

artursouza Jan 10, 2022

Choose a reason for hiding this comment

Taction commented Jan 8, 2022

jigargandhi commented Jan 8, 2022

beiwei30 commented Jan 12, 2022

codecov bot commented Jan 21, 2022

Codecov Report

jigargandhi commented Jan 21, 2022

jigargandhi commented Nov 26, 2021 •

edited