calico-kube-controllers fix concurrent map writes issue #8706
Conversation
marvin-tigera
added
release-note-required
|
|
✅ Deploy Preview for calico-v3-25 canceled.
|
|
/sem-approve |
|
Thanks for the PR. Could you show where the conflicting Write is happening on the resource? If I understand the issue & panic correctly, the panic is generated by If the concurrent write is also occurring in the same Do we know what component is writing to that On a side-note, can you submit a PR of the fix to master branch instead, rather than directly to the release branch, to ensure we don't regress in future releases. We can then backport the master patch to 3.26. |
| @@ -126,7 +126,9 @@ func (c *calicoCache) Set(key string, newObj interface{}) { | |||
| if reflect.TypeOf(newObj) != c.ObjectType { | |||
| c.log.Fatalf("Wrong object type received to store in cache. Expected: %s, Found: %s", c.ObjectType, reflect.TypeOf(newObj)) | |||
| } | |||
|
|
|||
| // lock the cache | |||
There was a problem hiding this comment.
Something isn't quite adding up to me here - the underlying map is within the threadSafeCache instance, which IIUC should already handle this.
Do you know why we need this second mutex?
Description
Add mutex lock to the Set function to prevent a race condition that causes the process to panic with fatal error: concurrent map read and map write
Related issues/PRs
fixes 8705
Todos
Release Note
Reminder for the reviewer
Make sure that this PR has the correct labels and milestone set.
Every PR needs one
docs-*label.docs-pr-required: This change requires a change to the documentation that has not been completed yet.docs-completed: This change has all necessary documentation completed.docs-not-required: This change has no user-facing impact and requires no docs.Every PR needs one
release-note-*label.release-note-required: This PR has user-facing changes. Most PRs should have this label.release-note-not-required: This PR has no user-facing changes.Other optional labels:
cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.