Fix ingesters with less tokens stuck in LEAVING

[marianafranco]

Signed-off-by: Mariana Franco marfram@amazon.com

What this PR does:

It's possible to have ingesters in the ring with less tokens than configured because of conflict resolution:

cortex/pkg/ring/model.go

Lines 354 to 361 in f930c7a

	// This function resolves token conflicts, if there are any.
	//
	// We deal with two possibilities:
	// 1) if one node is LEAVING or LEFT and the other node is not, LEVING/LEFT one loses the token
	// 2) otherwise node names are compared, and node with "lower" name wins the token
	//
	// Modifies ingesters map with updated tokens.
	func resolveConflicts(normalizedIngesters map[string]InstanceDesc) {

In this case, the ingester will get stuck in LEAVING when they are evicted and rescheduled.

Looking at previous changes, seems that there was not a good reason to check for NumTokens: #1921 (comment)

Which issue(s) this PR fixes:
Fixes #

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[friedrichg]

Thank you for your contribution.

I also notice that function is not completely tested with unit tests

[marianafranco]

I also notice that function is not completely tested with unit tests

Sorry, I don't have time to improve these unit tests now. I can create an issue if you want. Hopefully other contributors can help to improve the coverage.

[friedrichg]

Thanks!

[marianafranco]

@alanprot Could you take a look?