I have observed that injecting to user.css the following seems to fix it:

.ButtonInner-medium-iconOnly {
    background-color: var(--background-base);
    color: var(--spice-text);
}

Not sure if it cascades the problem somewhere else.

OK I did some more investigations on this. Here's why I observed.

CSS injection of the buttons are as follows:

Some color schemes such as base, dark, purple, etc, have same values for sidebar and button-active. Either we edit those files to ensure the contrast values make sense (which looks like a lot of effort). Or we inject above CSS, I haven't tested all schemes to check if the css don't break anything but that filed looks promising (until spotify decides to use that classname somewhere else).