More prominently exposing SPI score

[heckj]

I know it's a possibly very controversial subject, but I wanted to broach the idea of exposing the computed SPI score for a package more prominently.

If you know where to look and dig, it's available - now with a great layering of explanation thanks to Cyndi's work. At the same time, the number of github stars is featured for each package, both in the packages page, and in the results from a search. I personally think the computed SPI score has more value than the stars from GitHub, but it would be even more interesting to me to expose them together.

I'm certain this has been a topic of discussion at some point, probably repeatedly, but i wanted to ask here to see what y'all thought.

I was coming at it from the perspective of wanting to evaluate search results, noted that stars was presented, and then rather wanted the SPI package score alongside that. I think it would influence people looking for a package, and I'm a fan of the scoring criteria and values - so i think it's worthwhile generally and outside my wacky obsession with search relevance.

[daveverwer]

It can be a controversial subject, but it's also one that's worth discussing.

We considered where to put the package score at length while developing the new section on the maintainer's page. The initial plan was that it should be somewhere public on the package page, but there were two main factors behind moving it to the maintainer's page:

One score may not be good enough

A single score can be a little reductive when it comes to people deciding on a dependency. Two scores or grades might give a better picture, depending on what someone is looking for.

What I have in mind right now, and this has been flying around my head since before we even launched back in 2020, is that there are two primary dimensions that I'd like to know about if I am searching for a package:

1. How well-maintained is it?
2. How is the code quality?

Using an occasionally-maintained package can be perfectly valid if the code quality is good. Not every problem needs constant iteration, and while Swift versions used to necessitate updates, that's no longer the case. Equally, using a package that isn't quite there on code quality can be desirable if it's iterating quickly and being well maintained.

Even if splitting the score into two grades is a good idea, and I’m not 100% sure it is, I don’t think we’re ready to move to a system like this because of the second point.

Not enough indicators

We don’t have enough indicators to split the score and barely have enough to make it useful. Some of those indicators need significant improvement, too. For example, having a test target is a good first step towards knowing if a package has tests, but it’s a long way from:

1. Measuring whether the test target contains tests
2. Counting the number of tests
3. Running the tests
4. Measuring code coverage

I’m not saying we want to get to point 4 in that list, but we’re not even up to point 1 yet 😬

I don’t want to wait until I’m 100% convinced we have a foolproof algorithm for package quality because that’s not even the goal. I want to be closer than we are right now before making it more public.

You’re also correct that putting the stars on the page and the search results isn’t ideal, but I think I’d like to be a little further down this road before adding a score there because right now, I don’t think the scoring criteria would hold up well enough to the scrutiny it would bring them.

[daveverwer]

Pinging #2591 so we have these two threads linked together.

[finestructure]

I personally would prefer the package score (or scores) to remain a partially visible entity. We know it's only ever going to be an approximation of something resembling a quality score and giving it too much prominence will just invite too much obsession with it.

I think it's good that the breakdown highlights the areas of interest but I fear making the score appear all over the place would just lead folks to add empty test targets and the like. I believe relevance by obscurity is the only viable approach we have here.

Honestly, I'd be in favour of removing stars from the search results before we add the score. It's a flawed metric and probably shouldn't be that prominent either.