Generate graphical display of difference between pre- and post- sort orders

[Ortham]

Back when LOOT was BOSSv3 beta 8, it could generate an image of the plugin graph used to sort the load order. This functionality was removed because rendering the graph image was slow, required the bundling of a separate executable, and the result too complex to easily glean useful information from.

An idea that came up was to save the graph data each time LOOT calculated a load order, and then use that data to produce an image that displayed the difference between the previously calculated load order and the newly calculated order.

As the difference would be much less to render than the whole graph, this should solve the performance problem, and the difference would be of greater benefit in informing users about what changes had occurred than the full graph.

Boost's graph library can output GraphVis files, and a quick search comes up with a bunch of JavaScript renderer libraries that could be used to produce images from them, which would be neater than using a separate executable (and could be done asynchronously with greater ease, so that the whole process was not slowed down).

[Ortham]

Something else I just thought of: if this is done, then edges should have a "type" property, to distinguish between priority, conflict, etc. edges, and the edge types should also be displayed in the generated image.

[MacSplody]

Taken from discord

On the prompt to Apply sorting, the plug-in could have it's position before and after sorting displayed in the left panel. Just the position number would not be sufficient though. Maybe an arrow and position change count, with Ɵ for unchanged.

ElminsterAUToday at 04:23
I was going to write a request about that at some point
I would like to see the list basically doubled, with old and new order, and lines connecting the same file.
Ok, build the two lists, then align them as good as possible, finally draw lines between the same mod in both lists if they are not aligned

https://cdn.discordapp.com/attachments/473543945188802580/494356108131958784/unknown.png

Just my thought on it, but something built into the UI instead of a generated image might work instead, but I'm not sure if that would be more costly than generating an image. I mean you already have pre and post sort load order when apply is showing.

The second list might not even be necessary.
It would have to show which mods where moved, the direction and how far. While ignoring mods which moved relative to that change.
This could even be extended further to have labels for the rule type which caused the change.
Such as Group, Load after, Master.

[lmstearn]

LO history sounds like a great idea tailoring to such parameters as:

• Making the feature optional (off by default).
• Number of previous LOOT runs (runs performed within a specified time interval may not require logging).
• Extra focus on promoted/ demoted mods in past Masterlist revisions (pathing: requires more thought).
• Version changes of active mod X may also impact history collation of LOOT orders.
• Versioning: LOOT logging and D/B mgt.

[pStyl3]

Two months ago, we discussed this a bit more in Discord, and Ortham produced a couple more graphs. So let's post the discussion here as well, so that it doesn't get lost.

Ortham:

Hah, maybe I'll take a look at 377 now that the plugin graph has 2/3 fewer edges. Though the image is probably still a crazy number of lines all over the place.
I can cut edges some more, but doing that would probably make things slower again, unless I can think of an efficient way to do it...

pStyl3:

Maybe produce a graph with LOOT as it is right now and then go from there? Whatever the complexity, it wouldn't be too bad to see what such a graph would look like. Maybe we get new ideas on how to iterate on it from there.
Would it look like the graphs that you produced before?

The mentioned graphs:
Image 01 - Image 02 - Image 03

Ortham:

How it looks depends on the visualisation library I use, if I used the same library it would look similar (but with 1/3 the lines).
Getting the data from libloot for a rough stab at it should be fairly straightforward, the main thing is I need to decide what that interface would look like.

Ortham:

Here's the graph of a very small load order, visualised using the same library and algorithm as the groups editor uses.
Image 04
There's a small problem: I tried to generate one for my 400 plugin test load order, and Chromium killed the page after it ate 1.3 GB of RAM...
I also tried the library I was using a few years ago, and that just gave up instead of crashing.

Ortham:

It managed to finish rendering a circular 'breadthfirst' layout - orange isn't the background colour, that's all lines...
Image 05
But yeah, if this is going to go anywhere, I need to find a more efficient library...

Kerber

What is that library?

Ortham:

cytoscape.js

Ortham:

Using https://github.com/vasturiano/force-graph now, it uses WebGL to improve performance, so visualisation is feasible, just useless due to the sheer amount of information - and this is with a third as many lines as there would have been a couple of weeks ago!
Image 06
(ignore that the graph is overflowing its dialog box)
I think I need to try reducing the number of edges in the graph before I go any further with this.

[pStyl3]

pStyl3:

Those graphs include all plugins in the load order, right?
Basically, it's the load order (after a sort, that is).
What about a graph that looks like this:
Idea 01
Each point is a plugin. The size of each point represents the number of factors, why said plugin is in that place (master flag, load after, and so on).

If you right click one of the points, it will look like this:
Idea 02

And if I click on, say "group" it then expands to:
Idea 03

Would any of these ideas make any sense?
I'm trying to come up with a way to hide information if they aren't needed all the time.
Question would also be, if that would make any tangible sense for the user.
Maybe more, if there would be numbers to each node, indicating how many slots the individual plugin went up or down through sorting. Something like this:
Idea 04

Also, if the load order graph would be displayed after sorting, maybe mention it in the header bar
Show Graph | Accept | Cancel
Those 3 graphs you generated above look great, but as you also mentioned, the sheer amount of information makes it difficult to make anything from it.
And even if we assume that the average load order has 100 plugins in it, we still would need to make something out of a load order with 300 plugins and more (maxing out the usual slots + lots of .esl's).

Alternatively:
Idea 05
A right click on a node will reveal all connections to other plugins, with the color of each connection revealing what sort of connection it is (loadafter, group, etc).

And if there is a list of some sort at the side, detailing what colors will represent what (load after, groups, .. ) .. by clicking on said explanation one could maybe expand / collapse every connection of that sort in the load order, for all plugins.
Though I would imagine that expanding/collapsing so much information would probably take considerable pc horse power to do...

---

Ortham:

Filtering is possible and would help. Displaying all the plugins in a line that is their load order should also be doable, but I'll need to expose more info to do that. I don't think just having displacement numbers would help, it's more interesting to see why something has moved than how far, and anyway how do you tell how far anything has moved? (e.g. have a lot of plugins moved down one position, or has one plugin moved up many positions?)
And yes, this would work better as a "show graph" button when sorting, but was easier to hack in as a separate thing.