Ideas for pixel-perfect, non-blurry display

[shreeve]

Disclaimer: The pixel-perfect / blurry issue has many other tickets describing a whole variety of possible workarounds and tweaks, but I'm adding this as a new ticket in the hopes that some of these ideas can help further improve or refine the positioning in popper.

First, popper calculates it's initial positioning using getOuterSizes.js. Unfortunately, this uses element.offsetWidth and element.offsetHeight to calculate popper's dimensions. The problem is that these only return integer values, so calculations based on fine positioning (like that which may lead to blurriness) are often slightly off. It's this kind of "slightly off" stuff that makes you pull your hair out when things don't line up perfectly.

Instead of using these methods, we are calling element.getBoundingClientRect, which returns exact fractional (if necessary) values for the position of popper. Doing this allows the initial positioning values to be more accurate and allow better placement (with the chance for less blurriness) later on. We created a modifier, of sort order 50 (so it runs as the first modifier, not sure if this is correct or ideal). It runs before the other modifiers though. Here's what our popperOffsets call looks like:

            popperOffsets:
              order: 50
              enabled: true
              fn: (data, options) ->
                dom = data.instance.popper
                ref = data.offsets.reference
                pos = data.placement.split('-')[0]

                css = dom.ownerDocument.defaultView.getComputedStyle dom
                bcr = dom.getBoundingClientRect()

                [t, r, b, l, w, h] = 'top right bottom left width height'.split ' '

                vert = pos is t or b # top/bottom?
                main = pos is t or l # top/left?

                [start, half] = if vert then [l, w] else [t, h] # used to center
                [shift, full] = if vert then [t, h] else [l, w] # used to offset

                pop =
                  [t]: ref[t] # align top
                  [l]: ref[l] # align left
                  [w]: bcr[w] + parseFloat(css.marginLeft or 0) + parseFloat(css.marginRight  or 0)
                  [h]: bcr[h] + parseFloat(css.marginTop  or 0) + parseFloat(css.marginBottom or 0)
                  position: data.offsets.popper.position

                pop[start] += (ref[half] - pop[half]) / 2 # center on axis
                pop[shift] += if main then -pop[full] else ref[full] # adjust to edge

                data.offsets.popper = pop
                data

Second, the computeStyle modifier is used to calculate the actual positioning of the popper before rendering. It's this code that is responsible for figuring out exact placement and whether the GPU should be used (via translate3d) or if "normal" positioning (via top and left) should be used instead. The current code has a whole bunch of magic to try to determine how to correctly position the text to make it line up and not be blurry, etc.

What we've found is that GPU / hardware accelerated positioning (via transform: translate3d(...)), which is supposed to be much more efficient and generally way better, always leads to some blurry pixelation unless the positioning occurs on a "proper" pixel boundary. We've found that for a non-retina display, this means that we see pixelation unless the final position is integer pixel-aligned (ie - no decimal). On retina displays (which have a window.devicePixelRatio of 2), that we can do GPU position on integer pixel or 0.5 pixel boundaries and not see any blurriness. I assume if window.devicePixelRatio is 4, then blurriness would not occur on integer, 0.25, 0.5, or 0.75 pixel boundaries. In our code, if we are "off" by more than some small amount (0.001 of a pixel) from one of these boundaries, then we punt and failover to using "normal" positioning. But, if we are within the small delta or a proper pixel boundary, then we use the GPU via translate3d. This means that the GPU can be used dynamically, based on the context of that specific popper. The code for our computeStyle modifier is:

            computeStyle:
              order: 850
              enabled: true
              fn: (data, options) ->
                dom = data.instance.popper
                ref = data.offsets.reference
                pop = data.offsets.popper

                outer = dom.getBoundingClientRect()
                inner = dom.querySelector('.content-inner').firstChild.getBoundingClientRect()

                [t, r, b, l, w, h] = 'top right bottom left width height'.split(' ')

                x = pop[l] - inner[l] + outer[l]
                y = ref[t] - inner[t] + outer[t]

                # use gpu if possible
                dpr = window.devicePixelRatio
                gpu = ![x, y].find (n) => Math.abs(n - frac n, dpr) > 0.001

                styles =
                  if gpu
                    transform: "translate3d(#{x}px, #{y}px, 0)"
                    top: 0
                    left: 0
                    willChange: 'transform'
                  else
                    top: y
                    left: x
                    willChange: 'top, left'

                data.attributes  = { 'x-placement': data.placement, data.attributes... }
                data.styles      = { data.styles..., styles... }
                data.arrowStyles = { data.offsets.arrow..., data.arrowStyles... }
                data

The frac helper is used to for fractional steps, so frac 3.333, 4 would yield 3.25, since that's the closest 1/4th of an integer. Here's that code:

  frac = (val, n=1) -> if n > 1 then Math.round(val * n) / n else if n is 1 then (val | 0) else val

The end result, for us at least, has been pixel-perfect poppers that use the GPU when they can and failover to the not-optimized, but not-blurry either "normal" positioning. This gives us the best of both worlds, for only a small amount of tweaking of the code.

Hope this helps and sorry for creating a new ticket. It just didn't seem like this should be "tacked on" to the other closed tickets in the issue tracker.

[atomiks]

Amazing post. Would be great if you converted it to JS

I mostly did it: https://codepen.io/anon/pen/LaMPbo?editors=0110, but I don't understand the x & y variables in computeStyle (and probably the gpu variable is wrong too)

[FezVrasta]

I don't think I fully understand. We already round the offsets to the closest integer

[atomiks]

So I tried implementing this in core [but failed I think since the positions break when resizing] - what I noticed is that it can be 0.5px off using the fractional values compared to the current solution I made (using getRoundedOffsets). I tested using the playground from #715

[shreeve]

@FezVrasta - There are a few things that I learned over the past few days working on this issue on Chrome 72 on macOS Mojave.

It seems there are two main ways that browsers can position elements. The first is the "normal mode", in which the browser is in charge of all positioning, doesn't use the GPU, and aligns fractional pixels to it's own sub-pixel grid that is able to produce "not exact" positioning, but it's at least crisp and clear. We could use this mode all the time, but it would be nice to be able to improve efficiency by using the GPU to offload the work and get better performance. The problem is that, while "GPU mode" is more efficient and can place elements on exact sub-pixel locations, this can easily lead to blurriness of text. In order to prevent the blurriness of text in GPU mode, we need to make sure that the position aligns to a proper sub-pixel boundary, which is dependent on the device pixel ratio (via window.devicePixelRatio). If the dpr is 1, then the GPU must align to integer positions or the element will blur. If the dpr is 2, then we can align to integer or 0.5px without blur (4 would be 0.00, 0.25, 0.50, and 0.75; and so forth).

So, the basic idea is this: 1) make sure we determine the initial position correctly (by getting exact/fractional starting width/height via getBoundingClientRect() instead of the integer-only width/height via offsetWidth and offsetHeight), and 2) if we determine that the final popper position falls cleanly on one of the dpr-derived sub-pixel positions, then we can use the GPU to exactly position the element using transform: translate3d(x, y, 0), but otherwise we use the non-GPU based "normal mode" and use left: x/top: y to position the element.

[shreeve]

Amazing post. Would be great if you converted it to JS

I mostly did it: https://codepen.io/anon/pen/LaMPbo?editors=0110, but I don't understand the x & y variables in computeStyle (and probably the gpu variable is wrong too)

@atomiks - Sorry for leaving some extra sauce of mine in the code above. You are right that's not germane to the general solution.

What I'll work on now is a general solution in the form of a PR. I was just deep in the weeds and wanted to create an issue to track these concepts before they slipped my mind. :-)

[shreeve]

@atomiks - I just saw your code updates. I will check them out in more detail.

FWIW, the reason we are so interested in pixel-perfect display is because we have an existing element that contains several pieces of data, and when you hover on it, a new popper is displayed "on top of" the previous element but it contains a lot more information and details. So, if the new popper is even slightly off, it is really noticeable and obvious to tell that there has been a shift. So, we need the popper to be precisely positioned, so there is zero "shift" or jitter when it displays. We have been able to achieve this in our testing. If we can get these tweaks into the mainline/general codebase that would be awesome! I'll use your tweaks as a baseline. Thanks!

[shreeve]

@atomiks - The inner and outer are based on our internal use case, not needed here. The x and y are basically just the left and top. The code referring to the 0.001 just tries to determine if the final coordinates are "close enough" to being able to use the GPU or not. If they are close enough, then gpu will true. If not, then it'll be false and things will fail over to the "normal mode" placement (ie - no GPU involved).

[mreinstein]

@shreeve this is really cool! I think this logic could be generalized and useful in cases outside of popper. (e.g., I'm building a web game that has a pannable/zoomable map built on transform/translate/scale and my maps get blurry if I don't snap the positioning on correct increments, depending on the dpr.)

Maybe this could be exposed as a function in an npm module? The inputs and outputs are pretty clear.

[FezVrasta]

If anyone would like to try out these ideas on the next branch I think it would be great. The code base is way cleaner so it should be easier to experiment with these ideas.

[mreinstein]

I'm wondering, rather than go through the complexity of detecting gpu and non-gpu cases, would it be easier to simply snap the value to one of the allowed fractions?

e.g., you are on a screen with dpr 4, and you want to display at x=45.312, y=100.88. Snapping the values to x=45.25, y=101 would enable always using gpu for positioning, while always eliminating blurriness.

[mreinstein]

Have we considered non-integer device pixel ratios?
For example if you're on a non-hdpi screen, your normal dpr would be 1. If you zoom out to 80%, the dpr will be set to 0.8, or possibly even a crazy decimal value (on my screen I see 0.800000011920929)

How would we handle this?

[atomiks]

In v2, we're now forcing top/left properties for <@2x displays. This should effectively prevent blurry problems (when zooming or when the popper sits on a subpixel, or due to layer acceleration), since those properties round internally and use whole pixels.

There should now also be perfect positioning since we're using getBoundingClientRect and don't do any rounding. There will sometimes be 0.5px errors for low PPI displays since the popper can't sit on a subpixel due to blurriness (and we're using top/left, which round internally), if the popper/reference are at an odd/even size.

For >@2x displays where we use translate3d, the "subpixel" is actually a real pixel if the decimal ends in .5, so there is no blurriness. There may still technically be blurriness on high pixel density displays if the decimal is something like 1.128192px, but it's far less of an issue due to the high pixel density, you probably won't be able to tell... we'll have to continue investigating this further though