v5 Creating filters

Overview

V5 filters are awesome.

Coordinate systems

There are 3 unit measures of coordinates:

1. Normalized (0,0) is top-left corner, (1,1) is bottom-right.
2. Virtual units (CSS), its units that pixi use for containers, displayObjects and so on. (0,0) is left-top corner, (w,h) is right-bottom.
3. Physical pixels (pixels), basically same a screen units. Difference appears on retina screens. As an example, it helps in case you want to know physical pixel on the right/left/top/down of current one.

Here and in all docs we use normalized word for first type, pixels for third type. By default we are talking about virtual units (CSS).

There are 5 coordinates systems in pixi filters. Each can be used with any of three types.

1. Input coords - temporary pow2 texture that FilterSystem took from the pool. Used for texture2D sampling.
2. Screen coords - do not depend whether output is temporary texture or screen, whether there we are 10 filters away from screen, it is the screen coord.
3. Filter coords - (0,0) is mapped to top-left corner of part of screen that is covered by filter. For CSS or physical units it has the same scale but different offset than screen.
4. Sprite texture coords - sometimes there's extra sprite input in filter. Sprite is positioned in pixi stage tree and its area does not equal filter area. This is what we pass in texture2D() for extra sampler. Example is DisplacementFilter
5. Sprite atlas coords - sprite can use texture from an atlas, and just sprite texture coords aren't enough to get the correct value from texture2D. Example: SpriteMaskFilter

Default filter code

Filter constructor params include vertex and fragment shaders. What happens if we specify null or undefined in them?

new Filter(undefined, fragShader, myUniforms); // default vertex shader
new Filter(vertShader, undefined, myUniforms); // default fragment shader
new Filter(undefined, undefined, myUniforms);  // both default

Default vertex shader:

attribute vec2 aVertexPosition;

uniform mat3 projectionMatrix;

varying vec2 vTextureCoord;

uniform vec4 inputSize;
uniform vec4 outputFrame;

vec4 filterVertexPosition( void )
{
    vec2 position = aVertexPosition * max(outputFrame.zw, vec2(0.)) + outputFrame.xy;

    return vec4((projectionMatrix * vec3(position, 1.0)).xy, 0.0, 1.0);
}

vec2 filterTextureCoord( void )
{
    return aVertexPosition * (outputFrame.zw * inputSize.zw);
}

void main(void)
{
    gl_Position = filterVertexPosition();
    vTextureCoord = filterTextureCoord();
}

• aVertexPosition is normalized filter coord

• vTextureCoord is normalized input coord

• aVertexPosition * outputFrame.zw is filter coord

• vec2 position = aVertexPosition * max(outputFrame.zw, vec2(0.)) + outputFrame.xy is screen coord

@ivanpopelyshev :

1. I do not know why is there max
2. I recommend to put it in vFilterCoord varying

Default fragment code

varying vec2 vTextureCoord;

uniform sampler2D uSampler;

void main(void){
   gl_FragColor = texture2D(uSampler, vTextureCoord);
}

As mentioned above, vTextureCoord is normalized input coord that we pass to default sampler.

Default shaders in v4

v4 vertex shader

Default shaders in PixiJS v4 differ from the ones in v5. If you are porting filter from v4 please use the following code for vertex shader:

attribute vec2 aVertexPosition;
attribute vec2 aTextureCoord;

uniform mat3 projectionMatrix;

varying vec2 vTextureCoord;

void main(void) {
    gl_Position = vec4((projectionMatrix * vec3(aVertexPosition, 1.0)).xy, 0.0, 1.0);
    vTextureCoord = aTextureCoord;
}

aTextureCoord is normalized input coord that is passed through. If PixiJS v5 detects that attribute, it switches to v4-compatibility mode that adds old filter uniforms.

v4 default fragment shader

In case you are porting stuff from v4 and you used default fragment shader, you may notice that your filter look different in v5. That's because default v4 fragment code had a serious problem regarding double multiplication by alpha.

v4 dimensions uniform

There is very popular workaround with dimensions uniform in v4, it looks like that:

apply (filterManager, input, output, clear)
{
    this.uniforms.dimensions[0] = input.sourceFrame.width;
    this.uniforms.dimensions[1] = input.sourceFrame.height;
    filterManager.applyFilter(this, input, output, clear);
}

Unfortunately, in v5 it crashes because input is not RenderTarget but RenderTexture, it has no sourceFrame field. To fix the crash you have to:

1. replace input.sourceFrame to input.filterFrame.
2. add dimensions uniform in filter constructor params or body: this.uniforms.dimensions = new Float32Array(2);

However, I advice you to remove it completely in favor of built-in inputSize uniform, inputSize.xy is the size of filter area in pixels, exactly the same as dimensions. In that case you can remove extra code from constructor and apply function.

v4 padding difference

Padding in v5 applied before autoFit, in v4 it was applied after. That leads to the issue that blurFilter an other filters with padding are treated differently: https://github.com/pixijs/pixi.js/issues/5969

Multi-pass filters

Filters can use temporary renderTextures to apply shader two times, or to apply inner filters

apply(filterManager, input, output, clear) {
    let rt = filterManager.getFilterTexture();
    filterManager.applyFilter(this, input, rt, true);
    filterManager.applyFilter(this, rt, output, clear);
}

In v4 that function was getRenderTarget(clear, resolution). In v5 you can use getFilterTexture(resolution).

getFilterTexture() returns the texture the same size as the input. If you use resolution, filter area in new texture can have different normalized coordinate system! It is fine, unless you start to use it as an extra sampler.

Even more, due to fullscreen filters logic, there's no guarantee that getFilterTexture(0.5 * input.resolution) returns the texture that is exactly two times smaller than the input.

If you want to have it as an extra sampler, you have to use conversion functions.

Suppose we have inner filter that produces result in temporary texture with smaller resolution.

apply(filterManager, input, output, clear) {
    let rt = filterManager.getFilterTexture(0.5 * input.baseTexture.resolution);
    this._innerFilter.apply(filterManager, input, rt, true);
    this.uniforms.innerSampler = rt;
    this.uniforms.inputToTex = [input.width / rt.width, input.height / rt.height];
    filterManager.applyFilter(this, input, output, clear);
}

uniform sampler2D innerSampler;
uniform vec2 inputToTex;

{
    ...
    vec2 texCoord = vTextureCoord * inputToTex;
    vec4 inputColor = texture2D(uSampler, vTextureCoord);
    vec4 rtColor = texture2D(innerSampler, texCoord);
}

Fullscreen filters

This line forces pixi to use temporary renderTexture of the same size as screen:

filter.filterArea = renderer.screen; //same as app.screen

Also known as pixi-v3 emulation mode.

Input, output and screen coords are the same in that case, and you don't have to use conversion functions.

Conversion functions

vTextureCoord // normalized input
vTextureCoord * inputSize.xy // filter pixel(css)
vTextureCoord * inputSize.xy + outputFrame.xy // screen pixel(css) coord
vTextureCoord * inputSize.xy / outputFrame.zw // filter (normalized)
vTextureCoord * inputPixel.xy // filter pixel(physical)