What is the type for a file/blob/buffer?

[jsoneaday]

I am using Node/Fastify and Postgres. I would like to receive bytea from my postgres db and reply with a blob, arraybuffer, etc. In other words I need to return a file, but use the Fastify schema like this. It seems there is a type Any but is this the correct type to use?

schema: {
        params: Type.Object({
          userName: Type.String(),
        }),
        response: {
          200: Type.Object({
            userName: Type.String({ maxLength: 50 }),
            description: Type.String({ maxLength: 250 }),
            id: Type.BigInt(),
            createdAt: Type.Date(),
            updatedAt: Type.Date(),
            fullName: Type.String({ maxLength: 100 }),
            region: Type.Optional(Type.String({ maxLength: 50 })),
            mainUrl: Type.Optional(Type.String({ maxLength: 250 })),
            avatar: Type.Any(),
          }),
        },
      },

[sinclairzx81]

@jsoneaday Hi, Sorry for the delay on this (have just been catching up on things and noticed this went unanswered)

I would like to receive bytes from my postgres db and reply with a blob, arraybuffer, etc. In other words I need to return a file, but use the Fastify schema like this. It seems there is a type Any but is this the correct type to use?

There's a few ways you could type this response (and if any or unknown works for you, it's fine to use those), but if you're looking specifically to send binary data (the avatar) in the Json response payload, the best type to use (perhaps unintuitively) is string.

const T = Type.Object({
   avatar: Type.String({ contentEncoding: 'base64' })
})

This would require your Fastify route to base64 encode the image data prior to sending. Note also, when using Fastify, you should only use types prefixed with [Json] as extended types like Type.Uint8Array() are not understood by Ajv or the Json schema specification (even though this would be the most natural type to use in this scenario).

Generally, it's good to keep in mind the transmission format (Json in this case) when constructing the type, There are alternative formats (like msgpack, cbor) that do support actual binary encoding, but if using the default Fastify pipeline, it's better to limit the types to those representable in that pipeline.

Hope this helps
S

[jsoneaday]

Thank you. If I use Type.Any could I send back a Buffer as the avatar from Fastify?

[sinclairzx81]

@jsoneaday Hi,

If I use Type.Any could I send back a Buffer as the avatar from Fastify?

I don't think so (although you would need to check how Fastify encodes for buffers). Generally when writing types, the best place to start is imagining what the data will look like while in transit. In Fastify, the transmission format is typically JSON for structured data (as per schema). But I wouldn't expect it to be possible to send binary data in a json message. Below is a reduced version only including the buffer, and showing what happens when it is serialized to JSON.

const V = { data: new Uint8Array(8) } // where data in your image data

const J = JSON.stringify(V) 

console.log(J) // {"data":{"0":0,"1":0,"2":0,"3":0,"4":0,"5":0,"6":0,"7":0,"8":0,"9":0}}
               //         ^
               //         incorrect

So, clearly this isn't right. However if we encode binary data into base64, we have a way to embed the data in Json. This is a fixed version.

const V = { data: Buffer.from(new Uint8Array(8)).toString('base64') } // encode buffer as base64 string

const J = JSON.stringify(V) 

console.log(J) // {"data":"AAAAAAAAAAA="}
               //         ^
               //         correct

This is better, however the receiver of this data must be decode from base64 to Uint8Array (have a search for "base64 to bytes" online for how to do this on different environments). The main takeaway here is that "in order to serialize binary data in Json, you need to encode the data into something permissible on the Json format", which is strings in this case.

Here is how the Type should look if expressing the data in transit.

const T = Type.Object({
  data: Type.String({ contentEncoding: 'base64' }) // This schema informs that the binary 
                                                   // data will be base64 string encoded.
})

There are ways to help simplify some of the encoding and decoding in TypeBox, making some of this process automatic, but to make use of it, you would need to ensure data can be appropriately encoded and decoded manually using this base64 technique. Outside of this, there may be some features of Fastify that can help streamline this process also (you would need to check the Fastify documentation though)

Hope this helps!
S

[M-jerez]

Hi @sinclairzx81 quick related question.

Wondering if there is a particular reason for supporting Type.Uint8Array() but not the rest of Typed Arrays ?
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray#typedarray_objects