A good way to implement a communication flow of request/response with serialport

[alvaronhs]

Hello, i'm developing a project where i have a communication protocol divided in "packages" and i have to request the package and get the response using the serial port. I made a version in Python and everything went fine, but i'm having a really bad time trying to implement it in javascript. I checked some of the recommended code in stackoverflow and tried to mount up a scratch to discuss what my difficulties are. I'm gonna apologize in advance for i have minimal knowledge of working with Streams/EventEmitters/AsyncIterators and i'm learning as i go.

Main objective:

1. Send a request with the package in the comms protocol, ex: '^P003NOM'.
2. Get the response which for the request above would be something like this: ^D054120,600,120,600,01400,00840,0,2,096,144,01012015,01095
3. The request/response would be called in a function, for example let response = getNOM('^P003NOM').
   That's all.

Problems in implementation so far:

• Answer stacking, so instead of getting: ^D054120,600,120,600,01400,00840,0,2,096,144,01012015,01095 , i would get something like this: ^D0010^D054120,600,120,600,01400,00840,0,2,096,144,01012015,01095^D021136,136,,,,,,,,,,,,,, which are the answers of multiple function calls eg getNOM(...), getBT1(...),getTYP(...).
• Same answer for all the requests with the stacked answers when trying to use Promises.
• So then i tried to use timeouts and async/await and that solved the stacked answers problem but that then creates long timeout intervals before getting the replies correctly and also eventemitters memory leaks which i assume is because of the multiple functions calls made where i set the EventEmitters to wait on data and that creates a new EventEmitter.

.
. (7 previous calls [EventEmitters can only have 10 concurrently which explains the leak message below])
.
nom: ^D054120,600,120,600,01400,00840,0,2,096,144,01012015,01095
bt1: ^D021136,136,,,,,,,,,,,,,,
typ: ^D0010
(node:103393) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 open listeners added to [SerialPort]. Use emitter.setMaxListeners() to increase limit

• When not using a good interval between functions called the problem of stacked/same answers are still there, which seems to indicate that async/await is doing nothing.

setInterval(runMyNextSECFunction, 300);

nom: ^D054120,600,120,600,01400,00840,0,2,096,144,01012015,01095
bt1: ^D054120,600,120,600,01400,00840,0,2,096,144,01012015,01095
typ: ^D021136,136,,,,,,,,,,,,,,

What do i want with this discussion?
To understand how to work with serialport to get a simple request<->response comm flow going.

The code for where the last problems listed are apparent is below:

import SerialPort from 'serialport'
import InterByteTimeout from '@serialport/parser-inter-byte-timeout'

const port = new SerialPort('/dev/ttyACM0', {baudRate: 2400})
const parser = port.pipe(new InterByteTimeout({interval: 50}))

function sendSync(port, src) {
    return new Promise((resolve, reject) => {
        port.on('open', () => {
            port.flush()
        })
        port.write(src)
        
        parser.once('data', (data) => {
            resolve(data.toString());
        });
        
        port.once('error', (err) => {
            reject(err);
        });
    });
}

async function typ() {
    const response = await sendSync(port, '^P003TYP')
    console.log(`typ: ${response}`)
}

async function nom() {
    const response = await sendSync(port, '^P003NOM')
    console.log(`nom: ${response}`)
}

async function bt1() {
    const response = await sendSync(port, '^P003BT1')
    console.log(`bt1: ${response}`)
}

const secFunctions = [
    typ,
    nom,
    bt1,
];
  
let counter = 0;

function runMyNextSECFunction() {
    let func = secFunctions[counter];

    func();

    counter += 1;
    if (counter >= secFunctions.length) {
        counter = 0;
    }
}

setInterval(runMyNextSECFunction, 500);

node version: v14.17.3
serialport version: v9.2.8

[alvaronhs]

I fixed the stacked answer/same answer by removing the interval and relying entirely on async/await. But that still left me with a memory leak error because of the constant creation of eventemitters, by placing parser.removeListener('data', () => {}) i got rid of the eventemitter as soon as i receive the packet, i thought this was not going to work because it kept giving me errors of a new eventemitter being created, turns out i tried the same code in the next day and everything worked, it seemed the gargabe collector waited a bit to do it's thing and kill the eventemitters not being used. Now that everything works the main question still stands, is this a good way to manage a response/request comm?

import SerialPort from 'serialport'
import InterByteTimeout from '@serialport/parser-inter-byte-timeout'

const port = new SerialPort('/dev/ttyACM0', {baudRate: 2400})
const parser = port.pipe(new InterByteTimeout({interval: 50}))

port.once('error', (err) => {
    console.log(err);
});

const sendSync = (port, packet) => {
    return new Promise((resolve, reject) => {
        port.write(packet);
        try {
            parser.once('data', (data) => {
                parser.removeListener('data', () => {})
                resolve(data.toString());
            });
        } catch (err) {
            reject(err)
        }
    });
}

const loopSec = async (index) => {
    const typ = await sendSync(port, '^P003TYP')
    const nom = await sendSync(port, '^P003NOM')
    const bt1 = await sendSync(port, '^P003BT1')
    console.log(typ)
    console.log(nom)
    console.log(bt1)

    if (index < 10) {
        console.log(index)
        loopSec(index + 1)
    } 

};

loopSec(0)

^D0010
^D054120,600,120,600,01400,00840,0,2,096,144,01012015,01095
^D021136,136,,,,,,,,,,,,,,
0
^D0010
^D054120,600,120,600,01400,00840,0,2,096,144,01012015,01095
^D021136,136,,,,,,,,,,,,,,
1
^D0010
^D054120,600,120,600,01400,00840,0,2,096,144,01012015,01095
^D021136,136,,,,,,,,,,,,,,
2
^D0010
^D054120,600,120,600,01400,00840,0,2,096,144,01012015,01095
^D021136,136,,,,,,,,,,,,,,
3
.
.
.

[akujbida]

I'd like to bump this, please. I've done quite extensive research and found many folks with same question but never a solution.

Like alvaronhs, I've tried eventEmitter, though instead of looping, I've used a promise and put that into a race with a timeout. Then to get retries, I put the write into a recursive function.

The whole thing is a house of cards and leaves me with dangling promises and uncertain states and very convoluted code (partly my fault, I'm sure).

Every example I find of using this module shows a single message out and single reply. To me, this seems unrealistic, effectively requiring that a human always be present as if using minicom. Surely there must be a canonical tidy approach in the authors minds.

Could someone please kindly show a robust usage that includes:

1. wait for reply without using loops or intervals, so the next message can go out immediately upon reply
2. timeout
3. retry limit (recursion is fine I think)
4. no flags
5. no dangling promises or excessive housekeeping

In Perl and I think Python, all above together is one or two lines so I'm sure the capability is present, I just can't find it.

[akujbida]

To wrap this up, at least for me, here is one approach. I got it from Chat GPT after several prompts. Looks ok. During the chat, I noticed a weird use of push(null) onto the fastq and Chat apologized to me for the error!! I include that portion of the conversation here for your amusement:

In parser.on('data'...), you push null onto the queue. Why?
ChatGPT
Apologies for the confusion caused. In the previous response, pushing null onto the commandQueue in the parser.on('data'...) event handler was an oversight.

To clarify, the purpose of pushing null onto the queue in that context was to trigger the processing of the next command in the queue. However, since the fastq module handles the automatic processing of commands based on the defined concurrency level, there is no need to manually push null in the parser.on('data'...) event handler.

Here's the corrected version of the code, removing the unnecessary commandQueue.push(null):

const Readline = require('@serialport/parser-readline');
const prompt = require('prompt-sync')(); // For user input
const fastq = require('fastq');

// Configure serial port
const port = new SerialPort('/dev/ttyUSB0', {
  baudRate: 9600,
  dataBits: 8,
  stopBits: 1,
  parity: 'none'
});

// Create a parser to read lines from the serial port
const parser = port.pipe(new Readline({ delimiter: '\r\n' }));

// Queue to hold commands
const concurrency = 1; // Adjust the concurrency level as per your needs
const commandQueue = fastq.promise(async (command, callback) => {
  // Send the command
  port.write(command, (err) => {
    if (err) {
      console.error('Error while sending command:', err.message);
    } else {
      console.log('Command sent:', command);
    }

    // Signal completion by calling the callback
    callback();
  });
}, concurrency);

// Event handler for receiving data from the serial port
parser.on('data', (data) => {
  // Process received data
  console.log('Received data:', data.toString());
});

// Example usage: sending the first command
prompt('Press Enter to start sending commands...');
commandQueue.push('Command 1');

// Prompt user for new commands
while (true) {
  const userInput = prompt('Enter a command (or "exit" to quit): ');

  if (userInput.toLowerCase() === 'exit') {
    break;
  }

  commandQueue.push(userInput);
}

// Close the serial port when exiting
port.on('close', () => {
  console.log('Serial port closed');
});

// Close the program
port.on('error', (err) => {
  console.error('Serial port error:', err.message);
  process.exit(1);
});

Apologies for any confusion caused, and thank you for bringing it to my attention.
This line above really came from Chat GPT. Fascinating.

I hope this helps others.