feat: Add DataQueue class #151

markdirish · 2020-03-13T15:29:44Z

Add a DataQueue class that can be called similar to the Java toolkit
Experiment with pulling out API data into a const variable

PR opened to discuss work thus far and solicit input into how to improve pulling out these classes a la the Java Toolkit

Add a DataQueue class that can be called similar to the Java toolkit Experiment with pulling out API data into a const variable

markdirish · 2020-03-13T15:33:09Z

lib/DataQueue.js

+    program.addParam(this.name,    QRCVDTAQParameters.DataQueueName.type, { io: QRCVDTAQParameters.DataQueueName.io });
+    program.addParam(this.library, QRCVDTAQParameters.LibraryName.type,   { io: QRCVDTAQParameters.LibraryName.io });
+    program.addParam(length,       QRCVDTAQParameters.LengthOfData.type,  { io: QRCVDTAQParameters.LengthOfData.io });
+    program.addParam('',           `${this.maxLength}A`,                  { io: QRCVDTAQParameters.Data.io });


We talked about how to do this for "structures" (the array) to get the len and then using setlen, but can I do the same for single values? And should I here, with the LengthOfData parameter?

markdirish · 2020-03-13T15:33:49Z

lib/DataQueue.js

+// QRCVDTAQ Parameters
+const QRCVDTAQParameters = {
+  DataQueueName: { io: 'in',  type: '10A' },
+  LibraryName:   { io: 'in',  type: '10A' },
+  LengthOfData:  { io: 'in',  type: '5p0' },
+  Data:          { io: 'out', type: '10A' },
+  WaitTime:      { io: 'in',  type: '5p0' }
+}


Interested in what others think about pulling Parameter information out into a structure like this. Might work a little better after the this PR drops: #139

markdirish · 2020-03-13T15:35:31Z

lib/DataQueue.js

+  async _getDataQueueDescription(callback) {
+    const program = new ProgramCall('QMHQRDQD', { lib: 'QSYS' });
+
+    const receiverVariable = [
+      [0, '10i0'],
+      [0, '10i0'],
+      [0, '10i0']
+    ];
+
+    program.addParam(receiverVariable, { io: 'out', len: 'varLen'}); // Receiver variable
+    program.addParam(0,       '10i0', { setlen: 'varLen'}); // Length of receiver variable
+    program.addParam('RDQD0100',         '8A');   // Format name
+    program.addParam(this.qualifiedName, '20A');  // Qualified data queue name
+
+    this.connection.add(program)
+    this.connection.run((error, xmlOutput) => {
+      if (error) {
+        return callback(error, null);
+      }
+
+      const result = xmlToJson(xmlOutput);
+      this.maxLength = result[0].data[2].value
+      return callback(null, result);
+    });


Idea here is:

If we create a DataQueue using this API, we know what the max length is. But if we are consuming a DataQueue created elsewhere, we don't know the max size. So when we need to get the length the first time we call an API that requires us to specify the length of the receiver data

ThePrez · 2020-03-30T14:50:43Z

I like the idea, and if we can strive to be more like the revered Java Toolbox, that's great. However, now that we have the *DATA_QUEUE* SQL services, this is much less necessary
https://www.ibm.com/support/pages/ibm-i-services-sql
A Data Queue class and associated java types is still good for usability, so I'm not opposed to this enhancement, but want to doc that the SQL path is a food fallback if someone needs such function before this class is completed.

github-actions · 2020-04-30T00:09:23Z

👋 Hi! This pull request has been marked stale due to inactivity. If no further activity occurs, it will automatically be closed.

ghost · 2020-07-14T21:44:19Z

I like the idea, and if we can strive to be more like the revered Java Toolbox, that's great. However, now that we have the *DATA_QUEUE* SQL services, this is much less necessary
https://www.ibm.com/support/pages/ibm-i-services-sql
A Data Queue class and associated java types is still good for usability, so I'm not opposed to this enhancement, but want to doc that the SQL path is a food fallback if someone needs such function before this class is completed.

I'd like to see this class added since the SQL Data queue service aren't support on 7.2. Our upgrade is planned, but with the world as it it that may be delayed.

markdirish · 2021-03-04T14:18:10Z

I have a LUG customer concerned about the deprecation of iDataQueue, I'm going to make sure this PR is up to date and refresh any questions I have, then it might be a good idea to throw some developer minutes/hours at it to get it landed.

Signed-off-by: Mark Irish <mirish@ibm.com>

markdirish · 2021-03-04T16:30:44Z

Ok, I have tested the latest push:

Passing:

Creating a DataQueue with new DataQueue works
Creating a DTAQ on the system with create function works
Deleting a DTAQ on the system with delete function works
Receiving data with receive works
Sending data with send works

Failing:

When calling receive, if you don't pass a length, it will try to call QMHQRDQD to get information about the DTAQ. I am not sure how to add the params correctly to get it to work... particularly the receiver variable, type: 'ds', and the like.

I see I have some linting failing, I might have disabled eslint on my VSCode at some point. Will reenable and ensure Linting and DCO bot are happy. No idea why GitHub Actions tests are failing, I think I haven't touched anything outside of my file.

Lingering questions:

I see that xmlToJson was deprecated, along with all of the other functions we had. I have adjusted the class to return XML, but it seems like it would be best to return JSON? Since this is the first helper(?) class, maybe we can talk about how to do that best.
I see that everything is returning callback functions, and I have emulated that behavior. It is possible to have functions use callbacks or Promises, similar to how I do in the odbc package. Just check to see if a callback was passed as the final parameter, and if not, go to a Promise workflow. This would ensure API stability with anyone currently using the package, but would also allow developers to use the more modern Promise syntax.

abmusse

No idea why GitHub Actions tests are failing, I think I haven't touched anything outside of my file.

The build machine is currently down so fails to run the integration tests. We will need to create some tests for the new class though.

The PR linter check is failing because the title does not follow conventional commit type

I would suggest feat: Add DataQueue class

lib/DataQueue.js

abmusse · 2021-03-04T21:16:01Z

lib/DataQueue.js

+   * @param {number} maxLength
+   * @param {function} callback
+   */
+  async create(maxLength, callback) {


Should these methods be defined as async? I'm thinking this could lead to synchronization issues. Async function return a promise and within the function we call connection.run with is asynchronous function using the callback pattern. So we wouldn't wait for connection.run to complete before returning the promise.

So this goes with Issue #322 . I have always declared them as async since that's what they are. I wasn't aware the spec said I had to return a Promise, just that "Async functions can contain zero or more await expressions." Because of the latter I don't think using callbacks internally is a big deal. Will have to dwell on it, but maybe is a reason people don't mix callbacks/Promises in the same function

Signed-off-by: Mark Irish <mirish@ibm.com>

maghirian · 2021-03-05T01:18:28Z

Ok, I have tested the latest push:

Passing:

Creating a DataQueue with new DataQueue works

Creating a DTAQ on the system with create function works

Deleting a DTAQ on the system with delete function works

Receiving data with receive works

Sending data with send works

Failing:

When calling receive, if you don't pass a length, it will try to call QMHQRDQD to get information about the DTAQ. I am not sure how to add the params correctly to get it to work... particularly the receiver variable, type: 'ds', and the like.

I see I have some linting failing, I might have disabled eslint on my VSCode at some point. Will reenable and ensure Linting and DCO bot are happy. No idea why GitHub Actions tests are failing, I think I haven't touched anything outside of my file.

Lingering questions:

I see that xmlToJson was deprecated, along with all of the other functions we had. I have adjusted the class to return XML, but it seems like it would be best to return JSON? Since this is the first helper(?) class, maybe we can talk about how to do that best.

I see that everything is returning callback functions, and I have emulated that behavior. It is possible to have functions use callbacks or Promises, similar to how I do in the odbc package. Just check to see if a callback was passed as the final parameter, and if not, go to a Promise workflow. This would ensure API stability with anyone currently using the package, but would also allow developers to use the more modern Promise syntax.

Hey Mark....Regarding the lingering question on deprecated xmlToJson...If you have not figured it out by now :) use xml2js:

const { parseString } = require('xml2js');

connection.run((error, xmlOutput) => {
if (error) {
throw error;
}
parseString(xmlOutput, (parseError, result) => {
if (parseError) {
throw parseError;
}
console.log(JSON.stringify(result));
});
});_

Hope this helps.

[issue97] Add DataQueue class

fb66fda

Add a DataQueue class that can be called similar to the Java toolkit Experiment with pulling out API data into a const variable

markdirish requested review from kadler and abmusse March 13, 2020 15:29

markdirish commented Mar 13, 2020

View reviewed changes

DataQueue: remove some log statements

22a7885

kadler mentioned this pull request Apr 6, 2020

Data queue memory leak itoolkit 0.1.6 + idb-connector #230

Closed

github-actions bot added the stale label Apr 30, 2020

abmusse added the keep-open Exempts stale action from auto closing the issue/pr. label Apr 30, 2020

github-actions bot closed this May 8, 2020

abmusse mentioned this pull request May 8, 2020

Add exempt-pr-label #280

Closed

abmusse reopened this May 8, 2020

github-actions bot removed the stale label Oct 6, 2020

kadler linked an issue Oct 21, 2020 that may be closed by this pull request

Extract Data Queue functions in to a real DataQueue class #97

Open

markdirish added 2 commits March 4, 2021 08:46

Merge branch 'master' into issue97-dataqueue-class

32aec92

Signed-off-by: Mark Irish <mirish@ibm.com>

Update DataQueue class

646aa5c

Signed-off-by: Mark Irish <mirish@ibm.com>

abmusse mentioned this pull request Mar 4, 2021

Investigate supporting Promises and Callbacks #322

Open

abmusse reviewed Mar 4, 2021

View reviewed changes

lib/DataQueue.js Outdated Show resolved Hide resolved

abmusse reviewed Mar 4, 2021

View reviewed changes

markdirish changed the title ~~WIP: [issue97] Add DataQueue class~~ feat: Add DataQueue class Mar 4, 2021

Fix DataQueue implementation

37c37ac

Signed-off-by: Mark Irish <mirish@ibm.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add DataQueue class #151

feat: Add DataQueue class #151

markdirish commented Mar 13, 2020

markdirish Mar 13, 2020

markdirish Mar 13, 2020

markdirish Mar 13, 2020

ThePrez commented Mar 30, 2020

github-actions bot commented Apr 30, 2020

ghost commented Jul 14, 2020

markdirish commented Mar 4, 2021

markdirish commented Mar 4, 2021

abmusse left a comment •

edited

abmusse Mar 4, 2021

markdirish Mar 4, 2021

maghirian commented Mar 5, 2021

Passing:

Failing:

Lingering questions:

feat: Add DataQueue class #151

Are you sure you want to change the base?

feat: Add DataQueue class #151

Conversation

markdirish commented Mar 13, 2020

markdirish Mar 13, 2020

Choose a reason for hiding this comment

markdirish Mar 13, 2020

Choose a reason for hiding this comment

markdirish Mar 13, 2020

Choose a reason for hiding this comment

ThePrez commented Mar 30, 2020

github-actions bot commented Apr 30, 2020

ghost commented Jul 14, 2020

markdirish commented Mar 4, 2021

markdirish commented Mar 4, 2021

Passing:

Failing:

Lingering questions:

abmusse left a comment • edited

Choose a reason for hiding this comment

abmusse Mar 4, 2021

Choose a reason for hiding this comment

markdirish Mar 4, 2021

Choose a reason for hiding this comment

maghirian commented Mar 5, 2021

Passing:

Failing:

Lingering questions:

abmusse left a comment •

edited