Replace internal controls with @ldapjs/controls

[jsumners]

When this PR is completed, it will remove all code that implements LDAP controls internally and replace their usage with the @ldapjs/controls module. This is a step toward making the client and server code separate. The ultimate goal being:

const { server, client } = require('ldapjs') // deprecated

// The future!
const server = require('@ldapjs/server')
const client = require('@ldapjs/client')

[jsumners]

I'm running into some issues here. I do not think the VirtualListView control code in this repo is correct. The spec for the control is defined at https://datatracker.ietf.org/doc/html/draft-ietf-ldapext-ldapv3-vlv-07#section-6.1. The spec says that the target offset is a CHOICE sequence. The code in this repo starts the CHOICE sequence with 0xA0:

node-ldapjs/lib/controls/virtual_list_view_request_control.js

Line 76 in 9143456

writer.startSequence(0xa0)

But the docs I have been able to find indicate that CHOICE should start with 0xA1:

(See also https://www.oss.com/asn1/resources/asn1-made-simple/asn1-quick-reference/choice.html)

In order to determine the truth of the matter, I am attempting to build an integration test suite against a server that is known to implement the VLV control correctly. This has resulted in:

• https://github.com/ldapjs/docker-test-opendj
• https://github.com/ldapjs/docker-test-apacheds

So far, the apacheds repo is the only one in which I have been able to successfully create a running server with all of the test data added to it. However, if one reviews the readme, we can see that ApacheDS advertises support for VLV but doesn't actually implement it.

But the OpenDJ project can't even get that far. It has some sort of bug that prevents the initial bootstrapping of data. As we can see from its readme I thought the next best option would be RedHat's 389 Directory Server. But that thing does not support Docker well at all.

I am hesitant to use the slapo-sssvlv overlay for OpenLDAP for a couple reasons:

1. I am not clear what an overlay does. Does it actually implement any logic to provide the feature? Or would it simply accept a control request and generate a dummy control response? Which would be as effective as the current ApacheDS implemenation.
2. The documentation for it is basically non-existent. So I'm not at all clear how to even add it.

At the moment, I'm thinking the OpenDJ route is the better route IF I can get a response from that team about the bug I am encountering. Or if I can find a way around it.

@soisik since you implemented this control in #553, any input you can provide here would be helpful. I am also considering removing support for the control until such time as we can write proper integration tests for it. In fact, that is likely the path I will take so that I can get ldapjs prepared for the Node.js v12 end-of-life date in a few weeks.

[jsumners]

🤔 the UnboundId SDK includes a bare bones in-memory directory server: tools/in-memory-directory-server. This may be usable as it purports to support VLV.

[jsumners]

🤔 the UnboundId SDK includes a bare bones in-memory directory server: tools/in-memory-directory-server. This may be usable as it purports to support VLV.

To this end, I have started https://github.com/ldapjs/docker-test-unboundid. But I've done all I can stand to do on this right now. Next time I work on this, I will be skipping the VLV controls because they cannot be accurately tested. We can revisit them after v3 ships unless someone else wants to put in the time to build a working integration test image.

[jsumners]

After some further reading about ASN.1 due to working on another aspect this overall work, I have determined the answer to the above question about the CHOICE encoding for the VLV control.

First, we need the spec's ASN.1 declaration for the control:

VirtualListViewRequest ::= SEQUENCE {
          beforeCount    INTEGER (0..maxInt),
          afterCount     INTEGER (0..maxInt),
          CHOICE {
               byoffset            [0] SEQUENCE {
                    offset          INTEGER (0 .. maxInt),
                    contentCount    INTEGER (0 .. maxInt) },
               greaterThanOrEqual  [1] AssertionValue },
          contextID     OCTET STRING OPTIONAL }

The answer is actually in my above comment:

A value derived from a CHOICE type is encoded according to the rules for the chosen type.

This, for me, is a difficult to parse sentence that says: if the value of the CHOICE is a SEQUENCE, then the tag is the SEQUENCE tag. However, a bit of extra knowledge is required:

https://web.archive.org/web/20220418005126/https://letsencrypt.org/docs/a-warm-welcome-to-asn1-and-der/

Thus, if we are encoding the VLV request control for a byoffset value, the tag is 0xa0 (standard SEQUENCE tag). If we are encoding for a greaterThanOrEqual value, the tag is 0x81 (context specific tag for the second value).

Given the above, we can assume the code for the VLV control is correct. However, we still do not have a way to write a verifiable integration test. Therefore, I am likely to continue with the plan of shipping v3 with the VLV controls disabled.

[UziTech]

LGTM

[jsumners]

⚠️ This issue has been locked due to age. If you have encountered a recent
problem that seems to be covered by this issue, please open a new issue.

Please include a minimal reproducible example
when opening a new issue.