Mutable Access to Encoded Values

[azdle]

Would it be possible to add some mechanism to get mutable access to an encoded value? In my current use case it's specifically for an OCTET STRING where the length is known before encoding, but not the value.

Authentication using SNMPv3's USM has the unfortunate design that needs to encode a full message with the auth parameters zeroed out, take a hash, and then overwrite the zeros with the actual hash. (And the reverse for verifying.) This means that when using rasn I have to encode the message twice. (Actually following SNMP's Abstract Programming Interfaces, I have to encode, decode, and then encode again.) I could avoid this if there were some way to mutate (or replace) the authentication_parameters in USMSecurityParameters (which is itself an OctetString in Message).

I'm not really sure if/how this could be added. I pretty much expect this request to be closed as impossible / not fitting with rasn's design. I haven't even benchmarked anything to know if it's even anywhere near approaching a problem, it just feels unfortunate. But, if you have thoughts on some way this might be accomplished, I'm more than happy to handle the implementation myself.

[XAMPPRocky]

Thank you for your issue! I don't think we can provide that kind of interface, because our parser "basic" or permissive, while our encoder is canonical, in other words we don't guarantee to preserve the original encoding when re-encoding through rasn. Additionally there are some encodings where that would not be possible to provide a mutable view, as SNMP uses BER right?

I also found this part of the RFC that says this operation is required anyway.

If the fields in an incoming SNMPv3 message are changed by the Transport Model before passing it to the Security Model, then the Transport Model will need to decode the ASN.1 message, modify the fields, and re-encode the message in ASN.1 before passing the message on to the Message Dispatcher or to the transport layer. This would require an intimate knowledge of the message format and message versions in order for the Transport Model to know which fields could be modified. This would seriously violate the modularity of the architecture.

However just because this re-encoding is required that doesn't mean we can't make this easy for developers to do. We can add a convenience function that makes it easy to just modify the message and then have the function handling the decoding, rehashing, and encoding for you.

[azdle]

Thank you for your issue! I don't think we can provide that kind of interface, because our parser "basic" or permissive, while our encoder is canonical, in other words we don't guarantee to preserve the original encoding when re-encoding through rasn. Additionally there are some encodings where that would not be possible to provide a mutable view[...]

That's more or less what I figured. 👍

SNMP uses BER right?

Yep

I also found this part of the RFC that says this operation is required anyway.

If the fields in an incoming SNMPv3 message are changed by the Transport Model before passing it to the Security Model, then the Transport Model will need to decode the ASN.1 message, modify the fields, and re-encode the message in ASN.1 before passing the message on to the Message Dispatcher or to the transport layer. This would require an intimate knowledge of the message format and message versions in order for the Transport Model to know which fields could be modified. This would seriously violate the modularity of the architecture.

I think that is saying that the Transport (Subsystem) Model doesn't re-encode because that would violate the modularity that the various subsystems define.

Unfortunately the authors of RFC3414 weren't quite as careful with the Security Subsystem/Model, talking about the procedures for the USM Security Subsystem Model's generateRequestMsg and/or generateResponseMsg (second to last step):

8) a) If the securityLevel specifies that the message is to be
    authenticated, the message is authenticated according to the
    user's authentication protocol.  To do so a call is made to the
    authentication module that implements the user's authentication
    protocol according to the abstract service primitive:

    statusInformation =
      authenticateOutgoingMsg(
      IN  authKey               -- the user's localized authKey
      IN  wholeMsg              -- unauthenticated message
      OUT authenticatedWholeMsg -- authenticated complete message
          )

    statusInformation
      indicates if authentication was successful or not.

    authKey
      the user's localized private authKey is the secret key that
      can be used by the authentication algorithm.

    wholeMsg
      the complete serialized message to be authenticated.

    authenticatedWholeMsg
      the same as the input given to the authenticateOutgoingMsg
      service, but with msgAuthenticationParameters properly
      filled in.

    If the authentication module returns failure, then the message
    cannot be sent and an error indication (authenticationFailure)
    is returned to the calling module.

    If the authentication module returns success, then the
    msgAuthenticationParameters field is put into the
    securityParameters and the authenticatedWholeMsg represents the
    serialization of the authenticated message being prepared.

I think, it means to imply that the rest of the message is already otherwise fully encoded and encoding just msgAuthenticationParameters into the message means it's complete. That interpretation means that the security subsystem needs to be aware of the overall encoding of wholeMsg (aka Message) and not just the encoding of securityParameters. But there are a lot of places where you're told to use information that isn't given to you other than in an encoded wholeMsg, so I think that's an assumption I have to make in quite a few different places.

(Getting really off topic:) The only way that I've thought of to be able to make this work the way the RFC seem to be assuming, would be to make my own type that represents (for instance) Message along with its encoding. (This is basically the TypedOctetString<T> type I alluded to over in the discussions thread a couple weeks ago.) It would be something that has functions for getting and setting fields in a message that could be internally an encoded or decoded, sorta like a Cow, where that can internally be either owned or borrowed. And either way you can call .encoded() to get an encoded representation (Bytes) or .decoded() to get a decoded representation (T) regardless of the current internal representation.

However just because this re-encoding is required that doesn't mean we can't make this easy for developers to do. We can add a convenience function that makes it easy to just modify the message and then have the function handling the decoding, rehashing, and encoding for you.

IMO, I think that's probably only worth it if you think it might be able to be more efficient, even just for some cases. If you think that the end result is still doing a full decode, modify, full reencode; then I think it's probably not worth your trouble to implement that just so that the current 3 lines of code in my codebase can become 1.

---

Sorry if any of that doesn't make any sense, my brain is rather fried from spending 3+ weeks trying to make sense of these RFCs.