Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C#] Make message parsing primitives public #8036

Closed
6 tasks
zgramana opened this issue Nov 12, 2020 · 3 comments
Closed
6 tasks

[C#] Make message parsing primitives public #8036

zgramana opened this issue Nov 12, 2020 · 3 comments
Labels
c# inactive Denotes the issue/PR has not seen activity in the last 90 days.

Comments

@zgramana
Copy link

zgramana commented Nov 12, 2020

What language does this apply to?
C#

Describe the problem you are trying to solve.
Trying to manually marshall a message from a buffer, but unlike Java, several parsing primitives from CodedInputStream such as ReadRawVarint32 are not public. This significantly limits the utility of the new high-performance API's (e.g. IBufferWriter<byte>) or requires poor code hygiene (see the alternatives section below).

Describe the solution you'd like

At a very bare minimum, having some of the the same Java members available in dotnet, including:

CodedInputStream
  • public static int readRawVarint32(final InputStream input)
  • public static int readRawVarint32(final int firstByte, final InputStream input)
  • public static int decodeZigZag32(final int n)
  • public static long decodeZigZag64(final long n)
CodedOutputStream
  • public static int encodeZigZag32(final int n)
  • public static long encodeZigZag64(final long n)

Much, much, more practically for those using IBufferWriter<byte> to do custom (zero/minimal-copy) marshaling, it would be ideal to have access to many of the members of ReadingPrimitives and WritingPrimitives.

Describe alternatives you've considered

  1. Copy-and-pasting Protobuf source into my application. 😞
  2. Hand-rolling varint code.
  3. Casting IBufferWriter<byte> to ArrayBufferWriter<byte>, pinning the array, and then passing an UnmanagedMemoryStream to CodedOutputStream. This works, but is violating abstractions, requires an unsafe block, and breaks if you call SerializationContext.SetPayloadLength which causes SerializationContext.GetBufferWriter to return a HttpResponsePipeWriter instead of a ArrayBufferWriter.
  4. Both 1 and 2

Additional context
Related to:

@R030t1
Copy link

R030t1 commented Nov 29, 2020

I have a related but perhaps more easily encountered problem: providing a BytesBuffer object to satisfy a bytes field requires copying the data.[1] ByteString as well. A private constructor exists which does no copying. This constructor or another API should be exposed to elide the copies. This could be taken further and the C++ API mimicked so a pointer and length could be submitted to avoid copying in data via Marshal.

The latter was claimed to be problematic but this concern was not clearly articulated. Certainly C# is deficient in that all safe types must be constructed with copying but a workaround is worthwhile. Either 2 (unsafe) or 3 (safe) copies are needed before protobuf has the data.

Copy link

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please add a comment.

This issue is labeled inactive because the last activity was over 90 days ago.

@github-actions github-actions bot added the inactive Denotes the issue/PR has not seen activity in the last 90 days. label Feb 26, 2024
Copy link

We triage inactive PRs and issues in order to make it easier to find active work. If this issue should remain active or becomes active again, please reopen it.

This issue was closed and archived because there has been no new activity in the 14 days since the inactive label was added.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c# inactive Denotes the issue/PR has not seen activity in the last 90 days.
Projects
None yet
Development

No branches or pull requests

3 participants