Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use hex representation for binary data inside generated php files #10404

Conversation

Grebenschikov
Copy link

@Grebenschikov Grebenschikov commented Aug 14, 2022

For the sake of performance, the hex2bin call was removed from the generated PHP code in #8006
However, after this PR all autogenerated files contain binary data, which makes it hard or even impossible to see using common tools (git and github included (i.e. this file), as well as some code editors) since most software considers such files as binary ones and not as code.

At the same, there is a possibility to use hex representation of binary data inside PHP files, which doesn't affect the performance (because it compiles to the same OP codes by the interpreter).

package ~/Desktop/proto $ cat test1.php
<?php

$str = "teststring";
echo $str;%
package ~/Desktop/proto $ phpdbg -p\* test1.php

$_main:
     ; (lines=3, args=0, vars=1, tmps=1)
     ; /Users/package/Desktop/proto/test1.php:1-4
L0003 0000 ASSIGN CV0($str) string("teststring")
L0004 0001 ECHO CV0($str)
L0004 0002 RETURN int(1)
[Script ended normally]
package ~/Desktop/proto $ cat test2.php
<?php

$str = "\x74\x65\x73\x74\x73\x74\x72\x69\x6e\x67";
echo $str;%
package ~/Desktop/proto $ phpdbg -p\* test2.php

$_main:
     ; (lines=3, args=0, vars=1, tmps=1)
     ; /Users/package/Desktop/proto/test2.php:1-4
L0003 0000 ASSIGN CV0($str) string("teststring")
L0004 0001 ECHO CV0($str)
L0004 0002 RETURN int(1)
[Script ended normally]

So, this PR replaces binary data with hex representation to fix issues mentioned above

@google-cla
Copy link

google-cla bot commented Aug 14, 2022

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@Grebenschikov Grebenschikov force-pushed the do_not_store_binary_data_inside_php_files branch from 9dfce25 to ff31311 Compare August 14, 2022 21:16
@Grebenschikov Grebenschikov force-pushed the do_not_store_binary_data_inside_php_files branch from ff31311 to a832d2c Compare August 14, 2022 21:17
@Grebenschikov
Copy link
Author

Btw, I don't really like ptr magic in the code, but the implementation of the BinaryToPhpString function is basically a modified version of the old BinaryToHex function (see here)
Happy to rewrite this part if it's a problem.

@Grebenschikov
Copy link
Author

Also, I'm unsure, should I run php/generate_descriptor_protos.sh and commit changes manually?

@shaod2 shaod2 requested a review from haberman August 15, 2022 16:58
@shaod2 shaod2 added the php label Aug 15, 2022
@haberman
Copy link
Member

Thanks for the PR. Great analysis showing that everything is identical once things are compiled. Would you be able to also test the performance of PHP parsing the file? eg. if you have a .php file with a 100kB string, can you measure any difference in PHP's performance between parsing straight binary data vs. hex escaped?

One likely optimization that could help both parsing performance and code size is to only hex-escape non-printable characters. I would guess that 80% or so of most descriptors are printable characters.

@haberman
Copy link
Member

One optimization that I expect would help a lot is if we only escape non-printable characters. Printable characters can be emitted without escaping.

I would estimate that 80% or more of the characters in a typical descriptor are printable, so this could significantly reduce the size and (hopefully) parsing time.

Could you update the PR with this change?

@deannagarcia
Copy link
Member

Closing due to no response. Feel free to reopen if you want to address the comments.

@mikhainin
Copy link
Contributor

Hi there @haberman, @deannagarcia

I have just updated the patch by not-encoding printable characters commit. Although, I don't know how I can add a commit to this PR.

I also made a quick benchmark of the parsing. Basically, it shows that the parsing of the hexadecimal string is about 2.5 times slower than just using the binary string. But providing that this happens quite rarely, I reckon we can allow the slowdown in this place.

Would it be possible to move forward with this PR, please?

The percent of printable chars: 0.83869 (80112 vs 15408)
hex parsing: 14.63102
bin parsing: 1.68736
hex+ascii parsing: 4.31581

The benchmark (the source string is a real protobuf descriptor from our project):

<?php

$str = "\x0a\x95\x3e\x0a\x45\x73\x65\x72\x76\x69\x63\x65\x2f\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2f\x70\x72\x6f\x74\x6f\x2f\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x70\x72\x6f\x74\x6f\x12\x34\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x22\x2e\x0a\x1a\x47\x65\x74\x48\x69\x76\x65\x73\x4e\x65\x77\x41\x63\x74\x69\x76\x69\x74\x79\x52\x65\x71\x75\x65\x73\x74\x12\x10\x0a\x08\x68\x69\x76\x65\x5f\x69\x64\x73\x18\x01\x20\x03\x28\x04\x22\x73\x0a\x1b\x47\x65\x74\x48\x69\x76\x65\x73\x4e\x65\x77\x41\x63\x74\x69\x76\x69\x74\x79\x52\x65\x73\x70\x6f\x6e\x73\x65\x12\x54\x0a\x05\x68\x69\x76\x65\x73\x18\x01\x20\x03\x28\x0b\x32\x45\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x48\x69\x76\x65\x4e\x65\x77\x41\x63\x74\x69\x76\x69\x74\x79\x22\x2f\x0a\x1c\x47\x65\x74\x48\x69\x76\x65\x41\x6c\x6c\x6f\x77\x65\x64\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x71\x75\x65\x73\x74\x12\x0f\x0a\x07\x68\x69\x76\x65\x5f\x69\x64\x18\x01\x20\x01\x28\x04\x22\x7b\x0a\x1d\x47\x65\x74\x48\x69\x76\x65\x41\x6c\x6c\x6f\x77\x65\x64\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x73\x70\x6f\x6e\x73\x65\x12\x5a\x0a\x0f\x61\x6c\x6c\x6f\x77\x65\x64\x5f\x63\x6f\x6e\x74\x65\x6e\x74\x18\x01\x20\x03\x28\x0e\x32\x41\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x48\x69\x76\x65\x43\x6f\x6e\x74\x65\x6e\x74\x22\x3c\x0a\x0f\x48\x69\x76\x65\x4e\x65\x77\x41\x63\x74\x69\x76\x69\x74\x79\x12\x0f\x0a\x07\x68\x69\x76\x65\x5f\x69\x64\x18\x01\x20\x01\x28\x04\x12\x18\x0a\x10\x68\x61\x73\x5f\x6e\x65\x77\x5f\x61\x63\x74\x69\x76\x69\x74\x79\x18\x02\x20\x01\x28\x08\x22\xb1\x01\x0a\x19\x47\x65\x74\x55\x73\x65\x72\x43\x6f\x6e\x74\x65\x6e\x74\x4c\x69\x73\x74\x52\x65\x71\x75\x65\x73\x74\x12\x0f\x0a\x07\x75\x73\x65\x72\x5f\x69\x64\x18\x01\x20\x01\x28\x09\x12\x12\x0a\x0a\x70\x61\x67\x65\x5f\x74\x6f\x6b\x65\x6e\x18\x02\x20\x01\x28\x09\x12\x17\x0a\x0f\x70\x72\x65\x66\x65\x72\x72\x65\x64\x5f\x63\x6f\x75\x6e\x74\x18\x03\x20\x01\x28\x05\x12\x56\x0a\x09\x73\x6f\x72\x74\x65\x64\x5f\x62\x79\x18\x04\x20\x01\x28\x0e\x32\x43\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x53\x6f\x72\x74\x69\x6e\x67\x4f\x70\x74\x69\x6f\x6e\x22\xa8\x05\x0a\x1a\x47\x65\x74\x55\x73\x65\x72\x43\x6f\x6e\x74\x65\x6e\x74\x4c\x69\x73\x74\x52\x65\x73\x70\x6f\x6e\x73\x65\x12\x12\x0a\x0a\x70\x61\x67\x65\x5f\x74\x6f\x6b\x65\x6e\x18\x01\x20\x01\x28\x09\x12\x5a\x0a\x05\x70\x6f\x73\x74\x73\x18\x02\x20\x03\x28\x0b\x32\x47\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x50\x6f\x73\x74\x42\x02\x18\x01\x12\x60\x0a\x08\x63\x6f\x6d\x6d\x65\x6e\x74\x73\x18\x03\x20\x03\x28\x0b\x32\x4a\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x43\x6f\x6d\x6d\x65\x6e\x74\x42\x02\x18\x01\x12\x5d\x0a\x07\x72\x65\x70\x6c\x69\x65\x73\x18\x04\x20\x03\x28\x0b\x32\x48\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x52\x65\x70\x6c\x79\x42\x02\x18\x01\x12\x55\x0a\x0a\x68\x69\x76\x65\x5f\x70\x6f\x73\x74\x73\x18\x05\x20\x03\x28\x0b\x32\x41\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x48\x69\x76\x65\x50\x6f\x73\x74\x12\x5b\x0a\x0d\x68\x69\x76\x65\x5f\x63\x6f\x6d\x6d\x65\x6e\x74\x73\x18\x06\x20\x03\x28\x0b\x32\x44\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x48\x69\x76\x65\x43\x6f\x6d\x6d\x65\x6e\x74\x12\x58\x0a\x0c\x68\x69\x76\x65\x5f\x72\x65\x70\x6c\x69\x65\x73\x18\x07\x20\x03\x28\x0b\x32\x42\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x48\x69\x76\x65\x52\x65\x70\x6c\x79\x12\x4b\x0a\x06\x65\x76\x65\x6e\x74\x73\x18\x08\x20\x03\x28\x0b\x32\x3b\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x45\x76\x65\x6e\x74\x22\xae\x01\x0a\x16\x47\x65\x74\x50\x6f\x73\x74\x43\x6f\x6d\x6d\x65\x6e\x74\x73\x52\x65\x71\x75\x65\x73\x74\x12\x0f\x0a\x07\x70\x6f\x73\x74\x5f\x69\x64\x18\x01\x20\x01\x28\x03\x12\x12\x0a\x0a\x70\x61\x67\x65\x5f\x74\x6f\x6b\x65\x6e\x18\x02\x20\x01\x28\x09\x12\x17\x0a\x0f\x70\x72\x65\x66\x65\x72\x72\x65\x64\x5f\x63\x6f\x75\x6e\x74\x18\x03\x20\x01\x28\x05\x12\x56\x0a\x09\x73\x6f\x72\x74\x65\x64\x5f\x62\x79\x18\x04\x20\x01\x28\x0e\x32\x43\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x53\x6f\x72\x74\x69\x6e\x67\x4f\x70\x74\x69\x6f\x6e\x22\x9d\x03\x0a\x17\x47\x65\x74\x50\x6f\x73\x74\x43\x6f\x6d\x6d\x65\x6e\x74\x73\x52\x65\x73\x70\x6f\x6e\x73\x65\x12\x12\x0a\x0a\x70\x61\x67\x65\x5f\x74\x6f\x6b\x65\x6e\x18\x01\x20\x01\x28\x09\x12\x59\x0a\x04\x70\x6f\x73\x74\x18\x02\x20\x01\x28\x0b\x32\x47\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x50\x6f\x73\x74\x42\x02\x18\x01\x12\x60\x0a\x08\x63\x6f\x6d\x6d\x65\x6e\x74\x73\x18\x03\x20\x03\x28\x0b\x32\x4a\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x43\x6f\x6d\x6d\x65\x6e\x74\x42\x02\x18\x01\x12\x54\x0a\x09\x68\x69\x76\x65\x5f\x70\x6f\x73\x74\x18\x04\x20\x01\x28\x0b\x32\x41\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x48\x69\x76\x65\x50\x6f\x73\x74\x12\x5b\x0a\x0d\x68\x69\x76\x65\x5f\x63\x6f\x6d\x6d\x65\x6e\x74\x73\x18\x05\x20\x03\x28\x0b\x32\x44\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x48\x69\x76\x65\x43\x6f\x6d\x6d\x65\x6e\x74\x22\xb3\x01\x0a\x18\x47\x65\x74\x43\x6f\x6d\x6d\x65\x6e\x74\x52\x65\x70\x6c\x69\x65\x73\x52\x65\x71\x75\x65\x73\x74\x12\x12\x0a\x0a\x63\x6f\x6d\x6d\x65\x6e\x74\x5f\x69\x64\x18\x01\x20\x01\x28\x03\x12\x12\x0a\x0a\x70\x61\x67\x65\x5f\x74\x6f\x6b\x65\x6e\x18\x02\x20\x01\x28\x09\x12\x17\x0a\x0f\x70\x72\x65\x66\x65\x72\x72\x65\x64\x5f\x63\x6f\x75\x6e\x74\x18\x03\x20\x01\x28\x05\x12\x56\x0a\x09\x73\x6f\x72\x74\x65\x64\x5f\x62\x79\x18\x04\x20\x01\x28\x0e\x32\x43\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x53\x6f\x72\x74\x69\x6e\x67\x4f\x70\x74\x69\x6f\x6e\x22\xa5\x03\x0a\x19\x47\x65\x74\x43\x6f\x6d\x6d\x65\x6e\x74\x52\x65\x70\x6c\x69\x65\x73\x52\x65\x73\x70\x6f\x6e\x73\x65\x12\x12\x0a\x0a\x70\x61\x67\x65\x5f\x74\x6f\x6b\x65\x6e\x18\x01\x20\x01\x28\x09\x12\x5f\x0a\x07\x63\x6f\x6d\x6d\x65\x6e\x74\x18\x02\x20\x01\x28\x0b\x32\x4a\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x43\x6f\x6d\x6d\x65\x6e\x74\x42\x02\x18\x01\x12\x5d\x0a\x07\x72\x65\x70\x6c\x69\x65\x73\x18\x03\x20\x03\x28\x0b\x32\x48\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x52\x65\x70\x6c\x79\x42\x02\x18\x01\x12\x5a\x0a\x0c\x68\x69\x76\x65\x5f\x63\x6f\x6d\x6d\x65\x6e\x74\x18\x04\x20\x01\x28\x0b\x32\x44\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x48\x69\x76\x65\x43\x6f\x6d\x6d\x65\x6e\x74\x12\x58\x0a\x0c\x68\x69\x76\x65\x5f\x72\x65\x70\x6c\x69\x65\x73\x18\x05\x20\x03\x28\x0b\x32\x42\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x48\x69\x76\x65\x52\x65\x70\x6c\x79\x22\x40\x0a\x1c\x47\x65\x74\x50\x6f\x73\x74\x44\x65\x74\x61\x69\x6c\x73\x46\x6f\x72\x55\x73\x65\x72\x52\x65\x71\x75\x65\x73\x74\x12\x0f\x0a\x07\x70\x6f\x73\x74\x5f\x69\x64\x18\x01\x20\x01\x28\x04\x12\x0f\x0a\x07\x75\x73\x65\x72\x5f\x69\x64\x18\x02\x20\x01\x28\x09\x22\xbf\x02\x0a\x1d\x47\x65\x74\x50\x6f\x73\x74\x44\x65\x74\x61\x69\x6c\x73\x46\x6f\x72\x55\x73\x65\x72\x52\x65\x73\x70\x6f\x6e\x73\x65\x12\x14\x0a\x0c\x70\x6f\x73\x74\x5f\x73\x75\x62\x6a\x65\x63\x74\x18\x01\x20\x01\x28\x09\x12\x1a\x0a\x12\x70\x6f\x73\x74\x5f\x63\x72\x65\x61\x74\x65\x64\x5f\x61\x74\x5f\x74\x73\x18\x02\x20\x01\x28\x04\x12\x1c\x0a\x14\x70\x6f\x73\x74\x5f\x6c\x61\x73\x74\x5f\x63\x6f\x6d\x6d\x65\x6e\x74\x5f\x74\x73\x18\x03\x20\x01\x28\x04\x12\x19\x0a\x0d\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x5f\x69\x64\x18\x04\x20\x01\x28\x04\x42\x02\x18\x01\x12\x1b\x0a\x0f\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x5f\x6e\x61\x6d\x65\x18\x05\x20\x01\x28\x09\x42\x02\x18\x01\x12\x1f\x0a\x17\x75\x73\x65\x72\x5f\x76\x69\x73\x69\x74\x65\x64\x5f\x70\x6f\x73\x74\x5f\x61\x74\x5f\x74\x73\x18\x06\x20\x01\x28\x04\x12\x51\x0a\x09\x75\x73\x65\x72\x5f\x72\x6f\x6c\x65\x18\x07\x20\x01\x28\x0e\x32\x3e\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x55\x73\x65\x72\x52\x6f\x6c\x65\x12\x0f\x0a\x07\x68\x69\x76\x65\x5f\x69\x64\x18\x08\x20\x01\x28\x04\x12\x11\x0a\x09\x68\x69\x76\x65\x5f\x6e\x61\x6d\x65\x18\x09\x20\x01\x28\x09\x22\x33\x0a\x1b\x47\x65\x74\x4a\x6f\x69\x6e\x65\x64\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x52\x65\x71\x75\x65\x73\x74\x12\x10\x0a\x08\x75\x73\x65\x72\x5f\x69\x64\x73\x18\x01\x20\x03\x28\x09\x3a\x02\x18\x01\x22\x90\x01\x0a\x1c\x47\x65\x74\x4a\x6f\x69\x6e\x65\x64\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x52\x65\x73\x70\x6f\x6e\x73\x65\x12\x6c\x0a\x17\x75\x73\x65\x72\x5f\x6a\x6f\x69\x6e\x65\x64\x5f\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x18\x01\x20\x03\x28\x0b\x32\x4b\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x55\x73\x65\x72\x4a\x6f\x69\x6e\x65\x64\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x3a\x02\x18\x01\x22\x90\x01\x0a\x15\x55\x73\x65\x72\x4a\x6f\x69\x6e\x65\x64\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x12\x0f\x0a\x07\x75\x73\x65\x72\x5f\x69\x64\x18\x01\x20\x01\x28\x09\x12\x62\x0a\x12\x6a\x6f\x69\x6e\x65\x64\x5f\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x18\x02\x20\x03\x28\x0b\x32\x46\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x4a\x6f\x69\x6e\x65\x64\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x3a\x02\x18\x01\x22\x43\x0a\x10\x4a\x6f\x69\x6e\x65\x64\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x12\x15\x0a\x0d\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x5f\x69\x64\x18\x01\x20\x01\x28\x04\x12\x14\x0a\x0c\x6a\x6f\x69\x6e\x65\x64\x5f\x61\x74\x5f\x74\x73\x18\x02\x20\x01\x28\x03\x3a\x02\x18\x01\x22\x85\x02\x0a\x11\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x50\x6f\x73\x74\x12\x0a\x0a\x02\x69\x64\x18\x01\x20\x01\x28\x03\x12\x53\x0a\x06\x73\x74\x61\x74\x75\x73\x18\x02\x20\x01\x28\x0e\x32\x43\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x43\x6f\x6e\x74\x65\x6e\x74\x53\x74\x61\x74\x75\x73\x12\x12\x0a\x0a\x63\x72\x65\x61\x74\x65\x64\x5f\x74\x73\x18\x03\x20\x01\x28\x03\x12\x18\x0a\x10\x6c\x61\x73\x74\x5f\x6d\x6f\x64\x69\x66\x69\x65\x64\x5f\x74\x73\x18\x04\x20\x01\x28\x03\x12\x0f\x0a\x07\x73\x75\x62\x6a\x65\x63\x74\x18\x05\x20\x01\x28\x09\x12\x0c\x0a\x04\x62\x6f\x64\x79\x18\x06\x20\x01\x28\x09\x12\x0f\x0a\x07\x75\x73\x65\x72\x5f\x69\x64\x18\x07\x20\x01\x28\x09\x12\x14\x0a\x0c\x68\x61\x73\x5f\x63\x6f\x6d\x6d\x65\x6e\x74\x73\x18\x08\x20\x01\x28\x08\x12\x17\x0a\x0f\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x5f\x6e\x61\x6d\x65\x18\x09\x20\x01\x28\x09\x3a\x02\x18\x01\x22\xf5\x01\x0a\x0b\x42\x66\x66\x48\x69\x76\x65\x50\x6f\x73\x74\x12\x0a\x0a\x02\x69\x64\x18\x01\x20\x01\x28\x03\x12\x53\x0a\x06\x73\x74\x61\x74\x75\x73\x18\x02\x20\x01\x28\x0e\x32\x43\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x43\x6f\x6e\x74\x65\x6e\x74\x53\x74\x61\x74\x75\x73\x12\x12\x0a\x0a\x63\x72\x65\x61\x74\x65\x64\x5f\x74\x73\x18\x03\x20\x01\x28\x03\x12\x18\x0a\x10\x6c\x61\x73\x74\x5f\x6d\x6f\x64\x69\x66\x69\x65\x64\x5f\x74\x73\x18\x04\x20\x01\x28\x03\x12\x0f\x0a\x07\x73\x75\x62\x6a\x65\x63\x74\x18\x05\x20\x01\x28\x09\x12\x0c\x0a\x04\x62\x6f\x64\x79\x18\x06\x20\x01\x28\x09\x12\x0f\x0a\x07\x75\x73\x65\x72\x5f\x69\x64\x18\x07\x20\x01\x28\x09\x12\x14\x0a\x0c\x68\x61\x73\x5f\x63\x6f\x6d\x6d\x65\x6e\x74\x73\x18\x08\x20\x01\x28\x08\x12\x11\x0a\x09\x68\x69\x76\x65\x5f\x6e\x61\x6d\x65\x18\x09\x20\x01\x28\x09\x22\x87\x02\x0a\x14\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x43\x6f\x6d\x6d\x65\x6e\x74\x12\x0a\x0a\x02\x69\x64\x18\x01\x20\x01\x28\x03\x12\x53\x0a\x06\x73\x74\x61\x74\x75\x73\x18\x02\x20\x01\x28\x0e\x32\x43\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x43\x6f\x6e\x74\x65\x6e\x74\x53\x74\x61\x74\x75\x73\x12\x12\x0a\x0a\x63\x72\x65\x61\x74\x65\x64\x5f\x74\x73\x18\x03\x20\x01\x28\x03\x12\x18\x0a\x10\x6c\x61\x73\x74\x5f\x6d\x6f\x64\x69\x66\x69\x65\x64\x5f\x74\x73\x18\x04\x20\x01\x28\x03\x12\x0c\x0a\x04\x62\x6f\x64\x79\x18\x05\x20\x01\x28\x09\x12\x0f\x0a\x07\x75\x73\x65\x72\x5f\x69\x64\x18\x06\x20\x01\x28\x09\x12\x13\x0a\x0b\x68\x61\x73\x5f\x72\x65\x70\x6c\x69\x65\x73\x18\x07\x20\x01\x28\x08\x12\x0f\x0a\x07\x70\x6f\x73\x74\x5f\x69\x64\x18\x08\x20\x01\x28\x03\x12\x17\x0a\x0f\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x5f\x6e\x61\x6d\x65\x18\x09\x20\x01\x28\x09\x3a\x02\x18\x01\x22\xf7\x01\x0a\x0e\x42\x66\x66\x48\x69\x76\x65\x43\x6f\x6d\x6d\x65\x6e\x74\x12\x0a\x0a\x02\x69\x64\x18\x01\x20\x01\x28\x03\x12\x53\x0a\x06\x73\x74\x61\x74\x75\x73\x18\x02\x20\x01\x28\x0e\x32\x43\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x43\x6f\x6e\x74\x65\x6e\x74\x53\x74\x61\x74\x75\x73\x12\x12\x0a\x0a\x63\x72\x65\x61\x74\x65\x64\x5f\x74\x73\x18\x03\x20\x01\x28\x03\x12\x18\x0a\x10\x6c\x61\x73\x74\x5f\x6d\x6f\x64\x69\x66\x69\x65\x64\x5f\x74\x73\x18\x04\x20\x01\x28\x03\x12\x0c\x0a\x04\x62\x6f\x64\x79\x18\x05\x20\x01\x28\x09\x12\x0f\x0a\x07\x75\x73\x65\x72\x5f\x69\x64\x18\x06\x20\x01\x28\x09\x12\x13\x0a\x0b\x68\x61\x73\x5f\x72\x65\x70\x6c\x69\x65\x73\x18\x07\x20\x01\x28\x08\x12\x0f\x0a\x07\x70\x6f\x73\x74\x5f\x69\x64\x18\x08\x20\x01\x28\x03\x12\x11\x0a\x09\x68\x69\x76\x65\x5f\x6e\x61\x6d\x65\x18\x09\x20\x01\x28\x09\x22\xf0\x01\x0a\x12\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x52\x65\x70\x6c\x79\x12\x0a\x0a\x02\x69\x64\x18\x01\x20\x01\x28\x03\x12\x53\x0a\x06\x73\x74\x61\x74\x75\x73\x18\x02\x20\x01\x28\x0e\x32\x43\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x43\x6f\x6e\x74\x65\x6e\x74\x53\x74\x61\x74\x75\x73\x12\x12\x0a\x0a\x63\x72\x65\x61\x74\x65\x64\x5f\x74\x73\x18\x03\x20\x01\x28\x03\x12\x18\x0a\x10\x6c\x61\x73\x74\x5f\x6d\x6f\x64\x69\x66\x69\x65\x64\x5f\x74\x73\x18\x04\x20\x01\x28\x03\x12\x0c\x0a\x04\x62\x6f\x64\x79\x18\x05\x20\x01\x28\x09\x12\x0f\x0a\x07\x75\x73\x65\x72\x5f\x69\x64\x18\x06\x20\x01\x28\x09\x12\x0f\x0a\x07\x70\x6f\x73\x74\x5f\x69\x64\x18\x07\x20\x01\x28\x03\x12\x17\x0a\x0f\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x5f\x6e\x61\x6d\x65\x18\x08\x20\x01\x28\x09\x3a\x02\x18\x01\x22\xe0\x01\x0a\x0c\x42\x66\x66\x48\x69\x76\x65\x52\x65\x70\x6c\x79\x12\x0a\x0a\x02\x69\x64\x18\x01\x20\x01\x28\x03\x12\x53\x0a\x06\x73\x74\x61\x74\x75\x73\x18\x02\x20\x01\x28\x0e\x32\x43\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x43\x6f\x6e\x74\x65\x6e\x74\x53\x74\x61\x74\x75\x73\x12\x12\x0a\x0a\x63\x72\x65\x61\x74\x65\x64\x5f\x74\x73\x18\x03\x20\x01\x28\x03\x12\x18\x0a\x10\x6c\x61\x73\x74\x5f\x6d\x6f\x64\x69\x66\x69\x65\x64\x5f\x74\x73\x18\x04\x20\x01\x28\x03\x12\x0c\x0a\x04\x62\x6f\x64\x79\x18\x05\x20\x01\x28\x09\x12\x0f\x0a\x07\x75\x73\x65\x72\x5f\x69\x64\x18\x06\x20\x01\x28\x09\x12\x0f\x0a\x07\x70\x6f\x73\x74\x5f\x69\x64\x18\x07\x20\x01\x28\x03\x12\x11\x0a\x09\x68\x69\x76\x65\x5f\x6e\x61\x6d\x65\x18\x08\x20\x01\x28\x09\x22\xec\x01\x0a\x05\x45\x76\x65\x6e\x74\x12\x0a\x0a\x02\x69\x64\x18\x01\x20\x01\x28\x03\x12\x53\x0a\x06\x73\x74\x61\x74\x75\x73\x18\x02\x20\x01\x28\x0e\x32\x43\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x43\x6f\x6e\x74\x65\x6e\x74\x53\x74\x61\x74\x75\x73\x12\x12\x0a\x0a\x63\x72\x65\x61\x74\x65\x64\x5f\x74\x73\x18\x03\x20\x01\x28\x03\x12\x18\x0a\x10\x6c\x61\x73\x74\x5f\x6d\x6f\x64\x69\x66\x69\x65\x64\x5f\x74\x73\x18\x04\x20\x01\x28\x03\x12\x0d\x0a\x05\x74\x69\x74\x6c\x65\x18\x05\x20\x01\x28\x09\x12\x10\x0a\x08\x6c\x6f\x63\x61\x74\x69\x6f\x6e\x18\x06\x20\x01\x28\x09\x12\x0f\x0a\x07\x64\x65\x74\x61\x69\x6c\x73\x18\x07\x20\x01\x28\x09\x12\x0f\x0a\x07\x75\x73\x65\x72\x5f\x69\x64\x18\x08\x20\x01\x28\x09\x12\x11\x0a\x09\x68\x69\x76\x65\x5f\x6e\x61\x6d\x65\x18\x09\x20\x01\x28\x09\x22\x3d\x0a\x23\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x6d\x6f\x76\x61\x6c\x52\x65\x71\x75\x65\x73\x74\x12\x12\x0a\x0a\x63\x6f\x6e\x74\x65\x6e\x74\x5f\x69\x64\x18\x01\x20\x01\x28\x03\x3a\x02\x18\x01\x22\x2f\x0a\x19\x52\x65\x6d\x6f\x76\x65\x48\x69\x76\x65\x73\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x71\x75\x65\x73\x74\x12\x12\x0a\x0a\x63\x6f\x6e\x74\x65\x6e\x74\x5f\x69\x64\x18\x01\x20\x01\x28\x03\x22\x2a\x0a\x24\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x6d\x6f\x76\x61\x6c\x52\x65\x73\x70\x6f\x6e\x73\x65\x3a\x02\x18\x01\x22\x1c\x0a\x1a\x52\x65\x6d\x6f\x76\x65\x48\x69\x76\x65\x73\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x2d\x0a\x1b\x47\x65\x74\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x44\x65\x74\x61\x69\x6c\x73\x52\x65\x71\x75\x65\x73\x74\x12\x0a\x0a\x02\x69\x64\x18\x01\x20\x01\x28\x03\x3a\x02\x18\x01\x22\x5e\x0a\x1c\x47\x65\x74\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x44\x65\x74\x61\x69\x6c\x73\x52\x65\x73\x70\x6f\x6e\x73\x65\x12\x0a\x0a\x02\x69\x64\x18\x01\x20\x01\x28\x03\x12\x0c\x0a\x04\x6e\x61\x6d\x65\x18\x02\x20\x01\x28\x09\x12\x13\x0a\x0b\x64\x65\x73\x63\x72\x69\x70\x74\x69\x6f\x6e\x18\x03\x20\x01\x28\x09\x12\x0b\x0a\x03\x75\x72\x6c\x18\x04\x20\x01\x28\x09\x3a\x02\x18\x01\x2a\xa1\x01\x0a\x0d\x43\x6f\x6e\x74\x65\x6e\x74\x53\x74\x61\x74\x75\x73\x12\x1e\x0a\x1a\x43\x4f\x4e\x54\x45\x4e\x54\x5f\x53\x54\x41\x54\x55\x53\x5f\x55\x4e\x53\x50\x45\x43\x49\x46\x49\x45\x44\x10\x00\x12\x1a\x0a\x16\x43\x4f\x4e\x54\x45\x4e\x54\x5f\x53\x54\x41\x54\x55\x53\x5f\x43\x52\x45\x41\x54\x45\x44\x10\x01\x12\x1a\x0a\x16\x43\x4f\x4e\x54\x45\x4e\x54\x5f\x53\x54\x41\x54\x55\x53\x5f\x44\x45\x4c\x45\x54\x45\x44\x10\x02\x12\x1c\x0a\x18\x43\x4f\x4e\x54\x45\x4e\x54\x5f\x53\x54\x41\x54\x55\x53\x5f\x4d\x4f\x44\x45\x52\x41\x54\x45\x44\x10\x03\x12\x1a\x0a\x16\x43\x4f\x4e\x54\x45\x4e\x54\x5f\x53\x54\x41\x54\x55\x53\x5f\x49\x4e\x49\x54\x49\x41\x4c\x10\x04\x2a\x88\x01\x0a\x0d\x53\x6f\x72\x74\x69\x6e\x67\x4f\x70\x74\x69\x6f\x6e\x12\x1e\x0a\x1a\x53\x4f\x52\x54\x49\x4e\x47\x5f\x4f\x50\x54\x49\x4f\x4e\x5f\x55\x4e\x53\x50\x45\x43\x49\x46\x49\x45\x44\x10\x00\x12\x2a\x0a\x26\x53\x4f\x52\x54\x49\x4e\x47\x5f\x4f\x50\x54\x49\x4f\x4e\x5f\x43\x52\x45\x41\x54\x49\x4f\x4e\x5f\x44\x41\x54\x45\x5f\x41\x53\x43\x45\x4e\x44\x49\x4e\x47\x10\x01\x12\x2b\x0a\x27\x53\x4f\x52\x54\x49\x4e\x47\x5f\x4f\x50\x54\x49\x4f\x4e\x5f\x43\x52\x45\x41\x54\x49\x4f\x4e\x5f\x44\x41\x54\x45\x5f\x44\x45\x53\x43\x45\x4e\x44\x49\x4e\x47\x10\x02\x2a\x56\x0a\x08\x55\x73\x65\x72\x52\x6f\x6c\x65\x12\x19\x0a\x15\x55\x53\x45\x52\x5f\x52\x4f\x4c\x45\x5f\x55\x4e\x53\x50\x45\x43\x49\x46\x49\x45\x44\x10\x00\x12\x14\x0a\x10\x55\x53\x45\x52\x5f\x52\x4f\x4c\x45\x5f\x41\x55\x54\x48\x4f\x52\x10\x01\x12\x19\x0a\x15\x55\x53\x45\x52\x5f\x52\x4f\x4c\x45\x5f\x43\x4f\x4e\x54\x52\x49\x42\x55\x54\x4f\x52\x10\x02\x2a\x33\x0a\x0b\x48\x69\x76\x65\x43\x6f\x6e\x74\x65\x6e\x74\x12\x0f\x0a\x0b\x55\x4e\x53\x50\x45\x43\x49\x46\x49\x45\x44\x10\x00\x12\x08\x0a\x04\x50\x4f\x53\x54\x10\x01\x12\x09\x0a\x05\x45\x56\x45\x4e\x54\x10\x02\x32\xa6\x0f\x0a\x1f\x42\x75\x6d\x62\x6c\x65\x46\x72\x69\x65\x6e\x64\x73\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x53\x65\x72\x76\x69\x63\x65\x12\xb9\x01\x0a\x12\x47\x65\x74\x55\x73\x65\x72\x43\x6f\x6e\x74\x65\x6e\x74\x4c\x69\x73\x74\x12\x4f\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x55\x73\x65\x72\x43\x6f\x6e\x74\x65\x6e\x74\x4c\x69\x73\x74\x52\x65\x71\x75\x65\x73\x74\x1a\x50\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x55\x73\x65\x72\x43\x6f\x6e\x74\x65\x6e\x74\x4c\x69\x73\x74\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x00\x12\xb0\x01\x0a\x0f\x47\x65\x74\x50\x6f\x73\x74\x43\x6f\x6d\x6d\x65\x6e\x74\x73\x12\x4c\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x50\x6f\x73\x74\x43\x6f\x6d\x6d\x65\x6e\x74\x73\x52\x65\x71\x75\x65\x73\x74\x1a\x4d\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x50\x6f\x73\x74\x43\x6f\x6d\x6d\x65\x6e\x74\x73\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x00\x12\xb6\x01\x0a\x11\x47\x65\x74\x43\x6f\x6d\x6d\x65\x6e\x74\x52\x65\x70\x6c\x69\x65\x73\x12\x4e\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x43\x6f\x6d\x6d\x65\x6e\x74\x52\x65\x70\x6c\x69\x65\x73\x52\x65\x71\x75\x65\x73\x74\x1a\x4f\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x43\x6f\x6d\x6d\x65\x6e\x74\x52\x65\x70\x6c\x69\x65\x73\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x00\x12\xc2\x01\x0a\x15\x47\x65\x74\x50\x6f\x73\x74\x44\x65\x74\x61\x69\x6c\x73\x46\x6f\x72\x55\x73\x65\x72\x12\x52\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x50\x6f\x73\x74\x44\x65\x74\x61\x69\x6c\x73\x46\x6f\x72\x55\x73\x65\x72\x52\x65\x71\x75\x65\x73\x74\x1a\x53\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x50\x6f\x73\x74\x44\x65\x74\x61\x69\x6c\x73\x46\x6f\x72\x55\x73\x65\x72\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x00\x12\xcb\x01\x0a\x0d\x52\x65\x6d\x6f\x76\x65\x43\x6f\x6e\x74\x65\x6e\x74\x12\x59\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x6d\x6f\x76\x61\x6c\x52\x65\x71\x75\x65\x73\x74\x1a\x5a\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x42\x66\x66\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x6d\x6f\x76\x61\x6c\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x03\x88\x02\x01\x12\xb9\x01\x0a\x12\x52\x65\x6d\x6f\x76\x65\x48\x69\x76\x65\x73\x43\x6f\x6e\x74\x65\x6e\x74\x12\x4f\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x52\x65\x6d\x6f\x76\x65\x48\x69\x76\x65\x73\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x71\x75\x65\x73\x74\x1a\x50\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x52\x65\x6d\x6f\x76\x65\x48\x69\x76\x65\x73\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x00\x12\xc2\x01\x0a\x14\x47\x65\x74\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x44\x65\x74\x61\x69\x6c\x73\x12\x51\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x44\x65\x74\x61\x69\x6c\x73\x52\x65\x71\x75\x65\x73\x74\x1a\x52\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x44\x65\x74\x61\x69\x6c\x73\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x03\x88\x02\x01\x12\xc2\x01\x0a\x14\x47\x65\x74\x4a\x6f\x69\x6e\x65\x64\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x12\x51\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x4a\x6f\x69\x6e\x65\x64\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x52\x65\x71\x75\x65\x73\x74\x1a\x52\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x4a\x6f\x69\x6e\x65\x64\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x03\x88\x02\x01\x12\xbc\x01\x0a\x13\x47\x65\x74\x48\x69\x76\x65\x73\x4e\x65\x77\x41\x63\x74\x69\x76\x69\x74\x79\x12\x50\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x48\x69\x76\x65\x73\x4e\x65\x77\x41\x63\x74\x69\x76\x69\x74\x79\x52\x65\x71\x75\x65\x73\x74\x1a\x51\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x48\x69\x76\x65\x73\x4e\x65\x77\x41\x63\x74\x69\x76\x69\x74\x79\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x00\x12\xc2\x01\x0a\x15\x47\x65\x74\x48\x69\x76\x65\x41\x6c\x6c\x6f\x77\x65\x64\x43\x6f\x6e\x74\x65\x6e\x74\x12\x52\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x48\x69\x76\x65\x41\x6c\x6c\x6f\x77\x65\x64\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x71\x75\x65\x73\x74\x1a\x53\x2e\x63\x6f\x6d\x2e\x62\x75\x6d\x62\x6c\x65\x2e\x63\x6f\x6e\x74\x72\x61\x63\x74\x2e\x73\x65\x72\x76\x69\x63\x65\x2e\x62\x75\x6d\x62\x6c\x65\x66\x72\x69\x65\x6e\x64\x73\x63\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x2e\x47\x65\x74\x48\x69\x76\x65\x41\x6c\x6c\x6f\x77\x65\x64\x43\x6f\x6e\x74\x65\x6e\x74\x52\x65\x73\x70\x6f\x6e\x73\x65\x22\x00\x42\x54\x48\x01\xca\x02\x20\x42\x75\x6d\x62\x6c\x65\x41\x70\x69\x5c\x53\x65\x72\x76\x69\x63\x65\x5c\x42\x46\x46\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\xe2\x02\x2c\x42\x75\x6d\x62\x6c\x65\x41\x70\x69\x5c\x53\x65\x72\x76\x69\x63\x65\x5c\x42\x46\x46\x43\x6f\x6c\x6c\x65\x63\x74\x69\x76\x65\x73\x5c\x47\x50\x42\x4d\x65\x74\x61\x64\x61\x74\x61\x62\x06\x70\x72\x6f\x74\x6f\x33";

$str = str_repeat($str, 12); // 95520 bytes of raw string

$hexString = '';
$hexAsciiString = '';
$binChars = 0;
$printableChars = 0;
foreach (str_split($str) as $c) {
    $hexString .= sprintf('\\x%02x', ord($c));
    if (32 <= ord($c) && ord($c) <= 126) {
        if ($c === '\\') {
            $hexAsciiString .= '\\\\';
        } elseif ($c === '"') {
            $hexAsciiString .= '\\"';
        } elseif ($c === '$') {
            $hexAsciiString .= '\\$';
        } else {
            $hexAsciiString .= $c;
        }
        ++ $printableChars;
    } else {
        ++ $binChars;
        $hexAsciiString .= sprintf('\\x%02x', ord($c));
    }
}

printf("The percent of printable chars: %1.5f (%d vs %d)\n", (double)$printableChars / ($printableChars + $binChars), $printableChars, $binChars);

$hexString = "\"$hexString\";"; // 286560 bytes
$hexAsciiString = "\"$hexAsciiString\";"; // 126876 bytes
$str = str_replace('\'', '\\\'', $str);
$binaryString = "'$str';";

$time = microtime(true);

const REPEATS = 10000;
for ($i = 0; $i < REPEATS; $i++) {
    eval($hexString);
}

$timeEnd = microtime(true);

printf("hex parsing: %1.5f\n", $timeEnd - $time);

$time = microtime(true);

for ($i = 0; $i < REPEATS; $i++) {
    eval($binaryString);
}

$timeEnd = microtime(true);

printf("bin parsing: %1.5f\n", $timeEnd - $time);


$time = microtime(true);

for ($i = 0; $i < REPEATS; $i++) {
    eval($hexAsciiString);
}

$timeEnd = microtime(true);

printf("hex+ascii parsing: %1.5f\n", $timeEnd - $time);

@haberman
Copy link
Member

Thanks for the benchmark, that is really helpful.

But providing that this happens quite rarely, I reckon we can allow the slowdown in this place.

At startup, the entire schema will be loaded from these strings. For large schemas, that is a lot of data, and the speed of these initial operations can have a large impact on startup time. We have put significant effort into optimizing startup time.

The PR that initially created these binary strings was part of that optimization effort: #8006 That said, the PR says that this saved "a small amount of CPU" (it's not even quantified). Other optimizations such as #8000 had far more impact.

I modified your benchmark to also check hex2bin(), which is what we used to do. The hexAscii string is at least faster than hex2bin(), even if it's slower than binary:

The percent of printable chars: 0.83869 (80112 vs 15408)
hex parsing: 3.22371, length=382090
hex2Bin parsing: 1.18489, length=191059
hexAscii parsing: 0.89524, length=142306
binary parsing: 0.26437, length=95542
<?php

$str = "...";

$str = str_repeat($str, 12); // 95520 bytes of raw string

$hexString = '';
$hexAsciiString = '';
$hex2BinString = '';
$binChars = 0;
$printableChars = 0;
foreach (str_split($str) as $c) {
    $hexString .= sprintf('\\x%02x', ord($c));
    $hex2BinString .= sprintf('%02x', ord($c));
    if (32 <= ord($c) && ord($c) <= 126) {
        if ($c === '\\') {
            $hexAsciiString .= '\\\\';
        } elseif ($c === '"') {
            $hexAsciiString .= '\\"';
        } elseif ($c === '$') {
            $hexAsciiString .= '\\$';
        } else {
            $hexAsciiString .= $c;
        }
        ++ $printableChars;
    } else {
        ++ $binChars;
        $hexAsciiString .= sprintf('\\x%02x', ord($c));
    }
}

printf("The percent of printable chars: %1.5f (%d vs %d)\n", (double)$printableChars / ($printableChars + $binChars), $printableChars, $binChars);

$hexString = "return \"$hexString\";";
$hex2BinString = "return hex2bin(\"$hex2BinString\");";
$hexAsciiString = "return \"$hexAsciiString\";";
$str = str_replace('\'', '\\\'', $str);
$binaryString = "return '$str';";

const REPEATS = 1000;

@ini_set('zend.assertions', '1');

function bm($str, $name) {
  global $hexString;
  if (eval($str) != eval($hexString)) {
    throw new Exception('Incorrect output');
  }
  $time = microtime(true);
  for ($i = 0; $i < REPEATS; $i++) {
      eval($str);
  }
  $timeEnd = microtime(true);
  printf("%s parsing: %1.5f, length=%d\n", $name, $timeEnd - $time, strlen($str));
}

bm($hexString, "hex");
bm($hex2BinString, "hex2Bin");
bm($hexAsciiString, "hexAscii");
bm($binaryString, "binary");

Given that the original PR says this saved only a "small amount of CPU", I'm inclined to move forward with using the hexAscii string.

@mikhainin
Copy link
Contributor

@haberman, do you think we can reopen the PR? Or shall I open a new one?

@haberman haberman reopened this Aug 28, 2023
@haberman haberman requested a review from a team as a code owner August 28, 2023 22:35
@haberman
Copy link
Member

@mikhainin I initially reopened, but then I saw that this PR was closed due to lack of response from the original author.

If you have the bandwidth to submit another PR, please do so, and we'll review it.

@mikhainin
Copy link
Contributor

Thanks, @haberman, I opened #13911 to follow-up

copybara-service bot pushed a commit that referenced this pull request Jan 24, 2024
Follow up on #10404, the initial description:

> For the sake of performance, the hex2bin call was removed from the generated PHP code in #8006
However, after this PR all autogenerated files contain binary data, which makes it hard or even impossible to see using common tools (git and github included (i.e. [this file](https://github.com/protocolbuffers/protobuf/blob/main/php/src/GPBMetadata/Google/Protobuf/Struct.php)), as well as some code editors) since most software considers such files as binary ones and not as code.

Alexander suggested using a hex representation of string literals and I updated the original patch a bit, not hex-encoding printable characters.

Benchmarks from [#10404#issuecomment-1635939062](#10404 (comment))
```
The percent of printable chars: 0.83869 (80112 vs 15408)
hex parsing: 3.22371, length=382090
hex2Bin parsing: 1.18489, length=191059
hexAscii parsing: 0.89524, length=142306 (suggested option)
binary parsing: 0.26437, length=95542
```

Closes #13911

COPYBARA_INTEGRATE_REVIEW=#13911 from mikhainin:do_not_store_binary_data_inside_php_files 490958e
PiperOrigin-RevId: 601123255
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants