api: Extend and improve json-iterator usage

[beorn7]

For one, this pulls up the histogram-related json-iterator usage from prometheus/common into the API client. Previously, the only json-iterater usage was here in the API client. But then json-iterator was used for the native histogram additions directly in prometheus/common, see
https://github.com/prometheus/common/pull/440/files . This however meant that any user of prometheus/common/model would now link in json-iterator, even if they are not using the JSON marshaling at all. To keep prometheus/common/model more leightweight, this commit moves all the json-iterator usage into the API client itself, as it was done before for the normal float samples.

This commit also adds an unmarshaling function for native histograms, which didn't even exist in prometheus/common/model so far.

It also adds json-iterator marshaling and un-marshaling for model.SampleStream, which is only needed for the benchmark (BenchmarkSamplesJsonSerialization). This fixes the benchmark such that it actually compares json-iterator and std-lib json encoding (which didn't work before because the custom marshaling methods of model.SampleStream enforced std-lib json encoding for floats and json-iterator encoding for histograms in all cases).

I expect this to fix #1179.

[beorn7]

@zenador as discussed. I believe, together with prometheus/common#453 , this should still work out in Mimir. To try it out, just make sure you have vendored in both PRs.

[codesome]

LGTM after @krajorama's comments. I believe this code was ported from common, so I did not do a deep review

[beorn7]

It's not just ported. It also adds the unmarshal abilities. Also, it tries to fully embrace json-iterator, whereas the previous code in common was implementing json-iterator usage within a conventional marshaling method (which I believe only gets you some of the benefits of using json-iterator).

[beorn7]

OK, this should do the trick.

[beorn7]

Ah wait, now the order of labels is not deterministic (in fact, it never was, but we used only one in the tests so far). Will fix that, too.

[beorn7]

OK, we should be good now. @krajorama and @zenador, could you double check that this doesn't create any undue burdens for Mimir?

[codesome]

LGTM with small comments on the tests.

[kakkoyun]

LGTM.

Out of curiosity: Do we have result of the benchmark comparison anywhere?

[beorn7]

Note: Previously posted benchmark used wrong dependencies (and thus snuck in jsoniter again). Will post correct results soon.

[beorn7]

And here the benchmark in its full glory:

goos: linux
goarch: amd64
pkg: github.com/prometheus/client_golang/api/prometheus/v1
cpu: 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
BenchmarkSamplesJsonSerialization/10/10/marshal/encoding/json/floats-16                     8732            137981 ns/op           33760 B/op      1512 allocs/op
BenchmarkSamplesJsonSerialization/10/10/marshal/jsoniter/floats-16                         69432             16974 ns/op            6541 B/op        92 allocs/op
BenchmarkSamplesJsonSerialization/10/10/marshal/encoding/json/histograms-16                  694           1864528 ns/op          531641 B/op      16686 allocs/op
BenchmarkSamplesJsonSerialization/10/10/marshal/jsoniter/histograms-16                      7674            157222 ns/op           44613 B/op         92 allocs/op
BenchmarkSamplesJsonSerialization/10/10/unmarshal/encoding/json/floats-16                  10000            111661 ns/op           27961 B/op        664 allocs/op
BenchmarkSamplesJsonSerialization/10/10/unmarshal/jsoniter/floats-16                       34912             32038 ns/op           21737 B/op        550 allocs/op
BenchmarkSamplesJsonSerialization/10/10/unmarshal/encoding/json/histograms-16               1071           1126084 ns/op          186308 B/op       3576 allocs/op
BenchmarkSamplesJsonSerialization/10/10/unmarshal/jsoniter/histograms-16                    4600            242203 ns/op           95991 B/op       4320 allocs/op
BenchmarkSamplesJsonSerialization/10/100/marshal/encoding/json/floats-16                     878           1275515 ns/op          301719 B/op      15014 allocs/op
BenchmarkSamplesJsonSerialization/10/100/marshal/jsoniter/floats-16                        13528             83137 ns/op           28192 B/op         92 allocs/op
BenchmarkSamplesJsonSerialization/10/100/marshal/encoding/json/histograms-16                  56          18389181 ns/op         5742097 B/op     167858 allocs/op
BenchmarkSamplesJsonSerialization/10/100/marshal/jsoniter/histograms-16                      764           1577361 ns/op          374873 B/op         93 allocs/op
BenchmarkSamplesJsonSerialization/10/100/unmarshal/encoding/json/floats-16                  1263            924032 ns/op          236820 B/op       5164 allocs/op
BenchmarkSamplesJsonSerialization/10/100/unmarshal/jsoniter/floats-16                       5103            226944 ns/op          159039 B/op       4150 allocs/op
BenchmarkSamplesJsonSerialization/10/100/unmarshal/encoding/json/histograms-16                92          11965127 ns/op         1818293 B/op      34297 allocs/op
BenchmarkSamplesJsonSerialization/10/100/unmarshal/jsoniter/histograms-16                    508           2334601 ns/op          915824 B/op      42940 allocs/op
BenchmarkSamplesJsonSerialization/10/1000/marshal/encoding/json/floats-16                     99          12006212 ns/op         3039883 B/op     150021 allocs/op
BenchmarkSamplesJsonSerialization/10/1000/marshal/jsoniter/floats-16                        1899            635218 ns/op          243742 B/op         92 allocs/op
BenchmarkSamplesJsonSerialization/10/1000/marshal/encoding/json/histograms-16                  7         168927378 ns/op        60681275 B/op    1679953 allocs/op
BenchmarkSamplesJsonSerialization/10/1000/marshal/jsoniter/histograms-16                      86          13779806 ns/op         3797715 B/op         95 allocs/op
BenchmarkSamplesJsonSerialization/10/1000/unmarshal/encoding/json/floats-16                  130           9019330 ns/op         2329028 B/op      50166 allocs/op
BenchmarkSamplesJsonSerialization/10/1000/unmarshal/jsoniter/floats-16                       540           2151526 ns/op         1504689 B/op      40150 allocs/op
BenchmarkSamplesJsonSerialization/10/1000/unmarshal/encoding/json/histograms-16                9         112337969 ns/op        18556022 B/op     352435 allocs/op
BenchmarkSamplesJsonSerialization/10/1000/unmarshal/jsoniter/histograms-16                    46          23504944 ns/op         9059048 B/op     429947 allocs/op
BenchmarkSamplesJsonSerialization/100/10/marshal/encoding/json/floats-16                     867           1526717 ns/op          338736 B/op      15105 allocs/op
BenchmarkSamplesJsonSerialization/100/10/marshal/jsoniter/floats-16                         7323            200341 ns/op           63262 B/op        902 allocs/op
BenchmarkSamplesJsonSerialization/100/10/marshal/encoding/json/histograms-16                  66          19245647 ns/op         5736698 B/op     166859 allocs/op
BenchmarkSamplesJsonSerialization/100/10/marshal/jsoniter/histograms-16                      771           1659066 ns/op          415837 B/op        904 allocs/op
BenchmarkSamplesJsonSerialization/100/10/unmarshal/encoding/json/floats-16                  1000           1166025 ns/op          277906 B/op       6605 allocs/op
BenchmarkSamplesJsonSerialization/100/10/unmarshal/jsoniter/floats-16                       3750            320976 ns/op          227060 B/op       5500 allocs/op
BenchmarkSamplesJsonSerialization/100/10/unmarshal/encoding/json/histograms-16                98          11632679 ns/op         1865100 B/op      35828 allocs/op
BenchmarkSamplesJsonSerialization/100/10/unmarshal/jsoniter/histograms-16                    469           2453231 ns/op          941066 B/op      43204 allocs/op
BenchmarkSamplesJsonSerialization/100/100/marshal/encoding/json/floats-16                     99          11995562 ns/op         3073930 B/op     150113 allocs/op
BenchmarkSamplesJsonSerialization/100/100/marshal/jsoniter/floats-16                        1640            706291 ns/op          267577 B/op        902 allocs/op
BenchmarkSamplesJsonSerialization/100/100/marshal/encoding/json/histograms-16                  7         168068959 ns/op        58368977 B/op    1678138 allocs/op
BenchmarkSamplesJsonSerialization/100/100/marshal/jsoniter/histograms-16                      84          14098131 ns/op         3517504 B/op        904 allocs/op
BenchmarkSamplesJsonSerialization/100/100/unmarshal/encoding/json/floats-16                  134           9100753 ns/op         2371422 B/op      51621 allocs/op
BenchmarkSamplesJsonSerialization/100/100/unmarshal/jsoniter/floats-16                       519           2171140 ns/op         1554558 B/op      41506 allocs/op
BenchmarkSamplesJsonSerialization/100/100/unmarshal/encoding/json/histograms-16                9         111895584 ns/op        18624831 B/op     354185 allocs/op
BenchmarkSamplesJsonSerialization/100/100/unmarshal/jsoniter/histograms-16                    50          23371291 ns/op         8818471 B/op     429441 allocs/op
BenchmarkSamplesJsonSerialization/100/1000/marshal/encoding/json/floats-16                     9         118825623 ns/op        31470278 B/op    1500117 allocs/op
BenchmarkSamplesJsonSerialization/100/1000/marshal/jsoniter/floats-16                        187           6223831 ns/op         2336576 B/op        902 allocs/op
BenchmarkSamplesJsonSerialization/100/1000/marshal/encoding/json/histograms-16                 1        1684408325 ns/op        614336752 B/op  16798314 allocs/op
BenchmarkSamplesJsonSerialization/100/1000/marshal/jsoniter/histograms-16                      8         137553434 ns/op        35466424 B/op        904 allocs/op
BenchmarkSamplesJsonSerialization/100/1000/unmarshal/encoding/json/floats-16                  12          86076610 ns/op        23707380 B/op     501880 allocs/op
BenchmarkSamplesJsonSerialization/100/1000/unmarshal/jsoniter/floats-16                       57          20860301 ns/op        14418114 B/op     401553 allocs/op
BenchmarkSamplesJsonSerialization/100/1000/unmarshal/encoding/json/histograms-16                       1        1177192823 ns/op        223191256 B/op   4505026 allocs/op
BenchmarkSamplesJsonSerialization/100/1000/unmarshal/jsoniter/histograms-16                            5         231029658 ns/op        88360867 B/op    4299807 allocs/op
BenchmarkSamplesJsonSerialization/1000/10/marshal/encoding/json/floats-16                             90          12879048 ns/op         3468431 B/op     151012 allocs/op
BenchmarkSamplesJsonSerialization/1000/10/marshal/jsoniter/floats-16                                 832           1393033 ns/op          627944 B/op       9003 allocs/op
BenchmarkSamplesJsonSerialization/1000/10/marshal/encoding/json/histograms-16                          6         167692970 ns/op        58739218 B/op    1668134 allocs/op
BenchmarkSamplesJsonSerialization/1000/10/marshal/jsoniter/histograms-16                              76          15615271 ns/op         4096227 B/op       9005 allocs/op
BenchmarkSamplesJsonSerialization/1000/10/unmarshal/encoding/json/floats-16                           96          10761365 ns/op         2787004 B/op      66118 allocs/op
BenchmarkSamplesJsonSerialization/1000/10/unmarshal/jsoniter/floats-16                               367           3240022 ns/op         2067919 B/op      55054 allocs/op
BenchmarkSamplesJsonSerialization/1000/10/unmarshal/encoding/json/histograms-16                        9         113263048 ns/op        19124375 B/op     370453 allocs/op
BenchmarkSamplesJsonSerialization/1000/10/unmarshal/jsoniter/histograms-16                            46          24429008 ns/op         9221668 B/op     432305 allocs/op
BenchmarkSamplesJsonSerialization/1000/100/marshal/encoding/json/floats-16                             9         118519663 ns/op        33382637 B/op    1501017 allocs/op
BenchmarkSamplesJsonSerialization/1000/100/marshal/jsoniter/floats-16                                163           7082971 ns/op         2686341 B/op       9002 allocs/op
BenchmarkSamplesJsonSerialization/1000/100/marshal/encoding/json/histograms-16                         1        1682598676 ns/op        602379296 B/op  16780240 allocs/op
BenchmarkSamplesJsonSerialization/1000/100/marshal/jsoniter/histograms-16                              7         153306807 ns/op        63818085 B/op       9014 allocs/op
BenchmarkSamplesJsonSerialization/1000/100/unmarshal/encoding/json/floats-16                          12          87060307 ns/op        24282541 B/op     517923 allocs/op
BenchmarkSamplesJsonSerialization/1000/100/unmarshal/jsoniter/floats-16                               55          21591973 ns/op        13795235 B/op     415382 allocs/op
BenchmarkSamplesJsonSerialization/1000/100/unmarshal/encoding/json/histograms-16                       1        1176108660 ns/op        225636352 B/op   4540036 allocs/op
BenchmarkSamplesJsonSerialization/1000/100/unmarshal/jsoniter/histograms-16                            5         232410956 ns/op        86345472 B/op    4296809 allocs/op
BenchmarkSamplesJsonSerialization/1000/1000/marshal/encoding/json/floats-16                            1        1211173264 ns/op        346271808 B/op  15001059 allocs/op
BenchmarkSamplesJsonSerialization/1000/1000/marshal/jsoniter/floats-16                                16          66596466 ns/op        31334329 B/op       9005 allocs/op
BenchmarkSamplesJsonSerialization/1000/1000/marshal/encoding/json/histograms-16                        1        19388791733 ns/op       5904839048 B/op 167980293 allocs/op
BenchmarkSamplesJsonSerialization/1000/1000/marshal/jsoniter/histograms-16                             1        2034999217 ns/op        2233774152 B/op     9080 allocs/op
BenchmarkSamplesJsonSerialization/1000/1000/unmarshal/encoding/json/floats-16                          2         869687903 ns/op        260146264 B/op   5032520 allocs/op
BenchmarkSamplesJsonSerialization/1000/1000/unmarshal/jsoniter/floats-16                               5         205689092 ns/op        119564204 B/op   4019004 allocs/op
BenchmarkSamplesJsonSerialization/1000/1000/unmarshal/encoding/json/histograms-16                      1        11688071190 ns/op       2231900336 B/op 45050036 allocs/op
BenchmarkSamplesJsonSerialization/1000/1000/unmarshal/jsoniter/histograms-16                           1        2282696797 ns/op        882476976 B/op  43010020 allocs/op

[bwplotka]

Thanks! 💪🏽