perf: improve performance of PreparedStatement.setBlob, BlobInputStream, and BlobOutputStream with dynamic buffer sizing #3044

vlsi · 2023-11-29T10:27:18Z

This reverts commit 3af3b32 (see to #2376, #2422)

There are several extra fixes:

The buffer is created lazily, and it is created only in case the number of requested bytes is less than the buffer size
"small" read(byte[], int, int) can be buffered as well (e.g. if the buffer size exceeds the requested len)
Added locks to read calls for consistency with the locking in mark/reset

Note: this PR does not address "default blob buffer size" issue.

See #2376

I arranged commits as follows:

Build script-like changes that do not affect performance
Add benchmarks
Performance fixes for BlobInputStream
Performance fixes for BlobOuputStream

Relevant issues:

pgjdbc/src/main/java/org/postgresql/largeobject/BlobInputStream.java

davecramer · 2023-11-29T12:07:59Z

pgjdbc/src/main/java/org/postgresql/largeobject/LargeObject.java

@@ -210,12 +216,10 @@ public byte[] read(int len) throws SQLException {
   */
  public int read(byte[] buf, int off, int len) throws SQLException {
    byte[] b = read(len);


same comment about buf naming, dest is better and makes more sense later in the code.

davecramer · 2023-11-29T12:10:39Z

minor nits, other than that LGTM

pgjdbc/src/main/java/org/postgresql/largeobject/BlobOutputStream.java

davecramer · 2023-12-01T11:40:01Z

pgjdbc/src/main/java/org/postgresql/largeobject/BlobOutputStream.java

@@ -55,22 +59,37 @@ public BlobOutputStream(LargeObject lo) {
   */
  public BlobOutputStream(LargeObject lo, int bufferSize) {
    this.lo = lo;
-    this.bufferSize = bufferSize;
+    // Avoid "0" buffer size, and ensure the bufferSize will always be a power of two
+    this.bufferSize = Integer.highestOneBit(Math.max(bufferSize, 1));


Cool, didn't know this function existed

pgjdbc/src/main/java/org/postgresql/largeobject/BlobOutputStream.java

vlsi · 2023-12-01T15:39:38Z

with fix (ssl=on)

java 17.0.7, -Xmx1g, Apple M1

Benchmark        (bufferSize)  (readSize)     (size)  Mode  Cnt     Score      Error  Units
readArray              131072       65536  104857600  avgt    3  2612,057 ±  532,073  ms/op
readArray              131072      131072  104857600  avgt    3  2701,488 ±  740,033  ms/op
readArray              262144       65536  104857600  avgt    3  2299,744 ±  220,876  ms/op
readArray              262144      131072  104857600  avgt    3  2292,834 ±  371,013  ms/op
readArray              524288       65536  104857600  avgt    3  2142,774 ±  281,031  ms/op
readArray              524288      131072  104857600  avgt    3  2130,650 ±  304,826  ms/op
readArray              524288      524288  104857600  avgt    3  2123,672 ±  169,076  ms/op
readArrayRandom        131072       65536  104857600  avgt    3  2693,018 ±  683,909  ms/op
readArrayRandom        131072      131072  104857600  avgt    3  2736,947 ± 1867,688  ms/op
readArrayRandom        262144       65536  104857600  avgt    3  2296,865 ±  692,942  ms/op
readArrayRandom        262144      131072  104857600  avgt    3  2298,103 ± 1207,288  ms/op
readArrayRandom        524288       65536  104857600  avgt    3  2168,311 ±  563,236  ms/op
readArrayRandom        524288      131072  104857600  avgt    3  2178,537 ±  650,530  ms/op
readArrayRandom        524288      524288  104857600  avgt    3  2119,763 ±  358,512  ms/op

java 17.0.7, -Xmx256m, Apple M1

Benchmark        (bufferSize)  (readSize)    (size)  Mode  Cnt    Score     Error  Units
readArray              131072       65536  10485760  avgt    3  242,565 ±  33,217  ms/op
readArray              131072      131072  10485760  avgt    3  247,398 ±  65,393  ms/op
readArray              262144       65536  10485760  avgt    3  216,742 ±  69,122  ms/op
readArray              262144      131072  10485760  avgt    3  216,215 ±  50,428  ms/op
readArray              524288       65536  10485760  avgt    3  203,406 ±  30,419  ms/op
readArray              524288      131072  10485760  avgt    3  203,603 ±  53,533  ms/op
readArray              524288      524288  10485760  avgt    3  210,421 ± 107,382  ms/op
readArrayRandom        131072       65536  10485760  avgt    3  250,675 ±  67,252  ms/op
readArrayRandom        131072      131072  10485760  avgt    3  246,594 ±  44,621  ms/op
readArrayRandom        262144       65536  10485760  avgt    3  216,981 ±  75,521  ms/op
readArrayRandom        262144      131072  10485760  avgt    3  216,747 ±  47,019  ms/op
readArrayRandom        524288       65536  10485760  avgt    3  201,962 ±  49,681  ms/op
readArrayRandom        524288      131072  10485760  avgt    3  207,149 ± 201,073  ms/op
readArrayRandom        524288      524288  10485760  avgt    3  200,748 ±  10,300  ms/op

java 17.0.7, -Xmx256m, Apple M1
Benchmark        (bufferSize)  (readSize)   (size)  Mode  Cnt    Score     Error  Units
readArray                4096           1      100  avgt    3    0,423 ±   0,572  ms/op
readArray                4096           1     1024  avgt    3    0,446 ±   0,254  ms/op
readArray                4096           1   102400  avgt    3   14,340 ±   1,413  ms/op
readArray                4096           1  1048576  avgt    3  145,125 ±  45,594  ms/op
readArray                4096          10      100  avgt    3    0,395 ±   0,102  ms/op
readArray                4096          10     1024  avgt    3    0,422 ±   0,408  ms/op
readArray                4096          10   102400  avgt    3   12,890 ±   5,373  ms/op
readArray                4096          10  1048576  avgt    3  132,470 ±  62,526  ms/op
readArray                4096         100      100  avgt    3    0,402 ±   0,131  ms/op
readArray                4096         100     1024  avgt    3    0,412 ±   0,108  ms/op
readArray                4096         100   102400  avgt    3   12,566 ±   3,081  ms/op
readArray                4096         100  1048576  avgt    3  134,764 ±  65,968  ms/op
readArray                4096        1024     1024  avgt    3    0,413 ±   0,363  ms/op
readArray                4096        1024   102400  avgt    3   12,665 ±   3,668  ms/op
readArray                4096        1024  1048576  avgt    3  128,273 ±  20,562  ms/op
readArray                4096       10240   102400  avgt    3    6,512 ±   2,015  ms/op
readArray                4096       10240  1048576  avgt    3   68,848 ±  28,928  ms/op
readArray                4096       65536   102400  avgt    3    3,161 ±   0,971  ms/op
readArray                4096       65536  1048576  avgt    3   30,701 ±   6,700  ms/op
readArray                4096      131072  1048576  avgt    3   26,659 ±  20,594  ms/op
readArray                4096      524288  1048576  avgt    3   20,628 ±   5,077  ms/op
readArray                8192           1      100  avgt    3    0,947 ±  16,339  ms/op
readArray                8192           1     1024  avgt    3    0,455 ±   0,473  ms/op
readArray                8192           1   102400  avgt    3   10,332 ±  29,398  ms/op
readArray                8192           1  1048576  avgt    3   96,077 ±  25,719  ms/op
readArray                8192          10      100  avgt    3    0,413 ±   0,099  ms/op
readArray                8192          10     1024  avgt    3    0,481 ±   0,343  ms/op
readArray                8192          10   102400  avgt    3    8,792 ±   6,308  ms/op
readArray                8192          10  1048576  avgt    3   81,213 ±  20,323  ms/op
readArray                8192         100      100  avgt    3    0,433 ±   0,145  ms/op
readArray                8192         100     1024  avgt    3    0,497 ±   1,197  ms/op
readArray                8192         100   102400  avgt    3    8,203 ±   1,248  ms/op
readArray                8192         100  1048576  avgt    3   82,933 ±  48,143  ms/op
readArray                8192        1024     1024  avgt    3    0,437 ±   0,144  ms/op
readArray                8192        1024   102400  avgt    3    8,021 ±   3,399  ms/op
readArray                8192        1024  1048576  avgt    3   78,948 ±  43,795  ms/op
readArray               65536           1      100  avgt    3    0,424 ±   0,188  ms/op
readArray               65536           1     1024  avgt    3    0,442 ±   0,101  ms/op
readArray               65536           1   102400  avgt    3    5,522 ±  11,918  ms/op
readArray               65536           1  1048576  avgt    3   59,480 ±  43,147  ms/op
readArray               65536          10      100  avgt    3    0,464 ±   1,134  ms/op
readArray               65536          10     1024  avgt    3    0,430 ±   0,141  ms/op
readArray               65536          10   102400  avgt    3    3,428 ±   1,159  ms/op
readArray               65536          10  1048576  avgt    3   31,994 ±  11,375  ms/op
readArray               65536         100      100  avgt    3    0,397 ±   0,141  ms/op
readArray               65536         100     1024  avgt    3    0,430 ±   0,203  ms/op
readArray               65536         100   102400  avgt    3    3,248 ±   0,808  ms/op
readArray               65536         100  1048576  avgt    3   31,250 ±  20,935  ms/op
readArray               65536        1024     1024  avgt    3    0,462 ±   0,454  ms/op
readArray               65536        1024   102400  avgt    3    3,350 ±   1,773  ms/op
readArray               65536        1024  1048576  avgt    3   31,819 ±  11,687  ms/op
readArray               65536       10240   102400  avgt    3    3,402 ±   2,165  ms/op
readArray               65536       10240  1048576  avgt    3   30,709 ±   4,576  ms/op
readArray               65536       65536   102400  avgt    3    3,368 ±   3,431  ms/op
readArray               65536       65536  1048576  avgt    3   31,289 ±   9,536  ms/op
readArray              131072           1      100  avgt    3    0,403 ±   0,105  ms/op
readArray              131072           1     1024  avgt    3    0,428 ±   0,108  ms/op
readArray              131072           1   102400  avgt    3    3,847 ±   0,166  ms/op
readArray              131072           1  1048576  avgt    3   40,638 ±   6,620  ms/op
readArray              131072          10      100  avgt    3    0,444 ±   0,110  ms/op
readArray              131072          10     1024  avgt    3    0,466 ±   0,432  ms/op
readArray              131072          10   102400  avgt    3    3,486 ±  27,109  ms/op
readArray              131072          10  1048576  avgt    3   26,194 ±   6,041  ms/op
readArray              131072         100      100  avgt    3    0,403 ±   0,071  ms/op
readArray              131072         100     1024  avgt    3    0,433 ±   0,109  ms/op
readArray              131072         100   102400  avgt    3    2,510 ±   0,077  ms/op
readArray              131072         100  1048576  avgt    3   25,328 ±   9,075  ms/op
readArray              131072        1024     1024  avgt    3    0,449 ±   0,301  ms/op
readArray              131072        1024   102400  avgt    3    2,576 ±   0,884  ms/op
readArray              131072        1024  1048576  avgt    3   28,505 ±  78,335  ms/op
readArray              131072       10240   102400  avgt    3    2,563 ±   0,158  ms/op
readArray              131072       10240  1048576  avgt    3   25,243 ±   6,801  ms/op
readArray              131072       65536   102400  avgt    3    2,496 ±   0,186  ms/op
readArray              131072       65536  1048576  avgt    3   24,712 ±   4,630  ms/op
readArray              131072      131072  1048576  avgt    3   24,704 ±   5,953  ms/op
readArrayRandom          4096           1      100  avgt    3    0,404 ±   0,100  ms/op
readArrayRandom          4096           1     1024  avgt    3    0,543 ±   0,772  ms/op
readArrayRandom          4096           1   102400  avgt    3   22,745 ±   5,031  ms/op
readArrayRandom          4096           1  1048576  avgt    3  233,109 ±  37,117  ms/op
readArrayRandom          4096          10      100  avgt    3    0,397 ±   0,139  ms/op
readArrayRandom          4096          10     1024  avgt    3    0,418 ±   0,145  ms/op
readArrayRandom          4096          10   102400  avgt    3   13,606 ±   2,366  ms/op
readArrayRandom          4096          10  1048576  avgt    3  154,113 ± 154,377  ms/op
readArrayRandom          4096         100      100  avgt    3    0,411 ±   0,448  ms/op
readArrayRandom          4096         100     1024  avgt    3    0,434 ±   0,122  ms/op
readArrayRandom          4096         100   102400  avgt    3   13,635 ±   8,758  ms/op
readArrayRandom          4096         100  1048576  avgt    3  152,548 ± 357,130  ms/op
readArrayRandom          4096        1024     1024  avgt    3    0,471 ±   0,402  ms/op
readArrayRandom          4096        1024   102400  avgt    3   13,283 ±  14,442  ms/op
readArrayRandom          4096        1024  1048576  avgt    3  132,055 ±  43,991  ms/op
readArrayRandom          4096       10240   102400  avgt    3   10,688 ±   9,011  ms/op
readArrayRandom          4096       10240  1048576  avgt    3  107,033 ±  44,937  ms/op
readArrayRandom          4096       65536   102400  avgt    3    5,259 ±   1,576  ms/op
readArrayRandom          4096       65536  1048576  avgt    3   42,337 ±  19,377  ms/op
readArrayRandom          4096      131072  1048576  avgt    3   35,213 ±   5,993  ms/op
readArrayRandom          4096      524288  1048576  avgt    3   31,224 ±  80,357  ms/op
readArrayRandom          8192           1      100  avgt    3    0,438 ±   0,310  ms/op
readArrayRandom          8192           1     1024  avgt    3    0,547 ±   0,199  ms/op
readArrayRandom          8192           1   102400  avgt    3   18,082 ±   7,083  ms/op
readArrayRandom          8192           1  1048576  avgt    3  191,293 ±  64,624  ms/op
readArrayRandom          8192          10      100  avgt    3    0,419 ±   0,137  ms/op
readArrayRandom          8192          10     1024  avgt    3    0,434 ±   0,118  ms/op
readArrayRandom          8192          10   102400  avgt    3    9,516 ±   8,778  ms/op
readArrayRandom          8192          10  1048576  avgt    3   91,182 ±  55,948  ms/op
readArrayRandom          8192         100      100  avgt    3    0,435 ±   0,138  ms/op
readArrayRandom          8192         100     1024  avgt    3    0,598 ±   4,718  ms/op
readArrayRandom          8192         100   102400  avgt    3    9,259 ±   9,457  ms/op
readArrayRandom          8192         100  1048576  avgt    3   92,530 ±  93,122  ms/op
readArrayRandom          8192        1024     1024  avgt    3    0,475 ±   0,809  ms/op
readArrayRandom          8192        1024   102400  avgt    3    7,817 ±   1,036  ms/op
readArrayRandom          8192        1024  1048576  avgt    3   89,230 ± 142,636  ms/op
readArrayRandom         65536           1      100  avgt    3    0,432 ±   0,231  ms/op
readArrayRandom         65536           1     1024  avgt    3    0,584 ±   2,045  ms/op
readArrayRandom         65536           1   102400  avgt    3   12,879 ±   2,921  ms/op
readArrayRandom         65536           1  1048576  avgt    3  129,801 ±  25,642  ms/op
readArrayRandom         65536          10      100  avgt    3    0,477 ±   0,369  ms/op
readArrayRandom         65536          10     1024  avgt    3    0,469 ±   0,558  ms/op
readArrayRandom         65536          10   102400  avgt    3    4,110 ±   0,202  ms/op
readArrayRandom         65536          10  1048576  avgt    3   39,844 ±   2,779  ms/op
readArrayRandom         65536         100      100  avgt    3    0,518 ±   2,141  ms/op
readArrayRandom         65536         100     1024  avgt    3    0,421 ±   0,194  ms/op
readArrayRandom         65536         100   102400  avgt    3    3,352 ±   1,570  ms/op
readArrayRandom         65536         100  1048576  avgt    3   33,486 ±  28,457  ms/op
readArrayRandom         65536        1024     1024  avgt    3    0,517 ±   1,261  ms/op
readArrayRandom         65536        1024   102400  avgt    3    3,341 ±   2,051  ms/op
readArrayRandom         65536        1024  1048576  avgt    3   31,767 ±  20,218  ms/op
readArrayRandom         65536       10240   102400  avgt    3    3,383 ±   1,548  ms/op
readArrayRandom         65536       10240  1048576  avgt    3   30,749 ±   8,004  ms/op
readArrayRandom         65536       65536   102400  avgt    3    3,223 ±   0,040  ms/op
readArrayRandom         65536       65536  1048576  avgt    3   30,623 ±   5,045  ms/op
readArrayRandom        131072           1      100  avgt    3    0,394 ±   0,092  ms/op
readArrayRandom        131072           1     1024  avgt    3    0,509 ±   0,109  ms/op
readArrayRandom        131072           1   102400  avgt    3   11,791 ±   0,234  ms/op
readArrayRandom        131072           1  1048576  avgt    3  121,710 ±  11,969  ms/op
readArrayRandom        131072          10      100  avgt    3    0,388 ±   0,201  ms/op
readArrayRandom        131072          10     1024  avgt    3    0,418 ±   0,059  ms/op
readArrayRandom        131072          10   102400  avgt    3    3,287 ±   0,466  ms/op
readArrayRandom        131072          10  1048576  avgt    3   35,536 ±  37,736  ms/op
readArrayRandom        131072         100      100  avgt    3    0,415 ±   0,447  ms/op
readArrayRandom        131072         100     1024  avgt    3    0,479 ±   1,352  ms/op
readArrayRandom        131072         100   102400  avgt    3    2,599 ±   0,290  ms/op
readArrayRandom        131072         100  1048576  avgt    3   27,365 ±   5,724  ms/op
readArrayRandom        131072        1024     1024  avgt    3    0,424 ±   0,191  ms/op
readArrayRandom        131072        1024   102400  avgt    3    2,472 ±   0,641  ms/op
readArrayRandom        131072        1024  1048576  avgt    3   25,246 ±  15,033  ms/op
readArrayRandom        131072       10240   102400  avgt    3    2,543 ±   0,222  ms/op
readArrayRandom        131072       10240  1048576  avgt    3   24,610 ±   6,424  ms/op
readArrayRandom        131072       65536   102400  avgt    3    2,488 ±   0,428  ms/op
readArrayRandom        131072       65536  1048576  avgt    3   24,822 ±   6,981  ms/op
readArrayRandom        131072      131072  1048576  avgt    3   24,840 ±   7,371  ms/op
readByte                 4096           1      100  avgt    3    0,390 ±   0,156  ms/op
readByte                 4096           1     1024  avgt    3    0,422 ±   0,107  ms/op
readByte                 4096           1   102400  avgt    3   14,115 ±   3,842  ms/op
readByte                 4096           1  1048576  avgt    3  142,037 ±  70,279  ms/op
readByte                 8192           1      100  avgt    3    0,386 ±   0,083  ms/op
readByte                 8192           1     1024  avgt    3    0,427 ±   0,070  ms/op
readByte                 8192           1   102400  avgt    3   10,677 ±  14,612  ms/op
readByte                 8192           1  1048576  avgt    3   97,635 ±  67,902  ms/op
readByte                65536           1      100  avgt    3    0,395 ±   0,058  ms/op
readByte                65536           1     1024  avgt    3    0,445 ±   0,048  ms/op
readByte                65536           1   102400  avgt    3    4,818 ±   5,297  ms/op
readByte                65536           1  1048576  avgt    3   46,538 ±   7,323  ms/op
readByte               131072           1      100  avgt    3    0,425 ±   0,081  ms/op
readByte               131072           1     1024  avgt    3    0,469 ±   0,432  ms/op
readByte               131072           1   102400  avgt    3    4,017 ±   1,278  ms/op
readByte               131072           1  1048576  avgt    3   40,260 ±  18,467  ms/op

vlsi · 2023-12-01T19:43:07Z

@bokken , @sehrope , I'm more-or-less happy with the changes, I will post more measurements, and it would be great to hear comments from you as well

vlsi · 2023-12-01T20:15:57Z

Measuring writes is not trivial: ERROR: could not extend file "base/16385/2995": No space left on device

Here's what I have so far:

Java 17, with fix (the buffer size is the default 4096, so all writes exceeding 4096 skip the buffer):

Benchmark           (size)  (writeSize)  Mode  Cnt     Score      Error  Units
writeArray             100            1  avgt   15     0,547 ±    0,078  ms/op
writeArray            1024            1  avgt   15     0,601 ±    0,048  ms/op
writeArray            1024         1000  avgt   15     0,633 ±    0,034  ms/op
writeArray          102400            1  avgt   15    27,103 ±    7,153  ms/op
writeArray          102400         1000  avgt   15    18,125 ±    1,294  ms/op
writeArray          102400        10240  avgt   15    11,124 ±    1,535  ms/op
writeArray          102400        65536  avgt   15     4,457 ±    0,458  ms/op
writeArray         1048576            1  avgt   15   248,525 ±   78,202  ms/op
writeArray         1048576         1000  avgt   15   190,362 ±   14,992  ms/op
writeArray         1048576        10240  avgt   15    99,337 ±   10,410  ms/op
writeArray         1048576        65536  avgt   15    39,155 ±    2,623  ms/op
writeArray         1048576       131072  avgt   15    30,789 ±    1,730  ms/op
writeArray         1048576       524288  avgt   15    25,904 ±    0,426  ms/op
writeArray        10485760            1  avgt   15  3483,400 ± 1219,914  ms/op
writeArray        10485760         1000  avgt   15  2097,403 ±  343,091  ms/op
writeArray        10485760        10240  avgt   15  1174,638 ±  135,971  ms/op
writeArray        10485760        65536  avgt   15   375,045 ±    4,404  ms/op
writeArray        10485760       131072  avgt   15   312,606 ±    4,956  ms/op
writeArray        10485760       524288  avgt   15   260,203 ±    4,634  ms/op
writeArrayRandom       100            1  avgt   15     0,544 ±    0,051  ms/op
writeArrayRandom      1024            1  avgt   15     0,659 ±    0,039  ms/op
writeArrayRandom      1024         1000  avgt   15     0,558 ±    0,043  ms/op
writeArrayRandom    102400            1  avgt   15    32,211 ±    4,468  ms/op
writeArrayRandom    102400         1000  avgt   15    18,811 ±    1,551  ms/op
writeArrayRandom    102400        10240  avgt   15    14,412 ±    0,497  ms/op
writeArrayRandom    102400        65536  avgt   15     7,841 ±    0,778  ms/op
writeArrayRandom   1048576            1  avgt   15   566,594 ±  115,647  ms/op
writeArrayRandom   1048576         1000  avgt   15   205,777 ±   18,717  ms/op
writeArrayRandom   1048576        10240  avgt   15   164,506 ±   10,794  ms/op
writeArrayRandom   1048576        65536  avgt   15    72,674 ±    9,555  ms/op
writeArrayRandom   1048576       131072  avgt   15    52,077 ±    3,433  ms/op
writeArrayRandom   1048576       524288  avgt   15    32,059 ±    0,728  ms/op
writeArrayRandom  10485760            1  avgt   15  7131,848 ±  682,081  ms/op
writeArrayRandom  10485760         1000  avgt   15  2962,996 ±  529,110  ms/op
writeArrayRandom  10485760        10240  avgt   15  2152,527 ±  216,468  ms/op
writeArrayRandom  10485760        65536  avgt   15   997,330 ±  150,519  ms/op
writeArrayRandom  10485760       131072  avgt   15   676,197 ±   46,427  ms/op
writeArrayRandom  10485760       524288  avgt   15   327,190 ±   16,124  ms/op
writeByte              100            1  avgt   15     0,563 ±    0,065  ms/op
writeByte             1024            1  avgt   15     0,606 ±    0,038  ms/op
writeByte           102400            1  avgt   15    30,698 ±    7,283  ms/op
writeByte          1048576            1  avgt   15   374,051 ±   94,008  ms/op
writeByte         10485760            1  avgt   15  3885,431 ±  462,703  ms/op

Extra benchmarks for 10M and 100M objects:

Benchmark            (size)  (writeSize)  Mode  Cnt     Score    Error  Units
writeArray         10485760       262144  avgt   15   279,261 ±  4,612  ms/op
writeArray         10485760       524288  avgt   15   260,036 ±  1,847  ms/op
writeArray         10485760      1048576  avgt   15   253,352 ±  2,214  ms/op
writeArray         10485760      2097152  avgt   15   245,531 ±  0,942  ms/op
writeArray        104857600       262144  avgt   15  2786,183 ± 18,532  ms/op
writeArray        104857600       524288  avgt   15  2614,849 ± 18,063  ms/op
writeArray        104857600      1048576  avgt   15  2514,936 ± 10,540  ms/op
writeArray        104857600      2097152  avgt   15  2479,028 ±  9,808  ms/op
writeArrayRandom   10485760       262144  avgt   15   371,154 ±  4,084  ms/op
writeArrayRandom   10485760       524288  avgt   15   323,224 ±  3,892  ms/op
writeArrayRandom   10485760      1048576  avgt   15   288,717 ±  4,192  ms/op
writeArrayRandom   10485760      2097152  avgt   15   269,490 ±  3,292  ms/op
writeArrayRandom  104857600       262144  avgt   15  3736,244 ± 51,320  ms/op
writeArrayRandom  104857600       524288  avgt   15  3204,924 ± 39,514  ms/op
writeArrayRandom  104857600      1048576  avgt   15  2852,680 ± 21,795  ms/op
writeArrayRandom  104857600      2097152  avgt   15  2659,954 ± 24,693  ms/op

vlsi · 2023-12-02T19:58:29Z

The results for 10M blobs read as if we can go 10x faster if we use 512K writes instead of 4K writes

bokken · 2023-12-02T21:13:51Z

4kb seems too small, but 512kb seems too large. 64kb might be nice balance of throughput and contiguous memory allocations.

vlsi · 2023-12-02T21:23:16Z

I wonder if it is good to select buffer size based on the known blob size and/or extend it as it goes

pgjdbc/src/main/java/org/postgresql/largeobject/BlobInputStream.java

bokken · 2023-12-03T21:43:53Z

pgjdbc/src/main/java/org/postgresql/largeobject/BlobInputStream.java

+            // Refill the buffer and copy from it
+            buffer = lo.read(bufferSize);
+            // Note that actual number of bytes read may be less than requested
+            bytesRead = Math.min(len, buffer.length);


Shouldn’t this be comparing buffersize?

No - it is just the bytesRead naming which has me a bit confused

Any suggestions for the name?

vlsi · 2023-12-04T08:28:48Z

I've added PgPreparedStatement.setBlob benchmark. It creates a blob from ByteArrayInputStream.

Before

Java 17, ARM64, ssl=on, PgPreparedStatement.createBlob uses 4096 buffer

Benchmark                                     (size)  Mode  Cnt        Score       Error   Units
BlobWrite.fromByteArray                         1024  avgt    3        3,102 ±     6,648   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm      1024  avgt    3    32081,486 ±   154,631    B/op
BlobWrite.fromByteArray                        10240  avgt    3        4,221 ±     2,175   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm     10240  avgt    3    37822,788 ±   536,854    B/op
BlobWrite.fromByteArray                       102400  avgt    3       22,230 ±    31,183   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm    102400  avgt    3    99061,033 ±   954,955    B/op
BlobWrite.fromByteArray                      1048576  avgt    3      282,162 ±   598,196   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm   1048576  avgt    3   748192,000 ±  7225,620    B/op
BlobWrite.fromByteArray                     10485760  avgt    3     2728,593 ±  1512,529   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm  10485760  avgt    3  7218333,333 ± 10105,727    B/op

After with min(length, 8192) in `createBlob`

This is the selected implementation. It seems to be minimal changes to the existing createBlob, and it yields reasonable performance. createBlob could be optimized later, however, I would like to release the changes rather than polish createBlob with diminishing gains.

-byte[] buf = new byte[4096];
+byte[] buf = new byte[(int) Math.min(length, 8192)];

Java 17, ARM64, ssl=on

Benchmark                                     (size)  Mode  Cnt        Score      Error   Units
BlobWrite.fromByteArray                         1024  avgt    3        2,569 ±    0,798   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm      1024  avgt    3    27005,674 ±   39,830    B/op
BlobWrite.fromByteArray                        10240  avgt    3        3,135 ±    1,245   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm     10240  avgt    3    51002,100 ±  451,376    B/op
BlobWrite.fromByteArray                       102400  avgt    3        5,875 ±    3,729   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm    102400  avgt    3   287300,221 ±  433,371    B/op
BlobWrite.fromByteArray                      1048576  avgt    3       28,251 ±    0,808   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm   1048576  avgt    3  1146210,263 ± 7165,006    B/op
BlobWrite.fromByteArray                     10485760  avgt    3      263,814 ±   66,389   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm  10485760  avgt    3  1906706,667 ±   84,264    B/op

After

Java 17, ARM64, ssl=on, PgPreparedStatement.createBlob uses 4096 buffer

Benchmark                                     (size)  Mode  Cnt        Score      Error   Units
BlobWrite.fromByteArray                         1024  avgt    3        2,471 ±    1,833   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm      1024  avgt    3    30080,913 ±  103,388    B/op
BlobWrite.fromByteArray                      1048576  avgt    3       28,254 ±    5,625   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm   1048576  avgt    3  1147907,717 ± 7180,327    B/op
BlobWrite.fromByteArray                     10485760  avgt    3      258,289 ±   29,823   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm  10485760  avgt    3  1886762,667 ± 3117,773    B/op

The improvement comes from automatic buffer sizing in BlobOutputStream.

After with `createBlob` fix

-byte[] buf = new byte[4096];
+byte[] buf = new byte[(int) Math.min(length, 512*1024)];

Benchmark                                     (size)  Mode  Cnt        Score      Error   Units
BlobWrite.fromByteArray                         1024  avgt    3        2,518 ±    1,898   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm      1024  avgt    3    27008,910 ±  123,280    B/op
BlobWrite.fromByteArray                      1048576  avgt    3       28,197 ±    4,224   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm   1048576  avgt    3   629700,283 ± 7205,863    B/op
BlobWrite.fromByteArray                     10485760  avgt    3      260,591 ±   17,094   ms/op
BlobWrite.fromByteArray:gc.alloc.rate.norm  10485760  avgt    3  1387320,000 ±    0,001    B/op

…ects Previously TestUtil.java was declared in src/test/..., and there was no way to reuse it in the benchmarks.

…reateBlob

…uffering reads This reverts commit 3af3b32. Co-authored-by: Dave Cramer <davecramer@gmail.com> Co-authored-by: Brett Okken <brett.okken.os@gmail.com> There are several fixes on top of 3af3b32: 1) The buffer is created lazily, and it is created only in case the number of requested bytes is less than the buffer size 2) "small" read(byte[], int, int) can be buffered as well (e.g. if the buffer size exceeds the requested len) 3) Added locks to read calls for consistency with the locking in mark/reset 4) read(byte[], int, int) should return 0 when len=0 5) refilling the bufer should read "bufferSize" amount rather than "the remaining length"

We start with 65K buffer, and increase it as the user requests more reads. It reaches reasonable performance for "full stream readouts", and it avoids loading large amount of data when the user needs first bytes only.

… be written at once without copying

…arget) Previously, the implementation always created a Channel with Channels.newChannel(outputStream), however, OpenJDK (e.g. 22) still allocates arrays. The improvement is to access the arrays in case ByteBuffer allows so. DirectByteBuffer and read-only ByteBuffers forbid array access.

…rites, and allocating the buffer lazily Previously, each each blobOutput.write(byte[], ..) resulted in database call even in the case the write size is small. It could impact cases when users write in small chunks. At the same time, if they already write with big chunks, we should not create a buffer in BlobOutputStream, so now we create the buffer lazily.

…igns writes

It enables to cover variety of incoming write requests, and it does not incur much overhead as the buffer grows lazily.

vlsi · 2023-12-04T11:43:22Z

I'm okay with this, and I will let the dust settle.

We suggest using Blob APIs as they are standard.

vlsi · 2023-12-04T12:22:16Z

I noticed the documentation suggested 2048 buffers, so I reworded it in e0277f1 so it suggest using Blob APIs instead.

This MR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [flow-bin](https://github.com/flowtype/flow-bin) ([changelog](https://github.com/facebook/flow/blob/master/Changelog.md)) | devDependencies | minor | [`^0.223.0` -> `^0.224.0`](https://renovatebot.com/diffs/npm/flow-bin/0.223.0/0.224.0) | | [org.postgresql:postgresql](https://jdbc.postgresql.org) ([source](https://github.com/pgjdbc/pgjdbc)) | build | patch | `42.7.0` -> `42.7.1` | | [net.java.dev.jna:jna](https://github.com/java-native-access/jna) | compile | minor | `5.13.0` -> `5.14.0` | | [org.hibernate.orm:hibernate-envers](https://hibernate.org/orm) ([source](https://github.com/hibernate/hibernate-orm)) | build | patch | `6.4.0.Final` -> `6.4.1.Final` | | [org.hibernate.orm:hibernate-core](https://hibernate.org/orm) ([source](https://github.com/hibernate/hibernate-orm)) | build | patch | `6.4.0.Final` -> `6.4.1.Final` | | [com.diffplug.spotless:spotless-maven-plugin](https://github.com/diffplug/spotless) | build | patch | `2.41.0` -> `2.41.1` | | [io.quarkus:quarkus-maven-plugin](https://github.com/quarkusio/quarkus) | build | patch | `3.6.0` -> `3.6.3` | | [io.quarkus:quarkus-universe-bom](https://github.com/quarkusio/quarkus-platform) | import | patch | `3.6.0` -> `3.6.3` | --- ### Release Notes <details> <summary>flowtype/flow-bin</summary> ### [`v0.224.0`](flow/flow-bin@e730887...e6104a1) [Compare Source](flow/flow-bin@e730887...e6104a1) ### [`v0.223.3`](flow/flow-bin@6e1e3f7...e730887) [Compare Source](flow/flow-bin@6e1e3f7...e730887) ### [`v0.223.2`](flow/flow-bin@5bb7bcf...6e1e3f7) [Compare Source](flow/flow-bin@5bb7bcf...6e1e3f7) </details> <details> <summary>pgjdbc/pgjdbc</summary> ### [`v42.7.1`](https://github.com/pgjdbc/pgjdbc/blob/HEAD/CHANGELOG.md#4271-2023-12-06-083400--0500) ##### Changed - perf: improve performance of PreparedStatement.setBlob, BlobInputStream, and BlobOutputStream with dynamic buffer sizing [MR #3044](pgjdbc/pgjdbc#3044) ##### Fixed - fix: Apply connectTimeout before SSLSocket.startHandshake to avoid infinite wait in case the connection is broken [MR #3040](pgjdbc/pgjdbc#3040) - fix: support waffle-jna 2.x and 3.x by using reflective approach for ManagedSecBufferDesc [MR #2720](pgjdbc/pgjdbc#2720) Fixes [Issue #2690](pgjdbc/pgjdbc#2720). - fix: NoSuchMethodError on ByteBuffer#position When Running on Java 8 when accessing arrays, fixes [Issue #3014](pgjdbc/pgjdbc#3014) - Revert "[MR #2925](pgjdbc/pgjdbc#2925) Use canonical DateStyle name" [MR #3035](pgjdbc/pgjdbc#3035) Fixes [Issue #3008](pgjdbc/pgjdbc#3008) - Revert "[MR ##2973](pgjdbc/pgjdbc#2973) feat: support SET statements combining with other queries with semicolon in PreparedStatement" [MR #3010](pgjdbc/pgjdbc#3010) Fixes [Issue #3007](pgjdbc/pgjdbc#3007) - fix: avoid timezone conversions when sending LocalDateTime to the database [#2852](pgjdbc/pgjdbc#3010) Fixes [Issue #1390](pgjdbc/pgjdbc#1390) ,[Issue #2850](pgjdbc/pgjdbc#2850) Closes \[Issue [#1391](pgjdbc/pgjdbc#1391) </details> <details> <summary>java-native-access/jna</summary> ### [`v5.14.0`](https://github.com/java-native-access/jna/blob/HEAD/CHANGES.md#Release-5140) [Compare Source](java-native-access/jna@5.13.0...5.14.0) \============== ## Features - [#1556](java-native-access/jna#1556): Add `SetJob`, `SetPrinter` to `c.s.j.p.w.Winspool` - [@tresf](https://github.com/tresf). - [#1534](java-native-access/jna#1534): Add `GetMethod`, `Put`, `SpawnInstance` to `c.s.j.p.win32.COM.WbemCli#IWbemClassObject` and `ExecMethod` to `c.s.j.p.win32.COM.WbemCli#IWbemServices` - [@faddom](https://github.com/faddom). - [#1544](java-native-access/jna#1544): Add `GetPriorityClass`, `SetPriorityClass`, `GetThreadPriority`, `SetThreadPriority` and associated constants to `c.s.j.p.win32.Kernel32` - [@dEajL3kA](https://github.com/dEajL3kA). - [#1548](java-native-access/jna#1548): Make interface `c.s.j.p.mac.XAttr public` - [@matthiasblaesing](https://github.com/matthiasblaesing). - [#1551](java-native-access/jna#1551): Add `c.s.j.p.bsd.ExtAttr` and `c.s.j.p.bsd.ExtAttrUtil` to wrap BSD [\<sys/extattr.h>](https://man.freebsd.org/cgi/man.cgi?query=extattr\&sektion=2) system calls. [@rednoah](https://github.com/rednoah). - [#1517](java-native-access/jna#1517): Add missing `O_*` (e.g. `O_APPEND`, `O_SYNC`, `O_DIRECT`, ...) to `c.s.j.p.linux.Fcntl` - [@matthiasblaesing](https://github.com/matthiasblaesing). - [#1521](java-native-access/jna#1521): Shutdown CleanerThread once the last cleanable is removed - [@matthiasblaesing](https://github.com/matthiasblaesing). - [#1557](java-native-access/jna#1557): Build linux-riscv64 on Ubuntu focal to improve compatibility with older glibc versions - [@matthiasblaesing](https://github.com/matthiasblaesing). ## Bug Fixes - [#1501](java-native-access/jna#1501): `Library.OPTION_STRING_ENCODING` is ignore for string arguments function calls - [@matthiasblaesing](https://github.com/matthiasblaesing). - [#1504](java-native-access/jna#1504): Increase maximum supported fixed args on varargs calls from 3 to 255 - [@andrew-nowak](https://github.com/andrew-nowak). - [#1545](java-native-access/jna#1545): Fix Java 6 incompatibility in `c.s.j.p.win32.Kerne32Util` and `c.s.j.p.win32.DBT` - [@matthiasblaesing](https://github.com/matthiasblaesing). ## Important Changes - The interfaces between Java and native code have changed, so `libjnidispatch` must be rebuilt to be compatible with this release. - Release drops support for JDKs 6 + 7, so you'll need at least JDK 8 to update to use this version. </details> <details> <summary>hibernate/hibernate-orm</summary> ### [`v6.4.1.Final`](https://github.com/hibernate/hibernate-orm/blob/HEAD/changelog.txt#Changes-in-641Final-December-15-2023) [Compare Source](hibernate/hibernate-orm@6.4.0...6.4.1) https://hibernate.atlassian.net/projects/HHH/versions/32216 \*\* Bug \* \[HHH-17570] - Wrong name provided when checking Oracle autonomous JSON database \* \[HHH-17561] - NullPointerException at OracleServerConfiguration class \* \[HHH-17555] - equals hashcode combined with naturalId \* \[HHH-17553] - java.util.ConcurrentModificationException: null when loading an entity \* \[HHH-17551] - Gradle Enterprise improvements \* \[HHH-17528] - Explicit selection of an [@Embeddable](https://github.com/Embeddable) property containing associated collections doesn't work \* \[HHH-17525] - StackOverflowError when using [@ManyToMany](https://github.com/ManyToMany) with composite id \* \[HHH-17520] - Schema creation fails with interval second data type on PostgreSQL \* \[HHH-17519] - Initializing a lazy association with a non aggregate id causes a NPE \* \[HHH-17515] - DynamicInstantiationResult wrong java type constructor selected \* \[HHH-17499] - SemanticException when querying embedded id entity with positional parameter \* \[HHH-17494] - Metamodel generator produces an uncompilable class when an EntityGraph name contains a comma \* \[HHH-17491] - UnknownEntityTypeException thrown when multiple subclasses define an attribute with the same name and one is a MappedSuperclass \* \[HHH-17489] - ConcurrentModificationException after upgrading to 6.4.0 \* \[HHH-17466] - Exception on query: Could not convert 'java.util.Locale' to 'java.util.Locale' using 'org.hibernate.type.descriptor.java.LocaleJavaType' to wrap \* \[HHH-17464] - 6.4.0 regression: NullPointerException in DefaultEvictEventListener#onEvict \* \[HHH-17456] - java.util.UnknownFormatConversionException thrown why logging set to TRACE \* \[HHH-17445] - Subquery correlated path expressions do not work with nullness predicates \* \[HHH-17418] - Bytecode enchanced read accessor for a final field uses "missing" writer method \* \[HHH-17416] - Using tuples in WHERE clause throws SemanticException \* \[HHH-17413] - Invalid SQL with TYPE() function and CASE expression in SELECT clause \* \[HHH-17408] - Unproxying leads to uninitialized one-to-one associations in some cases involving polymorphism \* \[HHH-17407] - AssertionError when using JpaEntityJoin with correlated subqery if criteria copy tree is enabled \* \[HHH-17405] - Cannot resolve path of generic mapped-superclass association path \* \[HHH-17402] - UnknownEntityTypeException: Unable to locate persister - when comparing generic interface ToOne attribute to parameter \* \[HHH-17397] - Count with right join returns count on the left entity although requested by the right one \* \[HHH-17395] - Refresh with PESSIMISTIC_WRITE ignored for lazy loaded entity \* \[HHH-17393] - Incorrect return type for List attribute, wrap in additional list \* \[HHH-17392] - UnsupportedOperationException when using row_number() window function without top level GROUP BY \* \[HHH-17391] - AssertionError when using row_number() window function with ORDER BY \* \[HHH-17387] - Can't use EAGER ManyToOne on parent abstract class of joined table entity \* \[HHH-17379] - Right join does not return records from the right table if there is condition on ID field and there's no matching record in the from clause \* \[HHH-17359] - Query Cache contains null values for entity associations when entity was already loaded \* \[HHH-17334] - Duplicated columns when updating indexed [@ElementCollection](https://github.com/ElementCollection) with non updatable fields \* \[HHH-17293] - Schema validation fails on postgres interval seconds type \* \[HHH-17280] - UnknownTableReferenceException in subquery selecting embeddable foreign key \* \[HHH-16967] - Missing join on inheritance parent table to validate [@Where](https://github.com/Where) condition \* \[HHH-16756] - orphanRemoval does not work when bytecode enhancement is enabled \* \[HHH-16306] - CountExpressionTest.testCountDistinctTuple fails on Sybase \* \[HHH-16100] - Dynamic-map entity mode does not register a column for one-to-one associations \* \[HHH-1645] - refresh with LockMode on an unitialized proxy does not work \*\* Improvement \* \[HHH-17554] - No join on the table of the parent class with JOINED inheritance strategy when the select not contains a column used in [@Where](https://github.com/Where) or [@SQLRestriction](https://github.com/SQLRestriction) \* \[HHH-16809] - Add JavaType#createArray \*\* Task \* \[HHH-17565] - Add testing for JDK 22-ea \* \[HHH-17546] - Update Cockroach CI testing to 23.1.12 \* \[HHH-17545] - Update SAP HANA JDBC driver to 2.19.11 \* \[HHH-17544] - Update SAP HANA CI testing to 2.00.072.00.20231123.1 \* \[HHH-17543] - Update DB2 CI testing to 11.5.9.0 \* \[HHH-17542] - Update DB2 JDBC driver to 11.5.9.0 \* \[HHH-17541] - Update SQL Server JDBC driver to 12.4.2 </details> <details> <summary>quarkusio/quarkus</summary> ### [`v3.6.3`](https://github.com/quarkusio/quarkus/releases/tag/3.6.3) [Compare Source](quarkusio/quarkus@3.6.2...3.6.3) ##### Complete changelog - [#37690](quarkusio/quarkus#37690) - Regression in 3.6.2: ConfigDiagnostic `NullPointerException: Cannot invoke "java.lang.CharSequence.toString()" because "s" is null` - [#37688](quarkusio/quarkus#37688) - Do not use build cache when releasing - [#37669](quarkusio/quarkus#37669) - OIDC: Fix DEV UI startup when OIDC or Keycloak Dev Services is disabled - [#37667](quarkusio/quarkus#37667) - Disabling of Dev Services for Keycloak leads to startup error - [#37651](quarkusio/quarkus#37651) - Fix GraalVM version checker in order to accept other implementations - [#37563](quarkusio/quarkus#37563) - Consistent file URI for folders and jars - [#37559](quarkusio/quarkus#37559) - Quarkus throws NPE when trying to match unknown config files ### [`v3.6.2`](https://github.com/quarkusio/quarkus/releases/tag/3.6.2) [Compare Source](quarkusio/quarkus@3.6.1...3.6.2) ##### Complete changelog - [#37686](quarkusio/quarkus#37686) - Use standard URL when updating the website - [#37653](quarkusio/quarkus#37653) - Improve error message in REST Client when no matching readers found - [#37640](quarkusio/quarkus#37640) - Handle expireAfterWrite properly in the redis-cache config - [#37639](quarkusio/quarkus#37639) - Logging docs include build time config reference - [#37638](quarkusio/quarkus#37638) - Add scope tag to all injected MP Metrics - [#37623](quarkusio/quarkus#37623) - Add quarkus-funqy-amazon-lambda-deployment to the quarkus-bom - [#37617](quarkusio/quarkus#37617) - SmallRye GraphQL 2.6.1 - [#37611](quarkusio/quarkus#37611) - Fix configuration of custom port for Elasticsearch dev services - [#37610](quarkusio/quarkus#37610) - Elasticsearch devservice port incorrect binding - [#37605](quarkusio/quarkus#37605) - Fix type in HeartbeatFilter example of rest-client-reactive doc - [#37599](quarkusio/quarkus#37599) - Bump kafka3.version from 3.6.0 to 3.6.1 - [#37594](quarkusio/quarkus#37594) - Bump org.jetbrains.kotlinx:kotlinx-serialization-json from 1.6.1 to 1.6.2 - [#37590](quarkusio/quarkus#37590) - Microprofile Metric Missing "Scope" Tag Only When Using Constructor Injection - [#37588](quarkusio/quarkus#37588) - Update the import.cypher script to use newer syntax - [#37581](quarkusio/quarkus#37581) - Support using commas to add extensions with CLI - [#37570](quarkusio/quarkus#37570) - Prevent concurrently running Jacoco ReportCreators to avoid report corruption - [#37565](quarkusio/quarkus#37565) - ArC: prevent duplicate bean discovery & detect duplicate bean identifiers - [#37564](quarkusio/quarkus#37564) - Invalid documentation for 'quarkus extension add' - [#37557](quarkusio/quarkus#37557) - Make docs/sync-web-site.sh recoverable - [#37553](quarkusio/quarkus#37553) - Register OpenAPI custom filters for reflection - [#37549](quarkusio/quarkus#37549) - Bump commons-io:commons-io from 2.15.0 to 2.15.1 - [#37546](quarkusio/quarkus#37546) - Bump io.quarkus:quarkus-platform-bom-maven-plugin from 0.0.99 to 0.0.100 - [#37545](quarkusio/quarkus#37545) - Bump org.junit.jupiter:junit-jupiter from 5.10.0 to 5.10.1 - [#37536](quarkusio/quarkus#37536) - Fix != expression in `@PreAuthorize` check - [#37535](quarkusio/quarkus#37535) - Fix Snappy note about native - [#37526](quarkusio/quarkus#37526) - Spring security annotatiton PreAuthorize process equals and not equals in the same way - [#37520](quarkusio/quarkus#37520) - Properly match unknown config files for Windows - [#37513](quarkusio/quarkus#37513) - Save pathParamValues encoded and perform decoding when requested - [#37505](quarkusio/quarkus#37505) - Bump org.jboss.logmanager:log4j2-jboss-logmanager from 1.1.1.Final to 1.1.2.Final - [#37496](quarkusio/quarkus#37496) - Quarkus 3.6.0 generate wrong jacoco xml - [#37493](quarkusio/quarkus#37493) - upgrading from quarkus 3.5.3 to 3.6.0 java.lang.IllegalArgumentException: A case block for the string value already exist - [#37479](quarkusio/quarkus#37479) - Redis Cache ttl default value invalid setting - [#37357](quarkusio/quarkus#37357) - Runtime OpenAPI filters are not registered for reflection - [#37298](quarkusio/quarkus#37298) - Bump Keycloak version to 23.0.0 - [#37187](quarkusio/quarkus#37187) - Bump org.apache.commons:commons-compress from 1.24.0 to 1.25.0 - [#37129](quarkusio/quarkus#37129) - Bump org.jetbrains.kotlinx:kotlinx-serialization-json from 1.6.0 to 1.6.1 - [#37128](quarkusio/quarkus#37128) - Bump org.junit:junit-bom from 5.10.0 to 5.10.1 - [#37035](quarkusio/quarkus#37035) - Register methods of RESTeasy reactive parameter containers for reflection - [#36986](quarkusio/quarkus#36986) - Native not index method with SSE and throw NoSuchMethodException - [#36813](quarkusio/quarkus#36813) - config-yaml warning - unrecognized configuration file on Windows - [#36773](quarkusio/quarkus#36773) - "Unrecognized configuration file" when using YAML - [#35960](quarkusio/quarkus#35960) - PathParam URL encoded in quarkus-resteasy-reactive since quarkus 3.2.x ### [`v3.6.1`](https://github.com/quarkusio/quarkus/releases/tag/3.6.1) [Compare Source](quarkusio/quarkus@3.6.0...3.6.1) ##### Complete changelog - [#37486](quarkusio/quarkus#37486) - Revert "Unlist quarkus-resteasy-qute and quarkus-resteasy-reactive-qute" - [#37478](quarkusio/quarkus#37478) - Qute resteasy change in 3.6.0 - [#37454](quarkusio/quarkus#37454) - Config: detect injected config value mismatch for missing values - [#37453](quarkusio/quarkus#37453) - Fix Panache bytecode enhancement for `@Embeddable` records - [#37449](quarkusio/quarkus#37449) - Update Apache Maven to 3.9.6 - [#37447](quarkusio/quarkus#37447) - Add Content-Range header to 206 Partial Content file response - [#37444](quarkusio/quarkus#37444) - Injecting ConfigProperty of type Optional\<List<String>> not working in the native mode - [#37436](quarkusio/quarkus#37436) - Respect comma escapes in property value for `@RolesAllowed.` - [#37428](quarkusio/quarkus#37428) - Fix various minor issues in quarkus update - [#37426](quarkusio/quarkus#37426) - Fix typo configMapRefKey -> configMapKeyRef - [#37421](quarkusio/quarkus#37421) - Scheduler: register ApplicationNotRunning as bean even if quartz is used - [#37417](quarkusio/quarkus#37417) - Build fails when using the Quartz extension and `@Scheduled`(..., skipExecutionIf = Scheduled.ApplicationNotRunning.class) - [#37415](quarkusio/quarkus#37415) - Make Truffle from GraalVM 23.1 work in all Quarkus modes - [#37401](quarkusio/quarkus#37401) - Print correct profiles in native image - [#37400](quarkusio/quarkus#37400) - Improve reliability when downloading builder images from Quay.io - [#37398](quarkusio/quarkus#37398) - Revert "Build cache - Upload quarkus-ide-launcher-999-SNAPSHOT.jar" - [#37395](quarkusio/quarkus#37395) - Update Gradle to 8.5 - [#37393](quarkusio/quarkus#37393) - Disable new Http2RSTFloodProtectionConfigTest on Windows - [#37392](quarkusio/quarkus#37392) - Update gradle to 8.5 and backport if possible to finish Java 21 dependencies - [#37384](quarkusio/quarkus#37384) - Stop disabling unsafe in netty at native-executable runtime - [#37382](quarkusio/quarkus#37382) - Update MAX_LTS_SUPPORTED_BY_KOTLIN to 21 - [#37367](quarkusio/quarkus#37367) - Documentation: security auth mechanism guide must also list runtime form auth properties alongside build-time ones - [#37366](quarkusio/quarkus#37366) - Documentation: Fix OIDC credentials reference to secret key - [#37364](quarkusio/quarkus#37364) - Improve Docker Desktop detection - [#37361](quarkusio/quarkus#37361) - Do not fail the request in OidcClient filters if OidcClient is disabled - [#37360](quarkusio/quarkus#37360) - Disable OidcClientFilter at runtime for simpler application testing - [#37359](quarkusio/quarkus#37359) - Fix incorrect log dependency - [#37354](quarkusio/quarkus#37354) - Update Boucycastle to 1.77 and Boucycastle FIPS to 1.0.2.4 - [#37351](quarkusio/quarkus#37351) - Upgrade to Jandex 3.1.6 - [#37350](quarkusio/quarkus#37350) - Unlist quarkus-resteasy-reactive-qute and quarkus-resteasy-qute - [#37347](quarkusio/quarkus#37347) - Runtime (re)initialize Netty's PlatformDependent classes - [#37318](quarkusio/quarkus#37318) - Use batch mode for update-version.sh - [#37317](quarkusio/quarkus#37317) - Avoid asking for GPG passphrase on CI - [#37314](quarkusio/quarkus#37314) - Fix deprecated dev-v1 url with dev-ui in a few places - [#37300](quarkusio/quarkus#37300) - Prepare docs/sync-web-site.sh for automated releases - [#37294](quarkusio/quarkus#37294) - Fix the major version of Java 21 - [#37292](quarkusio/quarkus#37292) - Bump testcontainers to 1.19.3 and use docker-java-bom - [#37291](quarkusio/quarkus#37291) - Bump org.jetbrains.kotlin:kotlin-gradle-plugin-api from 1.9.20 to 1.9.21 in /devtools/gradle - [#37280](quarkusio/quarkus#37280) - Bump Microsoft SQL Server JDBC driver to 12.4.2 - [#37279](quarkusio/quarkus#37279) - Bump Microsoft SQL Server JDBC driver to 12.4.2 - [#37271](quarkusio/quarkus#37271) - Docs: add keywords to Vert.x guides - [#37270](quarkusio/quarkus#37270) - Vert.x: report exception for blocking message consumer methods - [#37268](quarkusio/quarkus#37268) - Reactive REST Client: check for ClientRequestFilter when skipping `@Provider` auto-discovery - [#37252](quarkusio/quarkus#37252) - Use the default tenant resolver if the custom one does not resolve a tenant - [#37251](quarkusio/quarkus#37251) - Add a test showing how OIDC ID token can be propagated - [#37247](quarkusio/quarkus#37247) - Update kindcontainer to 1.4.4 - [#37244](quarkusio/quarkus#37244) - Always execute a JPA password action - [#37228](quarkusio/quarkus#37228) - Add classes from additional JPA model build items to pre-generate proxies - [#37224](quarkusio/quarkus#37224) - Qute: dev mode - debug a problem with no-restart-template - [#37222](quarkusio/quarkus#37222) - RuntimeException in vertx is silently swallowed for blocking ConsumeEvents - [#37177](quarkusio/quarkus#37177) - QUARKUS_MROFILE=dev and '-Dquarkus.profile=dev' not working for native image - [#37108](quarkusio/quarkus#37108) - Warning about build time generated proxy for entity OutboxEvent in Hibernate Search - [#36892](quarkusio/quarkus#36892) - Update Kotlin to version 1.9.21, Mockito to 5.7.0 - [#36747](quarkusio/quarkus#36747) - NoSuchMethodException when reading `@Embeddable` record - [#36735](quarkusio/quarkus#36735) - Failure to bump BouncyCastle FIPS version to 1.0.2.4 due to a native build failure - [#36242](quarkusio/quarkus#36242) - Quarkus cannot load Truffle/polyglot languages with the custom class loader. - [#17839](quarkusio/quarkus#17839) - Invalid memory configuration for netty maxDirectMemory in native image </details> <details> <summary>quarkusio/quarkus-platform</summary> ### [`v3.6.3`](quarkusio/quarkus-platform@3.6.2...3.6.3) [Compare Source](quarkusio/quarkus-platform@3.6.2...3.6.3) ### [`v3.6.2`](quarkusio/quarkus-platform@3.6.1...3.6.2) [Compare Source](quarkusio/quarkus-platform@3.6.1...3.6.2) ### [`v3.6.1`](quarkusio/quarkus-platform@3.6.0...3.6.1) [Compare Source](quarkusio/quarkus-platform@3.6.0...3.6.1) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever MR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This MR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

vlsi added the performance label Nov 29, 2023

vlsi added this to the 42.7.1 milestone Nov 29, 2023