R2dbcNonTransientResourceException: Connection validation failed #209
Comments
|
Is there more to the stack trace? Typically, the cause shows the reason for the validation failure. |
|
Hi Mark,
Thanks for the response.
Here are more logs.
at io.r2dbc.pool.Validation.lambda$validate$2(Validation.java:45)
~[r2dbc-pool-1.0.0.RELEASE.jar!/:1.0.0.RELEASE]",""
*"4/9/2024, 3:57:10.276 PM","Caused by:
io.r2dbc.spi.R2dbcNonTransientResourceException: Connection validation
failed",""*
....
....
at
org.springframework.r2dbc.connection.ConnectionFactoryUtils.lambda$getConnection$0(ConnectionFactoryUtils.java:100)
~[spring-r2dbc-6.1.1.jar!/:6.1.1]",""
*"Caused by: org.springframework.dao.DataAccessResourceFailureException:
Failed to obtain R2DBC Connection",""*
*reactor.core.Exceptions$ErrorCallbackNotImplemented:
org.springframework.dao.DataAccessResourceFailureException: Failed to
obtain R2DBC Connection",""*
Thanks
…On Tue, Apr 9, 2024 at 1:08 AM Mark Paluch ***@***.***> wrote:
Is there more to the stack trace? Typically, the cause shows the reason
for the validation failure.
—
Reply to this email directly, view it on GitHub
<#209 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJIZE6JWBHGUXLQVXSKATFTY4OOWRAVCNFSM6AAAAABFZULG2WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBUGM4TKMRUG4>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
Maybe I will add more information, because I'm fighting with the same problem for the last couple of days. Versions
Stacktrace Pool configuration
Additional context This issue happens in my case 2-3 times a day in different day time. It is totally random when will it happen. What is more interesting, that after some time (10-20sec) everything goes back to normal work. So it "heals" itself, but during this 10-20sec non of my queries are executed. During my investigation I found that validation is performed 2 times.
In my case it happens only on 1 case, that's why query is not executed, because connection is not acquired from connection pool. Stacktrace is also not very helpful, because it points only to 45 line in Validation.class Steps to reproduce Restart database server between queries.
Related issues Possible Solution In that case the following But I'm afraid that this only hides/delays the problem and makes it even more unpredictable. |
|
Here is my configuration.
poolInitialSize=10
poolMaxSize=30
maxIdleTime=60 mins (1hr)
…-Thanks
On Wed, Apr 10, 2024 at 12:26 AM MBec ***@***.***> wrote:
Maybe I will add more information, because I'm fighting with the same
problem for the last couple of days.
*Versions*
- Driver: r2dbc-postgres
- Database: postgres
- Java: 21
- io.r2dbc:r2dbc-spi:1.0.0.RELEASE
- io.r2dbc:r2dbc-pool:1.0.1.RELEASE
*Stacktrace*
io.r2dbc.spi.R2dbcNonTransientResourceException: Connection validation failed
at io.r2dbc.pool.Validation.lambda$validate$2(Validation.java:45)
at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onNext(FluxHandleFuseable.java:179)
at reactor.core.publisher.MonoSupplier$MonoSupplierSubscription.request(MonoSupplier.java:145)
at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.request(FluxHandleFuseable.java:260)
at reactor.core.publisher.MonoIgnoreElements$IgnoreElementsSubscriber.onSubscribe(MonoIgnoreElements.java:72)
at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onSubscribe(FluxHandleFuseable.java:164)
at reactor.core.publisher.MonoSupplier.subscribe(MonoSupplier.java:48)
*Pool configuration*
default
*Additional context*
This issue happens in my case 2-3 times a day in different day time. It is
totally random when will it happen. What is more interesting, that after
some time (10-20sec) everything goes back to normal work. So it "heals"
itself, but during this 10-20sec non of my queries are executed. During my
investigation I found that validation is performed 2 times.
1. During connectionFactory.create()
https://github.com/r2dbc/r2dbc-pool/blob/3ee034a65e888e0e3fcb330777afc3715ced6797/src/main/java/io/r2dbc/pool/ConnectionPool.java#L135
2. After query during connectionFactory.close()
https://github.com/r2dbc/r2dbc-pool/blob/3ee034a65e888e0e3fcb330777afc3715ced6797/src/main/java/io/r2dbc/pool/PooledConnection.java#L66
In my case it happens only on 1 case, that's why query is not executed,
because connection is not acquired from connection pool. Stacktrace is also
not very helpful, because it points only to 45 line in Validation.class
https://github.com/r2dbc/r2dbc-pool/blob/3ee034a65e888e0e3fcb330777afc3715ced6797/src/main/java/io/r2dbc/pool/Validation.java#L45
*Steps to reproduce*
Restart database server between queries.
1. Start application
2. Execute query
3. Restart database server: sudo pg_ctl restart / Ubuntu: sudo
pg_ctlcluster 14 main restart
4. Execute query (should FAIL with provided stacktrace)
*Related issues*
#138 <#138>
#107 <#107>
*Possible Solution*
The only way to mitigate this problem I found was to increase default
acquireRetry property from 1 to 10, which is maxSize value.
In that case the following Mono<Connection> create will retry until it
find valid connection.
https://github.com/r2dbc/r2dbc-pool/blob/3ee034a65e888e0e3fcb330777afc3715ced6797/src/main/java/io/r2dbc/pool/ConnectionPool.java#L172
But I'm afraid that this only hides/delays the problem and makes it even
more unpredictable.
@mp911de <https://github.com/mp911de> correct me if I'm wrong. With my
solution, connection pool will try 10 times to acquire connection until it
founds valid connection, so basically "searches" through entire pool to
find what it needs. I assume that if none of connection is valid then this
exception would be fired.
*Not related*
I'm not able to set pool.acquireRetry property in spring-data-r2dbc in
application.yml. I had to create manual connection pool without
R2dbcAutoConfiguration.class.
—
Reply to this email directly, view it on GitHub
<#209 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJIZE6IJA2RCFVFP2N76QRTY4TSRVAVCNFSM6AAAAABFZULG2WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBWG4YTGMBXGI>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
Hi @mp911de , Do you have any update for me. This issue is acting like a blocker, please let me know if you need more details from me. |
|
Hi Mark,
Any suggestion, I tried with maxIdle from 60 mins to 10 mins but still I
see the issue.
We are struggling with this issue in the production environment. Please
help me in resolving this issue.
-Thanks
On Wed, Apr 10, 2024 at 10:58 AM menaka prabu ***@***.***>
wrote:
… Here is my configuration.
poolInitialSize=10
poolMaxSize=30
maxIdleTime=60 mins (1hr)
-Thanks
On Wed, Apr 10, 2024 at 12:26 AM MBec ***@***.***> wrote:
> Maybe I will add more information, because I'm fighting with the same
> problem for the last couple of days.
>
> *Versions*
>
> - Driver: r2dbc-postgres
> - Database: postgres
> - Java: 21
> - io.r2dbc:r2dbc-spi:1.0.0.RELEASE
> - io.r2dbc:r2dbc-pool:1.0.1.RELEASE
>
> *Stacktrace*
>
> io.r2dbc.spi.R2dbcNonTransientResourceException: Connection validation failed
> at io.r2dbc.pool.Validation.lambda$validate$2(Validation.java:45)
> at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onNext(FluxHandleFuseable.java:179)
> at reactor.core.publisher.MonoSupplier$MonoSupplierSubscription.request(MonoSupplier.java:145)
> at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.request(FluxHandleFuseable.java:260)
> at reactor.core.publisher.MonoIgnoreElements$IgnoreElementsSubscriber.onSubscribe(MonoIgnoreElements.java:72)
> at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onSubscribe(FluxHandleFuseable.java:164)
> at reactor.core.publisher.MonoSupplier.subscribe(MonoSupplier.java:48)
>
> *Pool configuration*
>
> default
>
> *Additional context*
>
> This issue happens in my case 2-3 times a day in different day time. It
> is totally random when will it happen. What is more interesting, that after
> some time (10-20sec) everything goes back to normal work. So it "heals"
> itself, but during this 10-20sec non of my queries are executed. During my
> investigation I found that validation is performed 2 times.
>
> 1. During connectionFactory.create()
>
> https://github.com/r2dbc/r2dbc-pool/blob/3ee034a65e888e0e3fcb330777afc3715ced6797/src/main/java/io/r2dbc/pool/ConnectionPool.java#L135
> 2. After query during connectionFactory.close()
>
> https://github.com/r2dbc/r2dbc-pool/blob/3ee034a65e888e0e3fcb330777afc3715ced6797/src/main/java/io/r2dbc/pool/PooledConnection.java#L66
>
> In my case it happens only on 1 case, that's why query is not executed,
> because connection is not acquired from connection pool. Stacktrace is also
> not very helpful, because it points only to 45 line in Validation.class
>
> https://github.com/r2dbc/r2dbc-pool/blob/3ee034a65e888e0e3fcb330777afc3715ced6797/src/main/java/io/r2dbc/pool/Validation.java#L45
>
> *Steps to reproduce*
>
> Restart database server between queries.
>
> 1. Start application
> 2. Execute query
> 3. Restart database server: sudo pg_ctl restart / Ubuntu: sudo
> pg_ctlcluster 14 main restart
> 4. Execute query (should FAIL with provided stacktrace)
>
> *Related issues*
>
> #138 <#138>
> #107 <#107>
>
> *Possible Solution*
> The only way to mitigate this problem I found was to increase default
> acquireRetry property from 1 to 10, which is maxSize value.
>
> In that case the following Mono<Connection> create will retry until it
> find valid connection.
>
> https://github.com/r2dbc/r2dbc-pool/blob/3ee034a65e888e0e3fcb330777afc3715ced6797/src/main/java/io/r2dbc/pool/ConnectionPool.java#L172
>
> But I'm afraid that this only hides/delays the problem and makes it even
> more unpredictable.
> @mp911de <https://github.com/mp911de> correct me if I'm wrong. With my
> solution, connection pool will try 10 times to acquire connection until it
> founds valid connection, so basically "searches" through entire pool to
> find what it needs. I assume that if none of connection is valid then this
> exception would be fired.
>
> *Not related*
> I'm not able to set pool.acquireRetry property in spring-data-r2dbc in
> application.yml. I had to create manual connection pool without
> R2dbcAutoConfiguration.class.
>
> —
> Reply to this email directly, view it on GitHub
> <#209 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AJIZE6IJA2RCFVFP2N76QRTY4TSRVAVCNFSM6AAAAABFZULG2WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBWG4YTGMBXGI>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
io.r2dbc r2dbc-pool 1.0.0.RELEASE io.r2dbc r2dbc-mssql 1.0.0.RELEASE I am seeing this issue in both pre-prod and production environment intermittently. Could you please help me with this? Thanks.I am getting this issue "org.springframework.dao.DataAccessResourceFailureException: Failed to obtain R2DBC Connection nested exception. Caused by io.r2dbc.spi.R2dbcNonTransientResourceException: Connection validation failed" in MSSQL. I am using below libraries(Version).
The text was updated successfully, but these errors were encountered: