Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose parameter status messages (GUC_REPORT) to the user #1435

Merged
merged 2 commits into from
Jul 5, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/documentation/head/ext.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ next: geometric.html
* [Large Objects](largeobjects.html)
* [Listen / Notify](listennotify.html)
* [Server Prepared Statements](server-prepare.html)
* [Parameter Status Messages](parameterstatus.html)
* [Physical and Logical replication API](replication.html)
* [Arrays](arrays.html)

Expand Down
63 changes: 63 additions & 0 deletions docs/documentation/head/parameterstatus.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
layout: default_docs
title: Physical and Logical replication API
header: Chapter 9. PostgreSQL™ Extensions to the JDBC API
resource: media
previoustitle: Parameter Status Messages
previous: server-prepare.html
nexttitle: Physical and Logical replication API
next: replication.html
---

# Parameter Status Messages

PostgreSQL supports server parameters, also called server variables or,
internally, Grand Unified Configuration (GUC) variables. These variables are
manipulated by the `SET` command, `postgresql.conf`, `ALTER SYSTEM SET`, `ALTER
USER SET`, `ALTER DATABASE SET`, the `set_config(...)` SQL-callable function,
etc. See [the PostgreSQL manual](https://www.postgresql.org/docs/current/config-setting.html).

For a subset of these variables the server will *automatically report changes
to the value to the client driver and application*. These variables are known
internally as `GUC_REPORT` variables after the name of the flag that enables
the functionality.

The server keeps track of all the variable scopes and reports when a variable
reverts to a prior value, so the client doesn't have to guess what the current
value is and whether some server-side function could've changed it. Whenever
the value changes, no matter why or how it changes, the server reports the new
effective value in a *Parameter Status* protocol message to the client. PgJDBC
uses many of these reports internally.

As of PgJDBC 42.2.6, it also exposes the parameter status information to user
applications via the PGConnection extensions interface.

## Methods

Two methods on `org.postgresql.PGConnection` provide the client interface to
reported parameters. Parameter names are case-insensitive and case-preserving.

* `Map PGConnection.getParameterStatuses()` - return a map of all reported
parameters and their values.

* `String PGConnection.getParameterStatus()` - shorthand to retrieve one
value by name, or null if no value has been reported.

See the `PGConnection` JavaDoc for details.

## Example

If you're working directly with a `java.sql.Connection` you can

import org.postgresql.PGConnection;

void my_function(Connection conn) {

System.out.println("My application name is " +
((PGConnection)conn).getParameterStatus("application_name"));

}

## Other client drivers

The `libpq` equivalent is the `PQparameterStatus(...)` API function.
4 changes: 2 additions & 2 deletions docs/documentation/head/replication.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ layout: default_docs
title: Physical and Logical replication API
header: Chapter 9. PostgreSQL™ Extensions to the JDBC API
resource: media
previoustitle: Server Prepared Statements
previous: server-prepare.html
previoustitle: Parameter Status Messages
previous: parameterstatus.html
nexttitle: Arrays
next: arrays.html
---
Expand Down
4 changes: 2 additions & 2 deletions docs/documentation/head/server-prepare.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ header: Chapter 9. PostgreSQL™ Extensions to the JDBC API
resource: media
previoustitle: Listen / Notify
previous: listennotify.html
nexttitle: Physical and Logical replication API
next: replication.html
nexttitle: Parameter Status Messages
next: parameterstatus.html
---

### Motivation
Expand Down
74 changes: 74 additions & 0 deletions pgjdbc/src/main/java/org/postgresql/PGConnection.java
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
import java.sql.SQLException;
import java.sql.Statement;

import java.util.Map;

/**
* This interface defines the public PostgreSQL extensions to java.sql.Connection. All Connections
* returned by the PostgreSQL driver implement PGConnection.
Expand Down Expand Up @@ -236,4 +238,76 @@ public interface PGConnection {
* @return replication API for the current connection
*/
PGReplicationConnection getReplicationAPI();

/**
* <p>Returns the current values of all parameters reported by the server.</p>
*
* <p>PostgreSQL reports values for a subset of parameters (GUCs) to the client
* at connect-time, then sends update messages whenever the values change
* during a session. PgJDBC records the latest values and exposes it to client
* applications via <code>getParameterStatuses()</code>.</p>
*
* <p>PgJDBC exposes individual accessors for some of these parameters as
* listed below. They are more backwarrds-compatible and should be preferred
* where possible.</p>
*
* <p>Not all parameters are reported, only those marked
* <code>GUC_REPORT</code> in the source code. The <code>pg_settings</code>
* view does not expose information about which parameters are reportable.
* PgJDBC's map will only contain the parameters the server reports values
* for, so you cannot use this method as a substitute for running a
* <code>SHOW paramname;</code> or <code>SELECT
* current_setting('paramname');</code> query for arbitrary parameters.</p>
*
* <p>Parameter names are <i>case-insensitive</i> and <i>case-preserving</i>
* in this map, like in PostgreSQL itself. So <code>DateStyle</code> and
* <code>datestyle</code> are the same key.</p>
*
* <p>
* As of PostgreSQL 11 the reportable parameter list, and related PgJDBC
* interfaces or accesors, are:
* </p>
*
* <ul>
* <li>
* <code>application_name</code> -
* {@link java.sql.Connection#getClientInfo()},
* {@link java.sql.Connection#setClientInfo(java.util.Properties)}
* and <code>ApplicationName</code> connection property.
* </li>
* <li>
* <code>client_encoding</code> - PgJDBC always sets this to <code>UTF8</code>.
* See <code>allowEncodingChanges</code> connection property.
* </li>
* <li><code>DateStyle</code> - PgJDBC requires this to always be set to <code>ISO</code></li>
* <li><code>standard_conforming_strings</code> - indirectly via {@link #escapeLiteral(String)}</li>
* <li>
* <code>TimeZone</code> - set from JDK timezone see {@link java.util.TimeZone#getDefault()}
* and {@link java.util.TimeZone#setDefault(TimeZone)}
* </li>
* <li><code>integer_datetimes</code></li>
* <li><code>IntervalStyle</code></li>
* <li><code>server_encoding</code></li>
* <li><code>server_version</code></li>
* <li><code>is_superuser</code> </li>
* <li><code>session_authorization</code></li>
* </ul>
*
* <p>Note that some PgJDBC operations will change server parameters
* automatically.</p>
*
* @return unmodifiable map of case-insensitive parameter names to parameter values
* @since 42.2.6
*/
Map<String,String> getParameterStatuses();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't getParameterStatus(String parameterName) just enough?
Is the returned Map live?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The map returned is an unmodifiable wrapper.

If we don't expose the map then we should provide a List<String>-returning call to list known params instead. Just exposing the map is simpler.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a bit wonky is the handling of case sensitivity.
While we can certainly provide an implementation of String getParameterStatus(String param) which is case insensitive to param, getting the Map.keySet() off the return value here would return some specific case when iterating values.
I think I would lean towards 1 method to return the known parameters and a separate method to get the value for some given parameter in a case insensitive way.
This provides the needed/desired functionality and hides the implementation detail.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks sane to return "server-provided" casing, and allow for case insensitive access, doesn't it?

ResultSet.getInt(String) allows for case-insensitive access.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vlsi, I am good with the getParamterStatus(String) method being case insensitive to the parameter. I am a bit conflicted on returning a Map with all values and making any statement on it providing a case insensitive get.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bokken 👍
I'm inclining to a map with properties of:

  1. "server-native" casing
  2. "forbidden modifications"
  3. "might become out of date"

WHYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vlsi, I think my preference would still be to return a collection of parameter names (with properties you list) to get the available names. But I am fine with the map you describe.

Copy link
Member Author

@ringerc ringerc Mar 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We got rid of forbidden modifications. And I really fail to see the point of the rest, it seems excessively complex.

I take your point about keySet() returning the server-provided cases. But I don't really see the problem with it. The case won't tend to vary as the server doesn't change the case it uses, and the keys will get the correct parameters when looked up. The only possible issue I see is someReturnedKey.equals("APPLICATION_NAME") not matching ... and that's no different to how the system already behaves with a query of pg_settings.


/**
* Shorthand for getParameterStatuses().get(...) .
*
* @param parameterName case-insensitive parameter name
* @return parameter value if defined, or null if no parameter known
* @see #getParameterStatuses
* @since 42.2.6
*/
String getParameterStatus(String parameterName);
}
6 changes: 6 additions & 0 deletions pgjdbc/src/main/java/org/postgresql/core/QueryExecutor.java
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import java.sql.SQLException;
import java.sql.SQLWarning;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.TimeZone;

Expand Down Expand Up @@ -452,4 +453,9 @@ Object createQueryKey(String sql, boolean escapeProcessing, boolean isParameteri
void setNetworkTimeout(int milliseconds) throws IOException;

int getNetworkTimeout() throws IOException;

// Expose parameter status to PGConnection
Map<String,String> getParameterStatuses();

String getParameterStatus(String parameterName);
}
45 changes: 45 additions & 0 deletions pgjdbc/src/main/java/org/postgresql/core/QueryExecutorBase.java
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@
import java.sql.SQLException;
import java.sql.SQLWarning;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Map;
import java.util.Properties;
import java.util.TreeMap;
import java.util.logging.Level;
import java.util.logging.Logger;

Expand Down Expand Up @@ -52,6 +55,10 @@ public abstract class QueryExecutorBase implements QueryExecutor {
private final LruCache<Object, CachedQuery> statementCache;
private final CachedQueryCreateAction cachedQueryCreateAction;

// For getParameterStatuses(), GUC_REPORT tracking
private final TreeMap<String,String> parameterStatuses
= new TreeMap<String,String>(String.CASE_INSENSITIVE_ORDER);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it a TreeMap?
Should the field be just Map and the value just HashMap?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TreeMap is used to permit String.CASE_INSENSITIVE_ORDER, which is not supported by HashMap. That ensures that parameter lookups will find the parameter no matter what case is used - datestyle or DateStyle for example.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pg's parameters use a mix of camel case and underscore separators. Users generally expect case-folding behaviour in postgres, too. So I'd strongly prefer to do this.


protected QueryExecutorBase(PGStream pgStream, String user,
String database, int cancelSignalTimeout, Properties info) throws SQLException {
this.pgStream = pgStream;
Expand Down Expand Up @@ -389,4 +396,42 @@ public void setFlushCacheOnDeallocate(boolean flushCacheOnDeallocate) {
protected boolean hasNotifications() {
return notifications.size() > 0;
}

@Override
public final Map<String,String> getParameterStatuses() {
return Collections.unmodifiableMap(parameterStatuses);
}

@Override
public final String getParameterStatus(String parameterName) {
return parameterStatuses.get(parameterName);
}

/**
* Update the parameter status map in response to a new ParameterStatus
* wire protocol message.
*
* <p>The server sends ParameterStatus messages when GUC_REPORT settings are
* initially assigned and whenever they change.</p>
*
* <p>A future version may invoke a client-defined listener class at this point,
* so this should be the only access path.</p>
*
* <p>Keys are case-insensitive and case-preserving.</p>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does Keys are case-insensitive mean?
Does backend send the same parameter with multiple casing variations?

Copy link
Member Author

@ringerc ringerc Mar 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applications may use parameters in any combination of letter cases and expect the same result. So the PgJDBC interface should be consistent with that, otherwise users will experience unexpected results if they

stmt.execute("SET Application_Name = 'ILoveCamelCase';");
// then in a more sensible part of the application
pgconn.getParameterStatus("application_name");

Observe

SET APPLICATION_NAME='myapp';
SHOW aPpLiCaTiON_name;
SHOW application_name;

As Java provides a trivial way to make map keys case-insensitive and case-preserving it makes sense to do so.

*
* <p>The server doesn't provide a way to report deletion of a reportable
* parameter so we don't expose one here.</p>
*
* @param parameterName case-insensitive case-preserving name of parameter to create or update
* @param parameterStatus new value of parameter
* @see org.postgresql.PGConnection#getParameterStatuses
* @see org.postgresql.PGConnection#getParameterStatus
*/
protected void onParameterStatus(String parameterName, String parameterStatus) {
if (parameterName == null || parameterName.equals("")) {
throw new IllegalStateException("attempt to set GUC_REPORT parameter with null or empty-string name");
}

parameterStatuses.put(parameterName, parameterStatus);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have any potential concurrency issue here?
IIRC, query executors are used/reused for life of connection, which can get used across threads (particularly when connections pooled). Here we are potentially mutating the map on different threads from where it is being accessed/read and giving a live "view" of the map on the read method.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bokken At any point where onParameterStatus may be called, the current thread holds the monitor of the object guarding the entrypoint. We can only get parameter status messages during query result processing or when processing notices by explicit user request in processNotifies(). The latter is synchronized on QueryExecutorImpl, and all the querying entry points also take the QueryExecutorImpl monitor. They have to, otherwise we'd have a horrible mess on our hands already.

No query executor can be used for more than one connection, and no connection may run more than one statement at a time or process results from more than one statement.

You could annotate it synchronized but that'd be (a) unnecessary and (b) if it was necessary, wrong and likely to cause deadlocks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ringerc, my concern was not necessarily concurrent threads calling onParamterStatus, but rather consistency between reading from the map on a different thread from where it was mutated.

}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2621,6 +2621,11 @@ public void receiveParameterStatus() throws IOException, SQLException {
LOGGER.log(Level.FINEST, " <=BE ParameterStatus({0} = {1})", new Object[]{name, value});
}

/* Update client-visible parameter status map for getParameterStatuses() */
if (name != null && !name.equals("")) {
onParameterStatus(name, value);
}

if (name.equals("client_encoding")) {
if (allowEncodingChanges) {
if (!value.equalsIgnoreCase("UTF8") && !value.equalsIgnoreCase("UTF-8")) {
Expand Down
11 changes: 11 additions & 0 deletions pgjdbc/src/main/java/org/postgresql/jdbc/PgConnection.java
Original file line number Diff line number Diff line change
Expand Up @@ -1726,4 +1726,15 @@ public PreparedStatement prepareStatement(String sql, String[] columnNames) thro
}
return ps;
}

@Override
public final Map<String,String> getParameterStatuses() {
return queryExecutor.getParameterStatuses();
}

@Override
public final String getParameterStatus(String parameterName) {
return queryExecutor.getParameterStatus(parameterName);
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,8 @@
CopyLargeFileTest.class,
ServerErrorTest.class,
UpsertTest.class,
OuterJoinSyntaxTest.class
OuterJoinSyntaxTest.class,
ParameterStatusTest.class
})
public class Jdbc2TestSuite {
}