Feature/urlparser improve3 pr2

[MarekUniq]

1. Compatibility with libpq is improved regarding URL syntax and resolving url values

1.1. user and password are supported

Complete syntax:
jdbc:postgresql://[[user][:password]@][host1[:port1][,host2[:port2]][,...]][/database][?property1=value1[&property2=value2][&...]]

URL may include:

user (Optional) is the user to connect. Defaults to operating system username.
password (Optional) is the password to connect. No default value.

1.2. property passfile is supported

Example:

Properties props = new Properties();
PGProperty.PASSFILE.set(props, "/mydir/mypass");
Connection conn = DriverManager.getConnection(url, props);

1.3. Code is more clear regarding Properties override rules (source and result are logged at FINE logging level)

There are multiple sources for connection properties. If same property is specified in multiple sources then highest priority source is used. Priority list is here:

1) URL arguments (values after `?` mark)
2) URL values (values before `?` mark)
3) Properties given to `DriverManager.getConnection()`
4) values provided by `service` (from resource `.pg_service.conf`)
5) values in Java System Properties
6) values in Operating System environment
7) values from driverconfig file(s) (`org/postgresql/driverconfig.properties`)
8) global defaults (`dbname`, `host`, `pgpass`, `port`, `user`)

1.4. References

Related to: #2260 #2278 #2398 #2393 #2395 #2424 #2569 #2641 #2644

All Submissions:

• Have you followed the guidelines in our Contributing document?
• Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

1. Does your submission pass tests?
2. Does ./gradlew autostyleCheck checkstyleAll pass ?
3. Have you added your new test classes to an existing test suite in alphabetical order?

Changes to Existing Features:

• Have you added an explanation of what your changes do and why you'd like us to include them?
• Have you written new tests for your core changes, as applicable?
• Have you successfully run tests with your changes locally?

[gitguardian]

⚠️ GitGuardian has uncovered 13 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
-		PostgreSQL Credentials	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret
-		Generic Password	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret
-		Generic Password	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret
-		PostgreSQL Credentials	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret
-		Generic Password	ab4e427	pgjdbc/src/test/resources/pg_service/pgservicefileProps.conf	View secret
-		Generic Password	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret
-		Generic Database Assignment	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret
-		Generic Password	ab4e427	pgjdbc/src/test/resources/pg_service/pgservicefileProps.conf	View secret
-		Generic Password	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret
-		Generic Password	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret
-		Generic Password	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret
-		Generic Password	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret
-		Generic Password	ab4e427	pgjdbc/src/test/java/org/postgresql/jdbcurlresolver/JdbcUrlResolverTest.java	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secrets safely. Learn here the best practices.
3. Revoke and rotate these secrets.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}

[MarekUniq]

@davecramer
Hi! What are merge plans? Should I resolve conflicts now or at a later time?

[davecramer]

@MarekUniq currently my plans are 1) merge in the issue with longs #2710 will be reopened and merged. A release with this fix will be released.
Then we plan on merging in #2635 and releasing as a new major version.

I would hold off rebasing until after the above happens.

Thanks for your patience

[MarekUniq]

@davecramer
Hi! The following pull requests are merged: #2710 #2635.
Any plans for this one?

[MarekUniq]

@davecramer Hi! What are merge plans?

[androa]

Is there any progress on merging this?

[davecramer]

I'm generally in favour of this PR. It will need to be rebased.

[MarekUniq]

@davecramer: I'm generally in favour of this PR. It will need to be rebased.
Rebased

[vlsi]

@MarekUniq , the change reads +19,305 −44,408. Please make sure you include only the needed changes. There's no way we merge something that touches 44 thousand lines.

[MarekUniq]

@vlsi - I agree that GitHub displayed this PR in a strange way. I saw it too. (was it a glitch or my mistake, I do not know)

But I noticed there were 5 new commits in master branch and I rebased branch feature/urlparser-improve3-pr2 again.
the change reads are back to normal now:
+2,031 −909

[vlsi]

@MarekUniq , please rebase commits so they appear "on top of the master branch". The project uses linear history rather than merge commits.

[vlsi]

The key question I have here is how do we test and ensure backward compatibility?

[MarekUniq]

@MarekUniq , please rebase commits so they appear "on top of the master branch". The project uses linear history rather than merge commits.

@vlsi I think I can't change the order of commits anymore.
However, in case this PR will be merged, there will be a "squash merge" and then there will be only one commit in the master branch, which will appear as the last commit in the master branch. Isn't it?

[vlsi]

What do you mean by "can't change the order of commits"? Could you please squash at your side?

[MarekUniq]

What do you mean by "can't change the order of commits"? Could you please squash at your side?

@vlsi Is this the result you were expecting? I did the following:

• git merge --squash ...
• git push --force ...

[vlsi]

Is this a backward compatible change?

[MarekUniq]

There is no one constant default value for given parameter.
The default value is operating system specific.
Yes, functionality is backward compatible (the same).

Can you recommend proper approach?
This enum is not meant to be used outside pgjdbc driver.
There should not be any promises assumed regarding this enum outside pgjdbc driver.
At the same time I can not remove keyword public.
How could we avoid "backward compatible" questions which are relevant only because of technical restrictions?

[vlsi]

This enum is not meant to be used outside pgjdbc driver.

The enum is public, it is located in a public org.postgresql package, nothing in its javadoc says anything regarding "for pgjdbc internal use only" and so on.

There might be usages of the enum outside of the driver.

---

How could we avoid "backward compatible" questions which are relevant only because of technical restrictions?

I do not understand the question.
I think the only case we can merge a PR is if the PR keeps full backward compatibility or if the breakage is minimal.

If there's a significant breakage of the backward compatibility, there's very little we can do.

[MarekUniq]

The enum is public, it is located in a public org.postgresql package, nothing in its javadoc says anything regarding "for pgjdbc internal use only" and so on.
There might be usages of the enum outside of the driver.

Exactly, this was the point of my question.

There is an enum. It is meant to be used only internally by pgjdbc project.
But, I have to make enum public because it is used by different packages inside pgjdbc project (technical/language limitation)
Javadoc etc is soft restriction, public enum can still be used.

I was wondering if there was an approach to the problem described above. How should we create classes/enums that are used by different packages in the pgjdbc project internally, but when renaming/moving/deleting them, the issue of "backward compatibility" is irrelevant.

[MarekUniq]

... is if the PR keeps full backward compatibility or if the breakage is minimal.

In my estimation, the real impact is close to zero.
The old value is incorrect and there is no common constant default value for this parameter across all operating systems.
If someone has adopted this default value and has tested their system, they must have realized that this value is wrong