possible problem with ies_phi_factor_file

[BJeannot1]

Hello,

I have a test case with observations of both piezometry (they are around 10² m) and concentrations (they are around 1e-9).
Also, the standard deviation of observations is set to 0.05 for all piezometries and to 1e-11 for all concentrations.
I use ies_phi_factor_file to change the weights so that the objective function is 50% piezometry and 50% concentrations.

I observe, something strange : the difference between the measured and actual objectives function became enormous as a result of the weight adjustment.

Before weight adjustment, the measured and actual objective functions are about the same order of magnitude :

--- pre-weight-adjustment initial phi summary ---
phi type mean std min max
measured 6233.62 7786.17 1031.19 23281.1
actual 5467.6 7613.79 166.446 22194.4
note: 'measured' phi reported above includes
realizations of measurement noise,
'actual' phi does not.

After weight adjustment, there is a 15 orders of magnitude difference :

--- initial phi summary ---
phi type mean std min max
measured 3.68989e+21 1.33153e+21 800702 4.68715e+21
actual 1.09352e+06 444877 625700 1.84434e+06
note: 'measured' phi reported above includes
realizations of measurement noise,
'actual' phi does not.

Does this look normal to you ?
Thank you for your help.
Best regards,
Benjamin JEANNOT

[jtwhite79]

Im guessing that really small initial concentration phi is doing weird things. Can you post your files somewhere for me to see what is going on?

[BJeannot1]

Unfortunately, I am affraid I cannot easily share my files, for confidentiality reasons.
However, I can confirm that multiplying all concentrations by 1e11 indeed solved the problem.
Thank you for your help.
Best regards,
B JEANNOT

[jtwhite79]

Yeah i was playing around earlier and noticed that I can generate some extreme measured phi values with weight adjustment but not the actual phi (the weight adjustment uses actual phi). So I think that means the noise realizations might be a place for you check...

[BJeannot1]

Checking the case.obs+noise file, I notice that the values look correct for the base realization (it looks like there is no observation noise for this realization anyway), but out of the place for other realizations (as a reminder, for concentrations, the standard deviation of noise in observations that i inputed is 1e-11 so the obs+noise realizations should be not far from my base realization). See a portion of my case.obs+noise.csv file below

Also, I notice that the obs+noise realizations also look strange when I multiply all concentrations by 1e11. Please see below the case.obs+noise.csv file I get with all concentrations multiplied by 1e11. You will notice a huge difference between base realization and the other realization, while the noise I inputed for the observations is only equal to 1.0

Sorry if this does not help a lot. I may try to make a sharable example when I have some time, but I am quite busy at the moment.

Best regards,
Benjamin JEANNOT

[jtwhite79]

All good. Ill run a few more checks on my end. How are you generating the noise? Using standard_deviation in obs data csv?

[BJeannot1]

Yes, I confirm I use the standard_deviation of the the obs_data.csv file to set my obs noise.
Thank you.

[BJeannot1]

So here is where we got so far:
-Using very small concentration values in the observations (1e-9), along with other observations of piezometry (around 10²m high) while activating ies_phi_factor_file generates a very high measured phi
-multiplying the concentrations by 1e11 solves this problem.
-However, in both cases, the contents of the file case.obs+noise.csv are strange (too much noise added to the observations)

And now, I write again to you because, just a few minutes ago, i tried to disable ies_phi_factor_file and noticed that the problem in case.obs+noise.csv remains. So this may not be related to ies_phi_factor_file but rather to the way case.obs+noise.csv is written.

Best regards,
B JEANNOT

[jtwhite79]

The obs+noise ensemble is generated separately from the weight adjustment so result that checks out. There are a few tests that cover the use of standard_deviation in the version 2 control file and I also just did some more testing and it seems to be working as expected, even with very small magnitude numbers.

If you check the rec file, do you see this:

Beyond the obs+noise issue (once we get it sorted out), Im wondering if it might be better to estimate and simulate the log of the concentration (with model post-processor to convert to log), just as a means to more naturally bring up the magnitude of the concentration residuals?

[BJeannot1]

Thank you for your insights.
I am amazed to notice that I do not have the text you provided in your picture in my rec file. And I think I understand why...
Please look at the first line of my case.obs_data.csv file :

There is a space I forgot to remove before the "s" of "standard_deviation". And this has been more than 1 year I have been using pestpp-ies with this mistake ! I guess my simulations will get a lot better from now on...

Thank you very much !!!!

I will correct this mistake and see if issues continue to arise. I keep you updated.

[BJeannot1]

I confirm that removing the space before "standard_deviation" solved the issue with the case.obs+noise.csv file.
Thank you for your help.
As regards the other comment you made, converting the conentrations obs in log is a relevant idea ideed. I might implement it very soon.

Also, returning to the topic which initiated this issue, I will soon provide you with inputs and outputs I can share so that you can understand why small concentration values cause a problem when activating ies_phi_factor_file.

Best regards,
Benjamin JEANNOT

[jtwhite79]

ah ok. I thought the column names in the external files were having leading and trailing whitespaces trimmed off - I'll check that...

[BJeannot1]

If it does not contradict the purpose of PEST++, it may also be useful to only accept as column names a list of specific strings which are to be defined in a whitelist. But this is up to your choice !

[jtwhite79]

Nah we want to allow unlimited additional columns in those external files as a means to carry metadata/aux information around thru the pest world...but the rec file does report explicitly when it finds something in an external file that it's going to use. So you can check for detection and usage...