Handle event meta data changes in streamer files

[Dr15Jones]

PR description:

Moved storage of meta data needed for events into the first EventMessage sent before the subsequent events which need that meta data.
This fixed a problem seen online where different HLT nodes were generating different event meta data for the same luminosity block but online the event data for one node is used.

PR validation:

All unit tests associated with streamer files pass.

[Dr15Jones]

please test

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44892/40158

• This PR adds an extra 248KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File CalibCalorimetry/EcalLaserSorting/test/streamIn_cfg.py modified in PR(s): [Dummy] Try creating big-enough cache to cause splitting #44368

[cmsbuild]

A new Pull Request was created by @Dr15Jones for master.

It involves the following packages:

• CalibCalorimetry/EcalLaserSorting (alca)
• DQMServices/StreamerIO (dqm)
• DataFormats/Streamer (core)
• EventFilter/Utilities (daq)
• IOPool/Streamer (core)

@saumyaphor4252, @perrotta, @nothingface0, @smuzaffar, @tjavaid, @syuvivida, @antoniovagnerini, @makortel, @smorovic, @consuegs, @rvenditti, @Dr15Jones, @emeschi can you please review it and eventually sign? Thanks.
@wddgit, @rovere, @ReyerBand, @argiro, @wang0jin, @yuanchao, @makortel, @Martin-Grunewald, @rsreds, @mmusich, @missirol, @tocheng, @rchatter, @barvic, @thomreis this is something you requested to watch as well.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-79e450/39218/summary.html
COMMIT: e3d9efe
CMSSW: CMSSW_14_1_X_2024-05-02-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44892/39218/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3331548
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3331525
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 202 log files, 165 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[makortel]

Part of addressing #44643

[smorovic]

+1

[makortel]

+core

[cmsbuild]

REMINDER @rappoccio, @sextonkennedy, @antoniovilela: This PR was tested with #44991, please check if they should be merged together

[antoniovilela]

ping @cms-sw/dqm-l2

[tjavaid]

+1

[rappoccio]

+1

[mmusich]

hold

is this still meant to be on hold?

[antoniovilela]

unhold

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

[smuzaffar]

@Dr15Jones , this is now available in today's 11h IB. There is a unit test failing with [a], could this PR be the cause of it? Note that IB is a patch build, may be we need a full build to avoid it?

[a] https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc12/CMSSW_14_1_X_2024-05-30-1100/unitTestLogs/RecoPPS/Local#/

----- Begin Fatal Exception 30-May-2024 13:09:44 CEST-----------------------
An exception of category 'FatalRootError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type NewEventStreamFileReader
   Additional Info:
      [a] Fatal Root Error: @SUB=TBufferFile::ReadClassBuffer
Could not find the StreamerInfo for version 11 of the class edm::SendJobHeader, object skipped at offset 33

----- End Fatal Exception -------------------------------------------------

[makortel]

There is a unit test failing with [a], could this PR be the cause of it?

It is.

Note that IB is a patch build, may be we need a full build to avoid it?

Full build won't help. Based on the logs the test seems to be reading streamer files (brittle), so either the streamer files need to be redone (through streamer -> root -> streamer cycle), or the files could be converted to root format if the streamer aspect is not super important for that test (or the test would be disabled). I'd suggest to follow up with the test authors in a new issue.

[makortel]

I'd suggest to follow up with the test authors in a new issue.

I opened an issue in #45101