Proposed CLR spec change: allow timestamp header to be deterministic hash with upper bit 1

[gafter]

We'd like the Roslyn compilers to be deterministic. One obstacle is the timestamp in the assembly header. We propose to use a deterministic hash of compiler inputs (or a hash of the rest of the assembly, excluding the timestamp) to produce bits to use in the timestamp, and we'll set the upper bit to 1. For this to be "legal", we need the spec to allow random-looking bits in the timestamp when the timestamp's upper bit is set to 1.

/cc @jaredpar @tmat

[jkotas]

This should be primarily request for PE file format spec change. AFAIK, the PE file format spec is maintained by Microsoft C++ compiler team. The CLI file format is meant to be a strict extension of PE file format.

ECMA-335 has the PE file headers as 'informative text only'. This informative text is essentially copy of the relevant part of PE file format spec.

cc @CarolEidt

[akoeplinger]

@gafter afaik the initial approach was to set timestamp to 0 (dotnet/roslyn@04462c4) and this is also how Mono's mcs implemented it, any reason why this didn't work out for you?

[gafter]

@gregg-miskelly You were the strongest push-back on using a timestamp of 0 for deterministic builds. Do you want to comment here?

[jaredpar]

Using a timestamp of 0 has proved problematic in the past. There are many tools which rely on timestamp being a non-zero value. In order to use 0 I think we'd need to have a very compelling use case.

[akoeplinger]

@jaredpar can you elaborate a bit more on "many tools"? Are those internal Microsoft tools or 3rd party ones? Since this is supposed to be an offset from the epoch I'd be very curious how 0 breaks those tools 😄

[jaykrell]

@akoeplinger right, building on January 1 1970 would seem legal.
Does a constant 1 work?
While more expensive, a hash should work too. Assuming 31 bits is enough. And tools are ok with future times. I think at some point the spec changed and that bit isn't the discriminant. I see hashes in PEs w/o the high bit set.
e.g.:

\bin\amd64\cdb /z %windir%\system32\kernel32.dll
:000> !lmi .
Time Stamp: 5f488a51 (This is a reproducible build file hash, not a true timestamp)

[akoeplinger]

probably something we can add to https://github.com/dotnet/runtime/blob/main/docs/design/specs/Ecma-335-Augments.md