Localize preprocessor feature + french locales ordinal day with long month format bug fix

[fturmel]

The PR introduces an optional preprocessor function that lives on the locale.localize object. After the format function parses and tokenizes, the preprocessor is invoked with the date and a parts array (the tokens and string literals found in the format string). The preprocessor can modify this array as needed.

This allows us to nicely fix the long standing #1391 french locale day ordinal with long months format regression. This is the best solution I could think of that would not break any public interface and be low overhead. Hope this can benefit other locales too!

As discussed in #3659, I brought back the unit tests from date-fns v1. I also added an additional date and format to the locale snapshots to better surface day ordinal issues (particularly related to the french language) - apologies for the added noise.

Bonus 1: somewhat unrelated, but working on format, I realized that the _originalDate argument sent to the formatters was never relevant (both date and options._originalDate were always the same object). I removed this to simplify things a bit.

Bonus 2: Fixed bad import in main branch that was breaking the tests

[antoinerousseau]

nice!

[fturmel]

For those specifically interested in the french locale fix, here is the isolated code:

date-fns/src/locale/fr/_lib/localize/index.ts

Lines 127 to 148 in ddac6a5

	const LONG_MONTHS_TOKENS = ["MMM", "MMMM"];
	export const localize: Localize = {
	preprocessor: (date, parts) => {
	// Replaces the `do` tokens with `d` when used with long month tokens and the day of the month is greater than one.
	// Use case "do MMMM" => 1er août, 29 août
	// see https://github.com/date-fns/date-fns/issues/1391
	if (date.getDate() === 1) return parts;
	const hasLongMonthToken = parts.some(
	(part) => part.isToken && LONG_MONTHS_TOKENS.includes(part.value),
	);
	if (!hasLongMonthToken) return parts;
	return parts.map((part) =>
	part.isToken && part.value === "do"
	? { isToken: true, value: "d" }
	: part,
	);
	},

[Dallas62]

Just to make sure, https://github.com/date-fns/date-fns/pull/3662/files#diff-eda4dba459a77e98278dfef21c803e0e1b94faef1d5a3a3c2b42657b8a4a6e58R128 there is an Invalid in the result of parse.
Since it's not the first one in this file I don't think it's blocking, but is it expected ?

[fturmel]

Just to make sure, https://github.com/date-fns/date-fns/pull/3662/files#diff-eda4dba459a77e98278dfef21c803e0e1b94faef1d5a3a3c2b42657b8a4a6e58R128 there is an Invalid in the result of parse. Since it's not the first one in this file I don't think it's blocking, but is it expected ?

Yes, it's been that way for a while now - I wrote about this in #3136 too

[Dallas62]

Thanks for the fix and for your time !

[fturmel]

Thanks for the fix and for your time !

Appreciate your review and all the efforts to surface this issue. Hope the approach is sound and we can eventually get this merged!

[fturmel]

Oh one other thing I forgot to mention, this only takes into account the MMM and MMMM tokens which are the format context. There are also stand-alone context tokens for long months: LLL and LLLL. Should these also be considered? (I assume not, but wanted to make sure)

https://date-fns.org/docs/format
https://www.unicode.org/reports/tr35/tr35-dates.html#dfst-month

[Dallas62]

I'm not sure to understand the difference between LLLL and MMMM, even after reading the documentation part about it. I guess it's not important since it's not for "format" ... 😄

[kossnocorp]

Love it, will try to review it carefully and ship with the next version 🙏

[kossnocorp]

The change is brilliant! I don't know if it will be helpful outside French, but I'll keep my eyes open. If not, and we meet some other nuance in a locale, I'll use a similar preprocessor approach.

[kossnocorp]

Wow, how did it work, and how didn't I miss it? 🤣

[kossnocorp]

Shipped with v3.3.0: https://github.com/date-fns/date-fns/releases/tag/v3.3.0