Faker 2.0 design

[erikgaal]

Hey all,

In fzaninotto's repo a discussion was started about the design of the next major version (fzaninotto#1807). Shall we reopen that here?

The three questions asked by @pimjansen were:

• Which PHP versions will be supported?
• What development tools are to be used?
• What does the actual architecture look like?

[mathieutu]

I think @fzaninotto is right saying the package is way to heavy, and should be splitted, with an extract of the languages.

I'm super concerned about the responsibility that we developers have on climate change. Over the years, Faker has probably emitted more than 11 Metric tons of CO2 equivalent. Being such a significant contributor to climate change kills me.

Most people only use 1 locale, so they only need a fraction of the library size. Yet, because it's designed as a multilingual library, there is no alternative: they have to download the 3MB even if they only need 10KB.

Faker should have been a small core library with no localized content. Then, developers from all over the world would have contributed localized packages to packagist, without requiring my supervision for merges. And users would just have to download the core and the locale they need.

With this organisation it could be easier to maintain separatly locale providers.

[erikgaal]

Do we want to split Faker and its locales into different repositories, or do we want to keep a single repo and split it into different Composer packages (like laravel/framework does with illuminate/*)? For the end user they achieve the same thing: smaller, composable packages. So the choice here has to be made in favour of the developer experience.

Monorepo:

• 👍 Easier to work on the entire project.
• ...?

Multirepo:

• 👍 Easier to work on a single locale.
• 👎 Might result in outdated locales, if they are not upgrades simultaneously.
• ...?

[mfn]

I think Laravel and Symfony are successful examples of such monorepos, using read-only repos for splitting/releasing (no "issues" there, for example).

I think a single repo for just for the issues is already better than having it spread and worst-case having issues in wrong repos.

I'm 👍 mono-repo if the split happens, because I figure you want to run the test suite also with all of them and that's why certain things are "just easier" and there's only the one-time complexity of setting everything up and making it work (with scripts).

[darthsoup]

I also believe that the Monorepo is the best approach for this project, just like Laravel does.

[erikgaal]

Maintainers, do we want to use a democratic system (using something like https://github.com/apex/gh-polls) to make the decisions here?

[mathieutu]

In my opinion, shared issues and pull request are here an inconvenient, instead of an advantage.

We're not talking about splitting a framework in different packages but having N independent locales which should evolve differently (respecting common interfaces, of course).

One of the pain point François raised is maintaining those locales, while not understanding the language.

With several repos, we can have different repo maintainers who are responsible for their own locale. It will so be easier to maintain the whole thing.

And it'll also be easier to find maintainers: if I have no time and no envy to maintain faker, I'm totally OK with handling the French locale, because smaller project, less and clearer notifications, and less responsibility.

Thoughts?

PS: I'm generally a monorepo guy, but I think that the Faker context is a enough special to handle it differently.

[ytilotti]

I agree with @mathieutu. It makes more sense to create differents repositories with differents mainteners. It's like in the translation business, you can only translate into your native language. It is not possible to manage languages that you can't understand.

A repository locale list on this repository would be consistent.

[mbabker]

Hi, some opinions from random onlooker who also recently went through the monorepo versus manyrepo debate.

So, I recently restructured Pagerfanta to support installing different optional elements as separate packages (akin to illuminate/* versus laravel/framework or symfony/* versus symfony/symfony). Part of my thinking in how I approached it is that if a portion of the monorepo gets deprecated in any way, all of the subsplit repos remain available and could be forked or become a canonical point of truth if someone were to want to become a maintainer for an individual package.

So, if I were a maintainer here, what I would personally do to start off is go with the monorepo approach while keeping all of the locales within this repo and supporting a subsplit, then if I found it to be too busy supporting those in one repo, in theory it should not take too much effort to turn the split locale repo into the point of truth and drop that portion of the code from the main repository.

[mathieutu]

if I found it to be too busy supporting those in one repo

Actually, in the faker context, we do have years of experience and feedbacks on it:

These PRs were in a language I don't speak, so I had no way of knowing whether the data was good or bad. I ended up closing many PRs for copyright reasons and blindingly merging the others.

[mbabker]

Right, but wasn't the original repo predominantly a single person overseeing things? There's a team here, so presumably one could use the code owners feature of GitHub to request reviews from team members who are "language specialists" (for lack of better terms) instead of expecting an English speaker to review all non-English changes. It wouldn't be much different than having those same team members review and merge pull requests to other repos other than the amount of traffic/noise on a single repository.

[pimjansen]

Im a fan to keep things seperate as much as possible. Where a mono repo is really cool but that is not the only thing. We always talk about Locale and stuff like that but Faker also holds so called Providers. Technically it is the same thing but the providers hold non language specific things.

For example there is an eCommerce provider as a suggestion. This provider will probably never make it to the Faker core but it should be easy to be pluggable.

I think if we provider proper interfaces for the locales we can ensure that the quality of those locales are good and that they hold all the required data.

[Nyholm]

This is an interesting discussion. I am a big fan of mono repos. I would say mono repo 7 days every week. However @mathieutu makes a really good point.

Since we have localized providers, we have a really hard time figuring out what is a good PR or not. I dont believe any of you could properly verify this PR to add Swedish municipalities.

I vote for doing separated repositories. I also think that if someone contributes we should make them a maintainer to that repository. Let's aim for at least 2-3 maintainers per locale.

[Bavragor]

I already started splitting up faker into faker-core, and one repository for each locale, also planned to add some more for ORM etc.

Like nyholm said, I also strongly believe that separate repositories are necessary to maintain language specific providers.

[pimjansen]

I already started splitting up faker into faker-core, and one repository for each locale, also planned to add some more for ORM etc.

Like nyholm said, I also strongly believe that separate repositories are necessary to maintain language specific providers.

I think it is important first to discuss on what to take into the core and define the interfaces on what we think is best. Same goes for the interface on the outside so that we can keep an easy migration plan available.

Graham already mentioned it earlier which i believe was totally right, that we need to use 2020 to ensure people migrate to the new repository and continue to support it. If we start breaking all stuff people will probably not move. Where also the old repo is still there and working fine for those

[stevebaros]

Small Isolated and localized packages would be the best Fit, The core Faker Framework should stand alone as a package Maintained by the core team. Core Team should give visibility to well crafted localized packages.

[emiliopedrollo]

I could maintain the pt_BR locale as well

[javabudd]

Please, split this up into a repo for each provider and assign a maintainer to own it.

[localheinz]

That's the plan!

[pimjansen]

please god split this up into a repo for each provider and let someone/s own it

This is an option but also grouping it for certain languages. For instance.

Nl_NL
Nl_BE

etc

[Nyholm]

Hey.
I’ve drafted some ideas of an architecture. Feel free to add comments or suggest changes with a or.

https://github.com/Nyholm/next-faker

[localheinz]

Everyone working on new features, can you take a look as well?

Perhaps it makes sense for you to hold off your work for a bit!

/cc @GlennM

[Nyholm]

I'll start working with these changes the next few weeks. Please share your opinions, good or bad.

[localheinz]

Quick heads-up, I have enabled discussions and converted this issue into a discussion.

I hope that makes sense for everyone involved!

[ibes]

I have not enough experience in structuring big projects - I am just a user.

I am German - so most likely would use de_DE and likely en_US as eg. in WordPress English is the source language even if my site ends up being German only.

I see that de_AT and de_DE will share a lot of similarities as both use German.
But eg. post codes, bank names, titles etc. will be different.

Then Germany is part of the EU.
So we share a currency, bank account numbers and a bunch of further things.

I imagine things like that will exist in a couple of locals -> sharing strong similarities with some, but be different in plenty.

Eg. de_AT extends some German generators:
https://github.com/FakerPHP/Faker/blob/main/src/Faker/Provider/de_AT/Text.php

If they are in separate packages the AT local would need to require the DE local - but what if the DE local extends another local - maybe an EU local or the EN local.

As a user I am pro lightweight but also pro simple.
If I need to define the locals I want to use, this should be easy.

Nice would be to require

• fakerphp/faker
• fakerphp/faker-local-de_DE

And all the rest is not mine to figure out.

If I miss the point of the discussion, be welcome to ignore my question.

[goodevilgenius]

This is a really good point. There could be a repo for a generic de locale, which would have everything that's common between the various German locales.

Then the de_DE, de_AT, de_CH, etc. could require and extend that locale repo. Those repos would be able to be very small, having only the differences.

[ibes]

Not sure if the same ist true for eg. Spanish and Portogues.
A lot of language strings will be the same but money related things might be completely different as eg. South America and West Europe are different regions.

[bambamboole]

Maybe the currency thing should be part of the main package and only the things which differ is in separate packages

[goodevilgenius]

Currency stuff could be a trait that's used by the necessary locales.

So, for example, there could be a EuroTrait with € currencies.

Then, es_ES would use that trait, but none of the other es locales would.

[devnix]

What about... voting? :D

[xwiz]

Although I voted for multi-repo, the monorepo is obviously the short term valid option until there is enough contributors to avoid premature death. Also I've been thinking about the data generators for over a year, fakers text and image generation needs to be completely rethought. In english, it's fairly easy to generate valid or near valid english words with a small morpheme library. With images it's better to generate dummy placeholders than connect to some external service.

[bigint]

English language support please!

[pimjansen]

Isnt that a default haha

[GlennM]

I'm curious about the current status of this, are there any updates?

[Erwane]

THIS need update. The main reasons of creator to stop faker (https://marmelab.com/blog/2020/10/21/sunsetting-faker.html) are still relevant, and more relevant now in 2022.

This should be discussed please.

[Erwane]

Very nice. There is a doc on "how to contribute" or the best way to regroup locales by languages ?

I saw FakerPHP\Dutch with 2 countries in it, this is the chosen method ?

[bram-pkg]

FakerPHP/Dutch is indeed a reference implementation of the new extension system, for now. Nothing is fully set in stone yet.

[pimjansen]

Very nice. There is a doc on "how to contribute" or the best way to regroup locales by languages ?

I saw FakerPHP\Dutch with 2 countries in it, this is the chosen method ?

The first step is to migrate all existing providers to Extensions. This is for all stuff that will stay in the Faker Core package.

After that we will adapt the en_US implementation in the core and remove all the rest.

I guess at that point we will release a first release candidate where others can implement a language extension and such.

For example at this point Faker generates images use an external website. This will be removed but i assume it would be good to build an extension that depends on GD and generates it locally

[DvDty]

FakerPHP/Dutch is indeed a reference implementation of the new extension system, for now. Nothing is fully set in stone yet.

I know it's still early, but when the time for separating each language into individual package comes, we could make a FakerPHP/Language template repository, from which the individual languages can generate their repositories. That way it will be easier to pull updates and keep consistent workflows (like the test runner for multiple PHP versions with code coverage, coding standards, etc.).

[pimjansen]

FakerPHP/Dutch is indeed a reference implementation of the new extension system, for now. Nothing is fully set in stone yet.

I know it's still early, but when the time for separating each language into individual package comes, we could make a FakerPHP/Language template repository, from which the individual languages can generate their repositories. That way it will be easier to pull updates and keep consistent workflows (like the test runner for multiple PHP versions with code coverage, coding standards, etc.).

This is indeed the plan. We now at a state where an initial reference implementation is done.

At this point we are publishing the interfaces and new generators. The current fakerphp/faker will contain the english language pack so upgrades will be easy

[GlennM]

Almost one year after the latest commits to the 2.x branch and the Dutch repo, I'm wondering about the progress for Faker 2.0.

[pimjansen]

The Poc for the dutch repo is finished but we need to finetune some stuff on the core repo and interfaces

[pimjansen]

Some info on the status. We had a solid meeting this evening concerning the 2.x progress and points left to do. We identified most of the leftover interfaces and we are now in the progress of implementing those for Core. The steps that are still open (roughly).

• Implement remaining interfaces in Core
• Rewrite fakerphp/faker to reflect the en_US locale and implement remaining interfaces
• Document everything and write a mapping for all 1.x to 2.x that got changed or removed
• Release first alpha version of Core and FakerPHP.

After all this we will start working on the language packs so we can have them ready when 2.x will release it first beta. The ones that are planned for now would be:

• English
• Dutch

We will ask if we are ready so that people can implement their own. We will add a boilerplate repo for people to start with to make implementations easy

[MrYamous]

Hello,
Do you have any information to share about the progress of 2.0 ? Can the community help you in any way ?

[localheinz]

@MrYamous

As far as I can see, we are in the uncomfortable situation where

• we have deprecations running the current code base on PHP 8.3 (see Fix: use default mode for mt_srand (MT_RAND_MT19937) #828 and similar)
• we have a disagreement regarding support of PHP versions
• we have valuable feedback from @TimWolla suggesting to create a version of fakerphp/faker that supports PHP 8.3 and drops support for previous versions (see Enhancement: Implement MersenneTwisterIntegerGenerator #769)

That is, it seems to me like we are in a situation where we are somewhat unable to move forward - some people call this death by committee.

[kcassam]

Can you explain the disagreement regarding support of PHP versions ?

[MrYamous]

PHP 8.3 support is indeed an important issue I see there are PRs for it but not sure to understand what's blocking progress ?

If we consider the design topic, IIIC probably the most important aspect of 2.0, is the new implementation with Core (en_US as default) and languages as independant packages functional ?