build: Non-recursive makefiles #652
Conversation
|
The goals of non-recursive make and making flex's build system portable to non-GNU make are (at least logically) separate. Can you point out where we could change things to make them not GNU-make specific? That seems worthwhile -- to the extent that automake allows this, that is. Given that po/ is handled by gettext, is there any possibility of incorporating po/ into the non-recursive scheme? I assume the answer is "no". The tests/ directory is just a mess of files that is only going to grow over time. I'm not sure we would want to incorporate it into a non-recursive scheme even if it were possible. If anything, I'd actually like to split tests/ into subdirectories just to make it much less of a sprawl. (But that's super low priority for me, so I haven't given any thought to how practical or good an idea this might turn out to be.) Given that we have po/ and tests/ as recursive, do we really want a build system that is sort-of-recursive, depending on which directory we're talking about? Having to repeatedly list subdirectories as the examples listed in the page you link seems like a huge net loss for no particular gain. Also, one of the idioms we have now is: cd doc && makeHow does your replacement build system provide for this kind of thing? I'm thinking particularly of the case where the files are edited in a subdirectory and the Makefile is not in the directory any longer. I think this would make running the Makefile from emacs harder with no particular upside. |
|
I don't care for this. It just looks different to be different. Hold off
until I answer your argument about make being POSIX. Work got busy.
…On Fri, May 10, 2024, 17:33 Will Estes ***@***.***> wrote:
The goals of non-recursive make and making flex's build system portable to
non-GNU make are (at least logically) separate.
Can you point out where we could change things to make them not GNU-make
specific? That seems worthwhile -- to the extent that automake allows this,
that is.
Given that po/ is handled by gettext, is there any possibility of
incorporating po/ into the non-recursive scheme? I assume the answer is
"no".
The tests/ directory is just a mess of files that is only going to grow
over time. I'm not sure we would want to incorporate it into a
non-recursive scheme even if it were possible. If anything, I'd actually
like to split tests/ into subdirectories just to make it much less of a
sprawl. (But that's super low priority for me, so I haven't given any
thought to how practical or good an idea this might turn out to be.)
Given that we have po/ and tests/ as recursive, do we really want a build
system that is sort-of-recursive, depending on which directory we're
talking about?
Having to repeatedly list subdirectories as the examples listed in the
page you link seems like a huge net loss for no particular gain.
Also, one of the idioms we have now is:
cd doc && make
How does your replacement build system provide for this kind of thing? I'm
thinking particularly of the case where the files are edited in a
subdirectory and the Makefile is not in the directory any longer. I think
this would make running the Makefile from emacs harder with no particular
upside.
—
Reply to this email directly, view it on GitHub
<#652 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAVJXIOAXTM6QTQZO45C7WDZBU4K7AVCNFSM6AAAAABHRHOK46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVGI4TOMZQGQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
To condense my questions down a fair bit and ask from a higher level perspective: What problem does this solve that flex is having? None of the reasons that the nonrecursive make page gives are relevant. Maybe the multi-threading thing is technically true, but if you're rebuilding flex so often that you notice, I would gently suggest that you're doing something else fundamentally wrong. |
The problem I was trying to address is when two 'make' instances running in parallel, and cross-directory 'make' calls exist, then two instances of 'make' might touch the same file at one time, causing contention. This problem is rare, I admit, but possible. For flex, the case is one can run (Unix 'make' is never designed to touch or build files atomically. The usual workaround is to tell users to be careful and not to invoke two 'make' instances that might touch the same file simultaneously.) That's how cross-directory dependency can be tricky. Recursive makefiles handle this problem by ensuring subdirectory makefiles be run only in a specific order, and no two subdirectory makefiles can be invoked at the same time (which limits parallelism). I wished there is an alternative way to handle this, and make the solution useful in the long run. And regarding the idiom you mentioned:
Yes, I am aware of that, and I was reluctant to change |
|
I understand the issue that you're having with the doc directory, but it
comes back to the misunderstanding we were discussing regarding the name of
the flex binary at various stages of the bootstrap and build process. Let
me take a crack at fixing the libtoolized make files before we go down this
path.
…On Fri, May 10, 2024, 20:27 Kang-Che Sung (宋岡哲) ***@***.***> wrote:
I'm sorry but I need some sleep before I have the energy to explain all
questions...
@westes <https://github.com/westes>
What problem does this solve that flex is having?
None of the reasons that the nonrecursive make page gives are relevant.
Maybe the multi-threading thing is technically true, but if you're
rebuilding flex so often that you notice, I would gently suggest that
you're doing something else fundamentally wrong.
The problem I was trying to address is when two 'make' instances running
in parallel, and cross-directory 'make' calls exist, then two instances of
'make' might touch the same file at one time, causing contention.
This problem is rare, I admit, but possible.
For flex, the case is one can run make -C doc/ all
and make -C src/ all almost at the same time, and because of
cross-directory dependency, two 'make' instances might try to build (with
compiler and linker) the same flex binary at the same time.
"make -C doc/ flex.1" -> flex.1
|
|
V
"make -C src/ flex" -----> flex (program)
(Unix 'make' is never designed to touch or build files atomically. The
usual workaround is to tell users to be careful and not to invoke two
'make' instances that might touch the same file simultaneously.) That's how
cross-directory dependency can be tricky. Recursive makefiles handle this
problem by ensuring subdirectory makefiles be run only in a specific order,
and no two subdirectory makefiles can be invoked at the same time (which
limits parallelism).
I wished there is an alternative way to handle this, and make the solution
useful in the long run.
And regarding the idiom you mentioned:
cd doc && make
Yes, I am aware of that, and I was reluctant to change tests/Makefile.am
for that very reason.
But for "doc" directory, this idiom is the problem.
Unless everything in the "doc" directory depends on nothing outside the
directory, I don't think it is safe to have a makefile to be invoked there.
(Just my opinion.)
—
Reply to this email directly, view it on GitHub
<#652 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAVJXILN75LIXREFE6PCXYTZBVQYTAVCNFSM6AAAAABHRHOK46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVGQYDMNZZGY>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Excuse me. What misunderstanding? (Just to attach the flex bootstrap diagram again for reference:) |
|
@westes
(a) Non-recursive makefiles have to be invoked from the top level; people would lose the ability to just rebuild a subdirectory. For subdirectories that are mostly self-contained (that depend on little from the outside), the lack of building a subdirectory is a loss. But, for subdirectories with a lot of inter-directory dependencies, the lack of such ability may be considered a feature, not a bug with the non-recursive approach. The "tests" directory in Flex does use some custom suffix rules, so the weakness (b) could become significant and thus is one of the reasons I cannot convert it to non-recursive now.
The answer is no. GNU Bison code, which I take for inspiration for the non-recursive makefiles like this, still has a recursive makefile for the "po" directory. |
|
I'm referring to the stage1scan.c naming scheme. You've chosen not to
change it back.
Your rationale for respecting LFLAGS is wrong. We are building flex, not
executing flex. If you wanted deterministic builds, letting users change
the build flags won't get you there.
You aren't using VPATH correctly in doc/ so you've had to copy the binary.
You also didn't notice that users don't regenerate the docs so you're
trying to solve a problem that doesn't (or shouldn't) exist.
You removed the ability to build in the src/, doc/, and examples/
directories. That was useful for unit testing the build system.
…On Sat, May 11, 2024, 07:38 Kang-Che Sung (宋岡哲) ***@***.***> wrote:
@Mightyjo <https://github.com/Mightyjo>
I understand the issue that you're having with the doc directory, but it
comes back to the misunderstanding we were discussing regarding the name of
the flex binary at various stages of the bootstrap and build process. Let
me take a crack at fixing the libtoolized make files before we go down this
path.
Excuse me. What misunderstanding?
(Just to attach the flex bootstrap diagram again for reference:)
e9e9b93e-9e3f-11e6-8319-e4e33d38049b.png (view on web)
<https://github.com/westes/flex/assets/2487263/5faacd54-8718-4cf5-aed9-0fb411299cb1>
—
Reply to this email directly, view it on GitHub
<#652 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAVJXII6HYZY2D6G7AGONQLZBX7MFAVCNFSM6AAAAABHRHOK46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVGY4DQNBXGY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
It's not my call for changing that naming, and I don't know what better name people would suggest then.
It's deterministic with respect to user's build flags now. Perhaps you were confused about the definition of "deterministic". I didn't mean the scanner the user generated (stage1scan.c) would be always identical to the stock one (scan.c) provided in the distribution, but the generated scanner (stage1scan.c) can reproduce itself (stage2scan.c) with the just built "flex" generator. A build is considered "deterministic" when, given the same source code, same compiler tools, and same build configurations (including CFLAGS and configure script options), then any person can produce bit-identical binaries regardless the machine they used. Build determinism mean eliminating non-deterministic factors such as system time, hostname, user names, and build paths (ideally, but some build paths can be tolerated to slip in binaries if they help debugging). Build determinism is a factor on whether users can trust the binaries especially for binaries from OS distributions. People don't want software backdoors or compiler viruses. (Compiler viruses are especially hard to uncover unless some kind of build determinism is forced.)
Nope. VPATH is nothing to do with this. The problem was the different name of flex binaries:
In CI servers the docs are frequency regenerated during build tests. Making the docs generated only during
There is nothing to build in the examples/ directory. For src/ and doc/, however, inter-directory dependencies would make just rebuilding a single directory of them a bad idea. If it was not these inter-directory dependencies, I would keep the makefiles as is and not suggest refactoring to non-recursive. |
You picked the current current naming. You could change it back. I suggest:
Ignoring determinism for now. The users should not be determining the LFLAGS used to build Flex itself.
That has been confusing. Thank you for defining your terms. "Determinism" carries a connotation that the steps to reach the end are important. You're more concerned that the end result is always the same. That's been difficult to reconcile with your goal of increasing parallelism.
Yep. That's what I'm saying you should have done instead of copying the binary into doc/. Not doing so was "using VPATH incorrectly."
Wrong. This works fine, since
Incorrect. The examples are in the examples/ directory. We direct users there to build and run the examples in the manual and when providing support.
Explain why this is a bad idea. |
I would not say the word "back" since the naming has been there when you made the PR about the bootstrap (#20).
Well, it's me who was addressing the problem on how to reach build parallelism and portability and the so called "determinism" at the same time. (And I almost got a way to do it.)
Which directory would you suggest to put the stage1flex object files in? You can use your proposed naming scheme here, for convenience.
Just to remind you that I didn't touch the makefile within the examples/manual subdirectory. It's still there, which is named "example/manual/Makefile.example" and same as before. |
Meh. I could run
Yes. This is what I intend. Also, I only intended for scan.c to go into the dist tarball. Seems that was malfunctioning. How do you have it working?
I don't put these objects in subdirectories. I use the automake/libtool prefixing to keep them all together in srcdir and builddir. That's how I avoided recursive Make modules up to this point.
Good. |
Well, I have explained why GNU make launches subdirectory makefiles one at the time, reducing parallelism to prevent inter-directory data contentions. And it invokes subdirectory makefiles in the order set by developers (in the SUBDIRS variable) and never out of order because the order dependencies are sometime not specified (developers know them but 'make' won't know them).
I can override the scan.c rule pregenerated by Automake, but this is not done yet. Notice that Automake generated "scan.l to scan.c" rule would gotta respect user's LFLAGS anyway. So I cannot make it ignore LFLAGS during the build. (The scan.c provided in distribution can only be used when LFLAGS is empty or
I have to make a correction in the last comment. Automake would kind of force me to put the stage1flex object (.o,.obj) files in the same directory as the source (.c) files. So I won't have that decision. And it becomes tricky to me, that Automake didn't document the exact scheme how the object file name prefixes are derived, so I might not be permitted to use the same executable file names (despite different directories) in Automake. Oh no...
I can add that, since the Makefile.am in that "examples/manual" directory go away, the example makefile can now occupy the main "Makefile" name without problem. |
This project was structured with all the executable and library files in the same directory. They're all handled by the same Makefile. Thus, Make is able to build the target objects in parallel. The tests are built after the main executables because it doesn't make any sense to build them sooner. Likewise, the documentation is built after the tests pass. You're arguing for parallelism where it isn't needed. I understand the argument for non-recursive make. But go re-read Recursive Make Considered Harmful and you'll see this project is already following those suggestions. Miller wasn't talking about recursion related to documentation and tests. He was arguing against separating libraries and functional code modules into directories build by recursive Makefiles. Flex doesn't do that.
Go read the Automake and Libtool manuals all the way through again. Specifying any of the _*FLAGS per-target variables overrides the user-defined versions of those variables. E.g. defining stage1flex_CFLAGS causes CFLAGS to go unapplied unless we specifically include it in the per-target definition. I don't think specifying target_CFLAGS overrides LFLAGS, but I need to check. @westes
LOL, you're just wrong. "So I won't have that decision." You're talking nonsense.
That's because the prefixes are a Libtool extension. I don't believe you've been reading the links I sent you. Read the Automake and Libtool manuals again.
|
|
Please, don't tell me to read the manual when I already did it. When I wrote this PR, I did a lot of build tests to make sure everything works within Automake, and sometimes I even rely on undocumented stuff in Automake. You won't see where the build breaks until you test them out yourself and analyze the generated makefiles. Here is one of them, if you're interested:
The doc/ and src/ directories are not completely separate in dependencies. The "help2man" part is the problem here. I mentioned in a previous comment (to @westes in particular), there is a data contention in flex build tree where two "make" instances can try to rebuild the same file at the same time, potentially breaking things. Whether it is about source files or documentation is irrelevant. As long as there is dependencies that resembles a diamond problem, the recursive makefiles would become a bad idea.
No. "target_CLAGS" only override AM_CLAGS, not CFLAGS. I even dumped out the makefiles to verify the fact. (An example: https://github.com/Explorer09/flex/actions/runs/9045080132/job/24854430365)
Did you look at the makefiles Automake generated for you? I can know what works and what doesn't. When I name the program "stage1flex/flex", and with a program specific CFLAGS (stage1flex_flex_CFLAGS), Automake generates object file names like this: "src/stage1flex_flex-*.o" Either in the src/ subdirectory of the top build directory, or in the top build directory itself (depending on "subdir-objects" automake option). I know this. The generated object files cannot relocate to an any directory of my choice, within Automake's scheme. |
Imagine this scenario:
The 03b5af1 change is only a partial solution to the inter-directory dependency problem like this. If a makefile in the doc/ directory can update something in the src/ directory, then the makefiles would not be "independent" as you might think when it come to unit tests. I hope this explains why. I didn't go non-recursive for the sake of it. And I know where non-recursive makefiles would not fit, as mentioned in a previous comment before. |
Respect gets respect. Throughout our discussions I've gotten the strong impression that you either don't strongly dislike Automake and Libtool or just don't write in them often. This PR reinforces that belief. I'll acknowledge that I feel a strong ownership for the current build system even though lots of people have contributed to it. So that's part of my reaction. Beyond that, your explanations for why you want to make this change don't seem valid. Making the Makefile marginally smaller; following patterns that we can follow in either build system; supporting a build sequence that you think is more correct. I trust you and respect your coding ability. I don't think these recent build system changes are better than what we already had. Being dismissive isn't helping.
What are you saying is wrong in this?
Not the point I'm making. Those directories are meant to be built sequentially. They don't fall into the "recursive" definition in Miller
Is this diagram what's happening now? flex.1 (via help2man) depends on the flex binary, not the other way around.
When did that behavior start? Please find the commit so we can understand the regression because I last cleaned that up when I got ESR's commits ready for merge. I think you posted a log of it before but I couldn't expand it for some reason. Try again?
In this tree, the subdirectory is relevant. Source and documentation builds should be sequential, not parallel, so diamond dependencies aren't every a problem. The timing is known before build.
Right, my mistake. So why did we ever need to worry about overriding LFLAGS in the first place? Unless it's in the bootstrap rules where I was overriding them specifically because we don't want the users changing our expectations of how the the new scan.c is made from scan.l?
Yes, I read them frequently when I'm working on the build system. That's one reason I prefer them to be scoped to the relevant subdirectories. Automake and Libtool generate those names during compilation, then Libtool demangles them at link. I like that. You clearly don't. It's a bad reason to throw one system out for another either way.
That's not the same place as the flex source files. Built sources and built objects being together in your builddir are fine. |
Changing the docs is a maintainer-level workflow. It's documented in README, INSTALL, and CONTRIBUTING. Assuming the user started from a dist tarball:
|
Can we have a discussion where we respect the work of each other? I just want to improve the flex build scheme of whatever it has right now, so I don't want to say anything about whether I like Automake or Libtool or not.
It's one of the build that I managed to make it work. And notice the eight build jobs there, testing the native build and cross build, with and without VPATH, and on different versions of Ubuntu distro. (Ubuntu 22.04 has an updated Autoconf, that uncovered a bug of ax_prog_cc_for_build, which I fixed in #649.)
Quote the definition from Miller, just for reference:
So as long as the top-level makefile invokes the makefiles subdirectory makefiles with another instance of And yes. The subdirectory makefiles are invoked in sequence specified by the developers. This doesn't invalidate my point:
Keep in mind the arrow in this diagram mean differently from the "bootstrap" diagram I given previously. Here, Thus I hope you understand what could happen: Two And also regarding the scenario I mentioned:
Saying it's a maintainer level workflow won't cut it. Maybe it's just me, but I wish the documentation can be rebuilt every time from a git repo checkout. That's also what CI is testing, after all. If the documentation is reproducible, it wouldn't hurt if users build it and build again and again, expecting the results be bit-for-bit identical to the original. (And yes I know this requires
The point is, scanners and parsers generated from ".l" and ".y" would always respect LFLAGS and YFLAGS during build. It's not maintainers who would rebuild the scanner and parser, but anyone who build from a checked-out git repository would gonna build them anyway, with the LFLAGS and YFLAGS. This assumption comes from Automake. It wouldn't make sense, for Flex during bootstrap, to not honor LFLAGS. And before you argue, I know what your vision is: Since you want scan.c to be included in the distribution, so that users don't need to rebuild the scanner C file (scan.c) and can skip the whole "stage1" process during normal build. My easy suggestion to this is making |
Not sure if you guys like discussing this further, but assuming the naming scheme like in this diagram, I have a few constraints:
|
I'd love to!
Understood. I'll look again.
Thanks! Miller uses "module" in the sense of C program modules, groups of source files compiled into a single object file. He doesn't talk about anything else. Flex's build system isn't recursive according to his paper because all the modules with common source code reside in the same source directory and are built by one Makefile at the same level. Building a system of projects loosely-related projects in the same tree doesn't meet his definition.
I think this diagram is still wrong given your explanation. As described it should be: (edited to fix the spacing here) This will never result in two instances of
If the diagram you provided is really what's happening there's a bug that needs attention right away. I will check.
This is what package maintainers have expected since I started writing OSS. It's a pain to build RPMS, DPKG, etc. specs from excessively opinionated source trees. Trying to maintain a packaged Flex is what got me started working on its build system. Docs are rebuilt every time from a git repo checkout, if you do the whole build the way we document it. That means:
The trouble with having users build the docs every time is the requirement that they install a TeX system. We got a lot of complaints about that. Everyone wants the PDF target but so few want to install the pre-reqs. I'm not opposed to having the docs be regenerated on every build if we can find a tiny, tiny TeX system to depend upon that's available everywhere.
See below.
We agree --disable-bootstrap being the better default. That's how it's supposed to work. I'll get that fixed. B But that's not my argument. It's incorrect for a compiler build process to honor the flags meant for users of that compiler. Flex is compiler for scanners. Flex's build process should not make special effort to honor Flex user flags. Whether we include scan.c in the dist or not, we should be in total control of the flags used to build it. |
No argument here. The source directory is writable when using our documented build commands for the dist tarball, so it hasn't been a problem.
Remind me what you mean by "bootstrap" again, please? I think we were close on this. I mean "regenerate scan.c from scan.l using a version of Flex built using the same version of scan.l (i.e. from stage1scan.c)." Using my definition, we only need to include scan.c and the "other C source files" in the dist tarball. Users who want to generate a .c from a .l need some version of Lex installed. That's the nature of bootstrapping. Unless we include something like a stage0scan.c that we almost never change and use that to build the bootstrap flex binary. I don't think that would be better than having users install Flex from their package managers.
(Reiterating my strong objection to allowing users to set LFLAGS while building Flex itself.)
I would love for the scan.c/stage2scan.c test to be part of |
The flex program has optional dependencies on We were already hitting the problems because of recursive makefiles that you might not be aware of. I know what scheme would be perfect for recursive makefiles, that would come with little or no dependency problem here: GCC source tree, with support libraries such as "libiberty", "libgcc", "libgfortran" individually configurable and buildable (i.e. they come with their own individial configure scripts). It would be still "recursive" as part of Miller's definition, but the dependencies would be "library-wide", not on individual source files. A quick way to analyze whether the problems described in Miller can appear is to read the makefile in each subdirectory, and see if there is any file dependency with "../" in its path (except for library-wide dependencies, e.g. "../foo.a", that can be converted to "
You missed my point. The real point is: We cannot expect user to run " In other words, drop support of this use case:
By removing doc/Makefile entirely.
Well, isn't "configure" able (or supposed) to detect whether user has "help2man" or "texi2any"? That way I can tweak the doc/Makefile.am to do this: if HAVE_HELP2MAN
flex.1: $(FLEX)
$(HELP2MAN) \
--name='$(PACKAGE_NAME)' \
--section=1 \
--source='The Flex Project' \
--manual='Programming' \
--output=$@ \
$(FLEX)
else
# See if the man page needs updating. If it does, print a friendly error.
flex.1: $(srcdir)/configure.ac $(srcdir)/src/cpp-flex.skl $(srcdir)/src/options.c $(srcdir)/src/options.h
@echo 'flex.1 man page is outdated; please install help2man and regenerate configure.'
@false
endif
It is the user who builds Flex, after all. When a compiler like gcc builds itself, it respects the user's CFLAGS, CXXFLAGS, and LDFLAGS as well.
Any scan.c from the source directory cannot be modified during "make" process. If it comes from dist tarball then it has to remain there (unless user performs "make maintainer-clean" to delete it). That's the constraint. We cannot say, ship scan.c within the dist tarball; allow user to overwrite it if he configures with "--enable-bootstrap". Nope, that won't work if user configures out of tree (i.e. VPATH build). With the aforementioned constraint, the best we could do is to not ship scan.c, but stage1scan.c only. |
Here is a summary of the three scanner C files:
|
The Makefile in lib/ only handled distribution. There was never a danger of compilation races. With that removed we're only getting distribution on lib/ through the CROSS version of libfl_SOURCES. That's awfully fragile.
Explain where. I keep asking and you haven't said.
This is rude.
This was irrelevant
You've changed the topic. We prescribe the command sequence users must run to consistently build Flex. It's in our INSTALL.md. Users can do anything with their source tree and we don't care. But we only promise we'll build consistently if they follow our instructions.
We already detect this in ./configure and issue warnings.
We're upstream maintainers. We are mainly supporting downstream package maintainers who do ship official binaries and call it a day. Doesn't seem like that with all the user support requests, but they're asking how to use Flex more than how to build it. Which of these defaults do you want users to override for their personal versions of Flex?
You haven't explained what you mean by "bootstrap". Please answer my question.
Like I said above. We prescribe the build commands that produce working binaries. Users can format their hard drives for all I care. It's not our problem to solve.
VPATH doesn't stop us from writing to srcdir. That's a distcheck constraint. It's not good practice, but we already mitigate it by delivering tarballs that won't require writing outside of the build directory. Again, we can't control users doing the Wrong Thing.
Repeating myself for emphasis. You are trying to solve users doing the Wrong Thing. This isn't our problem. We document the supported build process. We provide CI that follows the supported build process. Done. @westes |
I have an impression that you didn't take a look at the current source tree of Flex, and you made wrong assumptions. The lib/ directory of Flex source tree is NOT libfl. The directory contain two files that are compatibility functions that work like Gnulib. And I am now annoyed when you even get basic facts wrong. |
Explorer09
force-pushed
the
local-mk
branch
2 times, most recently
from
36a102a
to
4141ecf
Compare
Would you please open that as a separate issue? (Who does the pr is independent.) |
I think this will lead to no end of heartache. |
On the one hand, I'd love this to be the case as it makes it easier for most folks to build and use flex. On the other, those of us who routinely build from a clean git tree need it to not be the case during our normal operations. Ideally, the source distributions would be packaged with this as the default. I have not given any thought to doing this, however. |
|
With respect to the man page: As I understand it, the problem is that if cross-compiling, the built "flex" binary is built for a different system than the one being run on. Hence the mess with having to figure out what the name of the binary to run is. Also help2man doesn't have a way to say "run binary x but label the page y". |
I'm taking this for action. It's what I intended to happen all along. |
Agreed. Just meant to list this option, not endorse it. |
Sure |
Looking into this. Seems the man page is built during |
Help2man needs the binary to have the name that will appear in the man page. That was the catch requiring the install command. (Or building in a dedicated directory is another way to solve that problem.) |
|
I've added the last commit which shows a proof of concept of how a bootstrap can be done portably. And for the man pages, I made it so that both the stage1 program and the final binary to have the same file name to generate man page from. This is my ideal vision on how it can be done: 33d6725 |
I copy. |
Explorer09
force-pushed
the
local-mk
branch
3 times, most recently
from
4732d94
to
33d6725
Compare
How about this: check if |
|
That sounds correct so far as it goes. A good test case is if you get the correct result based on the current master branch. That new release is coming but you guys keep posting interesting questions/issues/comments on these pr's. So the current state of those macros in scan.c probably pushes the boundary of what your test is supposed to uncover and in exactly the right way. |
Group the files by "category", i.e. reasons for ignoring, and adjust some of the file name patterns.
This change is a prerequisite for converting src/Makefile.am for non-recursive use.
This change is a prerequisite for converting doc/Makefile.am for non-recursive use.
Avoid "target-specific variable values" (that looks like "target: variable=value") that works with GNU make only when building 'stage1flex'. A portable alternative is starting a sub-instance of 'make' with overridden variables (CC="$CC_FOR_BUILD" etc.). Make the bootstrap work even when EXEEXT and BUILD_EXEEXT might differ. The makefiles are designed that only the main instance of 'make' would handle the 'stage1bin/flex$(BUILD_EXEEXT)' so that no data contention can potentially occur.
Status of this branch
make distcheckpassed.Description
This converts the build system of flex to use non-recursive makefiles, except for "po" and "tests" directories which are technically too complex to convert. The main advantage of using non-recursive makefiles for Flex is that cross-directory dependencies can be handled better. The reduced total size of generated makefiles is a second advantage.
This is part of my rewrite of the bootstrap mechanism of Flex in order to make it portable to non-GNU "make".