javacc requires itself to build, bootstrap issue #40
Comments
|
??? That's what we do. We use the previous version to built the current
version. I don't understand what the issue is.
…
It seems javacc is using its own solution to build itself. This is not
really ideal as it creates a chicken and egg situation requiring use
of a pre-built binary. Ideally there should be a clean solution to
build from source. This maybe possible going back to an older version
and chaining up from there. I have no problem with javacc using javacc
for non-core aspects. But the initial build should not require javacc
,and then you can run what you just built to process any *.javacc
files. Thank you for consideration of this request.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#40>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/ATQdrU6mVpfEr3-9o7r4DRAoJWf6AKUJks5tABF6gaJpZM4RA8_A>.
|
|
There is no clean way to build from source without requiring a previously built binary. Which at that point, what is the point in building from source? If you have to use a pre-built binary. Might as well just run that binary. It is not just javacc, many java projects seem to have circular dependencies, or worse a custom language that they use for the very project like javacc. Kotlin is a horrible new example of this. I really hope someday it ends. It is not a good trend or software development process. |
|
Not sure what you are talking about here. This is bootstrapping 101. We
built a small core subset of javacc first and then bootstrapped using it
to build the next version. That's pretty well-understood development
paradigm.
… There is no clean way to build from source without requiring a
previously built binary
<https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-java/javacc/javacc-9999.ebuild#L51>.
Which at that point, what is the point in building from source? If you
have to use a pre-built binary. Might as well just run that binary.
It is not just javacc, many java projects seem to have circular
dependencies, or worse a custom language that they use for the very
project like javacc. Kotlin is a horrible new example of this. I
really hope someday it ends. It is not a good trend or software
development process.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#40 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ATQdrfE__lFd1BljqybeKExZksC2iym2ks5tACD3gaJpZM4RA8_A>.
|
|
The bootstrapping concept needs to end and die is what I am saying. Software needs to stop requiring bootstrapping, IMHO. It is not a well understood paradigm, if anything it maybe a Java specific paradigm. I rarely see this in other languages. Short of a core/base system, in Unix, glibc, gcc, binutils, etc. The bootstrapping paradigm needs to end. It is not good. It makes it almost impossible to fully audit code.... I guess in Java who cares what is in the binary you are running right? No need to audit that for security, modifications, etc. What is the point of open source if you are required to use pre-built binaries to build open source code? Why build from source at all then? |
|
A least in the area of compilers - it is pretty basic. I'm sure you need
gcc to build n old version of gcc. Check out the following thread:
https://stackoverflow.com/questions/9429491/how-are-gcc-g-bootstrapped
This has nothing to do with java. I do agree that it's not an easy
concept for many people but it's very effective way to do certain things.
… The bootstrapping concept needs to end and die is what I am saying.
Software needs to stop requiring bootstrapping, IMHO. It is not a well
understood paradigm, if anything it maybe a Java specific paradigm. I
rarely see this in other languages. Short of a core/base system, in
Unix, glibc, gcc, binutils, etc. The bootstrapping paradigm needs to
end. It is not good. It makes it almost impossible to fully audit code....
I guess in Java who cares what is in the binary you are running right?
No need to audit that for security, modifications, etc. What is the
point of open source if you are required to use pre-built binaries to
build open source code? Why build from source at all then?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#40 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ATQdrUQfcm5t1hQ3uf3VI1kJe7cRgEDGks5tACYzgaJpZM4RA8_A>.
|
|
That is core stuff for obvious reasons. Javacc does not have to use itself to build itself. Those javacc files could be coded in regular java. Such that it could build entirely from source. Or at least the core, and then use what was built to process any javacc files. IMHO it seems people are to eager to use their new shiny syntax, rather than Java code which it runs on. There is no need like in gcc. This is not like we are talking about the JDK itself. Which the java JDK can be 100% built from source going back to 1.5, using gcj, and gnu-classpath. So the gcc argument does not even apply to Java. If Java can be built 100% from source. Anything coded in Java should be able to do the same. This is a choice of the project developers. It is not a hard requirement like with gcc. Take antlr 2.7.7 as an example. That can build 100% from source. Then antlr 3.x requires 2.7.7 to build. Then you can use 3.x to build 4.x. There is just another stringtemplate there that causes circular issues/bootstraping. There are really a few Java projects out of hundreds that do this. I would like to see the trend and habit stop. Kotlin is a horrible new example coded almost entirely in Kotlin.... JRuby, Groovy, Jython and others are not as bad. Even Groovy can bootstrap itself without requiring a groovy binary. See me using a groovyc binary I just made, thus the relative path. That is what I am suggesting for javacc, and other projects as well. I am bringing this issue up with a number of projects. I packaged hundreds of things, mostly Java but not all. This invent a language on top of another and require that to bootstrap is very much a Java specific thing. I cannot believe the amount of stuff that generates Java code. It is tremendous. I have yet to run into anything along those lines for other stuff. Why people want to code in Java not using Java is another paradigm I will likely never understand. Rather than inventing a new language from scratch like Go or other. |
|
Of course javacc is a "compiler" that compiles javacc to Java code. It
will be stupid to hand-code any improvements to javacc syntax in each
version. Your argument holds zero value for JavaCC.
… That is core stuff for obvious reasons. Javacc does not have to use
itself to build itself. Those javacc files could be coded in regular
java. Such that it could build entirely from source. Or at least the
core, and then use what was built to process any javacc files.
IMHO it seems people are to eager to use their new shiny syntax,
rather than Java code which it runs on. There is no need like in gcc.
This is not like we are talking about the JDK itself. Which the java
JDK can be 100% built from source going back to 1.5, using gcj, and
gnu-classpath. So the gcc argument does not even apply to Java. If
Java can be built 100% from source. Anything coded in Java should be
able to do the same.
This is a choice of the project developers. It is not a hard
requirement like with gcc. Take antlr 2.7.7 as an example. That can
build 100% from source. Then antlr 3.x requires 2.7.7 to build. Then
you can use 3.x to build 4.x. There is just another stringtemplate
there that causes circular issues/bootstraping.
There are really a few Java projects out of hundreds that do this. I
would like to see the trend and habit stop. Kotlin is a horrible new
example coded almost entirely in Kotlin.... JRuby, Groovy, Jython and
others are not as bad. Even Groovy can bootstrap itself without
requiring a groovy binary. See me using a groovyc binary I just made
<https://github.com/Obsidian-StudiosInc/os-xtoo/blob/master/dev-java/groovy/groovy-9999.ebuild#L133>,
thus the relative path. That is what I am suggesting for javacc, and
other projects as well. I am bringing this issue up with a number of
projects.
I packaged hundreds of things, mostly Java but not all. This invent a
language on top of another and require that to bootstrap is very much
a Java specific thing. I cannot believe the amount of stuff that
generates Java code. It is tremendous. I have yet to run into anything
along those lines for other stuff. Why people want to code in Java not
using Java is another paradigm I will likely never understand. Rather
than inventing a new language from scratch like Go or other.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#40 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ATQdrWG95wojlfk1r4uBCS59m5V9bZ2uks5tACkrgaJpZM4RA8_A>.
|
|
Again javacc does not have to use javacc code to build itself. JDK/javac is a compiler that can be built from source. So your saying it would be impossible to build Javacc using 100% pure java code? I do not believe that to be the case at all. That seems to be more a deficiency of authors than the Java language. Which Java can build itself from source. I find it really hard to understand anything coded in Java cannot be built from source without requiring a binary to bootstrap. That seems like a bad design. Bad engineering. It is a development choice, not a requirement. Nothing says Javacc has to use javacc to build itself. That is a choice of the developers. It was not even possible till the first version of javacc was built. Even GCC, something else has to build that first. Before you could use gcc to build future gcc. All things have an origin. I am showing other things beyond jdk that can build from source, like antlr. There maybe some older version of javacc that is the same way. If I can figure out what ever version that is, I can chain up the versions to build the latest. I guess that is not something widely known. As javacc has likely bootstrapped since a very early version. A bad habit carried on for years. Javacc generates Java code. There is nothing saying that same java code could not be written, and/or committed to the project as Java code. I am not sure I even understand the benefit of generating such code as part of a compile. It does not change. Thus it could be generated and committed as part of the projects code for any tag/release just the same. Multiple ways to go about it. Again that is only a requirement because someone chose to code part of javacc using javacc compiler syntax/code. I find it hard to believe that had to be coded in javacc, and could not have been coded in pure Java. Its a choice, not a requirement. There is a difference. Again take it how ever you will. But its not helping to make things any better.It just leads to things like Kotlin. It needs to end and stop. If you do not want to help make things better and just continue to propagate the old bad habits. That is each projects choice. It is a CHOICE, not a requirement. Given that fact I would hope people to be more open minded and find better solutions. Again going back to why people even make compilers for other syntax and stuff on top of Java. That I will likely never understand nor like. I come across way to much of it. It seems like people are being lazy or something. I see no reason why that code has to be generated using the various tools. Only in rare cases like say apache poi converting office xml formats and what not into Java code. That makes sense. Most others thing do not. |
|
@wltjr - I wish you good luck in your crusade against bootstrapping. We all need our pet peeves! |
|
@matozoid Thanks :) But not really a crusade. Just going to bring it up and let upstreams decide. They have a choice to make software development better and friendlier to others or not. Seems most only care about their own needs not others. I am myself a selfish FOSS developer. But I would not ignore or not care about the needs of others using my stuff. I see that as a sign of being receptive to others. If I did not want others to use it, only for my needs, I do not need to make it open source or release to the world. Once I do, I should consider the request and needs of others. The bootstrapping concept in general I would think most all to want to avoid vs embrace. I have no idea why this has become so acceptable in Java where it is really not in most everything else. it is also interesting some of this is related to colleges, like antlr/stringtemplate, and java_cup/jflex. Which both also have circular bootstrapping issues. But if people really do not want to code in Java. Why they create a new language, syntax, compiler etc on top of Java does not make much sense either. Seems its to let people be lazy. Most any generated Java code could be hand coded just the same. What saves a developer time can cost all others. That is something I take into consideration. I rather cost myself more time once, and save all others time countless times. To much of software development has become the complete opposite Anyway to each their own. I just try to use common sense, and plea to others to consider needs beyond their own. What makes your life better, may make it worse on others. Just hope more developers keep that in mind :) |
|
To conclude, if javacc could consider one of the following approaches. That would be greatly appreciated!
Or other options, those 3 were just the obvious ones. Any would help considerably. Thank you for your consideration and time :) |
|
1 is a nonstarter. 2 is doable. Not sure about 3. I don't even understand the proposal.On Dec 13, 2017 1:03 PM, William L Thomson Jr <notifications@github.com> wrote:To conclude, if javacc could consider one of the following approaches. That would be greatly appreciated!
Modify code so that javacc is not required to build javacc.Generate files from javacc as part of release, like is done on maven sources. Not sure how feasible to do in repo with direct tarball download. If I use maven sources its not an issue. If the tarball can be made similar to that. Rather use direct sources from upstream/github than maven.Move anything that needs javacc to build to non core, so the main core javacc compiler can be built, and then use that to generate any needed code. Like is done with groovy.
Or other options, those 3 were just the obvious ones. Any would help considerably. Thank you for your consideration and time :)
—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.
|
|
FYI 1 and 3 are part of the recommended solutions to the chicken and egg problem Javacc is already coded in Java mostly. That it requires javacc for itself seems to go against recommended solution to the problem created. Why it starts with saying use language X for Y. Which is what Groovy and others are doing. Even early antlr and stringtemplate. Just in updates got lost. In 1 there is no javacc code, in 3 java code builds javacc so that non core parts can be coded using javacc format/syntax to be "compiled" by javacc. 2 is just a idea based on how maven sources are generated. That is an option for tarballs, but would not be for anyone checking out live sources. Which 1 and 3 would be better solutions for live sources. Not to mention always bad idea to have binaries in source control. Though it is nice that is provided for bootstrapping purposes. For other packages, usually grab a jar of the same version as the tarball to use the jar for bootstrapping. But that can cause other problems when the jar was made with a different version of Java. This requires a class file to be recompiled as the one in the jar has issues under Java 8. Hopefully most stuff is > 1.6, or will not even work in Java 9. Just some of the various issues that can arise from using pre-built binaries. Even for bootstrapping there can be issues. |
|
Hello William
Could you explain in which way using a javacc previous version as a bootstrap in the building process prevents you from fully audit the javacc newly built version (security, modifications…)?
When you audit a source code version you do not audit the developer or the IDE or the generator that wrote the code, don’t you? You do not try to foresee what could be the next version of that source code and try to make audit prophesies on it?
Marc
De : William L Thomson Jr [mailto:notifications@github.com]
Envoyé : mercredi 13 décembre 2017 20:32
À : javacc/javacc <javacc@noreply.github.com>
Cc : Subscribed <subscribed@noreply.github.com>
Objet : Re: [javacc/javacc] javacc requires itself to build, bootstrap issue (#40)
The bootstrapping concept needs to end and die is what I am saying. Software needs to stop requiring bootstrapping, IMHO. It is not a well understood paradigm, if anything it maybe a Java specific paradigm. I rarely see this in other languages. Short of a core/base system, in Unix, glibc, gcc, binutils, etc. The bootstrapping paradigm needs to end. It is not good. It makes it almost impossible to fully audit code....
I guess in Java who cares what is in the binary you are running right? No need to audit that for security, modifications, etc. What is the point of open source if you are required to use pre-built binaries to build open source code? Why build from source at all then?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub<#40 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AWGYi5TJxa7n5kgjYr8llS6ifhrKkLXsks5tACYzgaJpZM4RA8_A>.
|
|
@MarcMazas if there was a version of javacc I could build and then use that to bootstrap javacc I would be fine. There is no way to audit a binary jar. Which seems going back many versions there is no clean from source. I am working on this for other stuff, like jflex and javacup. I have found a old version of javacup, 10k that does not require jflex. Which I can use to then build new javacup 11b. Which then I use an older jflex 1.4.3 that has a pre-generated jflex file. Such that bootstrapping from that version is not required but I can use it for newer jflex. Just realized they are also providing a generated cup file so I likely do not even need javacup at all for jflex 1.4.3. Generated by the very version of javacup I am using 10k. But I also need javacup 10k. to build javacup 11b. That shows 2 different project and how they have addressed the chicken and egg situation. Now later versions of jflex >=1.5 no longer ships those pre-generated cups or jflex files. But at least a version can be used without any binaries. Same for javacup. Which 10k does not require any binaries to build. Which provides a clean path from source to build. Albeit through some older versions. They build and run fine under 9. Less an issue with jflex. Though seems more an issue with older jflex files or something. Like qdox 2 builds fine, but qdox 1.x has an error. The same one I get when using jflex 1.4.3 or 1.6.1 to build jflex. Which is ironic it has errors generating its own code :P |
|
I'm pretty sure we never released that very first seed version as we did
it when we were working at Sun Micro and by the time we open sourced it,
it was already bootstrapped. And that's the only version that's
available in the open source.
… @MarcMazas <https://github.com/marcmazas> if there was a version of
javacc I could build and then use that to bootstrap javacc I would be
fine. There is no way to audit a binary jar. Which seems going back
many versions there is no clean from source. I am working on this for
other stuff, like jflex and javacup
<jflex-de/jflex#271 (comment)>.
I have found a old version of javacup, 10k that does not require
jflex. Which I can use to then build jflex 1.4.3. Which then I use an
older jflex 1.4.3 that has a pre-generated jflex file
<https://github.com/jflex-de/jflex/blob/release_1_4_3/jflex/src/JFlex/LexScan.java>.
Such that bootstrapping from that version is not required but I can
use it for newer. Just realized they are also providing a generated
cup
<https://github.com/jflex-de/jflex/blob/release_1_4_3/jflex/src/JFlex/LexParse.java>
file so I likely do not even need javacup at all for jflex 1.4.3.
Generated by the very version of javacup I am using 10k. But I also
need javacup 10k. to build javacup 11b. That shows 2 different project
and how they have addressed the chicken and egg situation.
Now later versions of jflex >=1.5 no longer ships those pre-generated
cups or jflex files. But at least a version can be used without any
binaries. Same for javacup. Which 10k does not require any binaries to
build. Which provides a clean path from source to build. Albeit
through some older versions. They build and run fine under 9. Less an
issue with jflex. Though seems more an issue with older jflex files or
something. Like qdox 2 builds fine, but qdox 1.x has an error. The
same one I get when using jflex 1.4.1 or 1.6.1 to build jflex. Which
is ironic it has errors generating its own code :P
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#40 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ATQdrWZWDB2-z2w-N4gnP6sqgh-9ASJ1ks5tALyYgaJpZM4RA8_A>.
|
|
I kind of like the idea of having it generated and committed to vc between tags/released. Like anytime the javacc source files change, then run javacc against them. That helps out considerable and eliminates the need for any past versions. Then you can do straight javac, and you have a current javacc to do with as you desire :) That is done for maven repo sources, just not other tarballs, or committed to vc. I think ideal is committed to vc that way live building works as well, checkout from git, hack, etc. Anyway no worries. There are others far worse. Least javacc just needs itself. Others like jflex and cups are not to bad. antlr and stringtemplate seem to be the worst. More so given their relation. Though someday still have to address kotlin, presently preventing progress on spring 5, though never finished deps for spring 4. I was working on maven from source and got stuck on some modello/plexus stuff that seems to only be used by maven to build maven... Less that maybe able to build maven from source without requiring maven for bootstrap as many have said and project itself says as means to build. Not sure about gradle, but seems mostly in groovy and I can bootstrap groovy. Lots of bootstrapping.... :) Thanks for being receptive and sorry for all the noise. |
|
Well, in that case things look simple: I do not see why using the javacc previous version instead of using that old “bootstrap.jar” version would not work. The test should not be difficult ☺.
I do this for JTB, which is a superset of JavaCC. The only annoying thing is that when I change the (JTB) grammar, I must check that the generated parser works well and one good way is to try to generated the “next” version and check it works, and if it does not invalidate the build.
De : William L Thomson Jr [mailto:notifications@github.com]
Envoyé : jeudi 14 décembre 2017 07:14
À : javacc/javacc <javacc@noreply.github.com>
Cc : MAZAS Marc <marc.mazas@soprasteria.com>; Mention <mention@noreply.github.com>
Objet : Re: [javacc/javacc] javacc requires itself to build, bootstrap issue (#40)
@MarcMazas<https://github.com/marcmazas> if there was a version of javacc I could build and then use that to bootstrap javacc I would be fine. There is no way to audit a binary jar. Which seems going back many versions there is no clean from source. I am working on this for other stuff, like jflex and javacup<jflex-de/jflex#271 (comment)>. I have found a old version of javacup, 10k that does not require jflex. Which I can use to then build jflex 1.4.3. Which then I use an older jflex 1.4.3 that has a pre-generated jflex file<https://github.com/jflex-de/jflex/blob/release_1_4_3/jflex/src/JFlex/LexScan.java>. Such that bootstrapping from that version is not required but I can use it for newer. Just realized they are also providing a generated cup<https://github.com/jflex-de/jflex/blob/release_1_4_3/jflex/src/JFlex/LexParse.java> file so I likely do not even need javacup at all for jflex 1.4.3. Generated by the very version of javacup I am using 10k. But I also need javacup 10k. to build javacup 11b. That shows 2 different project and how they have addressed the chicken and egg situation.
Now later versions of jflex >=1.5 no longer ships those pre-generated cups or jflex files. But at least a version can be used without any binaries. Same for javacup. Which 10k does not require any binaries to build. Which provides a clean path from source to build. Albeit through some older versions. They build and run fine under 9. Less an issue with jflex. Though seems more an issue with older jflex files or something. Like qdox 2 builds fine, but qdox 1.x has an error. The same one I get when using jflex 1.4.1 or 1.6.1 to build jflex. Which is ironic it has errors generating its own code :P
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#40 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AWGYi1cPQ4NXfNyT6DS-Lig2bT7Pp2gZks5tALyYgaJpZM4RA8_A>.
|
|
@MarcMazas that is what I am doing now but its not really ideal. It is some what nice that the needed bootstrap jar is within the sources. Normally dislike bundled binaries. Though when it is from the same upstream its a bit different. Still no way to see what is in that binary, just have to run it blindly and hope for the best :) I know that is an extreme example. I doubt its doing anything nefarious. Can't get much worse than most mobile devices these days... Its more the point that its not ideal in open source to just run others binaries. More so to generate a file that likely does not really change. Not like the result changes, maybe with a different version of javacc, or jdk. Though even then the result maybe the same. |
|
By the way, there is a bunch of methods and classes generated by JavaCC that are not used by JavaC on parsing a grammar and generating a parser. Some look like old debug methods (dump* in SemanticChecker, LexGen. UnicodeConverter). The tool UCDetector helps finding them. For unused public classes it is little more difficult to point them (ExpansionTreeWalker, TreeWalkerOp, JavaFiles, LookaheadCalc, LookaheadWalk, MatchInfo, MetaParseException, OtherFilesGen, OutputFile, ParseEngine, ParseGen, Semanticize, in JavaCC 5.0). |
|
That is good to know. I just generate what ever is needed to build. Likely could be optional parts not needed. Mostly needed as other projects have their own javacc source files. Not sure many compile against javacc, just use the compiler. Just use this to build other stuff :) |
It seems javacc is using its own solution to build itself. This is not really ideal as it creates a chicken and egg situation requiring use of a pre-built binary. Ideally there should be a clean solution to build from source. This maybe possible going back to an older version and chaining up from there. I have no problem with javacc using javacc for non-core aspects. But the initial build should not require javacc ,and then you can run what you just built to process any *.javacc files. Thank you for consideration of this request.
The text was updated successfully, but these errors were encountered: