Community
Participate
Working Groups
I'm opening this bug from observations made in bug 460458. There, to name one example, we observed "errors" in the "main log" for org.eclipse.ant.ui, but they were not in the @dot.xml log file (which, we use to "summarize" our compiler results). It appears to occur for cases where there are more than one 'jar' produced per bundle ... only the output from the last jar processed is in the @dot.xml file. I will attach a large zip that contain "the whole" project, as it exists after being built, but suspect you can see the issue by looking in the @dot.xml file (and of course, might want to look at the pom definition, or manifest.mf file, or build.properties -- the later, I believe, is what's used to "drive" the compiling into multiple jars). The other tidbit -- and feel free to tell me if there is a better way -- but, ... still don't think the @dot.xml file should be "stepped on" ... we do "configure" our compiler in the parent pom, and specify the log file explicitly as ${project.build.directory}/@dot.xml Or, in context, <configuration> <compilerArgument>${code.ignoredWarnings}</compilerArgument> <compilerArguments> <verbose /> <inlineJSR /> <enableJavadoc /> <encoding>${project.build.sourceEncoding}</encoding> <proceedOnError /> <log>${project.build.directory}/@dot.xml</log> </compilerArguments> <showWarnings>true</showWarnings> <excludeResources> <exclude>**/package.html</exclude> </excludeResources> </configuration> If we didn't "specify" it, where would it go? (and what format would be used?) Is that the problem? That we specify the log file? (and, if you say yes, I still think all output should go there, and not "start over" for each jar compiled ... right?) Thanks,
Created attachment 250984 [details] zip of org.eclipse.ant.ui project and complete built target/
Created attachment 250994 [details] Simple test case showing the issue This is a minimal test case to show the issue in a simple say. The parent pom is a little overly complex, just because I left it as close to the "eclipse platform parent pom" as possible ... in case there is some subtle interactions? But, otherwise, it is a simple, "stand alone" build of one bundle, that consist of 3 Java files. One Java file is to be compiled into "library1.jar", another into "library2.jar" and the last into the 'root' of the bundle (similar to the ant.ui case). If you unzip the file, and run mvn clean verify -X | tee mvnout.txt You can see there are 14 warnings in the output file (mvnout.txt). [You would see the same 14 if you imported the org.eclipse.releng.bug460487 project into PDE.] But, if you look under eclipse-releng-test-parent/org.eclipse.releng.bug460487/target you will see there is only the one @dot.xml file (as expected) but it contains only 3 WARNINGS ... the ones for only the MainClass (which is compiled last). The "library" compilation warnings are "lost". My hope is this simple test case can help you isolate the cause of the problem ... and, who knows, maybe even a fix! :) Thank you very much.
Marking as critical given the definition of critical usually includes "lost data" ... and, this is "lost data" to us.
I am not sure if Tycho can solve this issue, since Tycho "just" calls the JDT compiler with the -log argument for each of the jars that gets created. Take a look at JDT batch Main class [1]. There is a method called "setLog(String logFileName)". Within this method, the outputstreams for the log files are created and in all cases (xml log, text log file) the FileOutputStream is created with "append=false": new FileOutputStream(logFileName, false). So maybe JDT has to add an flag to indicate if the log should be appended or not. [1] https://git.eclipse.org/c/jdt/eclipse.jdt.core.git/tree/org.eclipse.jdt.core/batch/org/eclipse/jdt/internal/compiler/batch/Main.java
(In reply to Martin Schreiber from comment #4) > I am not sure if Tycho can solve this issue, since Tycho "just" calls the > JDT compiler with the -log argument for each of the jars that gets created. > > Take a look at JDT batch Main class [1]. There is a method called > "setLog(String logFileName)". Within this method, the outputstreams for the > log files are created and in all cases (xml log, text log file) the > FileOutputStream is created with "append=false": new > FileOutputStream(logFileName, false). > So maybe JDT has to add an flag to indicate if the log should be appended or > not. > > [1] > https://git.eclipse.org/c/jdt/eclipse.jdt.core.git/tree/org.eclipse.jdt.core/ > batch/org/eclipse/jdt/internal/compiler/batch/Main.java Yes, you may be right -- that it will take some changes/effort from both teams. But, wanted to be positive what Tycho needed, and how it operates, before "requesting a change in JDT". For example, my first thought, too, was, "oh, thy must be "calling it three times, with different "input" and "output" parameters each time. But ... on a whim, I tried using <log>${project.build.outputDirectory}/dot.xml</log> instead of the original <log>${project.build.directory}/@dot.xml</log> Since, according to https://www.eclipse.org/tycho/sitedocs/BuildProperties.html the "main" (.) classes to go to /target/classes and "nested jars" to go to target/nestedjar-classes Even though target/nestedjar-classes is not where we would really want the compiler outout, I thought it would be a ?good? test if the problem was that the compiler was being called 3 separate, independent times (for this example). But, nothing was in target/nestedjar-classes but there was -- just the one -- log in /target/classes. So ... I have a feeling it's more complicated than "3 independent calls" to compiler? And, if not really independent, i.e. Tycho is using just one location for log, perhaps it is Tycho that can remember not to "close" and "re-open" the log? Seems as easy as knowing when to use "append" when not? (naively, perhaps?) [Apologies in advance, for not being familiar with the Tycho code, and how it works.]
(In reply to Martin Schreiber from comment #4) This might well be the case. I will confirm after trying out David's testcase.
I think separate log file per output jar is a better approach.
Tycho is building the different jars in a loop of compiler calls [1]. What Tycho can do is to use the <log> as it is for a single jar bundle and to prefix the filename specified in <log> with the jar name (or something similar) for the mutiple output jar case. [1] Line 251ff: https://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-compiler-plugin/src/main/java/org/eclipse/tycho/compiler/AbstractOsgiCompilerMojo.java
PDE build used to create multiple output files: http://archive.eclipse.org/eclipse/downloads/drops/R-3.8-201206081200/compilelogs/plugins/org.eclipse.ant.ui_3.5.300.v20120523-1742/@dot.xml http://archive.eclipse.org/eclipse/downloads/drops/R-3.8-201206081200/compilelogs/plugins/org.eclipse.ant.ui_3.5.300.v20120523-1742/lib/antrunner.jar.bin.xml http://archive.eclipse.org/eclipse/downloads/drops/R-3.8-201206081200/compilelogs/plugins/org.eclipse.ant.ui_3.5.300.v20120523-1742/lib/remoteAnt.jar.bin.xml Prepending the JAR name to the main log file name sounds good. For the case where two JARs with the same name are built (in different directories), you have to add a sequence number until the name is unique.
[Just read Markus's reply, after writing this, but will post anyway ... since ... he is referring to "3.5" :) ... and as far as I know, PDE may have offered a choice? (though, doubt it). (In reply to Igor Fedorenko from comment #7) > I think separate log file per output jar is a better approach. And ... why is that? I'm fairly sure that PDE put "all in one", and our current "summarizing scripts" (again, fairly sure) make the assumption that all is in one. So, we'd have to change our summarizing scripts, if that is no longer the case. (In reply to Martin Schreiber from comment #8) > Tycho is building the different jars in a loop of compiler calls [1]. > > What Tycho can do is to use the <log> as it is for a single jar bundle and > to prefix the filename specified in <log> with the jar name (or something > similar) for the mutiple output jar case. > > [1] Line 251ff: > https://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-compiler- > plugin/src/main/java/org/eclipse/tycho/compiler/AbstractOsgiCompilerMojo.java I suspect jar name prefix is the least of all evils, if it could not all be in one file. I was going to suggest "easier to trace back" if use source folder ... but ... in some cases, I think multiple source folders go into one jar (and, not really sure what you are looping over). The "deluxe solution" might be to offer choices, such as *perhaps* a new attribute "append" (true or false) and/or offer other "variables", besides current, ${project.build.directory}, such as ${nested.output.jar} or ${nested.source.directory} (which would be names only, not absolute, so we could write <log>${project.build.directory}/${nested.output.jar}@dot.xml</log> [Just a thought, not a strong request.] [I emphasize "*perhaps*" have an "append" attribute, but, perhaps JDT could weigh in here, since it seems to me from the "doc" that comes with compiler, the output file should ALWAYS be appended to, if it exists, but I might be reading too much into it. But, it says (in "Help" documentation): -log <file> Specify a log file in which all output from the compiler will be dumped. This is really useful if you want to debug the batch compiler or get a file which contains all errors and warnings from a batch build. If the extension is .xml, the generated log will be an xml file. But, the "help" from the jdt.core itself is less wordy: -log <file> log to a file. If the file extension is '.xml', then the log will be a xml file. If it was "always appended to", would that make Tycho's job easier? Or ... more complicated?
(In reply to David Williams from comment #10) > > (In reply to Igor Fedorenko from comment #7) > > I think separate log file per output jar is a better approach. > > And ... why is that? I'm fairly sure that PDE put "all in one", and our > current "summarizing scripts" (again, fairly sure) make the assumption that > all is in one. So, we'd have to change our summarizing scripts, if that is > no longer the case. > Single log file makes it harder to see messages related to individual compiler invocations.
(In reply to David Williams from comment #10) > [I emphasize "*perhaps*" have an "append" attribute, but, perhaps JDT could > weigh in here, since it seems to me from the "doc" that comes with compiler, > the output file should ALWAYS be appended to, if it exists, but I might be > reading too much into it. But, it says (in "Help" documentation): > > -log <file> Agree the doc is not explicit. Anyway, it would not be difficult to provide another option to append. But we would still keep the default behavior unchanged.
The JDT compiler can't just append output from another compilation to an existing log. The log can only contain the output of a single batch compiler invocation. See https://www.eclipse.org/jdt/core/compiler_32_004.dtd Since an XML document can only contain a single root, we'd need a different output format. And I agree with Igor that putting unrelated compile logs into one file is not a good idea anyway.
And ... what ever is "easy and quick" is fine ... I should emphasize. In fact, if I may, I don't know if you all are "close" to have 23 done? But if not, would you consider a 22.1 for this fix? The reason I ask is some people are "making changes" to these nested jars these days ... for the first time in years (which is probably why it has gone unnoticed for so long) so ideally we could get a "quick fix". So we'd better be able to spot those times when a nested jar was incorrectly upgraded. If not "quick", then would appreciate knowing (rough) idea of when ... since might want/need to do some "complete log parsing" (if I even could ... and, I have not really looked if that's feasible on our end). Thanks everyone for the quick feedback and analysis.
So, to summarize, I think we are all in agreement that we can't just "append logs" (technically, could "the plain" case, but we use "xml" anyway, and to "append" XML output would add a *lot* of complication for little gain) and even appending the plain case is probably too complicated to read easily. I think we all agree that "prefixing" name of log file, with nested-jar-file name is fine (as suggested in comment 8). Not only fine ... but, good! :) *might* want to consider prefixing with "nested jar relative path and name" (for the case where jars of same name, were in different paths within the same bundle? ... else as Markus suggested, may have to keep track if there already was one of same name, and then use an additional numeral or something to distinguish ... but, path seems clearer. Not sure we even have such cases, but, my thinking it the path separator would be replaced with underscore, or hyphen). This differs a little from the way PDE used to do it ... see comment 9 ... and I'm fine, either way, use directories or munge relative path names -- which ever is easiest. The other thing to "track" is the original extension. That should stay consistent, since JDT will produce xml output if log file ends with "xml", otherwise, plain text. So if original <log> parameter was @dot.log, then nested jar files would be, for example, lib/remoteAnt.jar.bin.log. (or, lib_remoteAnt.jar.log). Not to complicate things, but since part of the "log name" is under user control, and part is not, if easier for Tycho, I think it would be satisfactory to not rely on <log> passed to compiler, but to have your own, new model element, say <useLog format="plain|xml" /> and that you would simply provide the name. I can not imagine a better place to put the logs, other than under ${project.build.directory} and don't think the exact name of "main" log file is that bit an issue ... could hard code '@dot' for that main name, use ".xml" for XML format, and ".log" for plain format. But, again, I only mention this in case it's easier and less error prone. In conclusion, I'll ask again, does the Tycho team think this can be fixed soon? And provided in a 0.22.1 release? For anyone curious as to "what we do" with these logs, we transform them to HTML, and display them with each build. For an example, see http://download.eclipse.org/eclipse/downloads/drops4/I20150224-0800/testResults.php#PluginsErrors for an example. This allows us to apply a "project wide" standard to shoot for, for removing certain types of warnings (and, of course, errors). So, while any ERRORS would be caught through other means, we are likely missing many warnings in our many "nested jars". Much appreciated.
I prepared a patch but I need to do more tests and write unit tests for it. My solution works that way: If <log> is @dot.xml: _Single JAR bundle: Use the <log> as it is ( "@dot.xml" ) _Multi JAR Bundle: Prefix it with the jar filename (e.g. "library.jar@dot.xml") _Mulit JAR Bundles with same jar names in different directories: Use jar name, followed by an index (e.g. "mylib.jar@dot.xml", "mylib.jar1@dot.xml", "mylib.jar2@dot.xml",...) So the sample project (attached to this issue) would have 3 log files: @dot.xml (the one from the "root" jar) library1.jar@dot.xml (for library1) library2.jar@dot.xml (for library2) If it makes more sense to inlcude the path of the jar in the filename (using "_" instead of the file separator) I can change that. Than the log files for the attached project would be: @dot.xml lib_library1.jar@dot.xml lib_library2.jar@dot.xml One additional quesiton, the <compilerArguments> config parameter is deprecated, <compilerArgs> should be used instead. I tried to test my change also with <compilerArgs> <arg>-log=${project.build.directory}/@dot.xml</arg> <compilerArgs> but than the compiler is complaining that the -log is not a known compiler argument. Any ideas? Regarding the release process: I am a newby in the Tycho project, so I cannot decide if we can do a path release.
(In reply to David Williams from comment #15) > So, while any ERRORS would be caught through other means, we are likely > missing many warnings in our many "nested jars". We're also missing optional problems that were set to error. And we not only miss problems in nested jars, but even worse, we're missing the problems of the main jar if it has any nested jars. (In reply to Martin Schreiber from comment #16) Sounds good, thanks. > If it makes more sense to inlcude the path of the jar in the filename (using > "_" instead of the file separator) I can change that. > Than the log files for the attached project would be: > > @dot.xml > lib_library1.jar@dot.xml > lib_library2.jar@dot.xml That would be even better, and it would nicely avoid the problem with conflicting names. > One additional quesiton, the <compilerArguments> config parameter is > deprecated, <compilerArgs> should be used instead. I tried to test my change > also with > > <compilerArgs> > <arg>-log=${project.build.directory}/@dot.xml</arg> > <compilerArgs> > > but than the compiler is complaining that the -log is not a known compiler > argument. Any ideas? The compiler expects two separate arguments, see http://help.eclipse.org/luna/topic/org.eclipse.jdt.doc.user/tasks/task-using_batch_compiler.htm I guess that would have to look like this in Tycho: <compilerArgs> <arg>-log</arg> <arg>${project.build.directory}/@dot.xml</arg> <compilerArgs>
(In reply to Markus Keller from comment #17) > The compiler expects two separate arguments, see > http://help.eclipse.org/luna/topic/org.eclipse.jdt.doc.user/tasks/task- > using_batch_compiler.htm > I guess that would have to look like this in Tycho: > > <compilerArgs> > <arg>-log</arg> > <arg>${project.build.directory}/@dot.xml</arg> > <compilerArgs> If it helps confirm this, I've changed by example to use <compilerArgs> as below, and appears to work as before: <configuration> <compilerArgs> <arg>${code.ignoredWarnings}</arg> <arg>-verbose</arg> <arg>-inlineJSR</arg> <arg>-enableJavadoc</arg> <arg>-encoding</arg> <arg>${project.build.sourceEncoding}</arg> <arg>-proceedOnError</arg> <arg>-log</arg> <arg>${project.build.directory}/@dot.xml</arg> </compilerArgs>
Patch submitted : https://git.eclipse.org/r/42641 <compilerArgs> <arg>-log</arg> <arg>${project.build.directory}/@dot.xml</arg> </compilerArgs> or <compilerArguments> <log>${project.build.directory}/@dot.xml</log> </compilerArguments> will produce with the example project the log files: /path/to/build/directory/@dot.xml /path/to/build/directory/lib_library1.jar_@dot.xml /path/to/build/directory/lib_library2.jar_@dot.xml
(In reply to Martin Schreiber from comment #19) > Patch submitted : https://git.eclipse.org/r/42641 Thanks Martin for all your work on this! (I was going to say "what a huge number of changes" but then realized most were for the test case itself :) So ... Tycho team (I know it's not your decision alone, Martin) ... if/when this is in a "snapshot", for either 0.23.0 or 0.22.1? let me know and we'll give it a real life test! (We do prefer having a "released" version for our milestone - - due in about 3 weeks, March 17, -- but ... this is "critical", so we'll do what's needed. Thanks again, and thanks in advance.
(In reply to comment #19) > Patch submitted : https://git.eclipse.org/r/42641 > <compilerArgs> > <arg>-log</arg> > <arg>${project.build.directory}/@dot.xml</arg> > </compilerArgs> > > or > > <compilerArguments> > <log>${project.build.directory}/@dot.xml</log> > </compilerArguments> Sorry for nagging, but IMHO this approach includes way to much magic. Currently compilerArgs is very simple: it sets compiler arguments. Instead, we should add a new mojo parameter (= "model element" from comment #15) which triggers that Tycho generates a distinct -log compiler argument for every call. And this parameter could even be enabled by default. From what I could see, setting -log only enables additional logging. It e.g. doesn't disable the output to the console log.
I think you already said that you don't need to be able to control the exact file names, so I'd propose to add a mojo parameter which only allows the specify the folder for the logs. This parameter could be called "logDirectory" and would e.g. default to "${project.build.directory}/compile-logs". For a "Bundle-ClassPath: ., library.jar, lib/library.jar", Tycho would then generate the log files target/compile-logs/@dot.xml, target/compile-logs/library.jar.xml, and target/compile-logs/lib_library.jar.xml (In reply to comment #9) > For the case > where two JARs with the same name are built (in different directories), you have > to add a sequence number until the name is unique. IMHO, including the path in the log file name should be enough. I'd rather not include a sequence number - at least until someone can provide a really good use case for a Bundle-ClassPath: lib_library.jar, lib/library.jar (In reply to comment #10) > (...) offer other "variables", besides > current, ${project.build.directory}, such as ${nested.output.jar} or > ${nested.source.directory} (which would be names only, not absolute, so we could > write > <log>${project.build.directory}/${nested.output.jar}@dot.xml</log> For someone knowing Maven, this would be surprising if this worked. The reason is that variable expansion is done by Maven on project level, and Maven plugins then only see expanded values. But there is no way to correctly expand ${nested.output.jar} on a project level.
(In reply to Tobias Oberlies from comment #22) > I think you already said that you don't need to be able to control the exact > file names, so I'd propose to add a mojo parameter which only allows the > specify the folder for the logs. This parameter could be called > "logDirectory" and would e.g. default to > "${project.build.directory}/compile-logs". > > For a "Bundle-ClassPath: ., library.jar, lib/library.jar", Tycho would then > generate the log files target/compile-logs/@dot.xml, > target/compile-logs/library.jar.xml, and > target/compile-logs/lib_library.jar.xml Hard to argue against, since I originally suggested it :) But, would emphasize that "format" (xml, or plain) is still important (for the community at large). JDT simply takes the file extension as the "key" to what's wanted. If it ends in xml, xml is produced. Anything else is "plain". Hence, I think conceptually cleaner if the "element" was called <useLog ... /> (or, produceProjectLog, or what ever). and for that element to have two attributes, format (xml or plain, I'd suggest "plain" be the default, and Tycho would automatically use ".log" ... though, and alternative, if the goal was to "line up" better with JDT, would be to have the attribute be called 'extension' and then people could specify "xml" or whatever else they might currently be using, txt, log, out, whatever. and then an attribute for logDirectory which would default to what you described, if not specified. > (In reply to comment #9) > > For the case > > where two JARs with the same name are built (in different directories), you have > > to add a sequence number until the name is unique. > IMHO, including the path in the log file name should be enough. I'd rather > not include a sequence number - at least until someone can provide a really > good use case for a Bundle-ClassPath: lib_library.jar, lib/library.jar Yes, I think "we" have already implicitly agreed, if path is included in name (with underscores), we don't know of any use-cases.
(In reply to Tobias Oberlies from comment #21) > ... From what I could see, > setting -log only enables additional logging. It e.g. doesn't disable the > output to the console log. Yes, thought I should point out that <showWarnings>true</showWarnings> controls compiler warnings being printed in the console log. My preference would be to leave that as is, and then *add* this new functionality, even if a little redundant -- if nothing else, can provide a sanity check that the two agree with each other? Not to mention, I'm sure some in community are "used to" having the <showWarnings>. I assume that is implemented differently as well ... as a "listener" of some sort? (i.e. not all the same information is printed, as when -log is specified to batch compiler, such as literal parameters, totals, access warnings, etc. Just wanted to be explicit.
(In reply to Tobias Oberlies from comment #22) Yes, I think that would work for the Eclipse build. I fully agree that the sequence number is not necessary if the JAR name is prepended to the log file name. > > <log>${project.build.directory}/${nested.output.jar}@dot.xml</log> > For someone knowing Maven, this would be surprising if this worked. I agree. If we wanted to use a "late-expansion" variable, then it would have to use a different format, e.g.: > > <log>${project.build.directory}/@dot.xml</log> That's actually what the Eclipse build currently uses. PDE build used to generate "@dot.xml" by default when building the "." JAR. If we now considered "@dot" to be the "magic late-expansion" token that will be replaced by the JAR name, then this would work perfectly out of the box for everybody who transitions from PDE build. And then we'd be back to the proposal from comment 19...
(In reply to David Williams from comment #23) > Hence, I think conceptually cleaner if the "element" was called > > <useLog ... /> (or, produceProjectLog, or what ever). > > and for that element to have two attributes, > format (xml or plain, I'd suggest "plain" be the default, and Tycho would > automatically use ".log" ... though, and alternative, if the goal was to > "line up" better with JDT, would be to have the attribute be called > 'extension' and then people could specify "xml" or whatever else they might > currently be using, txt, log, out, whatever. > > and then an attribute for > logDirectory > which would default to what you described, if not specified. > AFAIK attributes within parameters (e.g. <useLog format="xml" logDirectory="/path/" /> are not supported by maven properties, so we have to introduce 2 parameters, one for the format/extension, one for the log directory, e.g. <configuration> <logDirectory>${project.build.directory}/compile-logs</logDirectory> <logExtension>xml</logExtension> </configuration> I would suggest to not use a default for the logDirectory. If <logDirectory> is not specified, no -log will be passed to the compiler (as it is the default right now if no compiler args are used). If a <logDirectory> is specified, logs will be put into that directory using <logExtension> as extension ("log" as default). Example 1: <configuration> <logDirectory>/path/to/logs</logDirectory> </configuration> Will produce: /path/to/logs/@dot.log /path/to/logs/library.jar.log /path/to/logs/libPath_library.jar.log Example 2: <configuration> <logDirectory>/path/to/logs</logDirectory> <logExtension>xml</logExtension> </configuration> Will produce: /path/to/logs/@dot.xml /path/to/logs/library.jar.xml /path/to/logs/libPath_library.jar.xml
(In reply to Martin Schreiber from comment #26) > > AFAIK attributes within parameters (e.g. <useLog format="xml" > logDirectory="/path/" /> are not supported by maven properties, so we have > to introduce 2 parameters, one for the format/extension, one for the log > directory, e.g. You are probably right ... conceptually, then, would this be a case to have names with dots or hypens in them? Not sure I've ever seen any "naming conventions" ... but, going by some examples, perhaps <log.directory> and <log.format> would get the idea across? (That is, that they were two aspects of "the same thing"?) > I would suggest to not use a default for the logDirectory. If <logDirectory> > is not specified, no -log will be passed to the compiler (as it is the > default right now if no compiler args are used). Sounds right to me, to maintain consistency of current behavior. And, to extend that "consistency" thought ... I'm assuming the implementation would be such that anyone currently using -log in compiler arguments, could continue to ... if they had no nested jars. And only in the case where <log.directory> specified would that "override" anything specified in -log as a compiler argument? (Or, else, should have some "error checking", and not sure that's easy to do?) I've sort of lost track ... but, is this <log configuration specific to "JDT" or "any compiler that supports it"? (just curious). > > Example 1: > <configuration> > <logDirectory>/path/to/logs</logDirectory> > </configuration> > > Will produce: > /path/to/logs/@dot.log > /path/to/logs/library.jar.log > /path/to/logs/libPath_library.jar.log > > Example 2: > <configuration> > <logDirectory>/path/to/logs</logDirectory> > <logExtension>xml</logExtension> > </configuration> > > Will produce: > /path/to/logs/@dot.xml > /path/to/logs/library.jar.xml > /path/to/logs/libPath_library.jar.xml Looks right to me.
(In reply to comment #26) > I would suggest to not use a default for the logDirectory. If <logDirectory> is > not specified, no -log will be passed to the compiler (as it is the default > right now if no compiler args are used). > If a <logDirectory> is specified, logs will be put into that directory using > <logExtension> as extension ("log" as default). The advantage of setting <logDirectory> by default is that we could define the folder name that then probably 99% of all builds will use. If we don't set a default, we'd get more variation, which potentially makes it harder to understand a foreign Tycho build. The same could be achieved with a third switch (logEnabled, in addition to logDirectory and logFormat/Extension), but this extra flexibility seems unnecessary. What are your concerns with enabling log output by default? Obviously, we'd need to make sure sure that this new default does not overwrite an explicit -log configuration in the compilerArgs or compilerArguments.
(In reply to Tobias Oberlies from comment #28) > (In reply to comment #26) > > I would suggest to not use a default for the logDirectory. If <logDirectory> is > > not specified, no -log will be passed to the compiler (as it is the default > > right now if no compiler args are used). > > If a <logDirectory> is specified, logs will be put into that directory using > > <logExtension> as extension ("log" as default). > > The advantage of setting <logDirectory> by default is that we could define > the folder name that then probably 99% of all builds will use. If we don't > set a default, we'd get more variation, which potentially makes it harder to > understand a foreign Tycho build. > > The same could be achieved with a third switch (logEnabled, in addition to > logDirectory and logFormat/Extension), but this extra flexibility seems > unnecessary. > > What are your concerns with enabling log output by default? Obviously, we'd > need to make sure sure that this new default does not overwrite an explicit > -log configuration in the compilerArgs or compilerArguments. I thought that currently if no compiler argument "-log" is specified, the compiler will not log into a file, just to sysout (not sure about that) and having no default value would keep that behaviour, but I am also fine with a default vaule. Then the compiler will produce a log file from now on in all cases (with a logDirectory specified and without). The logic would then be: If "-log" is in the compiler args, do not change anything If not use the logDirectory parameter + log file name as -log argument.
(In reply to comment #27) > You are probably right ... conceptually, then, would this be a case to have > names with dots or hypens in them? Not sure I've ever seen any "naming > conventions" ... but, going by some examples, perhaps > <log.directory> and <log.format> would get the idea across? (That is, that they > were two aspects of "the same thing"?) I haven't seen any mojo parameter in the form "log.directory" or "log-directory". Probably because it doesn't work: Maven by default maps parameter names to identical variable names in the mojo class. But if it worked (without complicating the code), I wouldn't object to the name with the dot. > I've sort of lost track ... but, is this <log configuration specific to "JDT" or > "any compiler that supports it"? (just curious). "javac -help -X" only reveals a "-Xstdout <filename>", so I suppose that -log is JDT specific. But this doesn't matter: using the tycho-compiler-plugin with other compilers than JDT is anyway a rather exotic use case.
(In reply to comment #29) > Then the compiler will produce a log file from now on in all > cases (with a logDirectory specified and without). Hm, I had in mind that setting logDirectory to an empty string would override the default - but it doesn't. There should be a way to disable the log, so I don't like this solution (although I proposed it...). Thank you Martin for pointing out this limitation. So back to the three parameters: * logDirectory * logFormat or logExtension (BTW: We could use "logFormat" and still use it as file name extension - just with "plain" substituted by "log". Then logFormat=plain and logFormat=xml would work, and we'd still be open for new formats in case JDT would decide to add support for them.) * logEnabled, or logToDisk, or logToFile, or writeLogFile (the last one would have the disadvantage that it wouldn't be next to the other parameters in the plugin documentation). With the extra parameter to enable the log, I would also leave it disabled by default. (My motivation was to have a default value for logDirectory, cf. comment #28). If we disable the generated -log argument by default, we could revise the precedence between compilerArgs and the new switch. I have a small precedence to have the new switch win (or fail the build in case of a collision), to not complicate its documentation too much.
You could have <logDirectory> with a default value and <logExtension> without a default. Then, users could enable logging by specifying a <logExtension> (file name postfix), and if they don't set a <logDirectory>, they will get their logs in the default location. The implementation needs to use a special file name prefix for the "." JAR. To match PDE build, it should use "@dot" in that case. File names like "lib_library1.jar@dot.xml" don't make much sense. Just use "lib_library1.xml". I don't think it's worth adding more magic like a special "plain" <logExtension> that gets replaced by ".log".
(In reply to comment #32) > I don't think it's worth adding more magic like a special "plain" <logExtension> > that gets replaced by ".log". I'm not too keen on the term logExtension because it makes me think of an "extender". logFile(s)Extension would be better, but also feels somewhat awkward. Plus, I'd prefer that Tycho users don't need to know that JDT determines its log format by the extension of the log file name. IMHO this is an implementation detail of JDT - it could also be that JDT has a -format parameter and this implied the file extension. So I'd rather have the parameter named after the primary concern, and this is the format.
(In reply to Tobias Oberlies from comment #31) > So back to the three parameters: > * logDirectory > * logFormat or logExtension (BTW: We could use "logFormat" and still use it > as file name extension - just with "plain" substituted by "log". Then > logFormat=plain and logFormat=xml would work, and we'd still be open for new > formats in case JDT would decide to add support for them.) > * logEnabled, or logToDisk, or logToFile, or writeLogFile (the last one > would have the disadvantage that it wouldn't be next to the other parameters > in the plugin documentation). Pushed another Patchset (https://git.eclipse.org/r/#/c/42641/) that does work as Tobias mentioned. Example configuration: <configuration> <logEnabled>true</logEnabled> <logDirectory>${project.build.directory}/logs</logDirectory> <logFormat>xml</logFormat> </configuration> will produce (for the example project attached): ${project.build.directory}/logs/@dot.xml ${project.build.directory}/logs/lib_library1.jar.xml ${project.build.directory}/logs/lib_library2.jar.xml If this configuration is set AND -log is specified in the compiler arguments, the new stuff wins.
(In reply to Martin Schreiber from comment #34) > If this configuration is set AND -log is specified in the compiler > arguments, the new stuff wins. Sounds good to me. First time I've seen <logEnabled> (which is fine). I'm wondering if that was related to "defaults" for directory? Just curious. Assume it will all be documented and in any case would satisfy what we need.
(In reply to David Williams from comment #35) > Sounds good to me. First time I've seen <logEnabled> (which is fine). I'm > wondering if that was related to "defaults" for directory? Just curious. > Assume it will all be documented and in any case would satisfy what we need. The alternative of having 3 parameters is having 2 <logDirectory> and <logFormat> and use the presents of one of these to enable the logging. E.g. if no <logDirectory> is specified, do not log, if one is specified use that and do persist the logs. And yes, whatever the final solution is, it will be documented.
(In reply to Martin Schreiber from comment #36) > The alternative of having 3 parameters is having 2 <logDirectory> and > <logFormat> and use the presents of one of these to enable the logging. E.g. > if no <logDirectory> is specified, do not log, if one is specified use that > and do persist the logs. I'd really like to have a default value for <logDirectory> so we can't use this parameter. But you are right that we could use <logFormat>: * logFormat unset -> no logging * logFormat=plain or logFormat=xml -> logging enabled
I'd call this parameter 'log', not 'logFormat' <configuration> <log>plain</log> <configuration>
(In reply to Tobias Oberlies from comment #37) > > But you are right that we could use <logFormat>: > * logFormat unset -> no logging > * logFormat=plain or logFormat=xml -> logging enabled With patchset 3 (https://git.eclipse.org/r/42641) the compiler mojo only has 2 parameters, <logDirectory> and <log>. <log> will enable logging (xml or plain), <logDirectory> the directory of the logs (default: ${project.build.directory}/compile-logs). The rest is the same as in patch set 2.
(In reply to Martin Schreiber from comment #39) I still see logEnabled in patch set 3 (in the code, but not in the commit message). (In reply to Tobias Oberlies from comment #33) > Plus, I'd prefer that Tycho users don't need to know that JDT determines its > log format by the extension of the log file name. IMHO this is an > implementation detail of JDT - it could also be that JDT has a -format > parameter and this implied the file extension. So I'd rather have the > parameter named after the primary concern, and this is the format. The whole -log parameter is an implementation detail of the JDT formatter. It would be better to just make this transparent, so that e.g. clients that used to generate *.txt logs in PDE build can continue to do so. PDE build had a "logExtension" property (default value: ".log"; the Eclipse build used ".xml").
<configuration> <log>xml</log> <logDirectory>path/to/log/</logDirectory> </configuration> will enable logging into the specified directory. Example log filenames: path/to/log/@dot.xml path/to/log/lib_mylibrary.jar.xml path/to/log/lib_myOtherLibrary.jar.xml Merged with Commit: https://git.eclipse.org/c/tycho/org.eclipse.tycho.git/commit/?id=983d89e0a1bcc3ace56de2df62dc25b3da80d205
Ok, now that it's "fixed", can you answer our question if you can put this in a 22.1 release? And/or remind me of your process? (Such as when this would be in a "snapshot" build? Is it better to ask about this on "tycho-dev"? Can you also explain your rationale for changing from "critical" to "major"? I mentioned my rationale for marking as "critical": this is "lost data", to us. Thanks,
(In reply to David Williams from comment #42) > Ok, now that it's "fixed", can you answer our question if you can put this > in a 22.1 release? And/or remind me of your process? (Such as when this > would be in a "snapshot" build? Is it better to ask about this on > "tycho-dev"? > > Can you also explain your rationale for changing from "critical" to "major"? > I mentioned my rationale for marking as "critical": this is "lost data", to > us. > > Thanks, David, the commit is already in the snapshot build. You can use the tycho snapshot by specifying the tycho snapshot repo as plugin repository in your pom: <pluginRepositories> <pluginRepository> <id>tycho-snapshots</id> <url>https://repo.eclipse.org/content/repositories/tycho-snapshots/</url> </pluginRepository> </pluginRepositories> Regarding a path release (22.1) please ask in the tycho-dev group, I cannot answer this.
*** Bug 460458 has been marked as a duplicate of this bug. ***
(In reply to Martin Schreiber from comment #43) > > Can you also explain your rationale for changing from "critical" to "major"? > > I mentioned my rationale for marking as "critical": this is "lost data", to > > us. > David, the commit is already in the snapshot build. You can use the tycho > snapshot by specifying the tycho snapshot repo as plugin repository in your > pom: .... Thanks Martin. I did give it a try today, but ran into other issues (unrelated to this bug) trying to use 0.23.0-snapshot (SWT compile problems, as documented in bug 461427 and bug 453446).
Confirming this appears to work exactly right. While a relatively temporary URL, you can see the *.jar.html files, in addition to the @dot.html files in http://download.eclipse.org/eclipse/downloads/drops4/I20150316-2000/testResults.php#PluginsErrors (The html is a simple conversion-for-display of the xml files produced by compiler). Thanks,
