Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 353317 - protected code messed up
Summary: protected code messed up
Status: CLOSED FIXED
Alias: None
Product: Acceleo
Classification: Modeling
Component: Core (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows Vista
: P3 normal
Target Milestone: RC1   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-07-28 08:32 EDT by Martin Eisengardt CLA
Modified: 2016-04-04 05:38 EDT (History)
6 users (show)

See Also:


Attachments
Sample project to reproduce behavior (22.59 KB, application/x-zip-compressed)
2011-08-11 17:09 EDT, Markku Saarela CLA
no flags Details
Regression after 3.1.1 SR1 (67.74 KB, application/x-zip-compressed)
2011-09-27 14:36 EDT, Markku Saarela CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Eisengardt CLA 2011-07-28 08:32:04 EDT
Build Identifier: 20110615-0604

According to Bug 347122. It is still there. One of the scripts produces the ACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION rather than line feeds every time and messes up the protected code. However it is not the main script for me but some imported script.
If the same template that causes this bug is called elsewhere (from another main script) it works well.

relevant template snippet:

     * <!-- [protected ('field ' + aProperty.name + ' comment -->')]
     * 
     * <!-- [/protected] -->


Reproducible: Always
Comment 1 Markku Saarela CLA 2011-08-02 16:18:04 EDT
I have same problem with imported templates. 

Second time generation produces: 
Start of user code angleInDegreesACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION    /**ACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATIONACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION     *ACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION     * @returnACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION     *ACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION     */ACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION    @OverrideACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION    public Double angleInDegrees()   {ACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION        throw new IllegalStateException(Not yet implemented: angleInDegrees);ACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION    }ACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION// End of user code

Third generation produces lost file.
Comment 2 Stephane Begaudeau CLA 2011-08-11 08:07:53 EDT
What version of Acceleo are you using? (with its complete qualifier like 3.1.0v20110607-0602). Do you have a small working example showing the problem because I couldn't reproduce this issue?
Comment 3 Martin Eisengardt CLA 2011-08-11 08:19:53 EDT
I will try to produce a simple example tonight. However if I do not get a small working sample to reproduce the bug I would like to give you a working snapshot of our eclipse plugins/ mtl files and model files as well as the project sources to reproduce the problems. Please contact me directly via mep_eisen@web.de because you would need to sign a NDA.
But maybe Markku has a small sample to reproduce it.
I used the indigo release version as available in the repositories three or two weeks before. I do not use any newer version or snapshot version of acceleo.
Comment 4 Markku Saarela CLA 2011-08-11 17:09:22 EDT
Created attachment 201355 [details]
Sample project to reproduce behavior

This project reproduce bug behavior. Generated classes GeodeticAngle, Latitude, Longitude produces .lost files. Look generated Position.java code that is not messed up.
Comment 5 Markku Saarela CLA 2011-08-11 17:11:02 EDT
That project i was using to develop Maven 3 plugin to compile templates and generate code. I can contribute that code if you want.
Comment 6 Stephane Begaudeau CLA 2011-08-12 05:00:29 EDT
This sample project was very helpful and I've found how to fix the problem. There was a piece of code to handle this problem and I've improved it to handle this new case but I don't know why this new situation appear. I'll investigate this issue a bit more and I'll push the fix with new unit tests after that.
Comment 7 Martin Eisengardt CLA 2011-08-12 05:25:13 EDT
Is your fix available in cvs so that I can test if it solves my issue too?
Comment 8 Stephane Begaudeau CLA 2011-08-12 11:08:58 EDT
With 120 new unit tests for protected areas, I have found some (all ?) combinations that can produce this problem. I have contributed this first version of the fix on git: git://git.eclipse.org/gitroot/m2t/org.eclipse.acceleo.git and I'll put some other unit tests later.

I've launched a nightly build with the fix, it is available here: https://hudson.eclipse.org/hudson/job/m2t-acceleo-3.1/230/
(build artifact -> Acceleo.p2.repository for an update site of this specific nightly build)

>>> Is your fix available in cvs so that I can test if it solves my issue too?

We are not using CVS anymore, we have switched to git with the Indigo release two months ago, you can find the details of our git repository here: http://git.eclipse.org/c/m2t/org.eclipse.acceleo.git/

(we are using the branch m3_1_maintenance for this fix)
Comment 9 Martin Eisengardt CLA 2011-08-13 05:00:05 EDT
ACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION disapperead. But now the identation itself does not work. within for loops it seems to increment the identation. Generating multiple methods results in following code (example):

/**
 */
public void foo() 
{
}
    /**
     */
    public void bar();
        /**
         */
        public void baz();


I used the git snapshot from about one hour ago.
Comment 10 Martin Eisengardt CLA 2011-08-13 07:17:16 EDT
I think a found out the problem. AcceleoEvaluationContext.getCurrentLineIndentation does not work if there are mixtures of DOS and UNIX Line feeds. It will find some DOS linefeed from within all of the generated code. Do not ask me why we got mixing up dos and unix linefeeds in our project and mtl files. But from my point of view the method should not fail finding the very last line of the generated code even if line feeds are mixed.
Comment 11 Stephane Begaudeau CLA 2011-08-31 10:06:25 EDT
An improved version of the fix has been contributed on R3_1_maintenance and master, a new nightly build is available for Acceleo 3.1.1: https://hudson.eclipse.org/hudson/job/m2t-acceleo-3.1/238/ and one for Acceleo 3.2.0 : https://hudson.eclipse.org/hudson/job/m2t-acceleo-master/19/ with the new fix.
Comment 12 Martin Eisengardt CLA 2011-09-01 03:42:30 EDT
changing AbstractAcceleoWriter:write and using Math.max to find the very last line seems to work.


if (str.contains(DOS_LINE_SEPARATOR)) {
			newLineIndex = str.lastIndexOf(DOS_LINE_SEPARATOR) + DOS_LINE_SEPARATOR.length();
		}
		if (str.contains(UNIX_LINE_SEPARATOR)) {
			newLineIndex = Math.max(newLineIndex, str.lastIndexOf(UNIX_LINE_SEPARATOR)
					+ UNIX_LINE_SEPARATOR.length());
		}
		if (str.contains(MAC_LINE_SEPARATOR)) {
			newLineIndex = Math.max(newLineIndex, str.lastIndexOf(MAC_LINE_SEPARATOR)
					+ MAC_LINE_SEPARATOR.length());
		}
Comment 13 Stephane Begaudeau CLA 2011-09-01 04:11:00 EDT
Do you still have the problem of the indentation or the problem of the protected code indentation marker with the latest build? (without your fix)
Comment 14 Martin Eisengardt CLA 2011-09-01 05:35:04 EDT
As I said before we did not see the marker with the latest git :)
Comment 15 Stephane Begaudeau CLA 2011-09-01 05:56:12 EDT
(In reply to comment #14)
> As I said before we did not see the marker with the latest git :)

Yes but the fix created an indentation problem, the improved version should correct this complex problem.

Your fix seems to address a problem when a module contains DOS and UNIX line feeds but the indentation problem is mainly coming from the way acceleo fix the indentation after a template call. The previous fix was used a bit early and thus Acceleo tried to fix the indentation of the content of the protected area but it should not always do so. See this example:

[template public myTemplate(...)]
    [templateCall()/]
[/template]

[template templateCall(...)]
my content
[/template]

After the call to templateCall, the text "my content" should be "fixed" to have four spaces before it to generate "    my content" but if we are using this template:

[template templateCall(...)]
my content
[protected ('protected')]
my protected content
[/protected]
[/template]

The first generation will be fine:
"    my content
    start of user code protected
    my protected content
    end of user code"

But since the next generation will find this in the previously generated protected area:
"start of user code protected
    my protected content
    end of user code"

This code will replace the regular protected area:
[protected ('protected')]
my protected content
[/protected

But its indentation should not be "fixed" anymore after the call to "templaceCall" since it would push the indentation of the code generated by the protected area to the right. And the use case that I've presented here, is a very simple one with only one template call.
Comment 16 Martin Eisengardt CLA 2011-09-01 06:32:25 EDT
(In reply to comment #15)
> Your fix seems to address a problem when a module contains DOS and UNIX line
> feeds but the indentation problem is mainly coming from the way acceleo fix the
> indentation after a template call.
Exactly. I do not know why we have this mixing line feeds. It seems to be a problem with line feeds we get from uml annotations. They seem not to respect the workspace settings (both utf-8 and unix line feeds). However I think making acceleo more robust for mixed up line feeds is a good idea.
Comment 17 Markku Saarela CLA 2011-09-01 12:49:31 EDT
Acceleo 3.1.1RC2 (2011/08/31) works for me.
Comment 18 Stephane Begaudeau CLA 2011-09-26 03:58:53 EDT
Marking as resolve for 3.1.1 and 3.2.0
Comment 19 Markku Saarela CLA 2011-09-26 13:19:52 EDT
Unfortunately I start using Eclipse 3.7.1 and install 3.1.1 Acceleo and that sample project is again messed up with this protected code. Also my Acceleo Maven project is producing same result. GeodeticAngle generated code is messed up no others.
Comment 20 Stephane Begaudeau CLA 2011-09-27 03:19:47 EDT
Do you still have the problem with the latest nightly available here:
https://hudson.eclipse.org/hudson/job/m2t-acceleo-master/57/artifact/Acceleo.p2.repository/
(update site of one nightly build)

If yes, do you have a sample project reproducing the problem?
Comment 21 Markku Saarela CLA 2011-09-27 14:36:37 EDT
Created attachment 204113 [details]
Regression after 3.1.1 SR1

3.1.1 SR1 was working but not current update site 3.1.1 release
Now ested with this nighly build (job 57 was already deleted)
https://hudson.eclipse.org/hudson/job/m2t-acceleo-master/64/artifact/Acceleo.p2.repository/

And still GeodeticAngle class produces this GeodeticAngle.java.lost file. Other classes are generated correctly. This must be some special case.
Comment 22 Stephane Begaudeau CLA 2011-09-28 04:12:18 EDT
The new version of the fix has the good algorithm to handle the problem but it was missing a condition. After a template call we need to fix the indentation as I've explained it previously. The original problem was that we did not look recursively to see if our protected area was in a file block.

This would work:
[template public myTemplate(...)]
[file (...)]
[protected (...)][/protected]
[/file]
[/template]

This would create a problem:
[template public myTemplate(...)]
[file (...)]
[myTemplate2()/]
[/file]
[/template]

[template public myTemplate2(...)]
[protected (...)][/protected]
[/template]

With the latest fixes we are now handling the indentation of the protected area after having completed the evaluation of myTemplate2. We are now in a file block, we can take care of the indentation of the protected block. Still a problem would remain with blocks nested in a file block like this:

[template public myTemplate(...)]
[file (...)]
[if (true)]
[if (true)]
[myTemplate2()/]
[/if]
[/if]
[/file]
[/template]

The latest fix solved that by checking recursively inside file blocks. Yet, an "else" block is a bit different than the "if"/"for"/"let" blocks and your problem appeared because of "[if o.isAbstract ]...[else][o.operationBody()/][/if]"

I will add new unit tests today to recursively check else/elselet blocks.
Thanks for reporting the problem. The fix will be available in a new nightly today and it will be available in Acceleo 3.2.0 which will be released soon (we are already testing release candidates).
Comment 23 Stephane Begaudeau CLA 2011-09-28 11:30:17 EDT
The fix has been pushed on HEAD, it is available in the nightly #66: https://hudson.eclipse.org/hudson/job/m2t-acceleo-master/66/
Comment 24 Markku Saarela CLA 2011-09-28 13:57:22 EDT
This head build 66 is working. Thanks.
Comment 25 Sebastien GABEL CLA 2011-11-04 11:24:20 EDT
Unfortunately, I still have some problems in 3.1.2 release. It should remain some unhandled cases. I spent some hours to finally found my problematic case.
My module structure looks like : 
[template]
[for]
  [let]
     [file]
	[if]
           [template1() /] --> OK
	[else]
           [template2() /] --> all protected areas of my sub-(sub-)templates generate the ACCELEO_PROTECTED_AREA_MARKER_FIT_INDENTATION keyword.
        [/if]
     [/file]
  [/let]
[/for]
[/template]

As a workaround, I split my the [if][else][/if] block in two distinct [if] blocks.
Hope it helps !
Comment 26 Bouchet Stéphane CLA 2012-07-23 10:45:02 EDT
Hi,

this is still there using 3.1.3... 

my templates are using multiple [if] and [for] and the marker annotation is generated after regeneration. first generation is OK.
Comment 27 Bouchet Stéphane CLA 2012-07-24 09:53:00 EDT
Hi,

i've proposed a fix that works on my use case .

see https://github.com/eclipse/acceleo/pull/2

cheers,
Comment 28 Laurent Goubet CLA 2016-04-04 05:38:32 EDT
Not reproducible with the same kind of structure as outlined in comment #c25