Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 325145

Summary: aggregator editor inappropriately changes EOL character
Product: [Technology] CBI Reporter: David Williams <david_williams>
Component: CBI p2 Repository AggregatorAssignee: CBI Inbox <cbi-inbox>
Status: RESOLVED FIXED QA Contact: David Williams <david_williams>
Severity: normal    
Priority: P2 CC: Ed.Merks, ed, Kenn.Hussey, thomas
Version: 1.1.0   
Target Milestone: ---   
Hardware: PC   
OS: Windows 7   
Whiteboard:

Description David Williams CLA 2010-09-13 13:59:23 EDT
When ever I make a change change to a model file, such as helios.b3aggr, such that all "included" files are re-written, they _all_ show up as "changed", even though only one or two actually have different content. 

As far as I can tell this is because it is changing the EOL in all the files based on using the "OS platform" default (I am running on windows) but the project's CRLF setting is set to Unix. 

If I "manually" convert back to unix CRLF (by action on File menu) and do 'refresh' and resync, it seems to eventually figure out there's "no changes" to all those other not-changed files. 

But pretty inconvenient. 

(I'll admit, I have cvs team set to ignore whitespace, so not sure why its still showing up as "changed" ... but that seems to happen to me frequently and would be a different bug ... even if that wasn't the case, I'd still like the files I intend to change have the CRLF I've set for the project). 

I'm sure at its heart, this is some EMF serialize issue, but hopefully there's an option to allow the editor to control how its serialized?
Comment 1 Thomas Hallgren CLA 2011-07-09 05:34:38 EDT
Kenn, is there any way to control how EMF serializes EOL characters? If not, then I think this bug should be moved to EMF.
Comment 2 Kenn Hussey CLA 2011-07-20 09:03:19 EDT
(In reply to comment #1)
> Kenn, is there any way to control how EMF serializes EOL characters? If not,
> then I think this bug should be moved to EMF.

Hmm, I don't know but maybe Ed does.
Comment 3 Ed Merks CLA 2011-07-20 10:47:34 EDT
No, there's no such control.  I don't think a SAX parser generally lets you even know what convention the file you're parsing is using.  It always converts to \n. I imagine the only really useful control would be to reproduce the convention being use on the input (much like the encoding is recorded and reused).

It all sounds a bit like a self-induced problem (setting the project's convention) given that version control systems handle this kind of conversion issue all the time.

I'm not so inclined to add more complexity to the XML serializer given this hasn't been an issue for anyone else these past 10 years.
Comment 4 Thomas Hallgren CLA 2011-07-20 16:08:31 EDT
Closing this as invalid since there's nothing that can be done about this problem within the realms of b3.
Comment 5 David Williams CLA 2011-07-20 18:31:31 EDT
(In reply to comment #3)

Its not worth arguing about ... or development effort if there's not already a way to do it ... but did want to explain a bit more ... [inline]

> 
> It all sounds a bit like a self-induced problem (setting the project's
> convention) given that version control systems handle this kind of conversion
> issue all the time.

Setting or not project's value, I still see "changes" show up, when only EOL changes. It depends on what others people have their version-control settings set to, I think. If we could get perfect consistency on how others set up their workbenches ... maybe wouldn't be an issue ... but, that is the purpose of "project level" settings ... to not have to worry about individuals' workspace settings.   


> 
> I'm not so inclined to add more complexity to the XML serializer given this
> hasn't been an issue for anyone else these past 10 years.

I agree its not worth added complexity ... I was assuming there was an easy/standard way with modern xml writers to control the EOL character used on output (but, googling around, I couldn't find anything obviously standard). 

But, I suspect ... just guessing ... the reason it is a problem here, and not elsewhere is that normally a developer changes model, and checks in all related files, and others check out all files associated with model. I was trying not to change individuals' projects source files, if people happened to have different settings than me.

Just wanted to explain.
Comment 6 David Williams CLA 2013-01-19 11:25:02 EST
Hmmm, looks like some were finally convinced -- 2 years after I reported :)

See bug 388418. 

Perhaps the aggregator editor could be updated, at the right time, with a recent EMF?
Comment 8 Thomas Hallgren CLA 2013-01-21 01:30:26 EST
Thanks for connecting the dots David. I'll look into this.
Comment 9 David Williams CLA 2013-12-16 10:11:18 EST
I've lost track ... I thought this was fixed. 

Seems to still be causing problems for people. See bug 418708 and bug 424041 for some recent cases.
Comment 10 Ed Willink CLA 2013-12-16 13:38:00 EST
(In reply to Thomas Hallgren from comment #4)
> Closing this as invalid since there's nothing that can be done about this
> problem within the realms of b3.

This isn't true. EMF (now) provides a LINE_DELIMITER option for resource saving.

The problem is that as a standalone application, EMF cannot sensibly know about workspace or project preferences so the save options default is blank giving the Java default to Windows.

However for Eclipse applications, something is failing to bridge the gap between a workspace preference/project setting being available and it being used. In the absence of an agreed EMF bridge, each EMF application such as the aggregator must do it.

Similarly for standalone JUnit launches no one is propagating the Eclipse new line preference to the Java launch environment (and if you try to do it yourself you get in a \\\\n mess.)
Comment 11 David Williams CLA 2013-12-16 19:19:43 EST
> Similarly for standalone JUnit launches no one is propagating the Eclipse
> new line preference to the Java launch environment (and if you try to do it
> yourself you get in a \\\\n mess.)

Do you know if a bug has been opened on that problem? I tried searching but didn't see anything. 

Thanks,
Comment 12 Ed Merks CLA 2013-12-17 00:03:24 EST
FYI

As of EMF 2.9 there is support for org.eclipse.emf.ecore.resource.Resource.OPTION_LINE_DELIMITER which can be used to specify exactly which line delimiter your want to save or can use org.eclipse.emf.ecore.resource.Resource.OPTION_LINE_DELIMITER_UNSPECIFIED as that save option's value, in which case EMF will try to determine which line delimiter is appropriate, e.g., by looking at the line delimiter currently used in the existing file, or by looking at the workspace preferences for a new file at the specified workspace location. Failing all those, EMF will use the value returned by the line.separator system property.  I think Ed W is alluding to the problem that it seems impossible (we don't know how) to specify the value of this system property on the command line, i.e., how to encode control characters?

But as I mentioned in the other bugzilla, if proper autocrlf=true is specified for git checkout and for git commit, using windows line feed conventions should not be a problem.
Comment 13 David Williams CLA 2013-12-17 14:06:01 EST
(In reply to David Williams from comment #11)
> > Similarly for standalone JUnit launches no one is propagating the Eclipse
> > new line preference to the Java launch environment (and if you try to do it
> > yourself you get in a \\\\n mess.)
> 
> Do you know if a bug has been opened on that problem? ...

Just in case not, I've opened bug 424257 for that.
Comment 14 Ed Willink CLA 2013-12-18 11:52:10 EST
(In reply to David Williams from comment #13)
> (In reply to David Williams from comment #11)
> > > Similarly for standalone JUnit launches no one is propagating the Eclipse
> > > new line preference to the Java launch environment (and if you try to do it
> > > yourself you get in a \\\\n mess.)
> > 
> > Do you know if a bug has been opened on that problem? ...
> 
> Just in case not, I've opened bug 424257 for that.

Bug 424257 is yet another issue.

Bug 424352 raised for workspace preference propagation.

Bug 424353 raised for manual setting of a Unix line ending property.
Comment 15 David Williams CLA 2016-09-16 15:43:39 EDT
[Bookkeeping change only. Moving bugs to the new "home" of aggregator, CBI.]
Comment 16 Ed Merks CLA 2022-06-25 05:14:32 EDT
Content type associations were added:

https://github.com/eclipse-cbi/p2repo-aggregator/commit/e401af46eebcd906044883169e6dffa8fb8bce32