Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 321444

Summary:	some bash script doesn't seem to work after build machine update
Product:	Community	Reporter:	David Williams <david_williams>
Component:	Servers	Assignee:	Eclipse Webmaster <webmaster>
Status:	RESOLVED NOT_ECLIPSE	QA Contact:
Severity:	normal
Priority:	P3	CC:	ccc, pwebster
Version:	unspecified
Target Milestone:	---
Hardware:	PC
OS:	Windows 7
Whiteboard:

Description David Williams

2010-07-31 21:56:41 EDT

This is probably related to the recent upgrade of OS on build server.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=302613
(since the same script worked the day before, but not after the upgrade).

After upgrade, I have a bash script that is failing while trying to do a regex function, namely
if [[ "$projectname" =~ "(.*)-(.*)-(.*)" ]]
(just doesn't parse ... not sure if at all, or what).

I thought it might be related the the shebang at top of file. I currently use
#!/usr/bin/env bash
instead of
#!/bin/bash
as I used to, because #!/bin/bash didn't work on a ubuntu machine I sometimes test on so was trying to make a "portable" version by using
#!/usr/bin/env bash.

And, I've noticed lately, the default shell script "title" is now "committer_shell" instead of the "bash" that it used to be (and, that changed a few days ago).

So, tried changing back to #!/bin/bash ... no help.

Then I tried changing to #!/bin/sh (just guessing from some blind internet searches I did) and it got a little further, but then failed elsewhere with a message that made it sound like either bash had change in new OS (which seems unlikely) or my scripts are no longer being interpreted by bash, but something else. The subsequent error message was
rsync-retry.sh: line 66: `rsync-retry': not a valid identifier

While I'm sure I can "brute force" fixes to the scripts ... I thought I should open this bug, to document the impact, and see if there is something that needs to be tweaked in the new OS, or if you have a recommendation for what shebang to use.

Thanks,

Comment 1 David Williams

2010-08-01 13:40:09 EDT

I think this has nothing to do with whatever the shebang is set to, (though, I would still like to know why $SHELL is set to /usr/local/bin/committer_shell (it's confusing, but I'm just curious ... as long as it is "bash" :) ... which I see it currently is ... set as a symlink pointing to /bin/bash ). 

what I have learned about the regex problem is that there seems to be some subtle change in "interpreting" the lines of code. In short, if I change

if [[ "$projectname" =~ "(.*)-(.*)-(.*)" ]]
to 
if [[ "$projectname" =~ (.*)-(.*)-(.*) ]]
(that is, dropped the quotes surrounding the regex pattern.  

then the regex expression works as before.

I checked, and at least on my red hat system that expression, with no quotes, still works fine. 

So, I'm not sure exactly what's changed to cause this, or if its a bug, or fixing a bug :) ... but, I didn't just dream up the quotes. In my trusty Bash Cookbook (page 123) they show quotes in their examples. 

Whoa ... I just checked on my Ubutu system, and there too I can not have the quotes to have that regex expression work ... so, guess not a bug. (Guess I've never executed "promote" on my Ubuntu system ... would have no reason to). 

Given the above, I think this can be closed as closed as 'invalid' but will leave open for a while, to foster discussion, if there is to be any. Though even if technically an invalid bug, it is certainly a valid impact of the upgrade. But since Ubuntu works the same way, I can't say its a bug in OS or that any thing needs to be changed. Hmm, not sure how to have an actual space in the regex expression ... but, guess that's a general user question, not a bug here. 

Thanks,

Comment 2 David Williams

2010-08-01 15:19:20 EDT

I did find some references that mention this change in bash 3.2.2
For example, 
http://tldp.org/LDP/abs/html/bashver3.html 
mentions its, gives several reference links, and concludes that you can revert to previous behavior using 
shopt -s compat31. 

(As far as I know, there's not reason to set shopt for this for the whole system, unless you get tons of failures due to this ... I myself only have a few scripts to update ... though you might list it in a "FAQ" for the upgrade, if you ever make one?) My redhat system actual uses bash greater than 3.2.2 ... but, guess they set the default to previous behavior?) 



<quote>

The =~ Regular Expression match operator no longer requires quoting of the pattern within [[ ... ]].

Caution	

In fact, quoting in this context is not advisable as it may cause regex evaluation to fail. Chet Ramey states in the Bash FAQ that quoting explicitly disables regex evaluation. See also the Ubuntu Bug List and Wikinerds on Bash syntax.

Setting shopt -s compat31 in a script causes reversion to the original behavior.
</quote>

Comment 3 David Williams

2010-08-01 16:07:19 EDT

Just for closure ... the best technical reference I can find for this issue is the in Chet Ramey's Bash FAQ, at 
ftp://ftp.cwru.edu/pub/bash/FAQ

Too long to quote, but see question E14: 
E14) Why does quoting the pattern argument to the regular expression matching
     conditional operator (=~) cause matching to stop working?

The advice there, to handle space in patterns, it is best to use an env variable, where you can quote and escape to your hearts content, and it still gets used it regex operator as one string, as usual. So, for example, the best way to right these types of expressions would be similar to 

PPAT="(.*)\ (.*)\ (.*)"
if [[ "${projectname}" =~ $PPAT ]] 
(if I expected spaces instead of hyphens in hypothetical "projectname") 

Good luck,

Comment 4 David Williams

2010-08-01 16:14:40 EDT

I think "not eclipse" is best resolution for this entry. 

I hope the information helps others transition. 

Thanks,

Comment 5 Denis Roy

2010-08-03 13:16:03 EDT

regexp in bash is nasty ..but this is good to know.  I have a few instances of this on various servers.

Thanks for the docs.