Comment 1 Kenn Hussey 2016-03-14 12:29:47 EDT

This is preventing me from producing an integration build for UML2 5.2.0. See related bug 481202 and bug 486050.

Comment 2 Mikaël Barbero 2016-03-15 04:30:38 EDT

I've re-done the same fix as previously, i.e. re-created mdt/uml2 folder. This time, I've changed the owner/group from genie.uml2:common to khussey:modeling.mdt.uml2 (i.e. a pattern similar to xtext and they don't have the same issue of "disappearing" folder). 

Let me know once you've successfully build and sign, I will have a look at the folder and check whether it has been removed or not. 

Matt, Denis, do we have some scripts that "clean" /shared/download-staging.priv from time to time?

Comment 3 Kenn Hussey 2016-03-15 10:09:56 EDT

Thanks, Mikael, but now I'm getting a different error:

ERROR: org.eclipse.core.runtime.CoreException: /opt/public/hipp/homes/genie.uml2/.eclipse/800847737/configuration/org.eclipse.osgi/bundles/25/1/.cp/org/eclipse/buckminster/jarprocessor/antscript/signing.ant:209: Failed to copy /home/hudson/genie.uml2/.hudson/jobs/mdt-uml2-master/workspace/buildroot/buckminster.temp/org.eclipse.uml2.build_1.0.0-eclipse.feature/temp/site_2007570495.zip to /shared/download-staging.priv/modeling/mdt/uml2/site_2007570495.zip due to java.io.FileNotFoundException /shared/download-staging.priv/modeling/mdt/uml2/site_2007570495.zip (Permission denied)
org.eclipse.core.runtime.CoreException: /opt/public/hipp/homes/genie.uml2/.eclipse/800847737/configuration/org.eclipse.osgi/bundles/25/1/.cp/org/eclipse/buckminster/jarprocessor/antscript/signing.ant:209: Failed to copy /home/hudson/genie.uml2/.hudson/jobs/mdt-uml2-master/workspace/buildroot/buckminster.temp/org.eclipse.uml2.build_1.0.0-eclipse.feature/temp/site_2007570495.zip to /shared/download-staging.priv/modeling/mdt/uml2/site_2007570495.zip due to java.io.FileNotFoundException /shared/download-staging.priv/modeling/mdt/uml2/site_2007570495.zip (Permission denied)
	at org.eclipse.buckminster.ant.AntRunner.handleInvocationTargetException(AntRunner.java:167)
	at org.eclipse.buckminster.ant.AntRunner.run(AntRunner.java:322)
	at org.eclipse.buckminster.ant.actor.AntActor.internalPerform(AntActor.java:254)
	at org.eclipse.buckminster.core.actor.AbstractActor.perform(AbstractActor.java:195)
	at org.eclipse.buckminster.core.internal.actor.PerformManager$DirectActionInvocation.execute(PerformManager.java:143)
	at org.eclipse.buckminster.core.internal.actor.PerformManager.internalPerform(PerformManager.java:454)
	at org.eclipse.buckminster.core.internal.actor.PerformManager.perform(PerformManager.java:293)
	at org.eclipse.buckminster.core.internal.actor.PerformManager.perform(PerformManager.java:305)
	at org.eclipse.buckminster.core.commands.Perform.internalRun(Perform.java:108)
	at org.eclipse.buckminster.core.commands.WorkspaceCommand.run(WorkspaceCommand.java:91)
	at org.eclipse.buckminster.cmdline.AbstractCommand.basicRun(AbstractCommand.java:200)
	at org.eclipse.buckminster.cmdline.Headless.run(Headless.java:350)
	at org.eclipse.buckminster.cmdline.Headless.run(Headless.java:145)
	at org.eclipse.buckminster.cmdline.Headless.start(Headless.java:165)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1438)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1414)
Caused by: /opt/public/hipp/homes/genie.uml2/.eclipse/800847737/configuration/org.eclipse.osgi/bundles/25/1/.cp/org/eclipse/buckminster/jarprocessor/antscript/signing.ant:209: Failed to copy /home/hudson/genie.uml2/.hudson/jobs/mdt-uml2-master/workspace/buildroot/buckminster.temp/org.eclipse.uml2.build_1.0.0-eclipse.feature/temp/site_2007570495.zip to /shared/download-staging.priv/modeling/mdt/uml2/site_2007570495.zip due to java.io.FileNotFoundException /shared/download-staging.priv/modeling/mdt/uml2/site_2007570495.zip (Permission denied)
	at org.apache.tools.ant.taskdefs.Copy.doFileOperations(Copy.java:914)
	at org.apache.tools.ant.taskdefs.Copy.execute(Copy.java:567)
	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
	at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
	at org.apache.tools.ant.Task.perform(Task.java:348)
	at org.apache.tools.ant.Target.execute(Target.java:392)
	at org.apache.tools.ant.Target.performTasks(Target.java:413)
	at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
	at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
	at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
	at org.eclipse.ant.internal.core.ant.EclipseDefaultExecutor.executeTargets(EclipseDefaultExecutor.java:32)
	at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
	at org.eclipse.ant.internal.core.ant.InternalAntRunner.run(InternalAntRunner.java:672)
	at org.eclipse.ant.internal.core.ant.InternalAntRunner.run(InternalAntRunner.java:498)
	at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.buckminster.ant.AntRunner.run(AntRunner.java:318)
	... 25 more
Caused by: java.io.FileNotFoundException: /shared/download-staging.priv/modeling/mdt/uml2/site_2007570495.zip (Permission denied)
	at java.io.FileOutputStream.open(Native Method)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:194)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:145)
	at org.apache.tools.ant.util.ResourceUtils.copyResource(ResourceUtils.java:523)
	at org.apache.tools.ant.util.FileUtils.copyFile(FileUtils.java:559)
	at org.apache.tools.ant.taskdefs.Copy.doFileOperations(Copy.java:899)
	... 45 more
/opt/public/hipp/homes/genie.uml2/.eclipse/800847737/configuration/org.eclipse.osgi/bundles/25/1/.cp/org/eclipse/buckminster/jarprocessor/antscript/signing.ant:209: Failed to copy /home/hudson/genie.uml2/.hudson/jobs/mdt-uml2-master/workspace/buildroot/buckminster.temp/org.eclipse.uml2.build_1.0.0-eclipse.feature/temp/site_2007570495.zip to /shared/download-staging.priv/modeling/mdt/uml2/site_2007570495.zip due to java.io.FileNotFoundException /shared/download-staging.priv/modeling/mdt/uml2/site_2007570495.zip (Permission denied)
Caused by: Recording test results
Archiving artifacts
Finished: FAILURE

Comment 4 Denis Roy 2016-03-15 10:39:35 EDT

Kenn, the HIPP system account wasn't in your project group. I've added it, but I think Hudson needs a restart.  Can you restart it from the HIPP Control at https://dev.eclipse.org/site_login and let me know if that works?

Comment 5 Mikaël Barbero 2016-03-15 10:56:00 EDT

My bad, I forgot to add genie.uml2 (the hudson owner) to the modeling.mdt.uml2 group. It was normal that he could not write to the folder. It should be ok now.

Comment 6 Mikaël Barbero 2016-03-15 10:56:28 EDT

I've already restarted the hipp manually ;)

Comment 7 Kenn Hussey 2016-03-15 13:46:41 EDT

Thanks, guys. Unfortunately, by build is still failing, albeit for a different reason now:

[ant] Queueing site_2085534926.zip for signing
[ant] Result: 1
[ant] Buildfile: /shared/download-staging.priv/modeling/mdt/uml2/site_2085534926.zip.ant.xml
[ant] 
[ant] doIt:
[ant] 
[ant] BUILD FAILED[addPackProperties] verbose logging is enabled
[ant] 
[ant] [addPackProperties] filenameonly: site_2085534926
[ant] [addPackProperties] Finding jars already signed/shared/download-staging.priv/modeling/mdt/uml2/site_2085534926.zip.ant.xml:9: Could not create temporary working directory: /site_2085534926
[ant] 
[ant] 
[ant] [addPackProperties] destinationdirectory: /site_2085534926/Total time: 0 seconds
[ant] Waiting for signing to complete. This may take more then 60 minutes
Build timed out (after 67 minutes). Marking the build as failed.
Recording test results
Archiving artifacts
Finished: FAILURE

Comment 8 Denis Roy 2016-03-15 14:11:35 EDT

> signed/shared/download-staging.priv/modeling/mdt/uml2/site_2085534926.zip.
> ant.xml:9: Could not create temporary working directory: /site_2085534926

That would make sense, since you're trying to create a temp directory directly under the root dir. If you have admin access, make sure the expected temp dir variables are set correctly.

Comment 9 Kenn Hussey 2016-03-15 14:29:12 EDT

Do you mean that my job needs to be updated somehow? This same job has been working all along, so I'm surprised I'd have to change it to get it to work. Did something change somewhere?

Comment 10 Kenn Hussey 2016-03-15 15:15:30 EDT

As far as I can tell, the temp root has always been set to /home/hudson/genie.uml2/.hudson/jobs/mdt-uml2-master/workspace/buckminster.temp. Are the right permissions in place to access that directory?

Comment 11 Denis Roy 2016-03-15 15:29:36 EDT

Sorry Kenn, I think one of our init scripts is broken. I have a patch under review. If you need to build RFN I can set you up temporarily. Just say the word.

Comment 12 Kenn Hussey 2016-03-15 15:43:34 EDT

I'd like to post a build no later than tomorrow, so that there's an opportunity to get feedback before Monday's M6 build.

Comment 13 Denis Roy 2016-03-15 16:06:43 EDT

I've set the JAVA_TMPDIR variable manually in the HIPP config, and your build is running now:

https://hudson.eclipse.org/uml2/job/mdt-uml2-master/249/console

Comment 14 Kenn Hussey 2016-03-15 17:11:17 EDT

Thanks, Denis, that did the trick. I've deleted that build and the one that failed before it, and kicked off a new integration build.

Comment 15 Denis Roy 2016-03-15 17:25:03 EDT

Great. I'll test the patch tomorrow on your HIPP by restarting it and removing the variable, then we should be all set.

Comment 16 Mikaël Barbero 2016-03-15 17:28:51 EDT

(In reply to Kenn Hussey from comment #9)
> Do you mean that my job needs to be updated somehow? This same job has been
> working all along, so I'm surprised I'd have to change it to get it to work.
> Did something change somewhere?

Yes. See bug 487857.

/usr/bin/sign now starts an Ant script that requires ANT_ARGS to be properly set. Due to the broken script, ANT_ARGS (and consequently java.io.tmpdir) was not set to the proper value.

Comment 17 Kenn Hussey 2016-03-15 17:58:37 EDT

My integration build was successful and is now published. Thanks again.

Comment 18 David Williams 2016-03-22 21:28:41 EDT

(In reply to Mikael Barbero from comment #2)
> I've re-done the same fix as previously, i.e. re-created mdt/uml2 folder.
> This time, I've changed the owner/group from genie.uml2:common to
> khussey:modeling.mdt.uml2 (i.e. a pattern similar to xtext and they don't
> have the same issue of "disappearing" folder). 
> 
> Let me know once you've successfully build and sign, I will have a look at
> the folder and check whether it has been removed or not. 
> 
> Matt, Denis, do we have some scripts that "clean"
> /shared/download-staging.priv from time to time?

I do think there is such a "cleaning" script. At least used to be. The reason I believe that is that in the past I made sure there was a directory under the main directory just to "hold a place". Could have been superstitious behavior on my part -- it was so long ago I do not recall what made me think they were "swept up". Just saying it is worth double checking if you find "disappearing folders".

Comment 19 Denis Roy 2016-03-23 08:48:09 EDT

Every morning at 6:30am we run this to delete files older than 14 days.



#!/bin/bash

find /opt/public/download-staging.priv/ -type f -mtime +14 -delete

# delete empty directories
# of course, directories containing directories won't be deleted because they're not empty
# but eventually they will get purged if not used
find /opt/public/download-staging.priv/ -mindepth 3 -type d -mtime +14 -delete


I suppose we could remove the second part, the one that deletes directories, but I fear as the years go by that we end up with a wicked mess of empty dirs that affects performance of something else.

Perhaps empty directories older that have an mtime > 90 days can be purged?

Comment 20 Kenn Hussey 2016-04-18 09:23:22 EDT

Ugh, UML2 builds are failing to sign again. :(

[ant] Queueing site_679677485.zip for signing
ERROR: org.eclipse.core.runtime.CoreException: /opt/public/hipp/homes/genie.uml2/.eclipse/800847737/configuration/org.eclipse.osgi/bundles/25/1/.cp/org/eclipse/buckminster/jarprocessor/antscript/signing.ant:208: Directory /shared/download-staging.priv/modeling/mdt/uml2 creation was not successful for an unknown reason
org.eclipse.core.runtime.CoreException: /opt/public/hipp/homes/genie.uml2/.eclipse/800847737/configuration/org.eclipse.osgi/bundles/25/1/.cp/org/eclipse/buckminster/jarprocessor/antscript/signing.ant:208: Directory /shared/download-staging.priv/modeling/mdt/uml2 creation was not successful for an unknown reason
	at org.eclipse.buckminster.ant.AntRunner.handleInvocationTargetException(AntRunner.java:167)
	at org.eclipse.buckminster.ant.AntRunner.run(AntRunner.java:322)
	at org.eclipse.buckminster.ant.actor.AntActor.internalPerform(AntActor.java:254)
	at org.eclipse.buckminster.core.actor.AbstractActor.perform(AbstractActor.java:195)
	at org.eclipse.buckminster.core.internal.actor.PerformManager$DirectActionInvocation.execute(PerformManager.java:143)
	at org.eclipse.buckminster.core.internal.actor.PerformManager.internalPerform(PerformManager.java:454)
	at org.eclipse.buckminster.core.internal.actor.PerformManager.perform(PerformManager.java:293)
	at org.eclipse.buckminster.core.internal.actor.PerformManager.perform(PerformManager.java:305)
	at org.eclipse.buckminster.core.commands.Perform.internalRun(Perform.java:108)
	at org.eclipse.buckminster.core.commands.WorkspaceCommand.run(WorkspaceCommand.java:91)
	at org.eclipse.buckminster.cmdline.AbstractCommand.basicRun(AbstractCommand.java:200)
	at org.eclipse.buckminster.cmdline.Headless.run(Headless.java:350)
	at org.eclipse.buckminster.cmdline.Headless.run(Headless.java:145)
	at org.eclipse.buckminster.cmdline.Headless.start(Headless.java:165)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1438)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1414)
Caused by: /opt/public/hipp/homes/genie.uml2/.eclipse/800847737/configuration/org.eclipse.osgi/bundles/25/1/.cp/org/eclipse/buckminster/jarprocessor/antscript/signing.ant:208: Directory /shared/download-staging.priv/modeling/mdt/uml2 creation was not successful for an unknown reason
	at org.apache.tools.ant.taskdefs.Mkdir.execute(Mkdir.java:70)
	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
	at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
	at org.apache.tools.ant.Task.perform(Task.java:348)
	at org.apache.tools.ant.Target.execute(Target.java:392)
	at org.apache.tools.ant.Target.performTasks(Target.java:413)
	at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
	at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
	at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
	at org.eclipse.ant.internal.core.ant.EclipseDefaultExecutor.executeTargets(EclipseDefaultExecutor.java:32)
	at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
	at org.eclipse.ant.internal.core.ant.InternalAntRunner.run(InternalAntRunner.java:672)
	at org.eclipse.ant.internal.core.ant.InternalAntRunner.run(InternalAntRunner.java:498)
	at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.buckminster.ant.AntRunner.run(AntRunner.java:318)
	... 25 more
/opt/public/hipp/homes/genie.uml2/.eclipse/800847737/configuration/org.eclipse.osgi/bundles/25/1/.cp/org/eclipse/buckminster/jarprocessor/antscript/signing.ant:208: Directory /shared/download-staging.priv/modeling/mdt/uml2 creation was not successful for an unknown reason
Caused by: Recording test results
Archiving artifacts
Finished: FAILURE

Comment 21 Kenn Hussey 2016-04-19 08:59:48 EDT

I just tried another build and it failed for the same reason. Is there something I can/should do to correct this problem?

Comment 22 David Williams 2016-04-19 09:15:25 EDT

(In reply to Kenn Hussey from comment #21)
> I just tried another build and it failed for the same reason. Is there
> something I can/should do to correct this problem?

This sounds like a permission problem, to me. 
If I was debugging this, I'd try to add a build step at the very beginning, 
just a "bash script" in Hudson that confirmed "current user" (say, execute 'whoami') and then try with bash to 
mkdir -p /opt/public/download-staging.priv/modeling/mdt/uml

If that fails, then I think it will take webmaster intervention to fix the permissions on 'mdt' directory. 

Just guessing -- but trying to help.

Comment 23 Kenn Hussey 2016-04-19 09:20:51 EDT

Thanks, David. I'm sure it's probably a permission problem. But seeing as I've made no changes to my build setup, and the job has been working (since it was fixed the last time), I don't see a lot of value in trying to debug this myself...

Comment 24 Eclipse Webmaster 2016-04-20 11:39:02 EDT

Something must have removed the uml2 directory since it wasn't there yesterday when I was working on something for etrice.  I've created it and given /shared/download-staging.priv/modeling/mdt to the signers group, so mdt HIPP users should be able to create new subdirs at will(the HIPPs may need a restart first)

-M.

Comment 25 Kenn Hussey 2016-04-20 17:50:19 EDT

Thanks, Matt, the job is working again now. Could the folder have been removed by the clean-up routine that Denis menioned in comment #19?

Comment 26 Eclipse Webmaster 2016-04-21 16:54:59 EDT

If it was empty and hadn't been used in more than 14 days, it's quite probable.

-M.

Comment 27 Kenn Hussey 2016-06-15 09:28:27 EDT

Thanks, Matt. I'll ensure that builds are done weekly to prevent the directories from being removed.