Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 539603

Summary: Create a CI instance for cognicrypt
Product: Community Reporter: Eric Bodden <eric>
Component: CI-JenkinsAssignee: CI Admin Inbox <ci.admin-inbox>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: chris.guindon, frederic.gurr, mikael.barbero, webmaster
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Mac OS X   
Whiteboard:
Attachments:
Description Flags
buildnow none

Description Eric Bodden CLA 2018-09-28 03:22:35 EDT
Hello.

The URL https://www.eclipse.org/cognicrypt/ is currently still a redirect but we shortly plan to replace this with an actual project website. For this we would like to use Hugo, as it is used on https://iot.eclipse.org.

Could you please enable Hugo support for this webspace? Thanks!
Comment 1 Eric Bodden CLA 2018-10-02 13:00:34 EDT
Would it be possible to give us a brief note stating the time frame within which we can expect this to be enabled? We have a larger marketing event coming up on Oct 15th and would really like to have the website online by now.

Also I have a second question: ideally we would like to delegate some of the website-editing to a person in our team who is not (yet) a committer. To obtain website access, would that person need to become a committer or are there other ways to provide such access?
Comment 2 Christopher Guindon CLA 2018-10-02 15:36:21 EDT
(In reply to Eric Bodden from comment #0)
> Hello.
> 
> The URL https://www.eclipse.org/cognicrypt/ is currently still a redirect
> but we shortly plan to replace this with an actual project website. For this
> we would like to use Hugo, as it is used on https://iot.eclipse.org.

Could you elaborate on "we would like to use Hugo, as it is used on https://iot.eclipse.org"? 

> Could you please enable Hugo support for this webspace? Thanks!

Are you requesting a Jenkins instance that will build and commit the html output from Hugo?
Comment 3 Eric Bodden CLA 2018-10-03 04:10:47 EDT
(In reply to Christopher Guindon from comment #2)
> Are you requesting a Jenkins instance that will build and commit the html
> output from Hugo?

Yes, sorry, that seems to be how it's realized for the other projects and this is, in fact, what we would like, too.
Comment 4 Eclipse Webmaster CLA 2018-10-04 13:44:51 EDT
(In reply to Eric Bodden from comment #1)

I've renamed and re-assigned this bug to better reflect what's needed right now.
 
> would that person need to become a committer or are there other ways to provide >such access?

They have to be a committer before they can get direct access.

-M.
Comment 5 Frederic Gurr CLA 2018-10-04 13:57:09 EDT
A JIPP for the cognicrypt project has already been provisioned: https://ci.eclipse.org/cognicrypt
Comment 6 Eric Bodden CLA 2018-10-05 03:47:00 EDT
(In reply to Frederic Gurr from comment #5)
> A JIPP for the cognicrypt project has already been provisioned:
> https://ci.eclipse.org/cognicrypt

Thanks! Could you still help us, though, setting up the required config for Hugo? That would be greatly appreciated.
Comment 7 Mikaël Barbero CLA 2018-10-05 03:48:22 EDT
Where are your hugo sources stored (which git repo)?
Comment 8 Eric Bodden CLA 2018-10-05 04:14:06 EDT
(In reply to Mikaël Barbero from comment #7)
> Where are your hugo sources stored (which git repo)?

This is the URL I am using to access the repo:
ssh://ebodden680@git.eclipse.org:29418/www.eclipse.org/cognicrypt.git

Currently the repo still contains a redirect (via index.php) but this we plan to remove once the Hugo-Job is running.
Comment 9 Mikaël Barbero CLA 2018-10-05 04:33:28 EDT
I see. The repo you're mentioning is actually the repo from which our infrastructure take the actual content of your website. Everything in the master branch will eventually be published on the http server at the URL https://www.eclipse.org/cognicrypt. For instance you can see that http://git.eclipse.org/c/www.eclipse.org/cognicrypt.git/tree/config.toml is already publish https://www.eclipse.org/cognicrypt/config.toml and http://git.eclipse.org/c/www.eclipse.org/cognicrypt.git/tree/content/documentation.md is published to https://www.eclipse.org/cognicrypt/content/documentation.md

We work this way because, historically, projects edited/pushed the HTML content of their website directly here: 

git.eclipse.org/www.eclipse.org/<projectname>.git => http server serving https://www.eclipse.org/<projectname>

Nowadays, the trend is to use a static website generator (e.g., Hugo) which generates the HTML website from another format (most frequently à la markdown). 

When doing this, we advise to store the Hugo (or any other static website generator) sources in a different repo, build the website, and push the result to git.eclipse.org/www.eclipse.org/<projectname>.git

<some repo with hugo source> => generates the HTML and push to => git.eclipse.org/www.eclipse.org/<projectname>.git => https://www.eclipse.org/<projectname>

I see that you already host your project source code under the eclipse-cognicrypt GitHub https://github.com/eclipse-cognicrypt. I suggest you ask for a new repo to be created there and move you hugo sources over here. I suggest something like https://github.com/eclipse-cognicrypt/hugo-website-sources.

What's nice here is that we can automatize the build process by setting up a Jenkins job. So, once you've setup your hugo-website-sources repo, I will submit a pull request to this repo with a Jenkinsfile pipeline script that will contain the necessary machinery to build your sources and do the push to git.eclipse.org/www.eclipse.org/<projectname>.git. 

When you're fine with this PR, we just have to create a pipeline job on your Jenkins instance that will be triggered on new every pushes. It will build your website and will push it to git.eclipse.org/www.eclipse.org/<projectname>.git. 

Finally, as our internal infrastructure is monitoring git.eclipse.org/www.eclipse.org/<projectname>.git repos on a regular basis, you will just have to wait a couple of min to see the changes built by the Jenkins job to appear on the website. 

WDYT?
Comment 10 Eric Bodden CLA 2018-10-05 04:58:16 EDT
Thanks a lot for the detailed explanation Mikaël.

(In reply to Mikaël Barbero from comment #9)
> I see that you already host your project source code under the
> eclipse-cognicrypt GitHub https://github.com/eclipse-cognicrypt. I suggest
> you ask for a new repo to be created there and move you hugo sources over
> here. I suggest something like
> https://github.com/eclipse-cognicrypt/hugo-website-sources.

That sounds great. Could you set this up for us this way? Or who else can create the repo for us?

> What's nice here is that we can automatize the build process by setting up a
> Jenkins job. So, once you've setup your hugo-website-sources repo, I will
> submit a pull request to this repo with a Jenkinsfile pipeline script that
> will contain the necessary machinery to build your sources and do the push
> to git.eclipse.org/www.eclipse.org/<projectname>.git. 
> 
> When you're fine with this PR, we just have to create a pipeline job on your
> Jenkins instance that will be triggered on new every pushes. It will build
> your website and will push it to
> git.eclipse.org/www.eclipse.org/<projectname>.git. 
> 
> Finally, as our internal infrastructure is monitoring
> git.eclipse.org/www.eclipse.org/<projectname>.git repos on a regular basis,
> you will just have to wait a couple of min to see the changes built by the
> Jenkins job to appear on the website. 
> 
> WDYT?

Sounds great, please let's do it that way.

Cheers
Eric
Comment 11 Mikaël Barbero CLA 2018-10-05 05:33:02 EDT
(In reply to Eric Bodden from comment #10)
> That sounds great. Could you set this up for us this way? Or who else can
> create the repo for us?

Done. https://github.com/eclipse-cognicrypt/hugo-website-sources

> Sounds great, please let's do it that way.

Please move your Hugo sources to the repository above. Once done, I'll submit a PR for the Jenkinfile. Thanks.
Comment 12 Eric Bodden CLA 2018-10-05 06:01:53 EDT
(In reply to Mikaël Barbero from comment #11)
> (In reply to Eric Bodden from comment #10)
> Please move your Hugo sources to the repository above. Once done, I'll
> submit a PR for the Jenkinfile. Thanks.

Done
Comment 13 Mikaël Barbero CLA 2018-10-05 06:41:10 EDT
Thanks. i'm almost done. The last thing I forgot is that we will need to run your Jenkins instance on the new infrastructure. It will be available @ https://jenkins.Eclipse.org/cognicrypt instead of https://ci.eclipse.org/cognicrypt. As you don't have anything running so far on https://ci.eclipse.org/cognicrypt, I guess this is not an issue, but I need your validation anyway. So, are you ok with that? Thanks.
Comment 14 Eric Bodden CLA 2018-10-05 06:45:05 EDT
(In reply to Mikaël Barbero from comment #13)
> Thanks. i'm almost done. The last thing I forgot is that we will need to run
> your Jenkins instance on the new infrastructure. It will be available @
> https://jenkins.Eclipse.org/cognicrypt instead of
> https://ci.eclipse.org/cognicrypt. As you don't have anything running so far
> on https://ci.eclipse.org/cognicrypt, I guess this is not an issue, but I
> need your validation anyway. So, are you ok with that? Thanks.

That's fine, thanks!
Comment 16 Eric Bodden CLA 2018-10-05 13:51:02 EDT
Thanks, it's merged now but it seems like the jenkins "build" is failing. It would be great if you could have another look. Thanks!
Comment 17 Eric Bodden CLA 2018-10-08 09:33:10 EDT
(In reply to Eric Bodden from comment #16)
> Thanks, it's merged now but it seems like the jenkins "build" is failing. It
> would be great if you could have another look. Thanks!

Just a kind reminder that this is still not working... Thanks!
Comment 18 Mikaël Barbero CLA 2018-10-08 09:48:56 EDT
Thanks fr reminder. A small issue with the ssh deploy key. You should be all set now (https://jenkins.eclipse.org/cognicrypt/job/build-and-publish-website/job/master/3/) minus the small PR to fix a stupid typo on my side (https://github.com/eclipse-cognicrypt/hugo-website-sources/pull/2).

Note that you will to remove the index.php from the www git repo in order to remove the redirection from http://www.eclipse.org/cognicypt/ to https://projects.eclipse.org/projects/technology.cognicrypt
Comment 19 Eric Bodden CLA 2018-10-08 10:05:58 EDT
(In reply to Mikaël Barbero from comment #18)
> Thanks fr reminder. A small issue with the ssh deploy key. You should be all
> set now
> (https://jenkins.eclipse.org/cognicrypt/job/build-and-publish-website/job/
> master/3/) minus the small PR to fix a stupid typo on my side
> (https://github.com/eclipse-cognicrypt/hugo-website-sources/pull/2).

I just merged this. Awesome, thanks!

> Note that you will to remove the index.php from the www git repo in order to
> remove the redirection from http://www.eclipse.org/cognicypt/ to
> https://projects.eclipse.org/projects/technology.cognicrypt

Yes I will do that once the site has been finalized.
Comment 20 Eric Bodden CLA 2018-10-10 11:09:14 EDT
I am sorry but I just notices that as I push changes to the master branch at  https://github.com/eclipse-cognicrypt/hugo-website-sources/ this does NOT yet automatically trigger the configured build at https://jenkins.eclipse.org/cognicrypt/

Mikaël are you able to see why that is? I am afraid that, as I am missing admin privileges on Github, I cannot really see myself what's going wrong.
Comment 21 Mikaël Barbero CLA 2018-10-10 11:23:26 EDT
https://github.com/eclipse-cognicrypt/hugo-website-sources/pull/3

I've added a cron trigger to configure jenkins to look for changes every 5 mintutes.
Comment 22 Mikaël Barbero CLA 2018-10-12 10:38:46 EDT
Did you have a look?
Comment 23 Eric Bodden CLA 2018-10-12 13:27:31 EDT
I am afraid it's still not working. I pushed a (trivial) change to the master branch of https://github.com/eclipse-cognicrypt/hugo-website-sources but it did not result in the Jenkins job being triggered.

P.S. Thanks for the clarification. Originally I thought that Jenkins were to be triggered by a post-commit hook on Github but I see that in terms of authorizations it might make more sense to regularly poll Github instead.
Comment 24 Mikaël Barbero CLA 2018-10-12 14:30:35 EDT
You did not merge my PR of comment #21 :)
Comment 25 Eric Bodden CLA 2018-10-16 01:55:53 EDT
Ah, stupid me - thanks! It's pushed now but Jenkins still does not seem to become active. Do we need to trigger the build manually once for the cron-job to be picked up?
Comment 26 Mikaël Barbero CLA 2018-10-16 06:54:27 EDT
Indeed, you need one manual build for jenkins to catch up with the changes.
Comment 27 Eric Bodden CLA 2018-10-16 07:18:23 EDT
(In reply to Mikaël Barbero from comment #26)
> Indeed, you need one manual build for jenkins to catch up with the changes.

Sorry for being such a Jenkins novice but how do I do that? I failed to find an appropriate UI widget.
Comment 28 Mikaël Barbero CLA 2018-10-16 07:24:11 EDT
Created attachment 276259 [details]
buildnow

You need to log into your jenkins instance https://jenkins.eclipse.org/cognicrypt/ (your Eclipse account email+password)

Then, navigate to the master branch (https://jenkins.eclipse.org/cognicrypt/job/build-and-publish-website/job/master/) and click on "Build now"
Comment 29 Eric Bodden CLA 2018-10-16 12:12:21 EDT
Great, this seems to work now! Turns out I had forgotten to log in hence the button was missing ;-)

Thanks!!
Comment 30 Frederic Gurr CLA 2018-10-26 11:30:18 EDT
The job should not be triggered but instead poll for changes every 5 minutes.

Please merge https://github.com/eclipse-cognicrypt/hugo-website-sources/pull/4.
Comment 31 Eric Bodden CLA 2018-10-27 15:34:51 EDT
(In reply to Frederic Gurr from comment #30)
> The job should not be triggered but instead poll for changes every 5 minutes.
> 
> Please merge
> https://github.com/eclipse-cognicrypt/hugo-website-sources/pull/4.

OK done. Thanks!