Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 430538 (WebkitJsExecSami)

Summary: [GTK3][webkit] Support Custom JavaScript execution in WebKit2 mode of SWT Browser
Product: [Eclipse Project] Platform Reporter: Sami Wagiaalla <swagiaal>
Component: SWTAssignee: Leo Ufimtsev <lufimtse>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: akurtakov, arunkumar.thondapu, grant_gayed, peter, sravankumarl
Version: 4.4   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
See Also: https://git.eclipse.org/r/78014
https://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=24607eb68067e6c91f3c5c7a3f9697729f23dc76
https://git.eclipse.org/r/78567
https://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=7003ce9e76832ab4cf8e13c1633c97edca922ae7
https://git.eclipse.org/r/80902
https://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=0c8f3ae60bcecede34223ec0c9aea6ca76f17dcf
https://git.eclipse.org/r/81349
https://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=688c0e346bc63c155a78983d3bf38fb76a8c5b6c
https://bugs.eclipse.org/bugs/show_bug.cgi?id=510905
Whiteboard:
Bug Depends on: 430914, 432944    
Bug Blocks: 441568    
Attachments:
Description Flags
Webcat: Native C Gtk execution of javascript
none
cflags for gtk, gio, glib. none

Description Sami Wagiaalla CLA 2014-03-17 11:58:27 EDT
The SWT Browser API provided a function (execute()) which allowed clients to execute custom JavaScript in the context of the main frame of the browser and get the result synchronously.

Because the WebKit APIs for manipulating and executing JavaScript have been moved to the Web Process [https://trac.webkit.org/wiki/WebKit2] SWT must use a webkit 'extension' to access those APIs

I have posted a preliminary patch here: https://git.eclipse.org/r/#/c/23416/

The patch creates a webkit extension for the SWT Browser,
Provides support for executing custom JS code,
Uses the extension to set up custom code needed for even notifications (Also a part of the Browser interface which must be supported),
Adds needed DBus bindings for communication between the extension and the main process.
Comment 1 Sami Wagiaalla CLA 2014-04-04 15:10:00 EDT
Update patch: https://git.eclipse.org/r/#/c/23416/
Contains the following:
 - Generate a unique DBus name for each instance.
 - Fixed test suite hangs.
 - Fixed delivery of the create-web-view signal
Comment 2 Sami Wagiaalla CLA 2014-04-16 14:29:00 EDT
Update patch: https://git.eclipse.org/r/23416 (Patch Set 11)
- use findResource to locate webkit extension
Comment 3 Leo Ufimtsev CLA 2016-07-04 17:03:07 EDT
Browser8 seems to be affected by this. Executing scripts has no effect.
I'll investigate.
Comment 4 Leo Ufimtsev CLA 2016-07-11 17:31:12 EDT
Created attachment 263032 [details]
Webcat: Native C Gtk execution of javascript

Status update:
I wrote a native C Gtk snippet that executes some javascript. (see webcast).

This version uses:
webkit_web_view_run_javascript 
(https://wiki.gnome.org/Projects/WebKitGtk/ProgrammingGuide/Cookbook#Executing_JavaScript)

It doesn't compile a seperate webkit extension, it just uses basic webkit2gtk and javascriptgtk (which is part of webkit2). 

Source:
https://github.com/LeoUfimtsev/LeoGtk3/blob/master/webkit.c

Make file uses pkg-config with: webkit2gtk-4.0 javascriptcoregtk-4.0,
See my makefile:
https://github.com/LeoUfimtsev/LeoGtk3/blob/master/makefile


The next step is to implement this functionality on the SWT level. 
The idea is to transform the asynchronous webkit_web_view_run_javascript into a synchronous call because Browser.execute() and Browser.evalute() are synchronous anyways. I'll have to find someway of dealing with the callback in a suitable fashion, perhaps with a spinlock.
Comment 5 Eclipse Genie CLA 2016-07-27 14:54:20 EDT
New Gerrit change created: https://git.eclipse.org/r/78014
Comment 7 Eclipse Genie CLA 2016-08-02 15:54:34 EDT
New Gerrit change created: https://git.eclipse.org/r/78351
Comment 8 Eclipse Genie CLA 2016-08-02 15:58:30 EDT
New Gerrit change created: https://git.eclipse.org/r/78352
Comment 9 Eclipse Genie CLA 2016-08-05 14:57:49 EDT
New Gerrit change created: https://git.eclipse.org/r/78567
Comment 10 Eclipse Genie CLA 2016-08-17 12:19:59 EDT
New Gerrit change created: https://git.eclipse.org/r/79213
Comment 11 Leo Ufimtsev CLA 2016-08-17 12:26:09 EDT
I'm gonna tackle this in stages.

Stage 1: Implement 'exec()' support.
https://git.eclipse.org/r/#/c/78567/
- Patch is ready for review. This fixes Browser 8 and Browser 3.
- Exact details are in the patch message.

Stage 2: Implement evaluate() support
- Handle return value from JS exec.


The major change in stage 1 are the makefile changes. They conditionally compile webkit2gtk4 only on build machines that have it installed. This should be ideally reviewed by experienced build-machine maintainers. I tested it compiles on F24 with webkit2gtk4 installed as well as on Fedora 21 without webkit2gtk4 installed, on both Gtk2 and Gtk3.

Once Stage 1 is merged, I will tackle evaluate() next.
Comment 12 Eclipse Genie CLA 2016-08-31 15:35:01 EDT
New Gerrit change created: https://git.eclipse.org/r/80159
Comment 14 Leo Ufimtsev CLA 2016-09-09 10:19:50 EDT
Stage 1 merged. Plan for stage 2 is to get 'evalute()' to work. Working on it...
Comment 15 Arun Thondapu CLA 2016-09-11 14:29:21 EDT
(In reply to Eclipse Genie from comment #13)
> Gerrit change https://git.eclipse.org/r/78567 was merged to [master].
> Commit:
> http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/
> ?id=7003ce9e76832ab4cf8e13c1633c97edca922ae7

These changes are probably causing the Linux native builds to fail on the current build machines with the following errors:

[sshexec] gcc -O -Wall -DSWT_VERSION=4711  -DLINUX -DGTK -I/bluebird/teamswt/swt-builddir/JDKs/x86_64/jdk1.5.0/include -I/bluebird/teamswt/swt-builddir/JDKs/x86_64/jdk1.5.0/include/linux -fPIC -DJNI64 `pkg-config --cflags gio-2.0`      -c webkitgtk.c
  [sshexec] Package gio-2.0 was not found in the pkg-config search path.
  [sshexec] Perhaps you should add the directory containing `gio-2.0.pc'
  [sshexec] to the PKG_CONFIG_PATH environment variable
  [sshexec] No package 'gio-2.0' found
  [sshexec] In file included from webkitgtk_structs.h:19
  [sshexec] ,
  [sshexec]                  from webkitgtk.c:20:
  [sshexec] webkitgtk.h:24:25: error: 
  [sshexec] glib-object.h: No such file or directory
  [sshexec] In file included from webkitgtk_structs.h:19,
  [sshexec]                  from webkitgtk.c:20:
  [sshexec] webkitgtk.h:84: error: expected specifier-qualifier-list before ‘gboolean’
  [sshexec] webkitgtk.h:89: error: expected specifier-qualifier-list before ‘GObject’
  [sshexec] webkitgtk.c: In function ‘Java_org_eclipse_swt_internal_webkit_WebKitGTK__1SoupMessage_1method’:
  [sshexec] webkitgtk.c:575: error: ‘SoupMessage’ has no member named ‘method’
  [sshexec] webkitgtk.c: In function ‘Java_org_eclipse_swt_internal_webkit_WebKitGTK__1SoupMessage_1request_1body’:
  [sshexec] webkitgtk.c:586: error: ‘SoupMessage’ has no member named ‘request_body’
  [sshexec] webkitgtk.c: In function ‘Java_org_eclipse_swt_internal_webkit_WebKitGTK__1SoupMessage_1request_1headers’:
  [sshexec] webkitgtk.c:598: error: ‘SoupMessage’ has no member named ‘request_headers’
  [sshexec] make: *** [webkitgtk.o] Error 1
  [sshexec] [0;31m*** GTK2 Build failed.[0m
Comment 16 Alexander Kurtakov CLA 2016-09-12 02:12:27 EDT
Is this a yet another instance of too old builder not allowing usage of features in the min GTK version as required by SWT case? Or gio-2.0.pc is not available on the system for other reason (it comes from glib-devel) ?
Comment 17 Arun Thondapu CLA 2016-09-12 05:37:51 EDT
(In reply to Alexander Kurtakov from comment #16)
> Is this a yet another instance of too old builder not allowing usage of
> features in the min GTK version as required by SWT case? Or gio-2.0.pc is
> not available on the system for other reason (it comes from glib-devel) ?

Its most likely the former because all the Linux builds have failed. Unfortunately, we will not be able to move to new build machines in time for M2.

Alex, are you fine with reverting this change for M2? I think we're close to having the new GTK build machines ready and should be able to push this again in early M3.
Comment 18 Alexander Kurtakov CLA 2016-09-12 05:51:09 EDT
(In reply to Arun Thondapu from comment #17)
> (In reply to Alexander Kurtakov from comment #16)
> > Is this a yet another instance of too old builder not allowing usage of
> > features in the min GTK version as required by SWT case? Or gio-2.0.pc is
> > not available on the system for other reason (it comes from glib-devel) ?
> 
> Its most likely the former because all the Linux builds have failed.
> Unfortunately, we will not be able to move to new build machines in time for
> M2.
> 
> Alex, are you fine with reverting this change for M2? I think we're close to
> having the new GTK build machines ready and should be able to push this
> again in early M3.

It's fine but git is not working for me so no idea when this revert will happen. If it works for others please do.
Comment 19 Eclipse Genie CLA 2016-09-12 06:36:27 EDT
New Gerrit change created: https://git.eclipse.org/r/80902
Comment 21 Leo Ufimtsev CLA 2016-09-12 09:34:37 EDT
Created attachment 264108 [details]
cflags for gtk, gio, glib.

Something to note is that gio is a subest of gtk2/gtk3.
(Please see attached cflags/libs).
gtk2/gtk3 is included in the make_linux.mak

I thought that since it's a subset, it should work.
I.e, if gtk2 compiles, then gio should compile. Theoretically it thus should build on older machines also.

However it's possible that gio devel libraries are not installed and pkg-config doesn't 'know' about gio, but the includes and flags should be present on the older machines.

I can see two solutions:
1) On all linux machines, install "libglib2.0-dev" package.
2) I could update the patch to use gtk2/gtk3 cflags/libs instead of gio. This way no linux machines have to be changed. I don't see any particular problem with that excep that webkit is compiled and linked with more symbols than neccessary.

Thoughts?
Comment 22 Eclipse Genie CLA 2016-09-19 10:52:00 EDT
New Gerrit change created: https://git.eclipse.org/r/81349
Comment 24 Leo Ufimtsev CLA 2016-09-19 11:00:39 EDT
(In reply to comment #23)
> Gerrit change https://git.eclipse.org/r/81349 was merged to [master].
> Commit:
> http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/?id=688c0e346bc63c155a78983d3bf38fb76a8c5b6c

Build servers were updated to Gtk2.24. These should include gio and thus the patch should build now.

I've pushed to master, let's see if things build well tomorrow.
Comment 25 Leo Ufimtsev CLA 2016-10-06 16:49:46 EDT
execute() works well. 

Closing for now. Will work on evaluate() in another bug.
Comment 26 Leo Ufimtsev CLA 2016-10-07 09:35:50 EDT
I'll look into backporting once I get evaluate() working, (as they're closley related).