Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 545682

Summary: org.eclipse.equinox.http.servlet and org.eclipse.equinox.servletbridge do not work together anymore in Tomcat 8.5.x
Product: [Eclipse Project] Equinox Reporter: Ivan Furnadjiev <ivan>
Component: CompendiumAssignee: Ivan Furnadjiev <ivan>
Status: RESOLVED FIXED QA Contact:
Severity: major    
Priority: P3 CC: a_mergey, hendel, mknauer, raymond.auge, tjwatson
Version: 4.11   
Target Milestone: 4.12 M1   
Hardware: PC   
OS: All   
See Also: https://git.eclipse.org/r/140089
https://git.eclipse.org/c/equinox/rt.equinox.bundles.git/commit/?id=5df7eda553d8b0a164e2f2050c1b59b83461eedf
Whiteboard:

Description Ivan Furnadjiev CLA 2019-03-22 13:22:41 EDT
RAP 3.8 application does not work in Tomcat 8.5.x. In fails with exception:
--------------
!ENTRY org.eclipse.equinox.http.servlet 4 0 2019-03-22 17:52:47.209
!MESSAGE FrameworkEvent ERROR
!STACK 0
org.osgi.framework.BundleException: Could not resolve module: org.eclipse.equinox.http.servlet [22]
  Unresolved requirement: Import-Package: javax.servlet; version="[3.1.0,5.0.0)"
--------------
RAP 3.8 is using:
- org.eclipse.equinox.servletbridge_1.4.200.v20181008-1820
- org.eclipse.equinox.http.servlet_1.6.0.v20190305-2225

With this commit [1], Raymond changed the package import of "javax.servlet" in org.eclipse.equinox.http.servlet MANIFEST.MF from "javax.servlet;version="[2.3.0,5.0.0)"," to "javax.servlet;version="[3.1.0,5.0.0)"".
Tomcat 8.5.x is based on Servlet 3.1.
The servletbridge FrameworkLauncher#generateExtensionBundle resolves "javax.servlet" 3.1 as 2.7, which is now out of range.

[1] https://github.com/eclipse/rt.equinox.bundles/commit/7d6da7c9bdf36bf132bd5384d909651b97c770ca
Comment 1 Ivan Furnadjiev CLA 2019-03-25 04:38:20 EDT
We have 2 options here:
1. Revert the "javax.servlet.http" package import version range in "org.eclipse.equinox.http.servlet" MANIFEST.MF to "[2.3.0,5.0.0)" (at least [2.6.0,5.0.0))
2. Fix the logic in "org.eclipse.equinox.servletbridge" FrameworkLauncher#generateExtensionBundle to not convert "javax.servlet" version from 3.1 to 2.7
Comment 2 Thomas Watson CLA 2019-03-25 09:08:21 EDT
(In reply to Ivan Furnadjiev from comment #1)
> We have 2 options here:
> 1. Revert the "javax.servlet.http" package import version range in
> "org.eclipse.equinox.http.servlet" MANIFEST.MF to "[2.3.0,5.0.0)" (at least
> [2.6.0,5.0.0))
> 2. Fix the logic in "org.eclipse.equinox.servletbridge"
> FrameworkLauncher#generateExtensionBundle to not convert "javax.servlet"
> version from 3.1 to 2.7

I prefer option 2, but it seems we have a bigger issue with this approach when consumers may have bundles expecting Servlet 3.1 containers to report servlet package versions to be 2.7.  Ray?
Comment 3 Ivan Furnadjiev CLA 2019-03-25 10:30:26 EDT
Here is the bug report https://bugs.eclipse.org/bugs/show_bug.cgi?id=360245, where the decision is taken Servlet 3.1 to have package version 2.7.
Comment 4 Raymond Auge CLA 2019-03-25 10:35:54 EDT
Other API providers stuck in similar scenario provide an export for every version the packages might take; like (fully expanded mind you):

javax.servlet.*;version=2.7,\
...
javax.servlet.*;version=3.0,\
javax.servlet.*;version=3.1,\
javax.servlet.*;version=4.0

as well as providing the contracts at all the appropriate levels.

Yes it's ugly, but it works for everyone.
Comment 5 Raymond Auge CLA 2019-03-25 10:39:09 EDT
The cleanest solution is of course the contracts approach where nobody imports by version, but uses contracts.

I'm not sure how tycho works with this however. My experience has not been good, but I've heard it is possible, we just need to find the right magic. (I'm told you do it with a p2.inf file or some such).
Comment 6 Ivan Furnadjiev CLA 2019-04-03 07:17:31 EDT
As this is a major issue for RAP users we need a fix as soon as possible. Is it possible to fix it in 4.12 M1?
Comment 7 Ivan Furnadjiev CLA 2019-04-04 04:34:23 EDT
Ray, why did you change the "org.eclipse.equinox.http.servlet" MANIFEST.MF at all? Do you want to exclude old servlets specs below 3.1? If yes than simply make it "javax.servlet;version="[2.7.0,5.0.0)"...
Comment 8 Raymond Auge CLA 2019-04-04 07:42:32 EDT
(In reply to Ivan Furnadjiev from comment #7)
> Ray, why did you change the "org.eclipse.equinox.http.servlet" MANIFEST.MF
> at all? Do you want to exclude old servlets specs below 3.1? If yes than
> simply make it "javax.servlet;version="[2.7.0,5.0.0)"...

I'm sorry but equinox.http had depended on servlet 3.1 for at least 2 releases now. Leaving the import at anything below 3.1 was simply incorrect.
Comment 9 Thomas Watson CLA 2019-04-04 09:33:23 EDT
(In reply to Raymond Auge from comment #8)
> (In reply to Ivan Furnadjiev from comment #7)
> > Ray, why did you change the "org.eclipse.equinox.http.servlet" MANIFEST.MF
> > at all? Do you want to exclude old servlets specs below 3.1? If yes than
> > simply make it "javax.servlet;version="[2.7.0,5.0.0)"...
> 
> I'm sorry but equinox.http had depended on servlet 3.1 for at least 2
> releases now. Leaving the import at anything below 3.1 was simply incorrect.

If we are taking the position to not use semantic versions for the servlet packages then I think we must update org.eclipse.equinox.servletbridge.FrameworkLauncher.generateExtensionBundle(File, String, String) to export the servlet packages for servlet 3.x at package versions 3.x.  We are still faced with a compatibility issue for existing bundles that may exist in the bridge scenario that import at sematic versioned packages for servlet 3.x specification (e.g. package version 2.6 == servlet 3.0 spec version).

My suggestion is to have the org.eclipse.equinox.servletbridge.FrameworkLauncher.generateExtensionBundle(File, String, String) logic export at both versions for 3.x containers implementations.

This area is obviously not tested which is a detriment to all that use it.  The unfortunate position we are in is that the main maintainers of the http.servlet bundle (mostly Ray and sometimes myself) do not use org.eclipse.equinox.servletbridge at all in our consumption of http.servlet.  We need help from interested parties (consumers) to develop automated tests for this component.  Otherwise I am sure it will get broken over and over each release.
Comment 10 Thomas Watson CLA 2019-04-04 09:37:17 EDT
*** Bug 546010 has been marked as a duplicate of this bug. ***
Comment 11 Raymond Auge CLA 2019-04-04 09:45:37 EDT
This is additional info I sent via email:

> The issue is that equinox.http.servlet needs to use at least Servlet API 3.1. and we are stuck with some legacy decision about how those are encoded.

> Next, equinox.http.servlet uses the encoding as defined by the dependencies in the Orbit repository which use the Servlet API version as the package version. By that same token we needed to express that it was no longer compatible with Servlet API versions below 3.1. Since we could not use Portable Java Contracts due to some limitations with Tycho/p2 we had to resort to raising the floor package import version.

> If we could manage to solve the Portable Java Contracts issue, equinox.http.servlet and friends could remove their Servlet import versions entirely.

> Finally, all RAP needs to do to solve this issue is to make its Servlet API provider export multiple versions of the Servlet API up to and including 3.1 (which is the API level of Tomcat 8.5 [1]).
Comment 12 Ivan Furnadjiev CLA 2019-04-04 10:08:31 EDT
Ray, the Servlet API provider is "org.eclipse.equinox.servletbridge". As Tom suggested, one of the options is to fix the logic in "org.eclipse.equinox.servletbridge.FrameworkLauncher.generateExtensionBundle(File, String, String)". But as this is contradictory with the decisions taken in bug 360245, Equinox team must find the best solution here.
Comment 13 Raymond Auge CLA 2019-04-04 10:14:39 EDT
(In reply to Ivan Furnadjiev from comment #12)
> But as this is contradictory
> with the decisions taken in bug 360245, Equinox team must find the best
> solution here.

I'm not sure I understand this comment. From what I could tell (since I've only been a maintainer for about 4-5 years), equinox.http.servlet has always assumed the package import encoding was against the Servlet API in Orbit which never assumed semantic versioning. The fact that equinox.http.servlet sort of accidentally worked for anyone making that assumption was just that, an accident.

But like I said, I'm open to fixing this using Portable Java Contracts which is the only other encoding which I think has any significant hope of working long term and across a wide range of consumers. Otherwise, we have to stick with the assumption that our gold standard is the Servlet API package version provided by Orbit Repo.
Comment 14 Raymond Auge CLA 2019-04-04 10:16:57 EDT
But If I'm overruled by a majority that's open source and I'll follow.
Comment 15 Thomas Watson CLA 2019-04-04 10:21:13 EDT
(In reply to Ivan Furnadjiev from comment #12)
> Ray, the Servlet API provider is "org.eclipse.equinox.servletbridge". As Tom
> suggested, one of the options is to fix the logic in
> "org.eclipse.equinox.servletbridge.FrameworkLauncher.
> generateExtensionBundle(File, String, String)". But as this is contradictory
> with the decisions taken in bug 360245, Equinox team must find the best
> solution here.

I think we lost the battle to semantically version the javax.servlet packages.  Not only Orbit bundles but most every other community moved on to just using the spec version for the packages.  The bug 360245 was back when there was a thought that the semantic versioning of these packages was possible and we thought we had the influence to actually make it happen.  That proved to be not true over time.  IMO we need to move on from the discussion in bug 360245.

I still think the most simple solution for now is to export at both 2.7/3.1 version for Servlet 3.1 container.  We can also generate a contract capability if we want, but should also update the orbit servlet bundles to also provide that contract.
Comment 16 Thomas Watson CLA 2019-04-04 10:24:14 EDT
Ivan, since you have a project that uses the servletbridge, can you provide a contribution that you have tested with your project and a servlet 3.1 implementation?  Doing the actual manual test of this for Ray or myself is not likely to happen in short order for M1.
Comment 17 Raymond Auge CLA 2019-04-04 10:25:26 EDT
(In reply to Thomas Watson from comment #15)
> We can also generate a contract
> capability if we want, but should also update the orbit servlet bundles to
> also provide that contract.

This is my preferred approach, but currently I feel we need some help to get this working in Tycho/P2 build model.
Comment 18 Eclipse Genie CLA 2019-04-05 04:40:47 EDT
New Gerrit change created: https://git.eclipse.org/r/140089
Comment 19 Ivan Furnadjiev CLA 2019-04-05 04:58:48 EDT
Tom, I created a Gerrit change the exports "javax.servlet.*" in  FrameworkLauncher#generateExtensionBundle as both 2.7 and 3.1. Please review.
Comment 20 Ivan Furnadjiev CLA 2019-04-05 05:37:42 EDT
... also I'm leaving to you the "org.eclipse.equinox.servletbridge" bundle version incrementation.
Comment 21 Thomas Watson CLA 2019-04-05 09:40:10 EDT
(In reply to Ivan Furnadjiev from comment #20)
> ... also I'm leaving to you the "org.eclipse.equinox.servletbridge" bundle
> version incrementation.

Thanks, just to confirm ... you were able to test this with RAP and tomcat 3.1 implementation?
Comment 22 Ivan Furnadjiev CLA 2019-04-05 09:46:14 EDT
Yes... I tested it in Tomcat 8.5.39
Comment 24 Thomas Watson CLA 2019-04-05 09:49:15 EDT
(In reply to Ivan Furnadjiev from comment #22)
> Yes... I tested it in Tomcat 8.5.39

Thanks, marking this bug as fixed.  I opened a follow on bug 546154 to add WAR tests for the servletbridge.  Help needed there.
Comment 25 Thomas Hendel CLA 2019-06-21 07:33:12 EDT
I'm running into this problem with a RAP application after upgrading from it RAP 3.7 to RAP 3.9. What's to do to get the fix working?

I'm a bit lost because it looks as if a newer "org.eclipse.equinox.http.servlet" bundle is now included (compared to WAR based on RAP 3.7) but the servlet bridge bundle (org.eclipse.equinox.http.servletbridge_1.1.100.v20180827-1235) is the same; and that's the newest version one can find...

OSGi console lists

22      RESOLVED    org.eclipse.equinox.servletbridge.extensionbundle_1.3.0
                    Master=0
23      INSTALLED   org.eclipse.equinox.http.servlet_1.6.100.v20190516-1504
24      INSTALLED   org.eclipse.equinox.http.servletbridge_1.1.100.v20180827-1235

and fails to start
osgi> start 23
org.osgi.framework.BundleException: Could not resolve module: org.eclipse.equinox.http.servlet [23]
  Unresolved requirement: Import-Package: javax.servlet; version="[3.1.0,5.0.0)"
Comment 26 Ivan Furnadjiev CLA 2019-06-24 03:37:05 EDT
Probably a cache issue... Just tested RAP 3.9 demos in Tomcat 8.5.39 and 9.0.10 without problems. If you still have issue please open a bug report against RAP.
Comment 27 Ivan Furnadjiev CLA 2019-06-24 03:39:22 EDT
You need org.eclipse.equinox.servletbridge_1.5.0.v20190405-1333
Comment 28 Thomas Hendel CLA 2019-06-24 10:50:30 EDT
Thanks a lot! It wasn't a cache issue, but a build configuration problem that caused the old version to be used.
Anyhow, it's solved, and our web app runs on RAP 3.9 on a tomcat 8.5 :-)
Great!