Bugzilla – Full Text Bug Listing
|
Description
Asaf Yaffe
suggest tagging version4.5 for 4.5 discussion. Setting target to 4.5, per request from Chris Elford. Setting "version" to 4.5 Here's a clarification to the scope of work needed: my numerous experiments in this field suggests that patching the byte-code offsets in the StackMapTable attribute is not enough. In most cases, the StackMapTable attribute needs to be recalculated from scratch. This adds an additional complexity to the implementation, and the original time estimate of 50 hours is incorrect. It will probably take at least one person month to implement correctly. Another thing to be considered is to leverage existing code from other projects (Eclipse-based or not) in order to implement this. For example, the Java compiler and AspectJ projects in Eclipse have an implementation for generating this attribute. I am not sure whether this code can be reused inside the Probekit engine though (the existing Eclipse code in written in Java, and the Probekit instrumentor is a native library). Apache Harmony also has similar functionality in its verifier. We've been toying with the idea of seeing if that can be modularized and leveraged here. Updated feature description URL and time estimates Approved by the AG for TPTP 4.5 with the following comments: -Is there an arrangement/agreement with the Apache Harmony project? For example, coordinated design/implementation, synchronized release schedules, contacts, implementation split, etc. For TPTP to benefit from this arrangement, we would need to modularize the code in the Harmony project and then request Eclipse legal approval for that module. -Have you contact the Eclipse JDT Team about using their solution. This code would be relatively easy to access and under the same license. -TPTP should redistribute the binaries for the Harmony module to keep the component separate and reduce built and support costs. -Does Apache Harmony support the same platforms as TPTP? -Bugzilla needs to be assigned to an active committer (currently Navid). -Sizings total 12.5 PW (not 13 PW). (In reply to comment #7) > Approved by the AG for TPTP 4.5 with the following comments: > > -Is there an arrangement/agreement with the Apache Harmony project? For > example, coordinated design/implementation, synchronized release schedules, > contacts, implementation split, etc. For TPTP to benefit from this > arrangement, we would need to modularize the code in the Harmony project and > then request Eclipse legal approval for that module. We are currently discussing all these issues with the Harmony development team. > -Have you contact the Eclipse JDT Team about using their solution. This code > would be relatively easy to access and under the same license. The key issue here is that the Probekit BCI engine is written in C++ while the Eclipse-based technologies (JDT, AJDT etc) are written in Java. > -TPTP should redistribute the binaries for the Harmony module to keep the > component separate and reduce built and support costs. I'll update the feature description document. > -Does Apache Harmony support the same platforms as TPTP? Apache Harmony currently supports 32/64 bit Windows and Linux. In this sense, it supports the same platforms currently supported by the TPTP JVMTI-based collectors (including the new JVMTI-based Probekit). > -Bugzilla needs to be assigned to an active committer (currently Navid). > -Sizings total 12.5 PW (not 13 PW). > I'll update this. (In reply to comment #8) > (In reply to comment #7) > > Approved by the AG for TPTP 4.5 with the following comments: > > > > -Is there an arrangement/agreement with the Apache Harmony project? For > > example, coordinated design/implementation, synchronized release schedules, > > contacts, implementation split, etc. For TPTP to benefit from this > > arrangement, we would need to modularize the code in the Harmony project and > > then request Eclipse legal approval for that module. > > We are currently discussing all these issues with the Harmony development team. Please provide an update to the PMC (via your Project Lead) when the issues are sorted out and/or you start work on the implementation. > > -Have you contact the Eclipse JDT Team about using their solution. This code > > would be relatively easy to access and under the same license. > > The key issue here is that the Probekit BCI engine is written in C++ while the > Eclipse-based technologies (JDT, AJDT etc) are written in Java. True, but the algorithm would be the same and could be easily ported to C++. Updated the Description Document provided by Asaf and updated the sizing to reflect the TPTP component of the sizing in the new Description Document (based on a 40 hour work week). (In reply to comment #9) > > The key issue here is that the Probekit BCI engine is written in C++ while the > > Eclipse-based technologies (JDT, AJDT etc) are written in Java. > > True, but the algorithm would be the same and could be easily ported to C++. > I spent some time looking at the JDT source code. It seems that the StackMapTable computation implementation is built into the compilation process and cannot be easily decoupled (for example, the process heavily depends on many data structures and types used by the Java compiler). Things are different in Harmony, where the verification process is quite modular and depends only on abstract interfaces which can be easily implemented on top of the Probekit BCI engine. (In reply to comment #10) > (In reply to comment #9) > > > The key issue here is that the Probekit BCI engine is written in C++ while the > > > Eclipse-based technologies (JDT, AJDT etc) are written in Java. > > > > True, but the algorithm would be the same and could be easily ported to C++. > > > > I spent some time looking at the JDT source code. It seems that the > StackMapTable computation implementation is built into the compilation process > and cannot be easily decoupled (for example, the process heavily depends on > many data structures and types used by the Java compiler). Things are different > in Harmony, where the verification process is quite modular and depends only on > abstract interfaces which can be easily implemented on top of the Probekit BCI > engine. > Thanks Asaf for investigating. Cannot be contained in 4.5.i5. Deferring to i6 Created attachment 88758 [details]
Updated Feature Description Document with detailed design
An updated FDD with detailed design
Created attachment 96544 [details]
This patch introduce new java 6 attributes parsing and interface to Harmony verifier
Created attachment 96545 [details]
this file contains declarations and definitions primitives for manipualting with class interface implementation
this file should be placed in org.eclipse.hyades.probekit/src-native/BCI/BCIEng/BCIEngJ
Created attachment 96546 [details]
includes headers from Harmony verifier
this file should be placed in org.eclipse.hyades.probekit/src-native/BCI/BCIEng/BCIEngJ
Created attachment 96547 [details]
class interface implementation for Harmony verifier
this file should be placed in org.eclipse.hyades.probekit/src-native/BCI/BCIEng/BCIEngJ
Created attachment 96548 [details]
This patch introduces java6 support in Martini allowing work of agents with Java6 binaries
This patch should be applied against org.eclipse.tptp.platform.jvmti.runtime
Created attachment 96761 [details]
patch to bciEng
Created attachment 96986 [details]
Java 6 StackMapTable support for Martini and Profilers
Patch enables optional Java 6 StackMapTable support in Martini and Profilers (CGProf, HeapProf, ThreadProf). The patch includes the following changes:
Martini
=======
- New Configure API for enabling StackMapTable calculation (default is OFF).
- Updates to JIE to utilize Harmony verifier code for calculating StackMapTable
- Updates to CGAdaptor to fix constructor method instrumentation and verification issues
Profilers
=========
- New command line option to JPIAgent: stackmap=true|false, to turn-on StackMapTable calculation. Default is OFF.
Alex, please review the Martini and Profile patch. "patch to bciEng" should be applied to directory "BCIEng" inside plugin, this is inconvenient. Please make next from plugin level.
I haven't managed to compile BCI after applying this patch. It doesn't contain some required files (at least BCIEngJ\class_interface.cpp and class_inerface_int.h). These classes are mentioned at patch beginning, probably some other from list also required.
? bci.1.patch
? BCIEngJ/BCIEngJ.plg
? BCIEngJ/bci.1.patch
? BCIEngJ/class_inerface_int.h
? BCIEngJ/class_interface.cpp
? BCIEngJ/class_interface_x.h
? BCIEngProbe/BCIEngProbe.plg
need to fix:
Copyrights notes should be updated and added to the new files
Lines 63-68 CModuleJ::~CModuleJ()
insert real free instead of /*XXX: free here*/
BCIEngJ/ModuleJ.cpp:274
remove useless malloc(50); and similar comments
BCIEngJ/ModuleJ.cpp:589
remove __asm {int 3}
BCIEngJ/ModuleJ.cpp:720 + some other places
remove commented code or add details about case
patch for Martini and Profiler looks Ok but also can't be compiled due absence of files
class_interface.cpp
class_inerface_int.h
Alex.
Created attachment 97124 [details]
Accumulative patch for BCI
Changes was made to meet Alexander's requests.
Alex could you please review the patch?
Created attachment 97125 [details]
BCI patch containing class_* files
Harmony contribution is not ready yet. The existing version we have has some blocking issues and cannot be used. Created attachment 97938 [details]
updated version of BCI-harmony verifier interface
Introduced changes in project files to compile against harmony verifier structure supposed to be following:
org.apache.harmony_vmcore_verifier\
build
lib\{window|linux}\{debug|release}\{IA-32|EM64T|IA64}
src\
include
vmcore\
include
src\verifier-3363
all patchs' shortcoming have been fixed Created attachment 98002 [details]
Introduces support of Stack Map (Java 6) callculcation includes Asaf patch with switch
This patch is updated version of Stack Map calculation patch for Martini with updated dependencies, described in "BCI-harmony verifier interface" patch
Created attachment 98071 [details]
Rising exception instead of asserion on fail of stack map recalculation
Introduced exception rising in case of verifier unable recalculate stack map
printf-debugging is removed
Created attachment 98143 [details]
Newest patch for org.eclipse.tptp.platform.jvmti.runtime (JVMTI Profiler/Martini)
Most up-to-date patch. Adapted to new Harmony contribution structure.
Open issues:
- Linux build still broken
Created attachment 98144 [details]
Newest patch for org.eclipse.hyades,probekit
Up-to-date patch. Adapted to new structure of Harmony code contribution.
Open issues:
- Linux build broken
- Unattended Windows IA64/EM64T build broken
- Unattended Windows IA32 debug build broken
Asaf, Where are you seeing the build errors? (In reply to comment #32) > Where are you seeing the build errors? There are currently no build errors in HEAD. Build errors will happen if you check-in the latest patch. I included these comments as a reminder (to me and others) that these issues must be resolved prior to checking-in these patches. Asaf Created attachment 98393 [details]
org.eclipse.hyades.probekit
1. Introduced global references for JNI objects (stability)
2. Enhanced ASCIIZ support (stability)
3. Includes build patch from Asaf
Created attachment 98824 [details]
Newest patch for org.eclipse.hyades,probekit
Updated patch.
1. Includes all fixes introduced in patch 98393 and patch 98144.
2. Fixes all build issues for all supported platforms (Windows and Linux)
Created attachment 98844 [details]
Newest patch for org.eclipse.hyades,probekit
Updated patch with fixed makefiles for Linux
Created attachment 98845 [details]
Newest patch for org.eclipse.tptp.platform.jvmti.runtime (JVMTI Profiler/Martini)
Updated patch with fixed makefiles for Linux
Created attachment 98935 [details] Newest patch for org.eclipse.tptp.platform.jvmti.runtime (JVMTI Profiler/Martini) Added workaround to bug 170075: when StackMapTable calculation is enabled, constructors are not instrumented for CGProf. Created attachment 99809 [details]
Newest patch for org.eclipse.tptp.platform.jvmti.runtime (JVMTI Profiler/Martini)
Fixed a Linux build issue
Created attachment 103085 [details]
Newest patch for org.eclipse.tptp.platform.jvmti.runtime (JVMTI Profiler/Martini)
Synchronized patch 99809 (now obsolete) with 4.5 HEAD revision.
Two patches committed to Head with PMC approval - copyrights updated on check in It seems like the latest probekit build doesnt work with a simple Hello World case. I get a standard windows error dialog with the message: "ProbeInstrumeter.exe has encountered a problem and needs to close. WE are sorry for the inconvinience. (With the option to "Send Error Report" and "Dont Send") " If I use any build before June 5, everything seems to work. I am using IBM Java 6. The patch for 220191 was commited around same time (May 30), so could it be that that patch impacted this i.e. StackMapTable wasn't properly patched for those return instructions Furthermore when I checked out the latest code, and tried to compile it using VisualStudio.Net I am getting "\org.eclipse.hyades.probekit\src-native\BCI\BCIEng\BCIEngJ\class_inerface_int.h(21): fatal error C1083: Cannot open include file: 'x_class_interface.h': No such file or directory" (In reply to comment #42) A separate bug was opened to track the problem (238427) which is now fixed.. Comment on attachment 98844 [details]
Newest patch for org.eclipse.hyades,probekit
adding iplog+ per Ganymede IP log
(In reply to comment #42) > Furthermore when I checked out the latest code, and tried to compile it using > VisualStudio.Net I am getting > "\org.eclipse.hyades.probekit\src-native\BCI\BCIEng\BCIEngJ\class_inerface_int.h(21): > fatal error C1083: Cannot open include file: 'x_class_interface.h': No such > file or directory" > Hi, I also encountered the same error. How did you fix this error? I used the latest version. Thanks, Yi check out the org.apache.harmony_vmcore_verifier package also. (In reply to comment #46) > check out the org.apache.harmony_vmcore_verifier package also. > Hi, I have checked out the harmony code to the same directory as Probekit component, i.e. the org.apache.harmony_vmcore_verifier/ and org.eclipse.hyades.probekit/ are under the same folder. The log is shown below: ProbeInstrumenter.cpp^M cl : Command line warning D9002 : ignoring unknown option '/ML'^M JVMUtils.cpp^M c:\cygwin\home\build_zy\TPTP\4.5.2\TPTP-4.5.2-200811100100\probekit\org.eclipse.hyades.probekit\src-native\BCI\BCIEng\BCIEngJ\class_inerface_int.h(21) : fatal error C1083: Cannot open include file: 'x_class_interface.h': No such file or directory^M Generating Code...^M NMAKE : fatal error U1077: 'cl.exe' : return code '0x2'^M Stop.^M NMAKE : fatal error U1077: 'c:\Progra~1\Micros~1\Bin\nmake.exe' : return code '0x2'^M Stop.^M NMAKE : fatal error U1077: 'c:\Progra~1\Micros~1\Bin\nmake.exe' : return code '0x2'^M Stop.^M NMAKE : fatal error U1077: 'c:\Progra~1\Micros~1\Bin\nmake.exe' : return code '0x2'^M Stop.^M Were you problem the same with mine? thanks, Yi (In reply to comment #47) yes I had the same problem. Howeve I was using VS and after checking out the verfier plugin, it worked.. Asaf: any ideas? (In reply to comment #48) Visual Studio .NET is not a supported compiler as far as I know. You must compile with VC++ 6. In my environment, I have no problems compiling with VC++ 6 after checking-out the harmony component. Asaf As of TPTP 4.6.0, TPTP is in maintenance mode and focusing on improving quality by resolving relevant enhancements/defects and increasing test coverage through test creation, automation, Build Verification Tests (BVTs), and expanded run-time execution. As part of the TPTP Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes), this enhancement/defect is verified/closed by the Project Lead since this enhancement/defect has been resolved and unverified for more than 1 year and considered to be fixed. If this enhancement/defect is still unresolved and reproducible in the latest TPTP release (http://www.eclipse.org/tptp/home/downloads/), please re-open. (In reply to comment #47) > > I have checked out the harmony code to the same directory as Probekit > component, i.e. the org.apache.harmony_vmcore_verifier/ and > org.eclipse.hyades.probekit/ are under the same folder. > > The log is shown below: > > ProbeInstrumenter.cpp^M > cl : Command line warning D9002 : ignoring unknown option '/ML'^M > JVMUtils.cpp^M > c:\cygwin\home\build_zy\TPTP\4.5.2\TPTP-4.5.2-200811100100\probekit\org.eclipse.hyades.probekit\src-native\BCI\BCIEng\BCIEngJ\class_inerface_int.h(21) > : fatal error C1083: Cannot open include file: 'x_class_interface.h': No such > file or directory^M > Generating Code...^M > NMAKE : fatal error U1077: 'cl.exe' : return code '0x2'^M > Stop.^M > NMAKE : fatal error U1077: 'c:\Progra~1\Micros~1\Bin\nmake.exe' : return code > '0x2'^M > Stop.^M > NMAKE : fatal error U1077: 'c:\Progra~1\Micros~1\Bin\nmake.exe' : return code > '0x2'^M > Stop.^M > NMAKE : fatal error U1077: 'c:\Progra~1\Micros~1\Bin\nmake.exe' : return code > '0x2'^M > Stop.^M > > Were you problem the same with mine? > > thanks, > Yi This problems occurs if you check out your sources in a subfolder too deeply nested in your local filesystem. This leads to the include not being found (as using /I "..\..\..\..\..\org.apache.harmony_vmcore_verifier\src\vmcore\src\verifier-3363\x_verifier" in the make file will make the already deeply nested path even longer). From our experiments it shows that using a path length of 42 characters will work, 58 characters will break. Your path is 53 characters long and will break as well. Boy, this hurts. |