Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 202546

Summary: Could not start Agent Controller. Empty serviceconfig.xml.
Product: z_Archived Reporter: Eugene Chan <ewchan>
Component: TPTPAssignee: Igor Alelekov <igor.alelekov>
Status: CLOSED FIXED QA Contact:
Severity: blocker    
Priority: P1 CC: asaf.yaffe, ewchan, guru.nagarajan, igor.alelekov, jcayne, jkubasta, kiryl.kazakevich, nmehrega, samwai
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Bug Depends on:    
Bug Blocks: 203472    
Attachments:
Description Flags
patch
none
patched jar
none
updated patch
none
updated patched jar
none
updated patched jar none

Description Eugene Chan CLA 2007-09-06 18:01:14 EDT
-driver tptp441 0906 build
- start Java process in command prompt using JVMTI throws error complaining Agent Controller. 
- problem does not exist with JVMPI agent.

>java -agentlib:JPIBootLoader=JPIAgent:server=controlled;CGProf:execdetai
ls=true PerformanceExample
Could not start Agent Controller.
Empty serviceconfig.xml file
Make sure you have created a valid serviceconfig.xml file by running the SetConf
ig script.
Comment 1 Eugene Chan CLA 2007-09-06 18:01:35 EDT
This is failing all the profile on server use cases.
Comment 2 Guru Nagarajan CLA 2007-09-07 01:10:57 EDT
This issue is a IAC/New Execution framework issue
Comment 3 Guru Nagarajan CLA 2007-09-07 01:17:19 EDT
Is this a duplicate of 202078 
Comment 4 Eugene Chan CLA 2007-09-07 09:59:35 EDT
This is different from bug 202078.

I start RAC before I do any profile for attach use cases, and all attach use case failed because of this. 

I was using IBM JVM 1.5.
Comment 5 Samson Wai CLA 2007-09-10 10:00:39 EDT
I cannot reproduce using the 0910 driver. Here is what I have done:
1. Start ACServer in a command prompt
2. Run "java -agentlib:JPIBootLoader=JPIAgent:server=controlled;CGProf:execdetails=true StartStop" in another command prompt with all required environment set
3. Start Eclipse workbench and attach to the running TI agent
4. Attached successfully and profiling data is collected

The "Could not start Agent Controller." message should not come from the TI agent process. This message only exist in the ACServer code, not the agent code.

Please retry the 0910 driver and let me know if the same problem occurs.
Comment 6 Eugene Chan CLA 2007-09-10 11:06:41 EDT
It works if environment variables are pointing to RAC instead instead of eclipse plugins.

it works if I use:
set TPTP_AC_HOME=D:\ibm\AgentController
set JAVA_PROFILER_HOME=D:\ibm\AgentController\plugins\org.eclipse.tptp.javaprofi
ler
set PATH=%TPTP_AC_HOME%\bin;%JAVA_PROFILER_HOME%;%PATH%
set CLASSPATH=%JAVA_PROFILER_HOME%;%CLASSPATH%

and fails if I use:
set TPTP_AC_HOME=D:\ibm\4410-0910\eclipse\plugins\org.eclipse.tptp.platform.ac.w
in_ia32_4.4.0.v200706020100\agent_controller
set JAVA_PROFILER_HOME=D:\ibm\4410-0906\eclipse\plugins\org.eclipse.tptp.platfor
m.jvmti.runtime_4.4.100.v200709060100\agent_files\win_ia32
set PATH=%TPTP_AC_HOME%\bin;%JAVA_PROFILER_HOME%;%PATH%
set CLASSPATH=%JAVA_PROFILER_HOME%;%CLASSPATH%
Comment 7 Eugene Chan CLA 2007-09-10 11:13:49 EDT
The failing use case works if I start IAC(by launching workbench first) and instead of RAC for the attach.
Comment 8 Eugene Chan CLA 2007-09-10 14:57:03 EDT
this problem is blocking the profile on server use case. There is a requirement to have AC started up before any attach scenarios, and profile on server is an attach scenario. Hence having user start up RAC and profile server does not work right now.
Comment 9 Eugene Chan CLA 2007-09-10 14:58:02 EDT
This problem does not exist in 4402, could be changes made in the IAC code.
Comment 10 Eugene Chan CLA 2007-09-10 15:17:40 EDT
Interestingly, the problem does not exist with Thread Analysis.
Comment 11 Samson Wai CLA 2007-09-11 09:45:58 EDT
Sizing estimates.
Comment 12 jkubasta CLA 2007-09-11 10:43:28 EDT
Igor, can you please help with this fix?
Comment 13 Eugene Chan CLA 2007-09-11 15:24:08 EDT
This could be a inconsistence between IAC and RAC packaging, in which there maybe file that is auto generated in IAC startup. 
Comment 14 Igor Alelekov CLA 2007-09-12 08:09:02 EDT
Eugene,

- Sorry, what is your use case? You are starting AC from eclipse\plugins\ac.. directory and AC can't find serviceconfig.xml, right?

However, you are successfuly launching both AC - standalone and IAC.
I'm not sure that AC should be launched in the above use case.
AC plugin in eclipse directory can't configure itself since it knows nothing about other required plugins - hyades, jvmti.runtime...

However, if IAC was previously launched and serviceconfig.xml was created,
after that, failed use case in Comment #6 works without problems.
Comment 15 Eugene Chan CLA 2007-09-12 12:52:50 EDT
Hi Igor, 

The use case I run is:

With a fresh workspace...
1. start RAC in command prompt. ie that standalone AC outside eclipse.

2. in command prompt, setup environment variables for JVMTI standalone profile and  point varialbles to IAC libraries, ie.
set TPTP_AC_HOME=D:\ibm\4410-0910\eclipse\plugins\org.eclipse.tptp.platform.ac.w
in_ia32_4.4.0.v200706020100\agent_controller
set
JAVA_PROFILER_HOME=D:\ibm\4410-0906\eclipse\plugins\org.eclipse.tptp.platfor
m.jvmti.runtime_4.4.100.v200709060100\agent_files\win_ia32
set PATH=%TPTP_AC_HOME%\bin;%JAVA_PROFILER_HOME%;%PATH%
set CLASSPATH=%JAVA_PROFILER_HOME%;%CLASSPATH%

3. in the same command prompt, try JVMTI standalone profile:
java -agentlib:JPIBootLoader=JPIAgent:server=controlled;CGProf:execdetai
ls=true PerformanceExample

I got error message in step 3.
If I in step 2, point to path and libraries in RAC, things works okay. It seems like there is a different in the libraries packaged in IAC and RAC and they don't give consistent result when used.
Comment 16 Igor Alelekov CLA 2007-09-12 13:41:02 EDT
Thanks, Eugene :)
I'll check it tomorrow.
Comment 17 Igor Alelekov CLA 2007-09-13 06:38:21 EDT
Hi Eugene,
I got your use case in Comment #15 worked with the following preliminary step:

0. Start Eclipse and initialize IAC by clicking "test connection" button in the Profile Dialog. Shutdown Eclipse (in order to shutdown IAC).

This step creates serviceconfig.xml configuration file.
You see, to find out AC, an agent needs to get some info from this config file.
Curently, IAC is intended to be configured from Eclipse workbench only. That is why step 0. is required.
Thanks,
Igor
Comment 18 Eugene Chan CLA 2007-09-13 09:33:42 EDT
Thanks Igor, I got it working from your suggestion. However in the profile on server scenario, server is started in attach mode where environment variables are point to the IAC path, and there is no IAC running until the first attach to the server happens with a fresh workspace. The IAC initializes as attach dialog is shown and a query for the available agents for attach starts.

Is there a way to trigger the initialization of the IAC, if it's not yet running, for the profile on server scenarios?
Comment 19 Navid Mehregani CLA 2007-09-13 09:59:27 EDT
(In reply to comment #15)
> Hi Igor, 
> 
> The use case I run is:
> 
> With a fresh workspace...
> 1. start RAC in command prompt. ie that standalone AC outside eclipse.
> 
> 2. in command prompt, setup environment variables for JVMTI standalone profile
> and  point varialbles to IAC libraries, ie.
> set
> TPTP_AC_HOME=D:\ibm\4410-0910\eclipse\plugins\org.eclipse.tptp.platform.ac.w
> in_ia32_4.4.0.v200706020100\agent_controller
> set
> JAVA_PROFILER_HOME=D:\ibm\4410-0906\eclipse\plugins\org.eclipse.tptp.platfor
> m.jvmti.runtime_4.4.100.v200709060100\agent_files\win_ia32
> set PATH=%TPTP_AC_HOME%\bin;%JAVA_PROFILER_HOME%;%PATH%
> set CLASSPATH=%JAVA_PROFILER_HOME%;%CLASSPATH%
> 
> 3. in the same command prompt, try JVMTI standalone profile:
> java -agentlib:JPIBootLoader=JPIAgent:server=controlled;CGProf:execdetai
> ls=true PerformanceExample
> 
> I got error message in step 3.
> If I in step 2, point to path and libraries in RAC, things works okay. It seems
> like there is a different in the libraries packaged in IAC and RAC and they
> don't give consistent result when used.
> 


Eugene, if I understand this correctly, it seems like you're mixing and matching IAC and AC.  You're starting AC, but asking it to use IAC libraries!  This isn't legal.  The paths in AC's serverconfig.xml will be different than what you set your environment variables to.  This will obviously confuse both AC and IAC.

I remember having a long discussion about the usage of IAC in the attach scenario with Asaf.  IMHO, IAC shouldn't even be used in the attach scenario.  IAC is meant for novice users for hiding all the complexities of the Agent Controller's life cycle.  This is possible to do in the launch scenario.  We know exactly when to configure IAC, start, and stop it.  But it's not possible in the attach scenario, since the user is launching an agent in a different process.  To use IAC in the attach scenario, the user would have to start IAC by clicking on the 'Test Connection' button or take another action that would launch IAC.  In essence they're manually starting it.  This defeats the whole purpose of IAC, since it's meant to hide these details from the user.  Advanced users who want to attach to an agent should use AC so they can have full control of AC's life cycle.

>Is there a way to trigger the initialization of the IAC, if it's not yet
>running, for the profile on server scenarios?

Are you looking to start it programmatically?  If so, see the AutoStartStop class in the IAC plugin
Comment 20 Eugene Chan CLA 2007-09-13 10:14:07 EDT
Navid, I agree with you that mixing IAC and RAC should not be promoted. However, in the profile on server use case, servers are started with the environment variables setup and point to the IAC libraries programmatically. As user is responsible to start an AC beforehand for the attach scenario and there is no guarantee on where the RAC will be installed on the host, the profile on server code has to use the IAC path in its setup. The server process starting in attach mode could be running before an IAC is initialized and started. It seems like in this use case, I have to initialize IAC to create the serviceconfig.xml file, even if RAC is used. I looked at the AutoStartStop class, can I call org.eclipse.tptp.platform.iac.administrator.internal.common.AdminUtil.generateConfigFile() directly to just generate the file needed?

Thanks.
Comment 21 Eugene Chan CLA 2007-09-13 10:16:31 EDT
I'm going to try add
		if (AdminUtil.isConfigEmpty()) {
			AdminUtil.generateConfigFile();
		}
in the profile on server code before a server process is started in profile mode.
Comment 22 Igor Alelekov CLA 2007-09-13 10:19:24 EDT
Eugene,
AC could start agents/applications/servers on its startup.
Isn't it appropriate for your scenario?
Comment 23 Navid Mehregani CLA 2007-09-13 10:25:48 EDT
Yes, you can call AdminUtil.generateConfigFile(), but you're asking for trouble whenever you mix up IAC and AC :)
Comment 24 Eugene Chan CLA 2007-09-13 10:29:10 EDT
There is no choice here as I have no idea which AC user is going to profile with and I only have the path information of IAC. 
Comment 25 Eugene Chan CLA 2007-09-13 10:31:08 EDT
I can get the server started now, but failed when try to attach to the process after it finished its start up process. Igor, does this ring your bell?

the console output I got is:
...
INFO: Server startup in 21925 ms
[Error: Attach failed]
Comment 26 Eugene Chan CLA 2007-09-13 10:32:49 EDT
Navid, where is the IAC's serviceconfig.xml file be saved? I removed the one under eclipse\plugins\org.eclipse.tptp.platform.ac.win_ia32_4.4.0.v200706020100\agent_controller\config
but there is no complain about missing file.
Comment 27 Eugene Chan CLA 2007-09-13 10:34:11 EDT
(In reply to comment #22)
> Eugene,
> AC could start agents/applications/servers on its startup.
> Isn't it appropriate for your scenario?
> 

Igor, could you please give more information here? Thanks.
Comment 28 Igor Alelekov CLA 2007-09-13 10:39:34 EDT
Any application can be configured to be started by AC (agents folder in $AC_HOME).
It is simple to allow agents/applications to be started automatically after AC starts - see enh. https://bugs.eclipse.org/bugs/show_bug.cgi?id=184277
Comment 29 Igor Alelekov CLA 2007-09-13 10:42:20 EDT
(In reply to comment #25)
> I can get the server started now, but failed when try to attach to the process
> after it finished its start up process. Igor, does this ring your bell?
> the console output I got is:
> ...
> INFO: Server startup in 21925 ms
> [Error: Attach failed]

Are you starting server with JVMTI agent before AC start-up?
Comment 30 Eugene Chan CLA 2007-09-13 10:44:06 EDT
Thanks Igor, Server processes are started manually by user in the UI, and your suggestion does not apply to the use case.
Comment 31 Eugene Chan CLA 2007-09-13 10:45:00 EDT
(In reply to comment #29)
> 
> Are you starting server with JVMTI agent before AC start-up?
> 

No, the server is started with JVMTI agent after RAC is start-up.
Comment 32 Eugene Chan CLA 2007-09-13 12:44:59 EDT
Igor, I got more information after several trials.

Even if I start IAC by profiling a simple helloworld and follow by a profile on server process, it failed.

here is what I got.
Sep 13, 2007 12:40:42 PM org.apache.catalina.storeconfig.StoreLoader load
INFO: Find registry server-registry.xml at classpath resource
[Error: Attach failed]
Unhandled exception
Type=Segmentation error vmState=0x009274a8
J9Generic_Signal_Number=00000004 ExceptionCode=c0000005 ExceptionAddress=43057F53 ContextFlags=0001003f
Handler1=7EFB0630 Handler2=7F057A80 InaccessibleAddress=00000000
EDI=4306D208 ESI=4306D1F8 EAX=00000000 EBX=448673B0
ECX=00000000 EDX=00000000
EIP=43057F53 ESP=0006FB74 EBP=0006FCCC
Module=D:\ibm\4410-0910\eclipse\plugins\org.eclipse.tptp.platform.jvmti.runtime_4.4.100.v200709060100\agent_files\win_ia32\CGProf.dll
Module_base_address=43050000 Offset_in_DLL=00007f53
Target=2_30_20070131_11312_lHdSMR (Windows XP 5.1 build 2600 Service Pack 2)
CPU=x86 (2 logical CPUs) (0x7fe5b000 RAM)
JVMDUMP006I Processing Dump Event "gpf", detail "" - Please Wait.
JVMDUMP007I JVM Requesting System Dump using 'D:\ibm\4410-0910\eclipse\core.20070913.124044.3200.dmp'
Unhandled exception
Type=Segmentation error vmState=0x00000000
J9Generic_Signal_Number=00000004 ExceptionCode=c0000005 ExceptionAddress=43057F53 ContextFlags=0001003f
Handler1=7EFB0630 Handler2=7F057A80 InaccessibleAddress=00000000
EDI=4306D208 ESI=4306D1F8 EAX=00000000 EBX=45885B50
ECX=00000000 EDX=00000000
EIP=43057F53 ESP=4960FD98 EBP=4960FEF0
Module=D:\ibm\4410-0910\eclipse\plugins\org.eclipse.tptp.platform.jvmti.runtime_4.4.100.v200709060100\agent_files\win_ia32\CGProf.dll
Module_base_address=43050000 Offset_in_DLL=00007f53
Target=2_30_20070131_11312_lHdSMR (Windows XP 5.1 build 2600 Service Pack 2)
CPU=x86 (2 logical CPUs) (0x7fe5b000 RAM)
JVMDUMP006I Processing Dump Event "gpf", detail "" - Please Wait.
Comment 33 Igor Alelekov CLA 2007-09-14 03:26:02 EDT
Hi Eugene,
It seems the exception was throwen in CGProf.dll module.
Please open a new bugzilla to track this.
BTW, the problem with config file is resolved, can we close this bug?
Comment 34 Navid Mehregani CLA 2007-09-14 09:23:30 EDT
(In reply to comment #26)
> Navid, where is the IAC's serviceconfig.xml file be saved? I removed the one
> under
> eclipse\plugins\org.eclipse.tptp.platform.ac.win_ia32_4.4.0.v200706020100\agent_controller\config
> but there is no complain about missing file.
> 

It should be in that location.  Try stopping IAC (by exiting workbench) and starting it again (by profiling).  It should complain then.
Comment 35 Eugene Chan CLA 2007-09-14 12:36:58 EDT
(In reply to comment #33)
> Hi Eugene,
> It seems the exception was throwen in CGProf.dll module.
> Please open a new bugzilla to track this.
> BTW, the problem with config file is resolved, can we close this bug?
> 

Igor, I will have to attach the patch of the fix in comment #21, test it on different platforms and get PMC approval before closing this bug.

I will also open a separate defect to track the problem in comment #32.
Comment 36 Eugene Chan CLA 2007-09-14 14:09:21 EDT
Created attachment 78454 [details]
patch
Comment 37 Eugene Chan CLA 2007-09-14 14:09:37 EDT
Created attachment 78455 [details]
patched jar
Comment 38 Eugene Chan CLA 2007-09-14 14:51:54 EDT
I have verified on WinXP and RHEL that the patch solve the problem and it's not complaining about the following anymore.
 Could not start Agent Controller.
 Empty serviceconfig.xml file
 Make sure you have created a valid serviceconfig.xml file by running the
 SetConfig script.

However, it still failed in the attach, and show the following error:
 ...
 INFO: Server startup in 21925 ms
 [Error: Attach failed]

I don't consistently get what is mentioned in comment #32. 

I have open bug 203472 to follow up on this, and mark this bug blocked by it.
Comment 39 Eugene Chan CLA 2007-09-14 14:56:39 EDT
To smooth the experience for user, when neither RAC or IAC is running, profile on server code should start IAC before server process is started to support the 'Running AC beforehand' requirement for profile on server use case.
Comment 40 Eugene Chan CLA 2007-09-17 10:58:35 EDT
Created attachment 78553 [details]
updated patch
Comment 41 Eugene Chan CLA 2007-09-17 10:59:16 EDT
Created attachment 78554 [details]
updated patched jar
Comment 42 Eugene Chan CLA 2007-09-17 10:59:20 EDT
Created attachment 78555 [details]
updated patched jar
Comment 43 Joel Cayne CLA 2007-09-17 16:14:45 EDT
I was able to reproduce the problem:
 INFO: Server startup in 21925 ms
 [Error: Attach failed]
on my Windows XP machine running the TPTP-4.4.1-200709101254 candidate driver with the latest version of the updated_patched.jar attached to the defect. 

Before running the test in the workbench, I ran the command line steps and also received the errors:
 Could not start Agent Controller.
 Empty serviceconfig.xml file
 Make sure you have created a valid serviceconfig.xml file by running the
 SetConfig script.

After running the test in the workbench, I did not see the error on the command line and the serviceconfig.xml file was not empty.
Comment 44 Igor Alelekov CLA 2007-10-25 09:47:26 EDT
Hi Eugene, do you have an update for this?
Comment 45 Eugene Chan CLA 2007-10-25 10:28:04 EDT
(In reply to comment #44)
> Hi Eugene, do you have an update for this?
> 

Hi Igor, the patch in comment #40 solve the problem of library references in the profile on server use case (please review and check it in). However, the complete use case is still blocked by bug 203472.
Comment 46 Igor Alelekov CLA 2008-01-24 05:53:39 EST
Hi Eugene,
The problem in this bugzilla seems to be fixed.
I'm going to resolve it.
Comment 47 Igor Alelekov CLA 2008-01-25 05:08:27 EST
Resolving as fixed
Comment 48 Eugene Chan CLA 2009-02-24 17:08:38 EST
CLOSE BUG