Bugzilla – Full Text Bug Listing
| Summary: | Synchronization fails for large directories | ||
|---|---|---|---|
| Product: | [Tools] PTP | Reporter: | Greg Watson <g.watson> |
| Component: | RDT.sync | Assignee: | Greg Watson <g.watson> |
| Status: | CLOSED FIXED | QA Contact: | |
| Severity: | critical | ||
| Priority: | P3 | CC: | jdeblen1, roland |
| Version: | 5.0 | ||
| Target Milestone: | 5.0.1 | ||
| Hardware: | Macintosh | ||
| OS: | Mac OS X | ||
| Whiteboard: | |||
|
Description
Greg Watson
It looks like the failure is happening because this command is trying to be run. I presume there is some maximum length being exceeded. Is there some other way this could be done? git add include/36r2/ifs/hltend2flx.intfb.h ifs/phys_radi/suecozcaqua.F90 include/36r2/ifs/ppinitztl.intfb.ok grib_api-1.8.1/share/grib_api/definitions/grib2/tables/4/4.2.0.20.table include/36r2/ifs/suallo.intfb.ok grib_api-1.8.1/share/grib_api/definitions/grib2/tables/5/4.2.0.0.table grib_api-1.8.1/definitions/grib1/grid_definition_14.def grib_api-1.8.1/definitions/mars/grib1.waef.fcmax.def ifs/phys_dmn/hlradia.F90 include/36r2/ifs/swad.intfb.h grib_api-1.8.1/definitions/grib2/tables/2/4.2.10.1.table ifsaux/module/yomerrtrap.F90 surf/offline/additional/include/suvdf.intfb.h include/36r2/ifs/rtl_oberror.intfb.ok surf/offline/function/fcttim.h wam/Alt/getclo.F ifs/adiab/gppre.F90 dummy/source/aeolus_l2bp_setup_ec.F ifs/module/yomdyna.F90 ifs/ald_inc/interface/sufpezo.intfb.h grib_api-1.8.1/share/grib_api/definitions/grib2/tables/2/4.2.0.3.table ifsaux/lfi/lfiofd.F wam/Wam_oper/mk_opts_blsp2grs_FUJITSU4 grib_api-1.8.1/src/.deps/action_class_late_when.Po wam/Wam_oper/check.F include/36r2/ifs/wrhfp.intfb.h ifs/module/yomspnrm.F90 wam/Wam_oper/wamcur.F include/36r2/ifs/suafn.intfb.ok include/36r2/ifs/grg_ak_tl.intfb.ok ifs/var/sujqcor.F90 include/36r2/ifs/gpgrxybad.intfb.h grib_api-1.8.1/tests/grib1to2.sh wam/Wam_oper/sinput.F include/36r2/ifs/callpar.intfb.ok ifs/adiab/gphluvad.F90 ifs/namelist/nammars.h ifs/phys_radi/mcica_cld_gen.F90 include/36r2/ifs/cpqsol.intfb.h include/36r2/ifs/su_so4_A1B2030.intfb.h ifs/setup/sunh_vertfe1dd.F90 include/36r2/ifs/aer_ssalt_ms.intfb.h grib_api-1.8.1/definitions/grib2/tables/5/5.7.table include/36r2/ifs/susrtaer.intfb.ok grib_api-1.8.1/share/grib_api/definitions/grib2/tables/1/3.10.table include/36r2/ifs/popinteger4.intfb.ok include/36r2/ifs/gp_kappatl.intfb.h wam/Wam_oper/readwind.F.save grib_api-1.8.1/share/grib_api/definitions/mars/grib1.maed.me.def ifs/nmi/reord.F90 include/36r2/ifs/cubasen2.intfb.h wam/Wam_parameter/parameter_rmedite_b01.h grib_api-1.8.1/share/grib_api/definitions/grib2/tables/5/4.1.table include/36r2/ifs/ppgeopad.intfb.h grib_api-1.8.1/share/grib_api/definitions/grib2/template.3.resolution_flags.def include/36r2/ifs/radpar.intfb.ok include/36r2/ifs/biascor_odb.intfb.h grib_api-1.8.1/share/grib_api/definitions/mars/grib1.enfo.svar.def include/36r2/ifs/lavabo.intfb.ok grib_api-1.8.1/share/grib_api/definitions/mars/grib1.enda.ef.def include/36r2/ifs/gpnorm_gfl.intfb.h ifs/ald_inc/interface/evarjk.intfb.h grib_api-1.8.1/examples/C/.deps/iterator.Po ifs/var/jgnr.F90 ifs/op_obs/emis_mw_n.F90 ifs/phys_ec/cuddrafntl.F90 include/36r2/ifs/srtm_srtm_224gp_mcica.intfb.ok grib_api-1.8.1/share/grib_api/definitions/grib2/tables/5/3.2.table gribex/gribex/exscal.F surf/module/yos_soil.F90 include/36r2/ifs/hirs_cld.intfb.h ifs/dia/fpspnorm.F90 ifs/phys_ec/aer_clim.F90 grib_api-1.8.1/share/grib_api/definitions/grib2/tables/1/4.0.table grib_api-1.8.1/definitions/mars/grib1.oper.ed.def grib_api-1.8.1/definitions/grib2/tables/2/4.9.table include/36r2/ifs/opk_obscor.intfb.h grib_api-1.8.1/tools/.deps/grib_diff.Po grib_api-1.8.1/definitions/grib1/2.98.128.table wam/module/yowmespas.F wam/Buoy/mc_rearrng_6centre_common.F ifs/dia/grnorma.F90 grib_api-1.8.1/definitions/mars/grib1.oper.ia.def include/36r2/ifs/cacsts.intfb.h algor/linalg/suhert.F90 grib_api-1.8.1/share/grib_api/definitions/grib2/boot_multifield.def ifs/canari/castas.F90 ifs/module/yoeaermap.F90 ifs/obs_preproc/nscatin.F90 grib_api-1.8.1/share/grib_api/definitions/mars/grib1.enfo.pd.def ifs/setup/sufa.F90 grib_api-1.8.1/share/grib_api/definitions/grib2/tables/5/5.7.table ifs/namelist/namfpios.h grib_api-1.8.1/share/grib_api/definitions/mars/grib1.mofm.fcmean.def grib_api-1.8.1/share/grib_api/definitions/mars/grib1.ewda.cm.def grib_api-1.8.1/definitions/grib2/tables/4/1.0.table include/36r2/ifs/cainsu.intfb.ok include/36r2/ifs/cnmitl.intfb.ok include/36r2/ifs/nox2no2tl.intfb.ok ifs/adiab/cputqysad.F90 ifs/setup/sunhbmat.F90 grib_api-1.8.1/src/.deps/grib_accessor_class_ksec1expver.Po include/36r2/ifs/susc2a.intfb.h grib_api-1.8.1/definitions/grib2/tables/local/ecmf/obstat.1.0.table algor/interface/suhert.h grib_api-1.8.1/definitions/grib2/tables/5/4.216.table include/36r2/ifs/laitritl.intfb.h trans/module/ftinv_ctlad_mod.F90 include/36r2/ifs/satemis.intfb.h grib_api-1.8.1/share/grib_api/definitions/grib2/tables/4/1.3.table grib_api-1.8.1/share/grib_api/definitions/grib2/tables/3/3.1.table ifs/parallel/dot_product_ctlvec.F90 surf/offline/driver/sufgrb.F90 dummy/source/rttov_ec_k.F include/36r2/ifs/wrphtrajm.intfb.ok include/36r2/ifs/cobstl.intfb.ok grib_api-1.8.1/definitions/grib2/tables/local/ecmf/4/1.2.table grib_api-1.8.1/share/grib_api/samples/sh_ml_grib2.tmpl include/36r2/ifs/inter0.intfb.h include/36r2/ifs/lwvdrtl.intfb.h include/36r2/ifs/suecradl.intfb.ok ifs/pp_obs/ppqad.F90 ifs/ald_inc/interface/sueldynb.intfb.h ifs/pp_obs/tjquaa.F90 grib_api-1.8.1/share/grib_api/definitions/mars/grib1.mfhw.cf.def trans/module/sutrle_mod.F90 include/36r2/ifs/sufpios.intfb.h grib_api-1.8.1/share/grib_api/definitions/grib2/tables/2/4.2.3.0.table grib_api-1.8.1/definitions/mars/grib1.ewda.cd.def dummy/source/int_setparam_obsort.F include/36r2/ifs/ppt2mtl.intfb.h Makedummy/unique.pl grib_api-1.8.1/share/grib_api/definitions/grib2/tables/0/4.2.1.1.table grib_api-1.8.1/data/tigge/tigge_sbsj_sfc_10u.grib ifs/utility/dealnmi.F90 ifs/module/yomnmia.F90 include/36r2/ifs/sunhbmat.intfb.h ifs/setup/su_events.F90 gribex/gribex/btvax.special include/36r2/ifs/slcset.intfb.ok grib_api-1.8.1/share/grib_api/definitions/mars/grib1.enfo.wes.def ifs/module/varbc_rad.F90 ifs/phys_radi/srtm_kgb23.F90 gribex/gribex/unbkin.F grib_api-1.8.1/share/grib_api/definitions/mars/grib1.waef.efic.def wam/Sar/asspar include/36r2/ifs/rdphtrsf.intfb.ok ifs/adiab/lascaw_clo_ad.F90 grib_api-1.8.1/definitions/grib2/template.4.derived.def ifs/adiab/lanhsi_geogw.F90 grib_api-1.8.1/definitions/mars/grib1.mnvr.me.def ifs/dia/primddh.F90 ifs/module/yomppc.F90 surf/offline/additional/include/sucld.intfb.h dummy/source/elarmes5.F ifs/utility/emptb.F90 include/36r2/ifs/sucmahfp.intfb.ok include/36r2/ifs/sitnuad.intfb.ok include/36r2/ifs/ppvafl.intfb.ok grib_api-1.8.1/share/grib_api/definitions/mars/grib1.enfo.ep.def include/36r2/ifs/filfbde.intfb.h ifs/adiab/gpgrptl.F90 grib_api-1.8.1/definitions/mars/grib1.oper.wes.def include/36r2/ifs/acptke.intfb.h ifs/pp_obs/ppstaad.F90 ifs/var/scaleae.F90 grib_api-1.8.1/definitions/grib2/template.4.8.def grib_api-1.8.1/share/grib_api/definitions/mars/grib1.enda.efi.def ifs/phys_radi/rrtm_cmbgb1.F90 ifs/ocean/wrcpl.F90 ifs/var/sujbchvar.F90 grib_api-1.8.1/definitions/mars/grib1.ewda.es.def dummy/source/ejbwav_wav2cv.F grib_api-1.8.1/src/.deps/grib_accessor_class_late_when.Po include/36r2/ifs/posddh.intfb.h include/36r2/ifs/sumode3.intfb.h grib_api-1.8.1/src/.deps/action_class_gen.Po ifs/setup/sufrag.F90 ifs/phys_radi/swr.F90 ifs/phys_dmn/aplmphys.F90 grib_api-1.8.1/src/grib_accessor_class_array.c grib_api-1.8.1/share/grib_api/definitions/grib2/tables/4/4.91.table include/36r2/ifs/satur_1d.intfb.ok grib_api-1.8.1/src/grib_jasper_encoding.c include/36r2/ifs/radheat15.intfb.h grib_api-1.8.1/Makefile ifs/var/jbvcor_waveletin.F90 include/36r2/ifs/sudim1.intfb.ok include/36r2/ifs/suhlturb.intfb.h include/36r2/ifs/restart_cnt3.intfb.h include/36r2/ifs/lwinterf.intfb.ok grib_api-1.8.1/share/grib_api/definitions/grib2/tables/4/4.2.0.13.table ifs/op_obs/gpscalc_nr2dtl.F90 include/36r2/ifs/gppwc.intfb.ok include/36r2/ifs/suobsaddr.intfb.h ifs/namelist/namparar.h grib_api-1.8.1/definitions/grib2/template.4.parameter.def include/36r2/ifs/fpgeo.intfb.h surf/offline/namelist/namphy.h include/36r2/ifs/o3chem.intfb.ok wam/Wam_oper/intpol.F dummy/source/fanouv.F ifs/ald_inc/interface/ebalstat.intfb.h grib_api-1.8.1/definitions/mars/grib1.mmaw.me.def grib_api-1.8.1/share/grib_api/definitions/mars/grib1.enda.fcmax.def dummy/source/clim_import.F ifs/canari/cainsu.F90 ifs/parallel/read_spec.F90 include/36r2/ifs/evjq.intfb.h ifs/phys_dmn/aplpassh.F90 grib_api-1.8.1/src/grib_lex.c include/36r2/ifs/rdmoderr.intfb.ok include/36r2/ifs/cvarbc.intfb.ok grib_api-1.8.1/share/grib_api/definitions/grib1/units.def include/36r2/ifs/cpg_gp_tl.intfb.ok gribex/gribex/grprs1.F dummy/source/ejghcori.F include/36r2/ifs/gpnoxtl.intfb.ok grib_api-1.8.1/definitions/grib2/tables/1/4.2.0.14.table wam/module/yowaltas.F grib_api-1.8.1/share/grib_api/definitions/grib2/tigge_short_name.def ifs/phys_radi/su_so4_A1B2060.F90 ifsaux/include/dr_hook_util_multi.h include/36r2/ifs/sufpsc2.intfb.h ifs/var/avarc.F90 include/36r2/ifs/ini3wrfp.intfb.h ifs/var/balverti.F90 trans/module/spnormd_mod.F90 include/36r2/ifs/wrspec.intfb.ok include/36r2/ifs/spconvert.intfb.ok dummy/source/suecuv.F dummy/source/unloc.F grib_api-1.8.1/share/grib_api/definitions/mars/grib1.ewda.fg.def include/36r2/ifs/acqwlsrtl.intfb.ok include/36r2/ifs/suallocuf.intfb.h grib_api-1.8.1/share/grib_api/definitions/grib2/local.98.20.def include/36r2/ifs/ctvtotad.intfb.h ifs/fullpos/sufpphy.F90 scripts/bld_dummy.job grib_api-1.8.1/share/grib_api/definitions/grib2/template.4.5.def ifs/var/cvtest.F90 grib_api-1.8.1/data/tigge/tigge_cwao_sfc_sd.grib include/36r2/ifs/gpsro_2dad.intfb.h include/36r2/ifs/cvar3inad.intfb.ok ifs/phys_dmn/mf_phys_prep.F90 include/36r2/ifs/caimma.intfb.ok include/36r2/ifs/acevolet.intfb.h grib_api-1.8.1/definitions/grib2/tables/1/4.2.2.0.table include/36r2/ifs/gppref.intfb.h grib_api-1.8.1/examples/F90/copy_message.sh scripts/include/makelib include/36r2/ifs/disgrid.intfb.h ifs/ald_inc/interface/aro_conv_mnh.h ifs/namelist/namfpdyh.h ifs/fullpos/dynfpos.F90 include/36r2/ifs/sucoaphy.intfb.h include/36r2/ifs/lwai.intfb.h grib_api-1.8.1/src/grib_dependency.c include/36r2/ifs/post_prsta.intfb.h grib_api-1.8.1/share/grib_api/definitions/grib1/local.98.7.def include/36r2/ifs/grtest.intfb.h gribex/gribex/dsect4a.F include/36r2/ifs/radaer.intfb.ok grib_api-1.8.1/definitions/grib2/tables/5/3.0.table grib_api-1.8.1/definitions/grib2/tables/5/4.2.0.1.table ifs/fullpos/sufpios.F90 ifsaux/support/adStack.c include/36r... Yes, this command has this needs to be split. And this can be easily fixed. I don't think the execution framework provides the maximum number of characters but I guess we can just assume that 1000 is safe (to split it even smaller might make it slow). At least this should only happens if the large code is on the remote side. Or do you also get this if it you add a large code on the local side? Do you consider to respin for this bug? I've only tried large code on the remote side so far. If you have a fix, please commit. I think at this point it will go into 5.0.1 (likely to be released soon after 5.0). Okay, I'll look into it. I've committed a patch that fixes the above. Please take a look and modify if you want to do it another way. I'm now getting the following exceptions. I presume these are similar. org.eclipse.ptp.rdt.sync.git.core.RemoteSyncException: at org.eclipse.ptp.rdt.sync.git.core.GitRemoteSyncConnection.syncLocalToRemote(GitRemoteSyncConnection.java:647) at org.eclipse.ptp.rdt.sync.git.core.GitServiceProvider.synchronize(GitServiceProvider.java:260) at org.eclipse.ptp.internal.rdt.sync.core.ResourceChangeListener$SynchronizeJob.run(ResourceChangeListener.java:73) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54) Caused by: org.eclipse.ptp.rdt.sync.git.core.RemoteExecutionException: Remote merge failed with message: at org.eclipse.ptp.rdt.sync.git.core.GitRemoteSyncConnection.syncLocalToRemote(GitRemoteSyncConnection.java:648) ... 3 more and org.eclipse.ptp.rdt.sync.git.core.RemoteSyncException: at org.eclipse.ptp.rdt.sync.git.core.GitRemoteSyncConnection.syncLocalToRemote(GitRemoteSyncConnection.java:647) at org.eclipse.ptp.rdt.sync.git.core.GitServiceProvider.synchronize(GitServiceProvider.java:260) at org.eclipse.ptp.internal.rdt.sync.core.ResourceChangeListener$SynchronizeJob.run(ResourceChangeListener.java:73) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54) Caused by: org.eclipse.ptp.rdt.sync.git.core.RemoteExecutionException: Remote merge failed with message: fatal: 'merge' is not possible because you have unmerged files. Please, fix them up in the work tree, and then use 'git add/rm <file>' as appropriate to mark resolution and make a commit, or use 'git commit -a'. at org.eclipse.ptp.rdt.sync.git.core.GitRemoteSyncConnection.syncLocalToRemote(GitRemoteSyncConnection.java:648) ... 3 more The patch looks good. That is similar to how I would have done it. I assume the remaining errors are caused by bug 348938. No, here's the actual message coming from "git merge ptp-push" (the exception is getting it's message from stderr, but this message is on stdin): Auto-merging .settings/org.eclipse.cdt.core.prefs CONFLICT (add/add): Merge conflict in .settings/org.eclipse.cdt.core.prefs Automatic merge failed; fix conflicts and then commit the result. I excluded .settings from the sync (in getRemoteFileStatus()), but maybe it needs to happen somewhere else also? Yes, I think it needs to be added to FileFilter as well. Yes, I think so. FileFilter filters files that are sync'ed local to remote, whereas your change filtered the other direction. (Eventually I hope we can have the same filtering mechanism for both directions.) (In reply to comment #9) > Yes, I think so. FileFilter filters files that are sync'ed local to remote, > whereas your change filtered the other direction. (Eventually I hope we can > have the same filtering mechanism for both directions.) I agree we definitely should use the FileFilter for both. Obviously we cannot do the binary check remote but we should just skip the on the remote side. But all the filename based checks should of the FileFilter should also be used on the remote side. This appears to have fixed the problem. I'm now able to sync the large project. I have committed this to HEAD, but it will probably have to wait for 5.0.1. |