Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 344233 - [DB] Get Max VARCHAR size from DBAdapter
Summary: [DB] Get Max VARCHAR size from DBAdapter
Status: CLOSED FIXED
Alias: None
Product: EMF
Classification: Modeling
Component: cdo.db (show other bugs)
Version: 4.2   Edit
Hardware: PC Windows XP
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Eike Stepper CLA
QA Contact: Eike Stepper CLA
URL:
Whiteboard: Lighter, Faster and Better
Keywords:
: 369612 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-04-29 01:28 EDT by Erdal Karaca CLA
Modified: 2013-06-27 03:33 EDT (History)
4 users (show)

See Also:


Attachments
Get column length from DB adapters (4.30 KB, patch)
2012-10-19 09:49 EDT, András Péteri CLA
no flags Details | Diff
Patch OracleAdapter (838 bytes, patch)
2012-11-11 07:54 EST, Silvestre Martins CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Erdal Karaca CLA 2011-04-29 01:28:28 EDT
In Oracle db VARCHAR has a max character count of 4000 which leads to errors when auto creating tables.
The proper code locations already states:

    // TODO: implement DBAdapter.getDBLength
    // mappingStrategy.getStore().getDBAdapter().getDBLength(type);
    // which should then return the correct default field length for the db type
    return type == DBType.VARCHAR ? 32672 : IDBField.DEFAULT;
    
I changed it to 4000 to work with Oracle.
Comment 1 Eike Stepper CLA 2011-06-23 03:59:07 EDT
Moving all open enhancement requests to 4.1
Comment 2 Eike Stepper CLA 2012-08-14 22:53:38 EDT
Moving all open issues to 4.2. Open bugs can be ported to 4.1 maintenance after they've been fixed in master.
Comment 3 András Péteri CLA 2012-10-19 09:49:02 EDT
Created attachment 222579 [details]
Get column length from DB adapters

The attached patch implements the TODO by introducing an additional interface.

The MySQL adapter would also need return a lower value than the default (when using multibyte characters, even a single column will exceed the internal 64kB row limit), but VARCHAR handling in general seems to be routed over to use LONGTEXTs currently in that adapter.
Comment 4 Eike Stepper CLA 2012-10-30 03:19:22 EDT
Hi Andras,

I can imagine that we'll have to add similar methods to IDBAdapter in the future. So I removed your extension interface and marked IDBAdapter @noimplement. I also changed the dbAdapters extension point definition to only accept subclasses of DBAdapter (rather than any implementations of IDBAdapter).

Thanks!
Comment 5 Eike Stepper CLA 2012-10-30 03:21:06 EDT
commit 2f8c489d32d48e9fb686a1e2ed2f092c8cebb823

Unfortunately we can't port this to maintenance because it's not forward compatible.
Comment 6 Eike Stepper CLA 2012-11-02 04:21:38 EDT
*** Bug 369612 has been marked as a duplicate of this bug. ***
Comment 7 Silvestre Martins CLA 2012-11-11 07:54:59 EST
Created attachment 223432 [details]
Patch OracleAdapter

The existing patch applies the necessary changes in order to define dynamically (per DBAdapter) the length of a specific field. However, it's still needed to implement/override in the specific Adapter, the method getFieldLength() in order to return the correct value for the specific DB.

Currently this was not causing any problem because the length for VARCHAR was directly defined (statically to 4000) in the method getTypeName() in the OracleAdapter.

My suggestion is to not define the length for VARCHAR hard-coded, but instead use the length defined for the field, for 2 reasons:

 1) In case the model defines a short string (less than 4000 bytes) on the ecore annotations, then we don't need to allocate always the 4000 bytes.

 2) In case the application is modeled to store more than 4000 bytes for a String attribute (though annotation), it would throw an error at the time of schema creation. Otherwise we could be hiding this fact here (at schema creation), and only when actually storing the value it would throw an error.
 Of course we cannot avoid the problem of model not being defined with a proper limit (when it's expected to have text higher than 4000) but we still can add an hint on the documentation as a good practice, to always define the maximum length for a String attribute through an annotation when using it with CDO, to optimize the schema.


Attached a patch for the OracleAdapter with the changes I suggested.
Please comment.
Comment 8 Eike Stepper CLA 2012-11-16 04:24:47 EST
What you suggest makes sense to me. I'll apply your patch. Please test it well, because I don't have Orcale.
Comment 9 Eike Stepper CLA 2012-11-16 04:27:06 EST
commit 09f5d478bbdc3a2740617c6d14caf8b25fb51e1f
Comment 10 Silvestre Martins CLA 2012-11-17 14:37:18 EST
(In reply to comment #8)
> What you suggest makes sense to me. I'll apply your patch. Please test it
> well, because I don't have Orcale.

Tested in Oracle XE: status Ok - table column is created with the defined length by the EAnnotation, or 4000 if not specified.

If there's a way to document this it would be nice, to give the hint to developers that they should always define the maximum length for the String types in the ecore model.
Comment 11 Silvestre Martins CLA 2012-11-17 14:40:54 EST
> If there's a way to document this it would be nice, to give the hint to
> developers that they should always define the maximum length for the String
> types in the ecore model.

Maybe we could add a comment in the wiki for CDO, under the setting for EMF:
http://wiki.eclipse.org/Tweaking_CDO_Performance#Setting_EMF_Parameters

Even we don't guarantee this will boost the application, is always an optimization we can do in the database.
What do you think?
Comment 12 Eike Stepper CLA 2012-11-18 00:46:40 EST
(In reply to comment #10)
> Tested in Oracle XE: status Ok - table column is created with the defined
> length by the EAnnotation, or 4000 if not specified.

Excellent. Thank you!

> If there's a way to document this it would be nice, to give the hint to
> developers that they should always define the maximum length for the String
> types in the ecore model.

Actually I'm not sure if this aspect is relevant for performance (see comment 11). What about adding something here: http://wiki.eclipse.org/CDO/DB_Store ?
Comment 13 Eike Stepper CLA 2012-11-18 00:48:26 EST
(In reply to comment #11)
> > If there's a way to document this it would be nice, to give the hint to
> > developers that they should always define the maximum length for the String
> > types in the ecore model.
> 
> Maybe we could add a comment in the wiki for CDO, under the setting for EMF:
> http://wiki.eclipse.org/Tweaking_CDO_Performance#Setting_EMF_Parameters
> 
> Even we don't guarantee this will boost the application, is always an
> optimization we can do in the database.
> What do you think?

We should keep the documentation in the separate DBStore page that I referred to in comment 12. But we could certainly link that from http://wiki.eclipse.org/Tweaking_CDO_Performance
Comment 14 Silvestre Martins CLA 2012-11-19 07:34:06 EST
(In reply to comment #12)
> (In reply to comment #10)
> > Tested in Oracle XE: status Ok - table column is created with the defined
> > length by the EAnnotation, or 4000 if not specified.
> 
> Excellent. Thank you!
> 
> > If there's a way to document this it would be nice, to give the hint to
> > developers that they should always define the maximum length for the String
> > types in the ecore model.
> 
> Actually I'm not sure if this aspect is relevant for performance (see
> comment 11). What about adding something here:
> http://wiki.eclipse.org/CDO/DB_Store ?

It seems ok for me. If you want, I can update it accordingly.
Comment 15 Eike Stepper CLA 2012-11-19 07:48:41 EST
That would be awesome ;-)

Thank you!
Comment 16 Silvestre Martins CLA 2012-11-20 18:03:12 EST
Wiki updated:
http://wiki.eclipse.org/index.php?title=CDO/DB_Store
Comment 17 Eike Stepper CLA 2013-06-27 03:33:27 EDT
Available in R20130613-1157 (4.2)