Issue 4294: tk_indirect (interop) Source: (Mr. Simon C. Nash, ) Nature: Uncategorized Issue Severity: Summary: I don't understand why tk_indirect isn't allowed as a top-level typecode. This causes servere performance penalties for RMI-IIOP because the Java to IDL mapping requires that Java objects of declared type java.lang.Object are marshalled as CORBA anys. In the case of a Vector or HashTable with 100 elements, this means that 100 anys must be marshalled. If all of these are of actual type foo, the restriction on tk_indirect means that all 100 of these data values must repeat the typeCode for foo, which could be very large. This causes very substantial overheads, since the space and time needed to marshal the TypeCode for foo can greatly exceed that needed to marshal the data for foo. I understand why a nested tk_indirect cannot refer to any TypeCode outside the scope of its enclosing top-level TypeCode. However, I don't think this restriction needs to apply to a top-level TypeCode. We have made this change experimentally without any adverse effects and we have discovered that using tk_indirect for all the top-level foo TypeCodes after the first one can speed up some common scenarios by at least a factor of 5 on end-to-end measurements. There seems to be no downside to making this change. I would therefore like to propose the following change to the section headed "Indirection: Recursive and Repeated TypeCodes" within section 15.3.5.1: Change the first bullet from: The indirection applies only to TypeCodes nested within some “top-level” TypeCode. Indirected TypeCodes are not “freestanding,” but only exist inside some other encoded TypeCode. by the following words: The indirection applies only to TypeCodes nested within some “top-level” TypeCode, or from one top-level TypeCode to another. Indirected TypeCodes nested within a top-level TypeCode can only reference TypeCodes that are part of the same top-level TypeCode, including the top-level TypeCode itself. Indirected top-level TypeCodes can reference other top-level TypeCodes but cannot reference TypeCodes nested within some other top-level TypeCode. If this change finds favour, then we need to work out how it can be brought into GIOP without causing problems interoperating with older ORBs that insist on the stronger restriction of the current spec. This could perhaps be added to the "wish list" for GIOP 1.3. Resolution: Close this issue and add it to the GIOP wish list Revised Text: Actions taken: May 4, 2001: received issue October 3, 2001: closed issue Discussion: The proposed change would also require encapsulations to have a CDR version (already idiscussed on the GIOP wish list). This is too big a change for RTF. Close this issue and add it to the GIOP wish list. End of Annotations:===== Date: Fri, 04 May 2001 13:18:57 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.72 [en] (Windows NT 5.0; I) X-Accept-Language: en MIME-Version: 1.0 To: issues@omg.org CC: interop@omg.org Subject: tk_indirect Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=iso-8859-1 X-UIDL: [;9!!dDU!!o@P!!8,Ce9 I don't understand why tk_indirect isn't allowed as a top-level typecode. This causes servere performance penalties for RMI-IIOP because the Java to IDL mapping requires that Java objects of declared type java.lang.Object are marshalled as CORBA anys. In the case of a Vector or HashTable with 100 elements, this means that 100 anys must be marshalled. If all of these are of actual type foo, the restriction on tk_indirect means that all 100 of these data values must repeat the typeCode for foo, which could be very large. This causes very substantial overheads, since the space and time needed to marshal the TypeCode for foo can greatly exceed that needed to marshal the data for foo. I understand why a nested tk_indirect cannot refer to any TypeCode outside the scope of its enclosing top-level TypeCode. However, I don't think this restriction needs to apply to a top-level TypeCode. We have made this change experimentally without any adverse effects and we have discovered that using tk_indirect for all the top-level foo TypeCodes after the first one can speed up some common scenarios by at least a factor of 5 on end-to-end measurements. There seems to be no downside to making this change. I would therefore like to propose the following change to the section headed "Indirection: Recursive and Repeated TypeCodes" within section 15.3.5.1: Change the first bullet from: The indirection applies only to TypeCodes nested within TypeCode. Indirected TypeCodes are some not but only exist inside some other encoded TypeCode. by the following words: The indirection applies only to TypeCodes nested within er. Indirected TypeCodes nested within a top-level TypeCode can only reference TypeCodes that are part of the same top-level TypeCode, including the top-level TypeCode itself. Indirected top-level TypeCodes can reference other top-level TypeCodes but cannot reference TypeCodes nested within some other top-level TypeCode. If this change finds favour, then we need to work out how it can be brought into GIOP without causing problems interoperating with older ORBs that insist on the stronger restriction of the current spec. This could perhaps be added to the "wish list" for GIOP 1.3. Simon -- Simon C Nash, Chief Technical Officer, IBM Java Technology Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb some Date: Fri, 18 May 2001 16:43:23 +1000 (EST) From: Michi Henning Reply-To: Interoperability RTF To: Interoperability RTF Subject: Re: issue 4294 -- Interop RTF issue In-Reply-To: <4.3.2.7.2.20010516145402.02539d80@emerald.omg.org> Message-ID: Organization: IONA Technologies MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-UIDL: +,c!!Y8-!!!Tn!!@#N!! Simon Nash wrote: > I would therefore like to propose the following change to the section headed > "Indirection: Recursive and Repeated TypeCodes" within section 15.3.5.1: > > Change the first bullet from: > > The indirection applies only to TypeCodes nested within some "top-level" > TypeCode. Indirected TypeCodes are not "freestanding," but only exist inside > some other encoded TypeCode. > > by the following words: > > The indirection applies only to TypeCodes nested within some "top-level" > TypeCode, or from one top-level TypeCode to another. Indirected TypeCodes > nested within a top-level TypeCode can only reference TypeCodes that are part > of the same top-level TypeCode, including the top-level TypeCode itself. > Indirected top-level TypeCodes can reference other top-level TypeCodes but > cannot reference TypeCodes nested within some other top-level TypeCode. > > If this change finds favour, then we need to work out how it can be brought > into GIOP without causing problems interoperating with older ORBs that insist > on the stronger restriction of the current spec. This could perhaps be > added to the "wish list" for GIOP 1.3. Simon, I like the idea in principle. Marshaling of TypeCodes and Anys is very inefficient, especially when sequences are involved or when you are sending lots of Anys containing the same type of value through an event channel. Improving on this would certainly be nice. Having said that, I don't think your suggestion would work, even if we do create a new GIOP revision: if a type code appears inside an encapsulation, the receiving ORB has no idea of how that type code was encoded. ORBs preceeding the hypothetical GIOP 1.3 version would then get confused about how to decode the type code. Basically, I don't see how we can touch anything at all in the CDR encoding of existing types as long as encapsulations can't indicate something like a CDR version number. Cheers, Michi. -- Michi Henning +61 7 3324 9633 Chief CORBA Scientist +61 4 1118 2700 (mobile) IONA Technologies +61 7 3324 9799 (fax) Total Business Integration http://www.ooc.com.au/staff/michi Date: Tue, 05 Jun 2001 12:53:56 -0400 From: Jishnu Mukerji Reply-To: jishnu_mukerji@hp.com Organization: Hewlett-Packard EIAL, Florham Park NJ, USA X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Simon Nash Cc: issues@omg.org, interop@omg.org Subject: Re: tk_indirect References: <3AF29E31.1BCD0D06@hursley.ibm.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=iso-8859-1 X-UIDL: Xm7e9*;9!!#i^d9b3S!! I would like to point out, for the sake of everyone's sanity, that there is no TCKind called tk_indirect, at least in the standard CORBA Core and Interop specifications. The value of 0xffffffff, which does not have a name (see table 15-2) is used as an indirection tag in CDR encoding. Please avoid calling this thing tk_indirect to reduce confusion. The actual TCKind-s with the tk_* form names have a one to one correspondence with enum values of type TCKind in TypeCodes. Merci, Jishnu. Simon Nash wrote: > > I don't understand why tk_indirect isn't allowed as a top-level typecode. > This causes servere performance penalties for RMI-IIOP because the Java to IDL > mapping requires that Java objects of declared type java.lang.Object are > marshalled as CORBA anys. In the case of a Vector or HashTable with 100 > elements, this means that 100 anys must be marshalled. If all of these are > of actual type foo, the restriction on tk_indirect means that all 100 of > these data values must repeat the typeCode for foo, which could be very large. > This causes very substantial overheads, since the space and time needed to > marshal the TypeCode for foo can greatly exceed that needed to marshal the > data for foo. > > I understand why a nested tk_indirect cannot refer to any TypeCode outside > the scope of its enclosing top-level TypeCode. However, I don't think this > restriction needs to apply to a top-level TypeCode. We have made this > change experimentally without any adverse effects and we have discovered that > using tk_indirect for all the top-level foo TypeCodes after the first one can > speed up some common scenarios by at least a factor of 5 on end-to-end > measurements. There seems to be no downside to making this change. > > I would therefore like to propose the following change to the section headed > "Indirection: Recursive and Repeated TypeCodes" within section 15.3.5.1: > > Change the first bullet from: > > The indirection applies only to TypeCodes nested within > TypeCode. Indirected TypeCodes are some not but only exist inside > some other encoded TypeCode. > > by the following words: > > The indirection applies only to TypeCodes nested within eCode to another. Indirected TypeCodes > nested within a top-level TypeCode can only reference TypeCodes that are part > of the same top-level TypeCode, including the top-level TypeCode itself. > Indirected top-level TypeCodes can reference other top-level TypeCodes but > cannot reference TypeCodes nested within some other top-level TypeCode. > > If this change finds favour, then we need to work out how it can be brought > into GIOP without causing problems interoperating with older ORBs that insist > on the stronger restriction of the current spec. This could perhaps be > added to the "wish list" for GIOP 1.3. > > Simon > -- > Simon C Nash, Chief Technical Officer, IBM Java Technology > Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England > Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb -- Jishnu Mukerji Senior Systems Architect EIAL, Hewlett-Packard Company 300 Campus Drive, MS 2E-62 Florham Park NJ 07932, USA +1 973 443 7528 jishnu_mukerji@hp.com Date: Tue, 05 Jun 2001 20:52:04 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.72 [en] (Windows NT 5.0; I) X-Accept-Language: en MIME-Version: 1.0 To: jishnu_mukerji@hp.com CC: interop@omg.org Subject: Re: tk_indirect References: <3AF29E31.1BCD0D06@hursley.ibm.com> <3B1D0EA4.7AA7C65C@hp.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=iso-8859-1 X-UIDL: `)-!!Gif!!$@2!!'K!!! Jishnu, Thanks for pointing this out. I stand corrected. Can I take the lack of feedback on this issue as consensus that the change is valid and has no problems other than how to introduce it without breaking compatibility? Simon Jishnu Mukerji wrote: > > I would like to point out, for the sake of everyone's sanity, that there > is no TCKind called tk_indirect, at least in the standard CORBA Core and > Interop specifications. The value of 0xffffffff, which does not have a > name (see table 15-2) is used as an indirection tag in CDR encoding. > Please avoid calling this thing tk_indirect to reduce confusion. The > actual TCKind-s with the tk_* form names have a one to one > correspondence with enum values of type TCKind in TypeCodes. > > Merci, > > Jishnu. > > Simon Nash wrote: > > > > I don't understand why tk_indirect isn't allowed as a top-level typecode. > > This causes servere performance penalties for RMI-IIOP because the Java to IDL > > mapping requires that Java objects of declared type java.lang.Object are > > marshalled as CORBA anys. In the case of a Vector or HashTable with 100 > > elements, this means that 100 anys must be marshalled. If all of these are > > of actual type foo, the restriction on tk_indirect means that all 100 of > > these data values must repeat the typeCode for foo, which could be very large. > > This causes very substantial overheads, since the space and time needed to > > marshal the TypeCode for foo can greatly exceed that needed to marshal the > > data for foo. > > > > I understand why a nested tk_indirect cannot refer to any TypeCode outside > > the scope of its enclosing top-level TypeCode. However, I don't think this > > restriction needs to apply to a top-level TypeCode. We have made this > > change experimentally without any adverse effects and we have discovered that > > using tk_indirect for all the top-level foo TypeCodes after the first one can > > speed up some common scenarios by at least a factor of 5 on end-to-end > > measurements. There seems to be no downside to making this change. > > > > I would therefore like to propose the following change to the section headed > > "Indirection: Recursive and Repeated TypeCodes" within section 15.3.5.1: > > > > Change the first bullet from: > > > > The indirection applies only to TypeCodes nested within > > TypeCode. Indirected TypeCodes are some not but only exist inside > > some other encoded TypeCode. > > > > by the following words: > > > > The indirection applies only to TypeCodes nested within > > TypeCode, or from one top-level TypeCode to another. Indirected TypeCodes > > nested within a top-level TypeCode can only reference TypeCodes that are part > > of the same top-level TypeCode, including the top-level TypeCode itself. > > Indirected top-level TypeCodes can reference other top-level TypeCodes but > > cannot reference TypeCodes nested within some other top-level TypeCode. > > > > If this change finds favour, then we need to work out how it can be brought > > into GIOP without causing problems interoperating with older ORBs that insist > > on the stronger restriction of the current spec. This could perhaps be > > added to the "wish list" for GIOP 1.3. > > > > Simon > > -- > > Simon C Nash, Chief Technical Officer, IBM Java Technology > > Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England > > Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb > > -- > Jishnu Mukerji > Senior Systems Architect > EIAL, Hewlett-Packard Company > 300 Campus Drive, MS 2E-62 > Florham Park NJ 07932, USA > +1 973 443 7528 > jishnu_mukerji@hp.com -- Simon C Nash, Chief Technical Officer, IBM Java Technology Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibm