Issue 3327: Preserving unencapsulated information (interop) Source: Triodia Technologies Pty Ltd (Mr. Michi Henning, michi(at)triodia.com) Nature: Uncategorized Issue Severity: Summary: Page 15-50 of 2.3.1: ORBs that suport only version 1.1. or 1.2 IIOP profiles must ignore, but preserve, any data in the profile that occurs after the components member. A 1.2 ORB is obliged to ignore, but preserve, any information that follows the components member in 1_1 profile body. I think the cost of implementing this is quite high. What follows after the components member is *not* in a separate encapsulation. So, an ORB that receives an IOR that indeed has data after the components member doesn't understand anything about that data and therefore must treat it as a blob of bits. However, the IOR might be little-endian and the receiving ORB might be big-endian. When the receiving ORB wants to send the IOR back out, it can't send it as a big-endian IOR because it can't know how to byte-swap the data that follows the components member. That means that a big-endian ORB is obliged to remarshal the IOR back as a little-endian IOR. That's inconvenient, to say the least. I don't think it makes sense to require an ORB to preserve something that is not encapsulated, simply because the cost of doing this is high. (The ORB has to keep a copy of the marshaled profile around because of the trailing bit it doesn't understand.) Resolution: Close with revision Revised Text: On page 15-50 in the Bullet item "IIOP version", change: " If the minor version number is 0, the encapsulation is fully described by the ProfileBody_1_0 structure. If the minor version number is 1 or 2, the encapsulation is fully described by the ProfileBody_1_1 structure. If the minor version number is greater than 1, then the length of the encapsulated profile may exceed the total size of components defined in this specification for profiles with minor version number 1. ORBs that support only version 1.1 or 1.2 IIOP profiles must ignore, but preserve, any data in the profile that occurs after the components member. " to: " If the minor version number is 0, the encapsulation is fully described by the ProfileBody_1_0 structure. If the minor version number is 1 or 2, the encapsulation is fully described by the ProfileBody_1_1 structure. If the minor version number is greater than 2, then the length of the encapsulated profile may exceed the total size of components defined in this specification for profiles with minor version number 1 or 2. ORBs that support only version 1.1 or 1.2 IIOP profiles must ignore, but preserve, any data in the profile that occurs after the components member, for IIOP profiles with minor version greater than 1.2. " Actions taken: February 17, 2000: received issue October 4, 2000: closed issue Discussion: The profile itself is an encapsulation. When an IOR is sent out, the encapsulated profiles *can* be remarshalled without knowing their full internal structure. All that is necessary is to only decode as much as you know of a profile when using it, ignoring any other data it might contain. And, keep the full encapsulated profile around for sending later. Preserving the entire profile encapsulation is only necessary when two conditions are both true: the minor version is unknown, and there is extra data in the encapsulation after demarshaling the members defined for the highest known minor version. This isn't going to cause any overhead at all until a new minor version of IIOP comes out that adds members to the profile. Given that anything that can be done with extra members could also be done with extra components, this is not very likely. However, it should be clarified that a legal IIOP 1.1 or 1.2 profile may not have any data after the sequence of components. End of Annotations:===== Date: Thu, 17 Feb 2000 13:35:12 +1000 (EST) From: Michi Henning Reply-To: interop@omg.org To: interop@omg.org cc: issues@omg.org Subject: Preserving unencapsulated information Message-ID: Organization: Object Oriented Concepts MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-UIDL: 7`kd9Z Date: Wed, 16 Feb 2000 20:03:31 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.7 [en] (X11; U; SunOS 5.5.1 sun4m) X-Accept-Language: en MIME-Version: 1.0 To: interop@omg.org Subject: Re: Preserving unencapsulated information References: Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: JCG!!n_:!!doLe95)bd9 Michi Henning wrote: > > Page 15-50 of 2.3.1: > > ORBs that suport only version 1.1. or 1.2 IIOP profiles must > ignore, but preserve, any data in the profile that occurs > after the components member. > > A 1.2 ORB is obliged to ignore, but preserve, any information that > follows > the components member in 1_1 profile body. I think the cost of > implementing > this is quite high. What follows after the components member is > *not* in > a separate encapsulation. So, an ORB that receives an IOR that > indeed has > data after the components member doesn't understand anything about > that data > and therefore must treat it as a blob of bits. > > However, the IOR might be little-endian and the receiving ORB might > be > big-endian. When the receiving ORB wants to send the IOR back out, > it > can't send it as a big-endian IOR because it can't know how to > byte-swap > the data that follows the components member. That means that a > big-endian > ORB is obliged to remarshal the IOR back as a little-endian IOR. > That's inconvenient, to say the least. > > I don't think it makes sense to require an ORB to preserve something > that > is not encapsulated, simply because the cost of doing this is high. > (The ORB has to keep a copy of the marshaled profile around because > of > the trailing bit it doesn't understand.) This same problem occurs for ORBs that only support IIOP 1.0, since they are also required to preserve information after the profile in the encapsulation. The simplest way to meet this requirement currently is for the ORB to preserve intact the encapsulations that contain each profile in the IOR. If you have the encapsulation sequence, there's no trouble. [And it makes the IOR rather simple to marshal as well. :-)] Granted that this does have unfortunate efficiency concerns depending on the ORB implementation's choice of strategies for its internal object reference format, but this is apparently the only reliable way to meet this requirement in the specification. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Date: Wed, 16 Feb 2000 23:09:15 -0500 From: Bob Kukura Organization: IONA Technologies X-Mailer: Mozilla 4.7 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: interop@omg.org Subject: Re: Preserving unencapsulated information References: Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: f+E!!)%~!!,Fh!!+"@!! Michi Henning wrote: > > Page 15-50 of 2.3.1: > > ORBs that suport only version 1.1. or 1.2 IIOP profiles must > ignore, but preserve, any data in the profile that occurs > after the components member. > > A 1.2 ORB is obliged to ignore, but preserve, any information that > follows > the components member in 1_1 profile body. I think the cost of > implementing > this is quite high. What follows after the components member is > *not* in > a separate encapsulation. So, an ORB that receives an IOR that > indeed has > data after the components member doesn't understand anything about > that data > and therefore must treat it as a blob of bits. > > However, the IOR might be little-endian and the receiving ORB might > be > big-endian. When the receiving ORB wants to send the IOR back out, > it > can't send it as a big-endian IOR because it can't know how to > byte-swap > the data that follows the components member. That means that a > big-endian > ORB is obliged to remarshal the IOR back as a little-endian IOR. > That's inconvenient, to say the least. I know its not that difficult because I implemented it in Orbix 2000. We don't keep the original octets around unless the version is greater than 1.2 and there also is data left over in the profile body after demarshaling the members defined for 1.1/1.2. > > I don't think it makes sense to require an ORB to preserve something > that > is not encapsulated, simply because the cost of doing this is high. > (The ORB has to keep a copy of the marshaled profile around because > of > the trailing bit it doesn't understand.) The rationale is that a similar rule was needed (retroactively, I'll admit) in 1.0 ORBs for them to not trash 1.1 profiles they passed through. If the 1.0 ORB simply dropped the extra data, yet left the version at 1.1, then a receiving 1.1 ORB would get a MARSHAL or INV_OBJREF error when the encapsulation was prematurely exhausted. The alternative would be for the ORB to remarshal the profile using the highest IIOP minor version number it knew whenever it had to disguard encapsulated data defined subsequently to that minor version. I wouldn't object to allowing ORBs to do that, but reducing the minor version would be absolutely required, and preserving the original profile intact (as we've implemented) should also remain conformant. -Bob > > Cheers, > > > Michi. > -- > Michi Henning +61 7 3891 5744 > Object Oriented Concepts +61 4 1118 2700 (mobile) > Suite 4, 904 Stanley St +61 7 3891 5009 (fax) > East Brisbane 4169 michi@ooc.com.au > AUSTRALIA > http://www.ooc.com.au/staff/michi-henning.html Date: Thu, 17 Feb 2000 14:49:28 +1000 (EST) From: Michi Henning To: Bob Kukura cc: interop@omg.org Subject: Re: Preserving unencapsulated information In-Reply-To: <38AB746B.AA2C7C86@iona.com> Message-ID: Organization: Object Oriented Concepts MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-UIDL: =!3!!632!!+K,e952Qe9 On Wed, 16 Feb 2000, Bob Kukura wrote: > I know its not that difficult because I implemented it in Orbix 2000. ORBacus gets it right too. I know that it is implementable. It's both the cost and the precedent it sets that worry me. > The rationale is that a similar rule was needed (retroactively, I'll > admit) in 1.0 ORBs for them to not trash 1.1 profiles they passed > through. If the 1.0 ORB simply dropped the extra data, yet left the > version at 1.1, then a receiving 1.1 ORB would get a MARSHAL or > INV_OBJREF error when the encapsulation was prematurely exhausted. > The > alternative would be for the ORB to remarshal the profile using the > highest IIOP minor version number it knew whenever it had to > disguard > encapsulated data defined subsequently to that minor version. I > wouldn't object to allowing ORBs to do that, but reducing the minor > version would be absolutely required, and preserving the original > profile intact (as we've implemented) should also remain conformant. I agree with your (and Jon's) reasoning. However, why is it that additional data wasn't required to be encapsulated, at least for 1.2 and beyond? It seems that by requiring the ORB to preserve something that is not encapsulated, we are setting a dangerous precedent. (And it makes the implementation unnecessarily bulky and inefficient.) This has bitten us before: because inside an Any, nothing is encapsulated, not only does an event channel have to do an insane amount of work in order to simply receive and pass on an event, it also means that a 1.0 channel cannot handle an Any that contains one of the new IDL types, such as a long double or a value type (because it won't recognize the type code). I would still suggest that this passage in the spec is in error. There is no need to allow the data past the components member to be unencapsulated. Further, it's not immediately clear to me why this requirement was introduced at all -- what is it needed for? Was this done with the intent to permit later changes to the profile body? If so, we are going about it in the wrong way, really. In fact, the entire way in which profile bodies are crafted together seems wrong because it is so hard to extend. It would have been better to use something like the TAG_MULTIPLE_COMPONENTS profile, which effectively contains a sequence of name-value pairs. With that, you get all the extensibility you could possibly want, without any of the problems caused by unencapsulated data... Cheers, Michi. -- Michi Henning +61 7 3891 5744 Object Oriented Concepts +61 4 1118 2700 (mobile) Suite 4, 904 Stanley St +61 7 3891 5009 (fax) East Brisbane 4169 michi@ooc.com.au AUSTRALIA http://www.ooc.com.au/staff/michi-henning.html From: Paul Kyzivat To: "'interop@omg.org'" Subject: RE: Preserving unencapsulated information Date: Mon, 13 Mar 2000 12:20:58 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: jM_!!:9'e9g#[!!K!-e9 I just came across this message again. Bob replied, but didn't answer Michi's issues. Michi, aren't you missing a key point? The profile itself is an encapsulation. When an IOR is sent out, the encapsulated profiles *can* be remarshalled without knowing their full internal structure. All that is necessary is to only decode as much as you know of a profile when using it, ignoring any other data it might contain. And, keep the full encapsulated profile around for sending later. Paul > -----Original Message----- > From: Michi Henning [mailto:michi@ooc.com.au] > Sent: Wednesday, February 16, 2000 10:35 PM > To: interop@omg.org > Cc: issues@omg.org > Subject: Preserving unencapsulated information > > > Page 15-50 of 2.3.1: > > ORBs that suport only version 1.1. or 1.2 IIOP profiles must > ignore, but preserve, any data in the profile that occurs > after the components member. > > A 1.2 ORB is obliged to ignore, but preserve, any information > that follows > the components member in 1_1 profile body. I think the cost > of implementing > this is quite high. What follows after the components member > is *not* in > a separate encapsulation. So, an ORB that receives an IOR > that indeed has > data after the components member doesn't understand anything > about that data > and therefore must treat it as a blob of bits. > > However, the IOR might be little-endian and the receiving ORB might > be > big-endian. When the receiving ORB wants to send the IOR back out, > it > can't send it as a big-endian IOR because it can't know how > to byte-swap > the data that follows the components member. That means that > a big-endian > ORB is obliged to remarshal the IOR back as a little-endian IOR. > That's inconvenient, to say the least. > > I don't think it makes sense to require an ORB to preserve > something that > is not encapsulated, simply because the cost of doing this is high. > (The ORB has to keep a copy of the marshaled profile around because > of > the trailing bit it doesn't understand.) > > Cheers, > > Michi. > -- > Michi Henning +61 7 3891 5744 > Object Oriented Concepts +61 4 1118 2700 (mobile) > Suite 4, 904 Stanley St +61 7 3891 5009 (fax) > East Brisbane 4169 michi@ooc.com.au > AUSTRALIA > http://www.ooc.com.au/staff/michi-henning.html > Date: Tue, 14 Mar 2000 13:20:36 +1000 (EST) From: Michi Henning To: Paul Kyzivat cc: "'interop@omg.org'" Subject: RE: Preserving unencapsulated information In-Reply-To: > <9B164B713EE9D211B6DC0090273CEEA926BCC9@bos1.noblenet.com> Message-ID: > Organization: Object Oriented Concepts MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-UIDL: L I just came across this message again. Bob replied, but didn't answer > Michi's issues. > > Michi, aren't you missing a key point? The profile itself is an > encapsulation. When an IOR is sent out, the encapsulated profiles *can* be > remarshalled without knowing their full internal structure. > > All that is necessary is to only decode as much as you know of a profile > when using it, ignoring any other data it might contain. And, keep the full > encapsulated profile around for sending later. Yes, what you suggest works. However, I can't say I like it. Basically, it now requires the ORB to preserve unencapsulated data by keeping the enclosing encapsulation around. That's inefficient and sets a dangerous precedent, IMO. I really don't like this... Cheers, Michi. -- Michi Henning +61 7 3891 5744 Object Oriented Concepts +61 4 1118 2700 (mobile) Suite 4, 904 Stanley St +61 7 3891 5009 (fax) East Brisbane 4169 michi@ooc.com.au AUSTRALIA http://www.ooc.com.au/staff/michi-henning.html Date: Mon, 13 Mar 2000 23:15:12 -0500 From: Bob Kukura Organization: IONA Technologies X-Mailer: Mozilla 4.7 [en] (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 To: Michi Henning CC: Paul Kyzivat , "'interop@omg.org'" Subject: Re: Preserving unencapsulated information References: Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: m=$!!:p(e9XMB!!36E!! Preserving the entire profile encapsulation is only necessary when two conditions are both true: the minor version is unknown, and there is extra data in the encapsulation after demarshaling the members defined for the highest known minor version. This isn't going to cause any overhead at all until a new minor version of IIOP comes out that adds members to the profile. Given that anything that can be done with extra members could also be done with extra components, I don't think this is very likely. Orbix 2000 does preserve the entire encapsulation, and uses that when remarshaling the profile, only when both the above mentioned conditions are true. This was not difficult to implement, and does not add overhead when either condition is false. Also, completely unknown profiles or components are always preserved as encapsulations. There only seem to be two other safe alternatives. One is to allow ORBs to remarshal the IIOP profile as a version they know, dropping the unknown stuff and reducing the version number. The other is to resolve that OMG will never add additional members to the 1.1/1.2 TAG_INTERNET_IOP profile structure, so the situation can't occur. But I don't see a compelling reason to change the current specification. -Bob Michi Henning wrote: > > On Mon, 13 Mar 2000, Paul Kyzivat wrote: > > > I just came across this message again. Bob replied, but didn't > answer > > Michi's issues. > > > > Michi, aren't you missing a key point? The profile itself is an > > encapsulation. When an IOR is sent out, the encapsulated profiles > *can* be > > remarshalled without knowing their full internal structure. > > > > All that is necessary is to only decode as much as you know of a > profile > > when using it, ignoring any other data it might contain. And, keep > the full > > encapsulated profile around for sending later. > > Yes, what you suggest works. However, I can't say I like > it. Basically, > it now requires the ORB to preserve unencapsulated data by keeping > the > enclosing encapsulation around. That's inefficient and sets a > dangerous > precedent, IMO. I really don't like this... > > Cheers, > > > Michi. > -- > Michi Henning +61 7 3891 5744 > Object Oriented Concepts +61 4 1118 2700 (mobile) > Suite 4, 904 Stanley St +61 7 3891 5009 (fax) > East Brisbane 4169 michi@ooc.com.au > AUSTRALIA > http://www.ooc.com.au/staff/michi-henning.html Date: Tue, 14 Mar 2000 14:26:08 +1000 (EST) From: Michi Henning To: Bob Kukura cc: Paul Kyzivat , "'interop@omg.org'" Subject: Re: Preserving unencapsulated information In-Reply-To: <38CDBCD0.8EA5A968@iona.com> Message-ID: Organization: Object Oriented Concepts MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-UIDL: VU7e9'Di!!aKS!!*1^!! On Mon, 13 Mar 2000, Bob Kukura wrote: > Preserving the entire profile encapsulation is only necessary when two > conditions are both true: the minor version is unknown, and there is > extra data in the encapsulation after demarshaling the members defined > for the highest known minor version. This isn't going to cause any > overhead at all until a new minor version of IIOP comes out that adds > members to the profile. Sorry, but I don't understand this. Right now, I get the impression that it would be legal for an ORB to add all sorts of trailing proprietary stuff following the last member and that we would be obliged to preserve it. > Given that anything that can be done with extra > members could also be done with extra components, I don't think this > is > very likely. Well, if we can do everything we want to with extra components (which is right, of course), why do we permit an ORB to tack on all this extra stuff following the last member? Wouldn't it be better to simply not allow anything to be added? I know -- the idea was to have an escape hatch that permits the structure to be grown later. However, what I'm saying is that to permit this for unencapsulated data is not very nice and constrains implementations unnecessarily. And, I have to write my ORB to preserve any trailing data right now; if I don't, then if GIOP 1.3, for example, adds additional members, my 1.2 servers and clients wouldn't interoperate. > Orbix 2000 does preserve the entire encapsulation, and > uses that when remarshaling the profile, only when both the above > mentioned conditions are true. This was not difficult to implement, > and > does not add overhead when either condition is false. Also, > completely > unknown profiles or components are always preserved as > encapsulations. Yes. For those, I have no problem. What concerns me is that we now have something that's unknown inside a known profile... > There only seem to be two other safe alternatives. One is to allow ORBs > to remarshal the IIOP profile as a version they know, dropping the > unknown stuff and reducing the version number. That would be totallly evil, IMO. > The other is to resolve > that OMG will never add additional members to the 1.1/1.2 > TAG_INTERNET_IOP profile structure, so the situation can't occur. > But I > don't see a compelling reason to change the current specification. When you look at the IIOP structure and the way we've extended it, it becomes clear that it probably should have been a list of name-value pairs all along. That way, there would be no problem with extending it, not understanding bits of it, etc... And there would be no need to resort to hacks like "if more than the expected amount of data arrives, the ORB must preserve it..." :-( I still think that these things are afterthoughts and ugly hacks. Before we know it, such stuff will come back to haunt us. I'd at least make a general rule (even if we don't touch the spec in this particular case) that if an ORB is expected to preserve something, it must be encapsulated. After all, that's what encapsulations are for... Cheers, Michi. -- Michi Henning +61 7 3891 5744 Object Oriented Concepts +61 4 1118 2700 (mobile) Suite 4, 904 Stanley St +61 7 3891 5009 (fax) East Brisbane 4169 michi@ooc.com.au AUSTRALIA http://www.ooc.com.au/staff/michi-henning.html From: "Rutt, T E (Tom)" To: Bob Kukura , "'Michi Henning'" Cc: Paul Kyzivat , "'interop@omg.org'" Subject: RE: Preserving unencapsulated information Date: Fri, 24 Mar 2000 15:44:54 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain X-UIDL: dend9bof!!;?\!!5aOd9 > ---------- > From: Michi Henning[SMTP:michi@ooc.com.au] > Sent: Monday, March 13, 2000 11:26 PM > To: Bob Kukura > Cc: Paul Kyzivat; 'interop@omg.org' > Subject: Re: Preserving unencapsulated information > > On Mon, 13 Mar 2000, Bob Kukura wrote: > > > Preserving the entire profile encapsulation is only necessary when > two > > conditions are both true: the minor version is unknown, and there > is > > extra data in the encapsulation after demarshaling the members > defined > > for the highest known minor version. This isn't going to cause > any > > overhead at all until a new minor version of IIOP comes out that > adds > > members to the profile. > > Sorry, but I don't understand this. Right now, I get the impression > that > it would be legal for an ORB to add all sorts of trailing > proprietary > stuff following the last member and that we would be obliged to > preserve > it. > > > Given that anything that can be done with extra > > members could also be done with extra components, I don't think > this is > > very likely. > > Well, if we can do everything we want to with extra components > (which is > right, of course), why do we permit an ORB to tack on all this extra > stuff following the last member? Wouldn't it be better to simply not > allow anything to be added? > this was intended as extensibility rules for future extensions. However, anytime a standard would extend it, it would have to encapsulate things to avoid the problems stated. There is no problem until some standard extension is not encapsulated. -- Tom Rutt Lucent Technologies - Bell Labs Rm 4L-336 Tel: +1(732)949-7862 101 Crawford Corner Rd Fax: +1(732)949-1196 Holmdel NJ, 07733 USA email: terutt@lucent.com Date: Thu, 30 Mar 2000 09:02:23 +1000 (EST) From: Michi Henning To: "Rutt, T E (Tom)" cc: Bob Kukura , Paul Kyzivat , "'interop@omg.org'" Subject: RE: Preserving unencapsulated information In-Reply-To: Message-ID: Organization: Object Oriented Concepts MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-UIDL: LTjd9poS!!iJFe9Q^ > Well, if we can do everything we want to with extra components (which is > > right, of course), why do we permit an ORB to tack on all this extra > > stuff following the last member? Wouldn't it be better to simply not > > allow anything to be added? > > > this was intended as extensibility rules for future extensions. > > However, anytime a standard would extend it, it would have to > encapsulate things to > avoid the problems stated. > > There is no problem until some standard extension is not > encapsulated. Tom, the whole point is that the extra stuff is *not* encapsulated (at least not separately). What we have is an encapsulation that contains a ProfileBody_1_1. The byte count for the encapsulation may be larger than the size of the ProfileBody_1_1. In other words, following the end of the structure, additional data can be present, but that data is not in its own encapsulation. Having re-read this thread, I agree that it is probably best not to muck around with this part of the spec because it is implementable as is. However, I want to raise at least a warning flag: the way profile bodies were extended places additional implementation cost on the ORB and, IMO, sets a dangerous precedent. We should at least keep an eye out for that kind of thing and not allow it to happen elsewhere. (If an ORB is required to preserve data that it doesn't understand, but that data is not in its own, separate encapsulation, implementations become more expensive in complexity and memory footprint.) Cheers, Michi. -- Michi Henning +61 7 3891 5744 Object Oriented Concepts +61 4 1118 2700 (mobile) Suite 4, 904 Stanley St +61 7 3891 5009 (fax) East Brisbane 4169 michi@ooc.com.au AUSTRALIA http://www.ooc.com.au/staff/michi-henning.html