Issue 3434: Valuetype in anys. Unmarshaling problem? (interop) Source: Borland Software Corporation (Mr. Vijaykumar Natarajan, nobody) Nature: Uncategorized Issue Severity: Summary: There seems to be a problem with valuetypes contained in anys. Consider the following: valuetype A { A a; }; valuetype B : A { private long long x; }; If an instance w/ the following structure: A { a = {B { x = {<value>}}}} is inserted into an any and the any is received by an event service. Given that the event service has no type information of the B encoded inside of A, the event service has no way of knowing how big A.a (an instance of B) is and how many bytes it needs to read (unless it contacts an IR) Am I missing something here? Resolution: Close this issue and place it into the GIOP wish list for future enhancements to GIOP. Revised Text: None required for GIOP 1.2 Actions taken: March 20, 2000: received issue February 27, 2001: closed issue Discussion: The fundamental problem comes down to the desire that Services like the event service be able to pass through valuetypes that it does not have compile time knowledge of. The obvious solution is to maintain the valuetype in its CDR encoded form, but the valuetype CDR encoding is not self-identifying in several ways: 1. The non-chunked encoding does not indicate the end of a valuetype (or the existence and extent of valuetypes encoded in its state). 2. The chunked encoding solves this problem, but still is not adequate. The encoded form cannot be reliably remarshalled without knowing the valuetype's TypeCode, since the byte order of the encoding is not part of the encoding, and even the chunked form doesn't make it possible to find and patch valuetype indirections in a valuetype's state. The only solution that applies to GIOP 1.2 is for the receiver to find the TypeCode for the valuetype in question. It can find the TypeCode in two ways: 1. Using a CodeBase reference it receives from the sender. 2. Doing its own lookup in the IFR. This is not as reliable as option 1, since it can suffer from version skew problems. If these methods of finding the TypeCode fails, then the receiver has no choice but to raise the MARSHAL or NO_IMPLEMENT exceptions as detailed in Chapter 5. Looking forward, if a future version of CDR encoding for valuetypes were modified (in GIOP 1.3?), then valuetypes could be held in CDR encoded-form and remarshalled without having to know the TypeCode (or quivalent information). However such CDR modifications are beyond the scope of this RTF, and should be place on the GIOP wish list. End of Annotations:===== Date: Mon, 20 Mar 2000 12:01:18 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: interop@omg.org, issues@omg.org Subject: Valuetype in anys. Unmarshaling problem? Content-Type: multipart/mixed; boundary="------------B6EF247094B7B9EBF922FD70" X-UIDL: [:hd9&/ld9^g,e9]M?!! There seems to be a problem with valuetypes contained in anys. Consider the following: valuetype A { A a; }; valuetype B : A { private long long x; }; If an instance w/ the following structure: A { a = {B { x = {}}}} is inserted into an any and the any is received by an event service. Given that the event service has no type information of the B encoded inside of A, the event service has no way of knowing how big A.a (an instance of B) is and how many bytes it needs to read (unless it contacts an IR) Am I missing something here? Vijay [] vijayn3.vcf X-Authentication-Warning: corvette.floorboard.com: jbiggar-u10.cisco.com [171.71.228.71] didn't use HELO protocol Sender: jbiggar@corvette.floorboard.com Date: Mon, 20 Mar 2000 12:26:43 -0800 From: Jonathan Biggar <"jon"@jon@biggar.org> X-Mailer: Mozilla 4.7 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Vijaykumar Natarajan CC: interop@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? References: <38D6838E.8E1F6122@inprise.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: */:e94(_d9L'6!!CPO!! Vijaykumar Natarajan wrote: > > There seems to be a problem with valuetypes contained in anys. > > Consider the following: > valuetype A { > A a; > }; > > valuetype B : A { > private long long x; > }; > > If an instance w/ the following structure: > > A { a = {B { x = {}}}} > > is inserted into an any and the any is received by an event > service. Given that > the event service has no type information of the B encoded inside of > A, the > event service has no way of knowing how big A.a (an instance of B) > is and how > many bytes it needs to read (unless it contacts an IR) > > Am I missing something here? No I don't think you are missing anything here. If the valuetype is not encoded in chunked format, it is impossible for any receiver that does not know the most derived valuetype to decode it properly. I think we ought to disallow non-chunked valuetype encodings. The penalty would be adding 8-11 bytes to valuetype encodings that could have been sent in non-chunked form. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Date: Mon, 20 Mar 2000 15:43:57 -0500 (EST) Message-Id: <200003202043.PAA05262@emerald.omg.org> To: Vijaykumar Natarajan , interop@omg.org, issues@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? From: "Jeffrey A. Marshall" X-Mailer: TWIG 2.1.1 Content-Type: text X-UIDL: 7]R!!?&,!!J5ad9$PIe9 Vijaykumar, This should work fine because the typecode for B & A are transmitted within the any. The only time this (a receiving ORB not having compile-time of a value) is an issue is when your value is "custom" -- we sent an issue on this earlier in the year, but I haven't seen it show up in the issues database yet. -jeff Vijaykumar Natarajan said: > There seems to be a problem with valuetypes contained in anys. > > Consider the following: > valuetype A { > A a; > }; > > valuetype B : A { > private long long x; > }; > > > If an instance w/ the following structure: > > A { a = {B { x = {}}}} > > is inserted into an any and the any is received by an event > service. Given that > the event service has no type information of the B encoded inside of > A, the > event service has no way of knowing how big A.a (an instance of B) > is and how > many bytes it needs to read (unless it contacts an IR) > > Am I missing something here? > > Vijay > Date: Mon, 20 Mar 2000 15:44:03 -0500 (EST) Message-Id: <200003202044.PAA05273@emerald.omg.org> To: Vijaykumar Natarajan , interop@omg.org, issues@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? From: "Jeffrey A. Marshall" X-Mailer: TWIG 2.1.1 Content-Type: text X-UIDL: W(Nd9EX_d99l7e9TGf!! Vijaykumar, This should work fine because the typecode for B & A are transmitted within the any. The only time this (a receiving ORB not having compile-time of a value) is an issue is when your value is "custom" -- we sent an issue on this earlier in the year, but I haven't seen it show up in the issues database yet. -jeff Vijaykumar Natarajan said: > There seems to be a problem with valuetypes contained in anys. > > Consider the following: > valuetype A { > A a; > }; > > valuetype B : A { > private long long x; > }; > > > If an instance w/ the following structure: > > A { a = {B { x = {}}}} > > is inserted into an any and the any is received by an event > service. Given that > the event service has no type information of the B encoded inside of > A, the > event service has no way of knowing how big A.a (an instance of B) > is and how > many bytes it needs to read (unless it contacts an IR) > > Am I missing something here? > > Vijay > Date: Mon, 20 Mar 2000 15:53:31 -0500 (EST) Message-Id: <200003202053.PAA05613@emerald.omg.org> To: "Jeffrey A. Marshall" , Vijaykumar Natarajan , interop@omg.org, issues@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? From: "Jeffrey A. Marshall" X-Mailer: TWIG 2.1.1 Content-Type: text X-UIDL: N5Ae9139!!SJ:e9#;]!! Oops, yes this is a problem -- as Joh pointed out. One addition which we raised in our earlier issue is that in addition to values always being chunked, the chunked data always needs to start on an eight byte boundry. -jeff "Jeffrey A. Marshall" said: > Vijaykumar, > This should work fine because the typecode for B & A are > transmitted within > the any. The only time this (a receiving ORB not having compile-time > of a > value) is an issue is when your value is "custom" -- we sent an > issue on this > earlier in the year, but I haven't seen it show up in the issues > database yet. > > -jeff > > Vijaykumar Natarajan said: > > > There seems to be a problem with valuetypes contained in anys. > > > > Consider the following: > > valuetype A { > > A a; > > }; > > > > valuetype B : A { > > private long long x; > > }; > > > > > > If an instance w/ the following structure: > > > > A { a = {B { x = {}}}} > > > > is inserted into an any and the any is received by an event > service. Given > that > > the event service has no type information of the B encoded inside > of A, the > > event service has no way of knowing how big A.a (an instance of B) > is and > how > > many bytes it needs to read (unless it contacts an IR) > > > > Am I missing something here? > > > > Vijay > > > > > Date: Mon, 20 Mar 2000 15:53:37 -0500 (EST) Message-Id: <200003202053.PAA05620@emerald.omg.org> To: "Jeffrey A. Marshall" , Vijaykumar Natarajan , interop@omg.org, issues@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? From: "Jeffrey A. Marshall" X-Mailer: TWIG 2.1.1 Content-Type: text X-UIDL: :oVd9Y;!e9AQNe9Aa8e9 Oops, yes this is a problem -- as Joh pointed out. One addition which we raised in our earlier issue is that in addition to values always being chunked, the chunked data always needs to start on an eight byte boundry. -jeff "Jeffrey A. Marshall" said: > Vijaykumar, > This should work fine because the typecode for B & A are > transmitted within > the any. The only time this (a receiving ORB not having compile-time > of a > value) is an issue is when your value is "custom" -- we sent an > issue on this > earlier in the year, but I haven't seen it show up in the issues > database yet. > > -jeff > > Vijaykumar Natarajan said: > > > There seems to be a problem with valuetypes contained in anys. > > > > Consider the following: > > valuetype A { > > A a; > > }; > > > > valuetype B : A { > > private long long x; > > }; > > > > > > If an instance w/ the following structure: > > > > A { a = {B { x = {}}}} > > > > is inserted into an any and the any is received by an event > service. Given > that > > the event service has no type information of the B encoded inside > of A, the > > event service has no way of knowing how big A.a (an instance of B) > is and > how > > many bytes it needs to read (unless it contacts an IR) > > > > Am I missing something here? > > > > Vijay > > > > > X-Authentication-Warning: corvette.floorboard.com: jbiggar-u10.cisco.com [171.71.228.71] didn't use HELO protocol Sender: jbiggar@corvette.floorboard.com Date: Mon, 20 Mar 2000 13:38:39 -0800 From: Jonathan Biggar <"jon"@jon@biggar.org> X-Mailer: Mozilla 4.7 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: "Jeffrey A. Marshall" CC: Vijaykumar Natarajan , interop@omg.org, issues@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? References: <200003202044.PAA05273@emerald.omg.org> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: P3Ce9!g,e9I4h!!+Bod9 "Jeffrey A. Marshall" wrote: > > Vijaykumar, > This should work fine because the typecode for B & A are > transmitted within > the any. The only time this (a receiving ORB not having compile-time > of a > value) is an issue is when your value is "custom" -- we sent an > issue on this > earlier in the year, but I haven't seen it show up in the issues > database yet. Jeff, this still doesn't cover all of the cases. Consider: valuetype A { }; valuetype B : A { public long l; }; struct S { A my_a; }; If an S with a my_a member which references a B is inserted into an any, then the typecode of S is encoded in the any, which does not reference the typecode for B at all. In this case, if the receiver does not already know valuetype B, it cannot unmarshal the any unless the valuetype is chunked. Now that I think about it some more, even the chunked form doesn't work as specified. The chunked form is only guaranteed to be marshaled on a four-byte boundary, because of the leading chunk tag. But members of the valuetype which need 8 byte or more alignment (long long, etc) may or may not need padding bytes. This means that a chunked valuetype must be remarshaled when inserted into another CDR stream, since the padding bytes may need to be different. Without the typecode information for the valuetype, it is impossible to do this remarshalling. This second problem can be solved by declaring that the in a valuetype chunk are treated like an encapsulation, i.e. that alignment is based on the beginning byte of the chunk, and that alignment is only guaranteed for 4 byte boundaries for primitives that usually use larger than 4 byte alignment. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org From: Paul Kyzivat To: interop@omg.org Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Mon, 20 Mar 2000 17:10:09 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: KS""!+/3e9e$8e9(fQ!! Jeff M and I had a discussion about this during the Denver meeting. As things stand, chunking can allow the receiver (e.g. the event service) to discard the part that is not understood. That is however less than ideal. It is preferable that the data not understood should be received, held, and retransmitted. However, there is insufficient information to do this. Minimally, each chunk would need to carry its byte order, so that it can be retransmitted preserving that, since the receiver that doesn't understand the data can't remarshal in a different order. Jeff is now going to have to help, because he came up with a reason why this is insufficient, but I don't recall the details. Paul > -----Original Message----- > From: Jonathan Biggar [mailto:"jon"@jon@biggar.org] > Sent: Monday, March 20, 2000 3:27 PM > To: Vijaykumar Natarajan > Cc: interop@omg.org > Subject: Re: Valuetype in anys. Unmarshaling problem? > > > Vijaykumar Natarajan wrote: > > > > There seems to be a problem with valuetypes contained in anys. > > > > Consider the following: > > valuetype A { > > A a; > > }; > > > > valuetype B : A { > > private long long x; > > }; > > > > If an instance w/ the following structure: > > > > A { a = {B { x = {}}}} > > > > is inserted into an any and the any is received by an event > service. Given that > > the event service has no type information of the B encoded > inside of A, the > > event service has no way of knowing how big A.a (an > instance of B) is and how > > many bytes it needs to read (unless it contacts an IR) > > > > Am I missing something here? > > No I don't think you are missing anything here. If the > valuetype is not > encoded in chunked format, it is impossible for any receiver that > does > not know the most derived valuetype to decode it properly. > > I think we ought to disallow non-chunked valuetype encodings. The > penalty would be adding 8-11 bytes to valuetype encodings that could > have been sent in non-chunked form. > > -- > Jon Biggar > Floorboard Software > jon@floorboard.com > jon@biggar.org > Date: Mon, 20 Mar 2000 17:27:22 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 CC: interop@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? References: <200003202044.PAA05273@emerald.omg.org> <200003202137.NAA23840@corvette.floorboard.com> Content-Type: multipart/mixed; boundary="------------462D55A50E097BCB56271BF1" X-UIDL: B) "Jeffrey A. Marshall" wrote: > > > > Vijaykumar, > > This should work fine because the typecode for B & A are > transmitted within > > the any. The only time this (a receiving ORB not having > compile-time of a > > value) is an issue is when your value is "custom" -- we sent an > issue on this > > earlier in the year, but I haven't seen it show up in the issues > database yet. > > Jeff, this still doesn't cover all of the cases. Consider: > > valuetype A { }; > > valuetype B : A { public long l; }; > > struct S { > A my_a; > }; > > If an S with a my_a member which references a B is inserted into an > any, > then the typecode of S is encoded in the any, which does not > reference > the typecode for B at all. In this case, if the receiver does not > already know valuetype B, it cannot unmarshal the any unless the > valuetype is chunked. > > Now that I think about it some more, even the chunked form doesn't > work > as specified. The chunked form is only guaranteed to be marshaled > on a > four-byte boundary, because of the leading chunk tag. But members > of > the valuetype which need 8 byte or more alignment (long long, etc) > may > or may not need padding bytes. This means that a chunked valuetype > must > be remarshaled when inserted into another CDR stream, since the > padding > bytes may need to be different. Without the typecode information > for > the valuetype, it is impossible to do this remarshalling. > This second problem can be solved by declaring that the in a > valuetype chunk are treated like an encapsulation, i.e. that alignment > is based on the beginning byte of the chunk, and that alignment is only > guaranteed for 4 byte boundaries for primitives that usually use larger > than 4 byte alignment. The first problem can be solved by requiring all valuetypes inside anys to be chunked (This is in addition to the suggestion that you and Jeff made about treating as encapsulations. Vijay [] vijayn4.vcf From: Paul Kyzivat To: interop@omg.org Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Tue, 21 Mar 2000 08:50:16 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: $3~!!3+Pe9pdRd9?L~!! > > This second problem can be solved by declaring that the > in a > > valuetype chunk are treated like an encapsulation, i.e. > that alignment > > is based on the beginning byte of the chunk, and that > alignment is only > > guaranteed for 4 byte boundaries for primitives that > usually use larger > > than 4 byte alignment. Now, if we accept that, we can make things a bit better by taking one of the free bits in the chunk header and using it as a byte-order bit, replacing the byte-order byte in true encapsulations. With this, it becomes possible to remarshal a chunked value without understanding its content. From: Paul Kyzivat To: "'Vijaykumar Natarajan'" , "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Thu, 23 Mar 2000 11:20:31 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: 6Ka!!F]id92Xad9cPV!! response below Paul > -----Original Message----- > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > Sent: Wednesday, March 22, 2000 6:54 PM > To: Paul Kyzivat > Subject: Re: Valuetype in anys. Unmarshaling problem? > > > Hi Paul, > > I presume you are talking about the value header itself, as > the chunk header is > just a long (a positive long) indicating the length of the > chunk. We do have 4 > reserved bits in the value header, of which we can use one, > to describe what > byte order the chunks in that value are using. Yes, that is what I meant. > However, while this solves the problem with anys, there still > exists a problem > with DynValues, because the DynValue representing the outer > valuetype will have > no information to interpret the nested derived valuetype and > this time we can no > longer treat the nested valuetype as a bunch of bytes. > > There is one solution we came up with. Seems a little > heavyweight, but works. > The ValueTag's second and third bits currently have 3 possible > values > 0 - No type info > 2 - Single repid > 6 - Multiple repid > > The solution involves adding a fourth value to this: > 4 - Typecode > > This would result in the typecode of the valuetype to be > marshaled instead of > any repository id. This is sufficient even in the cases of > truncatable types as > the typecode contains base information. This can also be > optimized out in the > case where the nested valuetype is the same as the declared > type. We could also > indirect this typecode to make it more compact. This will be > used only when > valuetypes are marshaled inside of anys. > > Once again, I understand that this is a heavy solution, but > we don't have a > solution that's better. > > To summarize, here are the four things needed to solve this > set of problems with > anys and dynvalues > 1. All valuetypes are chunked inside anys > 2. inside chunks are marshaled like encapsulations, > i.e data is aligned > to start of encapsulation > 3. One reserved bit in valuetag is used for byte order > 4. add another type info category that marshals the typecode > instead of rep ids, > which will be used inside of anys I don't have the time right now to think this through fully. It seems like there may be more here than needed, but I'm not sure. At the very least, this is incredibly ugly - something that typifies valuetypes in general. The notion of context sensitive marshalling - doing it differently inside an any than otherwise - greatly complicates implementation. Perhaps it doesn't make things substantially worse because the damage was already done for other features of valuetypes, but maybe not. So, for the moment I will reserve judgement and see what others have to say. Paul Date: Thu, 23 Mar 2000 10:59:48 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: "'Vijaykumar Natarajan'" , "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD41@bos1.noblenet.com> Content-Type: multipart/mixed; boundary="------------6C4145B077B425388A35F8A9" X-UIDL: /#U!!)~Oe9AiA!!#HQd9 Hi Paul, > > -----Original Message----- > > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > > Sent: Wednesday, March 22, 2000 6:54 PM > > To: Paul Kyzivat > > Subject: Re: Valuetype in anys. Unmarshaling problem? > > > I don't have the time right now to think this through fully. > It seems like there may be more here than needed, but I'm not sure. > > At the very least, this is incredibly ugly - something that typifies > valuetypes in general. The notion of context sensitive marshalling - > doing > it differently inside an any than otherwise - greatly complicates > implementation. Perhaps it doesn't make things substantially worse > because > the damage was already done for other features of valuetypes, but > maybe not. As I mentioned in my message, I think it is quite heavy weight myself and would love a simpler solution. However, I don't see one and so I proposed the one we found to work. As for typifying valuetypes, the repository ID in the header already does that to an extent. But tiers w/o static type information can derive relevant type information from the rep id only by going to the IR and I believe that's an unacceptable solution. Context sensitive marshaling does complicate things a bit, but the only alternative to that is to do it all the time, which I think is worse. Thanks, Vijay [] vijayn2.vcf Date: Thu, 23 Mar 2000 23:09:18 +0000 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat , "'Vijaykumar Natarajan'" CC: "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD41@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: !hg!!_k3!!O*=e9"7%!! Paul and Vijay, I didn't receive the original message to which this is a response. See my comments below. Simon Paul Kyzivat wrote: > > response below > > Paul > > > -----Original Message----- > > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > > Sent: Wednesday, March 22, 2000 6:54 PM > > To: Paul Kyzivat > > Subject: Re: Valuetype in anys. Unmarshaling problem? > > > > > > Hi Paul, > > > > I presume you are talking about the value header itself, as > > the chunk header is > > just a long (a positive long) indicating the length of the > > chunk. We do have 4 > > reserved bits in the value header, of which we can use one, > > to describe what > > byte order the chunks in that value are using. > > Yes, that is what I meant. > This would mean that valuetypes could have a different byte order from the message in which they are contained. If this is required, I think it should be done using an encapsulation, which was designed for this purpose, rather than inventing a new mechanism. Also, is this only needed for chunked valuetypes and not for unchunked valuetypes? > > However, while this solves the problem with anys, there still > > exists a problem > > with DynValues, because the DynValue representing the outer > > valuetype will have > > no information to interpret the nested derived valuetype and > > this time we can no > > longer treat the nested valuetype as a bunch of bytes. > > I don't understand the problem. Anys are marshalled with a TypeCode and a value. Isn't the same true for DynAnys and DynValues? If not, then I would think the marshaled form of a DynValue should be a > TypeCode and a value, rather than nesting a typecode inside a value. What is the problem with this? > > There is one solution we came up with. Seems a little > > heavyweight, but works. > > The ValueTag's second and third bits currently have 3 possible > values > > 0 - No type info > > 2 - Single repid > > 6 - Multiple repid > > > > The solution involves adding a fourth value to this: > > 4 - Typecode > > > > This would result in the typecode of the valuetype to be > > marshaled instead of > > any repository id. This is sufficient even in the cases of > > truncatable types as > > the typecode contains base information. This can also be > > optimized out in the > > case where the nested valuetype is the same as the declared > > type. We could also > > indirect this typecode to make it more compact. This will be > > used only when > > valuetypes are marshaled inside of anys. > > All anys, or just the DynValue case? Is this the TypeCode for the actual runtime type, or the declared type? > > Once again, I understand that this is a heavy solution, but > > we don't have a > > solution that's better. > > > > To summarize, here are the four things needed to solve this > > set of problems with > > anys and dynvalues > > 1. All valuetypes are chunked inside anys > > 2. inside chunks are marshaled like encapsulations, > > i.e data is aligned > > to start of encapsulation > > 3. One reserved bit in valuetag is used for byte order > > 4. add another type info category that marshals the typecode > > instead of rep ids, > > which will be used inside of anys > > I don't have the time right now to think this through fully. > It seems like there may be more here than needed, but I'm not sure. > > At the very least, this is incredibly ugly - something that typifies > valuetypes in general. The notion of context sensitive marshalling - > doing > it differently inside an any than otherwise - greatly complicates > implementation. Perhaps it doesn't make things substantially worse > because > the damage was already done for other features of valuetypes, but > maybe not. > > So, for the moment I will reserve judgement and see what others have > to say. > > Paul > I am also concerned, but I need a bit more information on the problems > that this is fixing and some more discussion of possible alternatives > before I can give a firm opinion. Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Date: Thu, 23 Mar 2000 18:04:07 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 CC: "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD41@bos1.noblenet.com> <38DAA41E.CCBC116@hursley.ibm.com> Content-Type: multipart/mixed; boundary="------------543CE001AE48E5A6E077F01D" X-UIDL: i8(e9b%S!!5oL!!'LF!! Hi Simon, Simon Nash wrote: > > > Hi Paul, > > > > > > I presume you are talking about the value header itself, as > > > the chunk header is > > > just a long (a positive long) indicating the length of the > > > chunk. We do have 4 > > > reserved bits in the value header, of which we can use one, > > > to describe what > > > byte order the chunks in that value are using. > > > > Yes, that is what I meant. > > > This would mean that valuetypes could have a different byte order > from the message in which they are contained. If this is required, > I > think it should be done using an encapsulation, which was designed > for > this purpose, rather than inventing a new mechanism. Also, is this > only needed for chunked valuetypes and not for unchunked valuetypes? All these requirements are only for marshaling inside anys as there are problems with unmarshaling anys w/ nested valuetypes where the declared type is different from runtime type. I have quoted the earlier relevant emails at the end of this mail. > > > > > However, while this solves the problem with anys, there still > > > exists a problem > > > with DynValues, because the DynValue representing the outer > > > valuetype will have > > > no information to interpret the nested derived valuetype and > > > this time we can no > > > longer treat the nested valuetype as a bunch of bytes. > > > > I don't understand the problem. Anys are marshalled with a TypeCode > and a value. Isn't the same true for DynAnys and DynValues? If not, > then I would think the marshaled form of a DynValue should be a TypeCode and > a value, rather than nesting a typecode inside a value. What is the > problem with this? This is needed because the typecode for the any does not provide sufficient information. > > > There is one solution we came up with. Seems a little > > > heavyweight, but works. > > > The ValueTag's second and third bits currently have 3 possible > values > > > 0 - No type info > > > 2 - Single repid > > > 6 - Multiple repid > > > > > > The solution involves adding a fourth value to this: > > > 4 - Typecode > > > > > > This would result in the typecode of the valuetype to be > > > marshaled instead of > > > any repository id. This is sufficient even in the cases of > > > truncatable types as > > > the typecode contains base information. This can also be > > > optimized out in the > > > case where the nested valuetype is the same as the declared > > > type. We could also > > > indirect this typecode to make it more compact. This will be > > > used only when > > > valuetypes are marshaled inside of anys. > > > > All anys, or just the DynValue case? All anys. Because you never know when an any w/ a value in it may be extracted using a DynValue. > Is this the TypeCode for the actual runtime type, or the declared type? Runtime type > >I am also concerned, but I need a bit more information on the problems that > this is fixing and some more discussion of possible alternatives before I can > give a firm opinion. Here's the original emails Vijaykumar Natarajan wrote: > There seems to be a problem with valuetypes contained in anys. > > Consider the following: > valuetype A { > A a; > }; > > valuetype B : A { > private long long x; > }; > > If an instance w/ the following structure: > > A { a = {B { x = {}}}} > > is inserted into an any and the any is received by an event service. Given that > the event service has no type information of the B encoded inside of A, the > event service has no way of knowing how big A.a (an instance of B) is and how > many bytes it needs to read (unless it contacts an IR) > > Am I missing something here? > > Vijay Jonathan Biggar wrote: > Vijaykumar Natarajan wrote: > > > > There seems to be a problem with valuetypes contained in anys. > > > > Consider the following: > > valuetype A { > > A a; > > }; > > > > valuetype B : A { > > private long long x; > > }; > > > > If an instance w/ the following structure: > > > > A { a = {B { x = {}}}} > > > > is inserted into an any and the any is received by an event > service. Given that > > the event service has no type information of the B encoded inside > of A, the > > event service has no way of knowing how big A.a (an instance of B) > is and how > > many bytes it needs to read (unless it contacts an IR) > > > > Am I missing something here? > > No I don't think you are missing anything here. If the valuetype is > not > encoded in chunked format, it is impossible for any receiver that > does > not know the most derived valuetype to decode it properly. > > I think we ought to disallow non-chunked valuetype encodings. The > penalty would be adding 8-11 bytes to valuetype encodings that could > have been sent in non-chunked form. > > -- > Jon Biggar > Floorboard Software > jon@floorboard.com > jon@biggar.org Jonathan Biggar wrote: > "Jeffrey A. Marshall" wrote: > > > > Vijaykumar, > > This should work fine because the typecode for B & A are > transmitted within > > the any. The only time this (a receiving ORB not having > compile-time of a > > value) is an issue is when your value is "custom" -- we sent an > issue on this > > earlier in the year, but I haven't seen it show up in the issues > database yet. > > Jeff, this still doesn't cover all of the cases. Consider: > > valuetype A { }; > > valuetype B : A { public long l; }; > > struct S { > A my_a; > }; > > If an S with a my_a member which references a B is inserted into an > any, > then the typecode of S is encoded in the any, which does not > reference > the typecode for B at all. In this case, if the receiver does not > already know valuetype B, it cannot unmarshal the any unless the > valuetype is chunked. > > Now that I think about it some more, even the chunked form doesn't > work > as specified. The chunked form is only guaranteed to be marshaled > on a > four-byte boundary, because of the leading chunk tag. But members > of > the valuetype which need 8 byte or more alignment (long long, etc) > may > or may not need padding bytes. This means that a chunked valuetype > must > be remarshaled when inserted into another CDR stream, since the > padding > bytes may need to be different. Without the typecode information > for > the valuetype, it is impossible to do this remarshalling. > > This second problem can be solved by declaring that the in > a > valuetype chunk are treated like an encapsulation, i.e. that > alignment > is based on the beginning byte of the chunk, and that alignment is > only > guaranteed for 4 byte boundaries for primitives that usually use > larger > than 4 byte alignment. > > -- > Jon Biggar > Floorboard Software > jon@floorboard.com > jon@biggar.org Vijaykumar Natarajan wrote: > Hi Jon, > > See comments below. > > Jonathan Biggar wrote: > > > "Jeffrey A. Marshall" wrote: > > > > > > Vijaykumar, > > > This should work fine because the typecode for B & A are transmitted within > > > the any. The only time this (a receiving ORB not having compile-time of a > > > value) is an issue is when your value is "custom" -- we sent an issue on this > > > earlier in the year, but I haven't seen it show up in the issues database yet. > > > > > Jeff, this still doesn't cover all of the cases. Consider: > > > > valuetype A { }; > > > > valuetype B : A { public long l; }; > > > > struct S { > > A my_a; > > }; > > > > If an S with a my_a member which references a B is inserted into an any, > > then the typecode of S is encoded in the any, which does not reference > > the typecode for B at all. In this case, if the receiver does not > > already know valuetype B, it cannot unmarshal the any unless the > > valuetype is chunked. > > > > Now that I think about it some more, even the chunked form doesn't work > > as specified. The chunked form is only guaranteed to be marshaled on a > > four-byte boundary, because of the leading chunk tag. But members of > > the valuetype which need 8 byte or more alignment (long long, etc) may > > or may not need padding bytes. This means that a chunked valuetype must > > be remarshaled when inserted into another CDR stream, since the padding > > bytes may need to be different. Without the typecode information for > > the valuetype, it is impossible to do this remarshalling. > > > This second problem can be solved by declaring that the in a > > valuetype chunk are treated like an encapsulation, i.e. that alignment > > is based on the beginning byte of the chunk, and that alignment is only > > guaranteed for 4 byte boundaries for primitives that usually use larger > > than 4 byte alignment. > > The first problem can be solved by requiring all valuetypes inside anys to be > chunked (This is in addition to the > suggestion that you and Jeff made about treating as encapsulations. > > Vijay Paul Kyzivat wrote: > Jeff M and I had a discussion about this during the Denver meeting. > > As things stand, chunking can allow the receiver (e.g. the event service) to > discard the part that is not understood. That is however less than ideal. It > is preferable that the data not understood should be received, held, and > retransmitted. > > However, there is insufficient information to do this. > Minimally, each chunk would need to carry its byte order, so that it can be > retransmitted preserving that, since the receiver that doesn't understand > the data can't remarshal in a different order. > > Jeff is now going to have to help, because he came up with a reason why this > is insufficient, but I don't recall the details. > > Paul > > > -----Original Message----- > > From: Jonathan Biggar [mailto:"jon"@jon@biggar.org] > > Sent: Monday, March 20, 2000 3:27 PM > > To: Vijaykumar Natarajan > > Cc: interop@omg.org > > Subject: Re: Valuetype in anys. Unmarshaling problem? > > > > > > Vijaykumar Natarajan wrote: > > > > > > There seems to be a problem with valuetypes contained in anys. > > > > > > Consider the following: > > > valuetype A { > > > A a; > > > }; > > > > > > valuetype B : A { > > > private long long x; > > > }; > > > > > > If an instance w/ the following structure: > > > > > > A { a = {B { x = {}}}} > > > > > > is inserted into an any and the any is received by an event > > service. Given that > > > the event service has no type information of the B encoded > > inside of A, the > > > event service has no way of knowing how big A.a (an > > instance of B) is and how > > > many bytes it needs to read (unless it contacts an IR) > > > > > > Am I missing something here? > > > > No I don't think you are missing anything here. If the > > valuetype is not > > encoded in chunked format, it is impossible for any receiver that does > > not know the most derived valuetype to decode it properly. > > > > I think we ought to disallow non-chunked valuetype encodings. The > > penalty would be adding 8-11 bytes to valuetype encodings that could > > have been sent in non-chunked form. > > > > -- > > Jon Biggar > > Floorboard Software > > jon@floorboard.com > > jon@biggar.org > > Paul Kyzivat wrote: > > > This second problem can be solved by declaring that the > > in a > > > valuetype chunk are treated like an encapsulation, i.e. > > that alignment > > > is based on the beginning byte of the chunk, and that > > alignment is only > > > guaranteed for 4 byte boundaries for primitives that > > usually use larger > > > than 4 byte alignment. > > Now, if we accept that, we can make things a bit better by taking > one of the > free bits in the chunk header and using it as a byte-order bit, > replacing > the byte-order byte in true encapsulations. With this, it becomes > possible > to remarshal a chunked value without understanding its content. I think I missed sending this to the rtf list. Paul Kyzivat wrote: > response below > > Paul > > > -----Original Message----- > > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > > Sent: Wednesday, March 22, 2000 6:54 PM > > To: Paul Kyzivat > > Subject: Re: Valuetype in anys. Unmarshaling problem? > > > > > > Hi Paul, > > > > I presume you are talking about the value header itself, as > > the chunk header is > > just a long (a positive long) indicating the length of the > > chunk. We do have 4 > > reserved bits in the value header, of which we can use one, > > to describe what > > byte order the chunks in that value are using. > > Yes, that is what I meant. > > > However, while this solves the problem with anys, there still > > exists a problem > > with DynValues, because the DynValue representing the outer > > valuetype will have > > no information to interpret the nested derived valuetype and > > this time we can no > > longer treat the nested valuetype as a bunch of bytes. > > > > There is one solution we came up with. Seems a little > > heavyweight, but works. > > The ValueTag's second and third bits currently have 3 possible values > > 0 - No type info > > 2 - Single repid > > 6 - Multiple repid > > > > The solution involves adding a fourth value to this: > > 4 - Typecode > > > > This would result in the typecode of the valuetype to be > > marshaled instead of > > any repository id. This is sufficient even in the cases of > > truncatable types as > > the typecode contains base information. This can also be > > optimized out in the > > case where the nested valuetype is the same as the declared > > type. We could also > > indirect this typecode to make it more compact. This will be > > used only when > > valuetypes are marshaled inside of anys. > > > > Once again, I understand that this is a heavy solution, but > > we don't have a > > solution that's better. > > > > To summarize, here are the four things needed to solve this > > set of problems with > > anys and dynvalues > > 1. All valuetypes are chunked inside anys > > 2. inside chunks are marshaled like encapsulations, > > i.e data is aligned > > to start of encapsulation > > 3. One reserved bit in valuetag is used for byte order > > 4. add another type info category that marshals the typecode > > instead of rep ids, > > which will be used inside of anys > > I don't have the time right now to think this through fully. > It seems like there may be more here than needed, but I'm not sure. > > At the very least, this is incredibly ugly - something that typifies > valuetypes in general. The notion of context sensitive marshalling - doing > it differently inside an any than otherwise - greatly complicates > implementation. Perhaps it doesn't make things substantially worse because > the damage was already done for other features of valuetypes, but maybe not. > > So, for the moment I will reserve judgement and see what others have to say. > > Paul Vijaykumar Natarajan wrote: > Hi Paul, > > > > -----Original Message----- > > > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > > > Sent: Wednesday, March 22, 2000 6:54 PM > > > To: Paul Kyzivat > > > Subject: Re: Valuetype in anys. Unmarshaling problem? > > > > > I don't have the time right now to think this through fully. > > It seems like there may be more here than needed, but I'm not sure. > > > > At the very least, this is incredibly ugly - something that typifies > > valuetypes in general. The notion of context sensitive marshalling - doing > > it differently inside an any than otherwise - greatly complicates > > implementation. Perhaps it doesn't make things substantially worse because > > the damage was already done for other features of valuetypes, but maybe not. > > As I mentioned in my message, I think it is quite heavy weight myself and would > love a simpler solution. However, I don't see one and so I proposed the one we > found to work. As for typifying valuetypes, the repository ID in the header > already does that to an extent. But tiers w/o static type information can derive > relevant type information from the rep id only by going to the IR and I believe > that's an unacceptable solution. Context sensitive marshaling does complicate > things a bit, but the only alternative to that is to do it all the time, which I > think is worse. > > Thanks, > Vijay [] vijayn7.vcf From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Fri, 24 Mar 2000 19:36:49 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: emWd9X`nd9h9Pe9D74!! > From: Simon Nash [mailto:nash@hursley.ibm.com] > Paul and Vijay, > I didn't receive the original message to which this is a response. > See my comments below. > > > I presume you are talking about the value header itself, as > > > the chunk header is > > > just a long (a positive long) indicating the length of the > > > chunk. We do have 4 > > > reserved bits in the value header, of which we can use one, > > > to describe what > > > byte order the chunks in that value are using. > > > > Yes, that is what I meant. > > > This would mean that valuetypes could have a different byte order > from the message in which they are contained. If this is required, > I > think it should be done using an encapsulation, which was designed > for > this purpose, rather than inventing a new mechanism. Also, is this > only needed for chunked valuetypes and not for unchunked valuetypes? In a sense, chunks are just a variation on an encapsulation, but they forgot about the byte order, so they can't be remarshalled transparently. My thought here was to get something that was upwardly compatible, which rules out using actual encapsulations. Also, true encapsulations would be even more heavy weight. Yes, this is needed only for chunked valuetypes, because chunking is there to deal with stuff you don't understand, and it is when you don't understand the content that you need this. > I don't understand the problem. Anys are marshalled with a TypeCode > and a value. Isn't the same true for DynAnys and DynValues? If > not, > then I would think the marshaled form of a DynValue should be > a TypeCode and > a value, rather than nesting a typecode inside a value. What is the > problem with this? You have to see the original problem statement to understand. The typecode only describes an outermost valuetype. There is a case described where that typecode doesn't contain a full description of nested valuetypes. > I am also concerned, but I need a bit more information on the > problems that > this is fixing and some more discussion of possible > alternatives before I can > give a firm opinion. I sent you (privately) the prior messages in the thread. Paul From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Fri, 24 Mar 2000 19:48:56 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: I^W!!@iE!!_Z!e9?7Rd9 > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > > This would mean that valuetypes could have a different byte order > > from the message in which they are contained. If this is > required, I > > think it should be done using an encapsulation, which was > designed for > > this purpose, rather than inventing a new mechanism. Also, is > this > > only needed for chunked valuetypes and not for unchunked > valuetypes? > > All these requirements are only for marshaling inside anys as > there are problems > with unmarshaling anys w/ nested valuetypes where the > declared type is different > from runtime type. I have quoted the earlier relevant emails > at the end of this mail. I disagree that this is only about valuetypes in anys, though that is an important case. Some servers (e.g. the Event Service) act as intermediaries for receiving Anys and passing them on, even though they don't have direct knowledge of the contained type. It is important that they don't lose data while doing this. Similarly, I imagine there will be servers acting as intermediaries for valuetypes that they don't have full knowledge of. And it will be equally important that they not lose data. Currently this cannot be guaranteed. Vijay has described a case where the typecode in an any isn't sufficient. He has also proposed a solution that involves marshalling the typecode as part of the valuetype itself. Another problem case is with custom valuetypes. There is no typecode that describes their contents in a sufficient way to marshal them. These problems could be solved if a way can be found to permit chunks that are not understood to be held opaquely and remarshalled on demand. Paul Date: Fri, 24 Mar 2000 17:08:38 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD60@bos1.noblenet.com> Content-Type: multipart/mixed; boundary="------------E55F260A58D1A43640CE5B91" X-UIDL: p'=!!$*Ge9f!9!!AF9e9 Paul, Paul Kyzivat wrote: > > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > > > > This would mean that valuetypes could have a different byte order > > > from the message in which they are contained. If this is > > required, I > > > think it should be done using an encapsulation, which was > > designed for > > > this purpose, rather than inventing a new mechanism. Also, is this > > > only needed for chunked valuetypes and not for unchunked valuetypes? > > > > All these requirements are only for marshaling inside anys as > > there are problems > > with unmarshaling anys w/ nested valuetypes where the > > declared type is different > > from runtime type. I have quoted the earlier relevant emails > > at the end of this mail. > > I disagree that this is only about valuetypes in anys, though that is an > important case. > > Some servers (e.g. the Event Service) act as intermediaries for receiving > Anys and passing them on, even though they don't have direct knowledge of > the contained type. It is important that they don't lose data while doing > this. > > > Similarly, I imagine there will be servers acting as intermediaries for > valuetypes that they don't have full knowledge of. And it will be equally > important that they not lose data. Currently this cannot be guaranteed. I don't see the problem here unless you talking about servers that deal w/ base valuetypes which have truncatable derived valuetypes. Otherwise, the problem you describe exists w/ all types, not just valuetypes. > Vijay has described a case where the typecode in an any isn't sufficient. He > has also proposed a solution that involves marshalling the typecode as part > of the valuetype itself. > > Another problem case is with custom valuetypes. There is no typecode that > describes their contents in a sufficient way to marshal them. Good catch! This case is problematic only with DynValues for anys containing nested types that are custom. > These problems could be solved if a way can be found to permit chunks that > are not understood to be held opaquely and remarshalled on demand. This will be an interesting addition to OBV to allow truncatable types to not lose derived state when passed through middle tiers that don't understand derived valuetypes. Vijay [] vijayn4.vcf From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Fri, 24 Mar 2000 20:21:19 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: ':fd9(D7!!1Idd9Q?1!! > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > I don't see the problem here unless you talking about servers > that deal w/ base > valuetypes which have truncatable derived valuetypes. > Otherwise, the problem you > describe exists w/ all types, not just valuetypes. Yep, that's what I'm talking about. From: Jeffrey Mischkinsky Message-Id: <200003250340.TAA13345@wheel.dcn.davis.ca.us> Subject: Re: Valuetype in anys. Unmarshaling problem? To: vnatarajan@inprise.com (Vijaykumar Natarajan) Date: Fri, 24 Mar 2000 19:40:45 -0800 (PST) Cc: paulk@roguewave.com (Paul Kyzivat), interop@omg.org ("Interop RTF (E-mail)") In-Reply-To: <38DC1196.76C3AA0B@inprise.com> from "Vijaykumar Natarajan" at Mar 24, 2000 05:08:38 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: SBAe9U^(!!c>>e93l6e9 'Vijaykumar Natarajan' writes: > > This will be an interesting addition to OBV to allow truncatable > types to not > lose derived state when passed through middle tiers that don't > understand > derived valuetypes. yup. This is/was a known deficiency that we decided we would deal with "later". Actually i think the problem arises whether or not the type is truncatable. valuetype derived:base { ... } Pass a derived to a server which only knows about base. (The extreme case is base is valuebase.) Let's say all the server will do is pass on the value untouched -- as in a event service. One theory was as long as the server "knows" that the value will be untouched, it can squirrel away the original value it got, and simply pass it on. (The complications come if the value is modified before being passed on, and you don't know/have the implementation, etc., etc.) So one extension we contemplated was a way to declare the parameter "readonly" so that the server (read recipient of a valuetype) would know that it should save the original and pass it on untouched. Hopefully there is a way to do this so that it doesn't have to remarshal it, but can just pass it on unchanged. I don't think making derived truncatable materially changes the above scenario. There are other more complex scenarios, but just providing a solution for this would go a long way. cheers, jeff -- Jeff Mischkinsky jmischki@dcn.davis.ca.us +1 530-758-9850 jeff@persistence.com +1 650-372-3604 Date: Mon, 27 Mar 2000 10:05:18 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Jeffrey Mischkinsky CC: Vijaykumar Natarajan , Paul Kyzivat , "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <200003250340.TAA13345@wheel.dcn.davis.ca.us> Content-Type: multipart/mixed; boundary="------------F114F29343F8F2D10F30F180" X-UIDL: n]n!!#F'!!l7bd9%K5!! Hi Jeff, Jeffrey Mischkinsky wrote: > 'Vijaykumar Natarajan' writes: > > > > This will be an interesting addition to OBV to allow truncatable > types to not > > lose derived state when passed through middle tiers that don't > understand > > derived valuetypes. > > yup. This is/was a known deficiency that we decided we would deal > with > "later". > Actually i think the problem arises whether or not the type is > truncatable. > > valuetype derived:base { ... } > > Pass a derived to a server which only knows about base. (The extreme > case > is base is valuebase.) I am not so sure about that. If the type is not truncatable, passing derived type to a server which knows only about the base should be an error, as it would be unable to read the valuetype. Vijay [] vijayn.vcf From: Jeffrey Mischkinsky Message-Id: <200003271821.KAA07269@wheel.dcn.davis.ca.us> Subject: Re: Valuetype in anys. Unmarshaling problem? To: vnatarajan@inprise.com (Vijaykumar Natarajan) Date: Mon, 27 Mar 2000 10:20:01 -0800 (PST) Cc: vnatarajan@inprise.com (Vijaykumar Natarajan), paulk@roguewave.com (Paul Kyzivat), interop@omg.org ("Interop RTF (E-mail)") In-Reply-To: <38DFA2DE.3457CB0C@inprise.com> from "Vijaykumar Natarajan" at Mar 27, 2000 10:05:18 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: aE&e9S5_!!Cp-!!Eo&!! 'Vijaykumar Natarajan' writes: > > Hi Jeff, > > > Jeffrey Mischkinsky wrote: > > > 'Vijaykumar Natarajan' writes: > > > > > > This will be an interesting addition to OBV to allow truncatable > types to not > > > lose derived state when passed through middle tiers that don't > understand > > > derived valuetypes. > > > > yup. This is/was a known deficiency that we decided we would deal > with > > "later". > > Actually i think the problem arises whether or not the type is > truncatable. > > > > valuetype derived:base { ... } > > > > Pass a derived to a server which only knows about base. (The > extreme case > > is base is valuebase.) > > I am not so sure about that. If the type is not truncatable, passing > derived type > to a server which knows only about the base should be an error, as > it would be > unable to read the valuetype. > > Vijay Doesn't it know enough to be able to blindly "read in" the whole > thing? It does know how long it is. correct? So if it never has to do anything but send the stuff it read in to someone else (w/o remarshaling it--could be a kicker), then couldn't that scenario work? jeff -- Jeff Mischkinsky jmischki@dcn.davis.ca.us +1 530-758-9850 jeff@persistence.com +1 650-372-3604 Date: Mon, 27 Mar 2000 10:54:55 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Jeffrey Mischkinsky CC: Vijaykumar Natarajan , Paul Kyzivat , "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <200003271821.KAA07269@wheel.dcn.davis.ca.us> Content-Type: multipart/mixed; boundary="------------B3698C7AB05DB9E6EEBB3276" X-UIDL: g4)!!(Gj!!,(Pe9[0Yd9 Actually, ignore my previous post. Come to think of it, it is not even theoretically possible for us to handle valuetypes in this way w/o it being truncatable. This is because when we pass a non-truncatable derived type over the wire, we take into account the type information to optimize the value header. That means, on the wire we will receive either no rep id or one rep id which is that of the most derived type. The server has no way of knowing determining that type's relationship w/ a base type potentially known to the server. Vijay Jeffrey Mischkinsky wrote: > 'Vijaykumar Natarajan' writes: > > > > Hi Jeff, > > > > > > Jeffrey Mischkinsky wrote: > > > > > 'Vijaykumar Natarajan' writes: > > > > > > > > This will be an interesting addition to OBV to allow > truncatable types to not > > > > lose derived state when passed through middle tiers that don't > understand > > > > derived valuetypes. > > > > > > yup. This is/was a known deficiency that we decided we would > deal with > > > "later". > > > Actually i think the problem arises whether or not the type is > truncatable. > > > > > > valuetype derived:base { ... } > > > > > > Pass a derived to a server which only knows about base. (The > extreme case > > > is base is valuebase.) > > > > I am not so sure about that. If the type is not truncatable, > passing derived type > > to a server which knows only about the base should be an error, as > it would be > > unable to read the valuetype. > > > > Vijay > Doesn't it know enough to be able to blindly "read in" the whole > thing? > It does know how long it is. correct? > > So if it never has to do anything but send the stuff it read in to > someone else > (w/o remarshaling it--could be a kicker), then couldn't that > scenario work? > > jeff > > -- > Jeff Mischkinsky > jmischki@dcn.davis.ca.us +1 530-758-9850 > jeff@persistence.com +1 650-372-3604 [] vijayn2.vcf Date: Mon, 27 Mar 2000 10:57:46 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: interop@omg.org Subject: [Fwd: Valuetype in anys. Unmarshaling problem?] Content-Type: multipart/mixed; boundary="------------8345747BA41ABABAF96877A4" X-UIDL: k$"!!&*(e9ElMd9i6Le9 The previous post I was talking about(missed cc'ing the rtf list) ;-) Vijay Vijaykumar Natarajan wrote: > Jeffrey Mischkinsky wrote: > > > 'Vijaykumar Natarajan' writes: > > > > > > I am not so sure about that. If the type is not truncatable, passing derived type > > > to a server which knows only about the base should be an error, as it would be > > > unable to read the valuetype. > > > > > > Vijay > > Doesn't it know enough to be able to blindly "read in" the whole thing? > > It does know how long it is. correct? > > > > So if it never has to do anything but send the stuff it read in to someone else > > (w/o remarshaling it--could be a kicker), then couldn't that scenario work? > > Theoretically yes. But that's like treating any type hierarchy as implicitly > truncatable. Since you are more familiar with the history here, wasn't the introduction > of the truncatable keyword a way for users to indicate when it was safe to truncate a > type and still maintain semantic validity of the type, and when it wasn't? Isn't the > above achieving the same thing in the case where it should not be valid? > > My understanding is that truncatability of a type was a user defined property and had > nothing to do with the ORB's ability to do so. Since we could always truncate any type, > it was necessary to have the users tell us when it was safe to do so. > > Vijay Message-ID: <38DFADB1.8F0DCE24@inprise.com> Date: Mon, 27 Mar 2000 10:51:29 -0800 From: Vijaykumar Natarajan X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Jeffrey Mischkinsky Subject: Re: Valuetype in anys. Unmarshaling problem? References: <200003271821.KAA07269@wheel.dcn.davis.ca.us> Content-Type: multipart/mixed; boundary="------------54F89E65A5E435015C8D8253" Jeffrey Mischkinsky wrote: > 'Vijaykumar Natarajan' writes: > > > > I am not so sure about that. If the type is not truncatable, > passing derived type > > to a server which knows only about the base should be an error, as > it would be > > unable to read the valuetype. > > > > Vijay > Doesn't it know enough to be able to blindly "read in" the whole > thing? > It does know how long it is. correct? > > So if it never has to do anything but send the stuff it read in to > someone else > (w/o remarshaling it--could be a kicker), then couldn't that > scenario work? Theoretically yes. But that's like treating any type hierarchy as implicitly truncatable. Since you are more familiar with the history here, wasn't the introduction of the truncatable keyword a way for users to indicate when it was safe to truncate a type and still maintain semantic validity of the type, and when it wasn't? Isn't the above achieving the same thing in the case where it should not be valid? My understanding is that truncatability of a type was a user defined property and had nothing to do with the ORB's ability to do so. Since we could always truncate any type, it was necessary to have the users tell us when it was safe to do so. Vijay [] vijayn3.vcf [] vijayn4.vcf From: Paul Kyzivat To: interop@omg.org Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Mon, 27 Mar 2000 16:34:09 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: 3>"e9EYgd9I9!e9~id!! I remember now what the other problem was that we discovered when talking about this in Denver: In addition to the byte-order problem, there is the issue of indirections to previously marshalled valuetypes in the same message. I don't offhand see how these can be fixed. Paul > -----Original Message----- > From: Jeffrey Mischkinsky [mailto:jmischki@wheel.dcn.davis.ca.us] > Sent: Monday, March 27, 2000 1:20 PM > To: vnatarajan@inprise.com > Cc: vnatarajan@inprise.com; paulk@roguewave.com; interop@omg.org > Subject: Re: Valuetype in anys. Unmarshaling problem? > > > 'Vijaykumar Natarajan' writes: > > > > Hi Jeff, > > > > > > Jeffrey Mischkinsky wrote: > > > > > 'Vijaykumar Natarajan' writes: > > > > > > > > This will be an interesting addition to OBV to allow > truncatable types to not > > > > lose derived state when passed through middle tiers > that don't understand > > > > derived valuetypes. > > > > > > yup. This is/was a known deficiency that we decided we > would deal with > > > "later". > > > Actually i think the problem arises whether or not the > type is truncatable. > > > > > > valuetype derived:base { ... } > > > > > > Pass a derived to a server which only knows about base. > (The extreme case > > > is base is valuebase.) > > > > I am not so sure about that. If the type is not > truncatable, passing derived type > > to a server which knows only about the base should be an > error, as it would be > > unable to read the valuetype. > > > > Vijay > Doesn't it know enough to be able to blindly "read in" the > whole thing? > It does know how long it is. correct? > > So if it never has to do anything but send the stuff it read > in to someone else > (w/o remarshaling it--could be a kicker), then couldn't that > scenario work? > > jeff > > > -- > Jeff Mischkinsky > jmischki@dcn.davis.ca.us +1 530-758-9850 > jeff@persistence.com +1 650-372-3604 > From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Mon, 27 Mar 2000 18:17:31 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: k)(!!a""!!"O]!!:PM!! > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > Actually, ignore my previous post. Come to think of it, it is > not even theoretically > possible for us to handle valuetypes in this way w/o it being > truncatable. This is > because when we pass a non-truncatable derived type over the > wire, we take into account > the type information to optimize the value header. That > means, on the wire we will > receive either no rep id or one rep id which is that of the > most derived type. The > server has no way of knowing determining that type's > relationship w/ a base type > potentially known to the server. I thought the only time the repid could be omitted altogether was when the most derived type exactly matches the declared type of whatever contains its reference. If so, things should be ok. The recipient must know declared type it is receiving, even if it is only ValueBase. Paul P.S. I believe something is wrong with the mail here at RW. I have only received 3-4 messages since Friday. So please excuse if I am not replying promptly. Date: Tue, 28 Mar 2000 13:35:03 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD6B@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: 5X2e9F6e!!*#1e9*d/!! Paul, Paul Kyzivat wrote: > > > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > > > Actually, ignore my previous post. Come to think of it, it is > > not even theoretically > > possible for us to handle valuetypes in this way w/o it being > > truncatable. This is > > because when we pass a non-truncatable derived type over the > > wire, we take into account > > the type information to optimize the value header. That > > means, on the wire we will > > receive either no rep id or one rep id which is that of the > > most derived type. The > > server has no way of knowing determining that type's > > relationship w/ a base type > > potentially known to the server. > > I thought the only time the repid could be omitted altogether was > when the > most derived type exactly matches the declared type of whatever > contains its > reference. > > If so, things should be ok. The recipient must know declared type it > is > receiving, even if it is only ValueBase. > Right. The more interesting case is where a subtype of the declared > type has been sent, and the receiver does not have knowledge of the subtype. > In this case, since supertype state is always marshalled first, the receiving ORB could force truncation to the base type even for derived types not declared truncatable, at the cost of violating semantic integrity of > the valuetype's data. There are a couple of issues with this, though: 1. In the case where the declared type is ValueBase or an abstract valuetype, this forced truncation would discard all the valuetype's data, which does not seem very useful. 2. Valuetypes not declared truncatable may be marshalled with non-chunked data, which would leave a truncating receiver unable to know how long the received data is so that it can skip to the end of it. So the receiver must either understand the actual type sent, or the actual type must be declared truncatable to a type that the receiver does understand. Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Date: Tue, 28 Mar 2000 13:56:05 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: interop@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD6A@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: Bi=!!;3bd9pa[d9194e9 Paul, Paul Kyzivat wrote: > > I remember now what the other problem was that we discovered when > talking > about this in Denver: > > In addition to the byte-order problem, there is the issue of > indirections to > previously marshalled valuetypes in the same message. > I don't offhand see how these can be fixed. > This one is trickier than byte order. The only way to do this would > be to find all the indirections, find the transitive closure of all the > valuetypes they point to, and remarshal this combination into a new message with > the indirections suitably fixed up. I think this means that any approach based on trying to treat a valuetype parameter as a pseudo-enacapsulation that can be remarshalled without fully parsing it is not viable. In order to fully parse it, the unmarshaller must 1. know the actual marshalled type, or 2. be sent a full description of the actual marshalled type, such as a TypeCode or FullValueDescription, or 3. be able to query the sender for a full description of the actual marshalled type, such as a TypeCode or FullValueDescription RMI-IIOP uses approach 3. The SendingContextCodeBase object allows a receiver to call back to the sender to obtain full type information for all marshalled valuetypes from their repids. This is more efficient than approach 2, since type information is only sent when necessary and can be cached by the receiver so that it only needs to be sent once for each different type. The SendingContextCodeBase mechanism is not specific to RMI but can also be used for IDL valuetypes. So I believe we already have a solution for this problem and do not need to invent a new mechanism. Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb From: Paul Kyzivat To: interop@omg.org Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Tue, 28 Mar 2000 16:07:34 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: ]/ld90d*e93@+!!d[Le9 > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > I don't remember exactly, but I remember a discussion earlier > that indicated > that Anys should be completely self contained. I.e. they > should not have > indirections that point outside the any? If that's true, > isn't this not a problem? That sounds slightly familiar, but in conflict with my other understandings: I thought that if the same valuetype was referenced multiple times in a request, no matter where, on the server side the same relationship must be restored. If the valuetype happens to be inside an Any I wouldn't expect this to change. Paul > > Vijay > > Paul Kyzivat wrote: > > > I remember now what the other problem was that we > discovered when talking > > about this in Denver: > > > > In addition to the byte-order problem, there is the issue > of indirections to > > previously marshalled valuetypes in the same message. > > I don't offhand see how these can be fixed. Date: Tue, 28 Mar 2000 21:03:27 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Jeffrey Mischkinsky CC: Vijaykumar Natarajan , Paul Kyzivat , "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <200003250340.TAA13345@wheel.dcn.davis.ca.us> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: \1)e9g%Be9^^0!!-lPe9 Jeff, Jeffrey Mischkinsky wrote: > > 'Vijaykumar Natarajan' writes: > > > > This will be an interesting addition to OBV to allow truncatable > types to not > > lose derived state when passed through middle tiers that don't > understand > > derived valuetypes. > > yup. This is/was a known deficiency that we decided we would deal > with > "later". > Actually i think the problem arises whether or not the type is > truncatable. > > valuetype derived:base { ... } > > Pass a derived to a server which only knows about base. (The extreme > case > is base is valuebase.) Let's say all the server will do is pass on > the > value untouched -- as in a event service. > > One theory was as long as the server "knows" that the value will be > untouched, > it can squirrel away the original value it got, and simply pass it > on. > (The complications come if the value is modified before being passed > on, > and you don't know/have the implementation, etc., etc.) > So one extension we contemplated was a way to declare the parameter > "readonly" so that the server (read recipient of a valuetype) would > know > that it should save the original and pass it on untouched. > > Hopefully there is a way to do this so that it doesn't have to > remarshal > it, but can just pass it on unchanged. > That would be nice, but I don't think it's possible. See my other > recent posting on this. Indirections in the truncated portion are the > killer. > I don't think making derived truncatable materially changes the above > scenario. > I think it does make a difference whether the valuetype is truncatable. If it were not truncatable, there is no guarantee that unmarshaling only the supertype portion of the marshaled state will produce a semantically valid instance of the supertype, even for read operations. Another reason why the derived type has to be truncatable is that this guarantees that it will be marshalled in chunked form. This allows the receiver to know how much data has been sent. Simon > There are other more complex scenarios, but just providing a solution > for this would go a long way. > > cheers, > jeff > > -- > Jeff Mischkinsky > jmischki@dcn.davis.ca.us +1 530-758-9850 > jeff@persistence.com +1 650-372-3604 -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb From: Jeffrey Mischkinsky Message-Id: <200003282017.MAA24972@wheel.dcn.davis.ca.us> Subject: Re: Valuetype in anys. Unmarshaling problem? To: nash@hursley.ibm.com (Simon Nash) Date: Tue, 28 Mar 2000 12:17:33 -0800 (PST) Cc: vnatarajan@inprise.com (Vijaykumar Natarajan), paulk@roguewave.com (Paul Kyzivat), interop@omg.org ("Interop RTF (E-mail)") In-Reply-To: <38E1100F.40A4E228@hursley.ibm.com> from "Simon Nash" at Mar 28, 2000 09:03:27 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: %)]d9EI0!!Rpb!!5jNe9 Simon, My questions are interspersed below. jeff 'Simon Nash' writes: > > Jeff, > > Jeffrey Mischkinsky wrote: > > > > 'Vijaykumar Natarajan' writes: > > > > > > This will be an interesting addition to OBV to allow truncatable > types to not > > > lose derived state when passed through middle tiers that don't > understand > > > derived valuetypes. > > > > yup. This is/was a known deficiency that we decided we would deal > with > > "later". > > Actually i think the problem arises whether or not the type is > truncatable. > > > > valuetype derived:base { ... } > > > > Pass a derived to a server which only knows about base. (The > extreme case > > is base is valuebase.) Let's say all the server will do is pass on > the > > value untouched -- as in a event service. > > > > One theory was as long as the server "knows" that the value will > be untouched, > > it can squirrel away the original value it got, and simply pass it > on. > > (The complications come if the value is modified before being > passed on, > > and you don't know/have the implementation, etc., etc.) > > So one extension we contemplated was a way to declare the > parameter > > "readonly" so that the server (read recipient of a valuetype) > would know > > that it should save the original and pass it on untouched. > > > > Hopefully there is a way to do this so that it doesn't have to > remarshal > > it, but can just pass it on unchanged. > > > That would be nice, but I don't think it's possible. See my other > recent > posting on this. Indirections in the truncated portion are the > killer. I saw that. I was afraid of that. > > > I don't think making derived truncatable materially changes the > above > > scenario. > > > I think it does make a difference whether the valuetype is > truncatable. > If it were not truncatable, there is no guarantee that unmarshaling > only the > supertype portion of the marshaled state will produce a semantically > valid instance of the supertype, even for read operations. I'm not sure i understand this part of the argument. I'm not talking about the (immediate) receiver doing anything with the value, except passing it on. To do just that, it only needs to have its length, ignoring remarshaling issues for the moment. I agree that if the immediate receiver trys to use the value for anything (even if it is only to do a read) it will have find a valid implementation which may only be possible if it were truncatable. > > Another reason why the derived type has to be truncatable is that > this > guarantees that it will be marshalled in chunked form. This allows > the receiver to know how much data has been sent. We could possible change this rule for the case under discussion, which since it would involve an addition to IDL syntax would be upwardly compatible. > > Simon > > > There are other more complex scenarios, but just providing a > solution > > for this would go a long way. > > > > cheers, > > jeff -- Jeff Mischkinsky jmischki@dcn.davis.ca.us +1 530-758-9850 jeff@persistence.com +1 650-372-3604 From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Tue, 28 Mar 2000 17:47:31 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: okFe9Y::e9m=kd9djP!! > From: Simon Nash [mailto:nash@hursley.ibm.com] > Right. The more interesting case is where a subtype of the > declared type has > been sent, and the receiver does not have knowledge of the > subtype. In this > case, since supertype state is always marshalled first, the > receiving > ORB could force truncation to the base type even for derived types > not > declared truncatable, at the cost of violating semantic > integrity of the valuetype's data. There are a couple of issues > with > this, though: Personally, I think truncation was a mistake. It effectively says: "you can throw away anything you don't understand, because if you don't understand it you don't need it." That may be true if the usage of your copy is limited to code with the same understanding that you have. But if you then pass it on to somebody that understands more, the missing information can be important to them. Also, introducing truncation seems to have been used as an excuse to not solve the problem for non-truncatable valuetypes. This should be handled in a way that intermediaries are able to receive and pass on things they don't understand, whether truncatable or not. This is a revelation - the need to do this has been known for a long time with Any. Why would anyone think it different with valuetype. And the fact that it doesn't work for valuetypes has now broken this property for Any as well, because Anys can contain valuetypes. Paul Date: Tue, 28 Mar 2000 12:22:24 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: interop@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD6A@bos1.noblenet.com> Content-Type: multipart/mixed; boundary="------------396DFDB95170D9257D1B5DDE" X-UIDL: 'NG!![GF!!JdH!!c'Ae9 I don't remember exactly, but I remember a discussion earlier that indicated that Anys should be completely self contained. I.e. they should not have indirections that point outside the any? If that's true, isn't this not a problem? Vijay Paul Kyzivat wrote: > I remember now what the other problem was that we discovered when talking > about this in Denver: > > In addition to the byte-order problem, there is the issue of indirections to > previously marshalled valuetypes in the same message. > I don't offhand see how these can be fixed. > > Paul > > > -----Original Message----- > > From: Jeffrey Mischkinsky [mailto:jmischki@wheel.dcn.davis.ca.us] > > Sent: Monday, March 27, 2000 1:20 PM > > To: vnatarajan@inprise.com > > Cc: vnatarajan@inprise.com; paulk@roguewave.com; interop@omg.org > > Subject: Re: Valuetype in anys. Unmarshaling problem? > > > > > > 'Vijaykumar Natarajan' writes: > > > > > > Hi Jeff, > > > > > > > > > Jeffrey Mischkinsky wrote: > > > > > > > 'Vijaykumar Natarajan' writes: > > > > > > > > > > This will be an interesting addition to OBV to allow > > truncatable types to not > > > > > lose derived state when passed through middle tiers > > that don't understand > > > > > derived valuetypes. > > > > > > > > yup. This is/was a known deficiency that we decided we > > would deal with > > > > "later". > > > > Actually i think the problem arises whether or not the > > type is truncatable. > > > > > > > > valuetype derived:base { ... } > > > > > > > > Pass a derived to a server which only knows about base. > > (The extreme case > > > > is base is valuebase.) > > > > > > I am not so sure about that. If the type is not > > truncatable, passing derived type > > > to a server which knows only about the base should be an > > error, as it would be > > > unable to read the valuetype. > > > > > > Vijay > > Doesn't it know enough to be able to blindly "read in" the > > whole thing? > > It does know how long it is. correct? > > > > So if it never has to do anything but send the stuff it read > > in to someone else > > (w/o remarshaling it--could be a kicker), then couldn't that > > scenario work? > > > > jeff > > > > > > -- > > Jeff Mischkinsky > > jmischki@dcn.davis.ca.us +1 530-758-9850 > > jeff@persistence.com +1 650-372-3604 > > [] vijayn.vcf From: Paul Kyzivat To: interop@omg.org Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Tue, 28 Mar 2000 18:17:07 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: lK1e9B5]d9oo;e97O_!! > From: Simon Nash [mailto:nash@hursley.ibm.com] > I think this means that any approach based on trying to treat > a valuetype > parameter as a pseudo-enacapsulation that can be remarshalled > without fully > parsing it is not viable. In order to fully parse it, the > unmarshaller must > 1. know the actual marshalled type, or > 2. be sent a full description of the actual marshalled type, > such as a > TypeCode or FullValueDescription, or > 3. be able to query the sender for a full description of the actual > > marshalled type, such as a TypeCode or FullValueDescription > > RMI-IIOP uses approach 3. The SendingContextCodeBase object allows > a > receiver to call back to the sender to obtain full type > information for > all marshalled valuetypes from their repids. This is more > efficient than > approach 2, since type information is only sent when necessary and > can > be cached by the receiver so that it only needs to be sent > once for each > different type. The SendingContextCodeBase mechanism is not > specific to > RMI but can also be used for IDL valuetypes. > > So I believe we already have a solution for this problem and > do not need to invent a new mechanism. This only works if you have Java on both sides, which is not a useful solution. I will be very surprised if *anybody* implements codebase for any language binding except Java. Even if they did, it would be more amazing if any one implementation provided code for more than a single language. So, if somebody implements a derived valuetype in C++ and passes it to a Java program, the java program is going to have difficulty getting code to support the derived type. We need a solution that actually preserves the language neutrality of Corba. Paul From: Jeffrey Mischkinsky Message-Id: <200003282356.PAA14426@wheel.dcn.davis.ca.us> Subject: Re: Valuetype in anys. Unmarshaling problem? To: paulk@roguewave.com (Paul Kyzivat) Date: Tue, 28 Mar 2000 15:56:36 -0800 (PST) Cc: interop@omg.org ("Interop RTF (E-mail)") In-Reply-To: <9B164B713EE9D211B6DC0090273CEEA926BD76@bos1.noblenet.com> from "Paul Kyzivat" at Mar 28, 2000 05:47:31 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: ke5!!E)[d9D4@!!E[@e9 'Paul Kyzivat' writes: > > > From: Simon Nash [mailto:nash@hursley.ibm.com] > > > Right. The more interesting case is where a subtype of the > > declared type has > > been sent, and the receiver does not have knowledge of the > > subtype. In this > > case, since supertype state is always marshalled first, the > receiving > > ORB could force truncation to the base type even for derived types > not > > declared truncatable, at the cost of violating semantic > > integrity of the valuetype's data. There are a couple of issues > with > > this, though: > > Personally, I think truncation was a mistake. > It effectively says: "you can throw away anything you don't > understand, > because if you don't understand it you don't need it." > That may be true if the usage of your copy is limited to code with > the same > understanding that you have. But if you then pass it on to somebody > that > understands more, the missing information can be important to them. Truncatability is an option that the designer/definer of the type chooses (or not) depending upon the semantics of the type they are defining. When used appropriately it can mitigate against the "brittleness" of subtyping which arises when types evolve and it becomes difficult to manage the deployement of implementations. Another approach is the Java approach which effectively requires that implemenations (of subtypes) be downloadable and available. But that approach has it's own set of issues in a multi-languge environment. > Also, introducing truncation seems to have been used as an excuse to not > solve the problem for non-truncatable valuetypes. > > This should be handled in a way that intermediaries are able to receive and > pass on things they don't understand, whether truncatable or not. > I agree. > This is a > revelation - the need to do this has been known for a long time with > Any. what's the revelation?, that it doesn't work, or that it would be nice to get it to work? > Why would anyone think it different with valuetype. And the fact that it > doesn't work for valuetypes has now broken this property for Any as well, > because Anys can contain valuetypes. -- Jeff Mischkinsky jmischki@dcn.davis.ca.us +1 530-758-9850 jeff@persistence.com +1 650-372-3604 Date: Wed, 29 Mar 2000 15:06:45 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: interop@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD77@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: O~/!!-1Vd9:p;!!k*md9 Paul, The SendingContextCodeBase described in chapter 5 of the core is a language neutral solution. If it weren't, I would not have proposed it to solve this problem. Specifically, I am talking about using the "meta" method which returns a FullValueDescription. This has no dependency on Java and does not need to be implemented multiple times for multiple languages. The information in the FullValueDescription is sufficient to parse any IDL valuetype sent by any language. Simon Paul Kyzivat wrote: > > > From: Simon Nash [mailto:nash@hursley.ibm.com] > > > I think this means that any approach based on trying to treat > > a valuetype > > parameter as a pseudo-enacapsulation that can be remarshalled > > without fully > > parsing it is not viable. In order to fully parse it, the > > unmarshaller must > > 1. know the actual marshalled type, or > > 2. be sent a full description of the actual marshalled type, > > such as a > > TypeCode or FullValueDescription, or > > 3. be able to query the sender for a full description of the > actual > > marshalled type, such as a TypeCode or FullValueDescription > > > > RMI-IIOP uses approach 3. The SendingContextCodeBase object > allows a > > receiver to call back to the sender to obtain full type > > information for > > all marshalled valuetypes from their repids. This is more > > efficient than > > approach 2, since type information is only sent when necessary and > can > > be cached by the receiver so that it only needs to be sent > > once for each > > different type. The SendingContextCodeBase mechanism is not > > specific to > > RMI but can also be used for IDL valuetypes. > > > > So I believe we already have a solution for this problem and > > do not need to invent a new mechanism. > > This only works if you have Java on both sides, which is not a > useful > solution. > > I will be very surprised if *anybody* implements codebase for any > language > binding except Java. > > Even if they did, it would be more amazing if any one implementation > provided code for more than a single language. > > So, if somebody implements a derived valuetype in C++ and passes it > to a > Java program, the java program is going to have difficulty getting > code to > support the derived type. > > We need a solution that actually preserves the language neutrality > of Corba. > > Paul -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Date: Wed, 29 Mar 2000 15:22:53 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: interop@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD71@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: J3 > > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > > > I don't remember exactly, but I remember a discussion earlier > > that indicated > > that Anys should be completely self contained. I.e. they > > should not have > > indirections that point outside the any? If that's true, > > isn't this not a problem? > > That sounds slightly familiar, but in conflict with my other > understandings: > > I thought that if the same valuetype was referenced multiple times > in a > request, no matter where, on the server side the same relationship > must be > restored. If the valuetype happens to be inside an Any I wouldn't > expect > this to change. > That's correct. This came up as an issue in the last RTF, and the > resolution was as you say. Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Date: Wed, 29 Mar 2000 16:05:56 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Jeffrey Mischkinsky CC: Vijaykumar Natarajan , Paul Kyzivat , "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <200003282017.MAA24972@wheel.dcn.davis.ca.us> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: c%W!!4\$e9_6'e9K6V!! Jeff, Jeffrey Mischkinsky wrote: > > Simon, > My questions are interspersed below. > jeff > > 'Simon Nash' writes: > > > > Jeff, > > > > Jeffrey Mischkinsky wrote: > > > > > > 'Vijaykumar Natarajan' writes: > > > > > > > > This will be an interesting addition to OBV to allow > truncatable types to not > > > > lose derived state when passed through middle tiers that don't > understand > > > > derived valuetypes. > > > > > > yup. This is/was a known deficiency that we decided we would > deal with > > > "later". > > > Actually i think the problem arises whether or not the type is > truncatable. > > > > > > valuetype derived:base { ... } > > > > > > Pass a derived to a server which only knows about base. (The > extreme case > > > is base is valuebase.) Let's say all the server will do is pass > on the > > > value untouched -- as in a event service. > > > > > > One theory was as long as the server "knows" that the value will > be untouched, > > > it can squirrel away the original value it got, and simply pass > it on. > > > (The complications come if the value is modified before being > passed on, > > > and you don't know/have the implementation, etc., etc.) > > > So one extension we contemplated was a way to declare the > parameter > > > "readonly" so that the server (read recipient of a valuetype) > would know > > > that it should save the original and pass it on untouched. > > > > > > Hopefully there is a way to do this so that it doesn't have to > remarshal > > > it, but can just pass it on unchanged. > > > > > That would be nice, but I don't think it's possible. See my other > recent > > posting on this. Indirections in the truncated portion are the > killer. > > I saw that. I was afraid of that. > > > > > > I don't think making derived truncatable materially changes the > above > > > scenario. > > > > > I think it does make a difference whether the valuetype is > truncatable. > > If it were not truncatable, there is no guarantee that > unmarshaling only the > > supertype portion of the marshaled state will produce a > semantically > > valid instance of the supertype, even for read operations. > > I'm not sure i understand this part of the argument. I'm not talking > about > the (immediate) receiver doing anything with the value, except > passing it > on. To do just that, it only needs to have its length, ignoring > remarshaling > issues for the moment. I agree that if the immediate receiver trys > to use > the value for anything (even if it is only to do a read) it will > have > find a valid implementation which may only be possible if it were > truncatable. > In that case, the correct receiver declaration is not "readonly". > It's more like "opaque". Also, since no valuetype semantics can be assumed by > the receiver, this seems to imply that the receiver's declared type must be > ValueBase. > > > > Another reason why the derived type has to be truncatable is that > this > > guarantees that it will be marshalled in chunked form. This > allows > > the receiver to know how much data has been sent. > > We could possible change this rule for the case under discussion, > which since > it would involve an addition to IDL syntax would be upwardly > compatible. > OK, but any proposed solution must take into account indirections to previously marshaled valuetypes in the same message. I think this > means that the receiver must fully parse the contents of the marshalled "opaque" > valuetype and any others to which indirections refer. Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb From: Paul Kyzivat To: interop@omg.org Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Wed, 29 Mar 2000 12:25:10 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: %=i!!,Qad9"3kd9=_=!! > From: Simon Nash [mailto:nash@hursley.ibm.com] > The SendingContextCodeBase described in chapter 5 of the core is a > language neutral solution. If it weren't, I would not have proposed > > it to solve this problem. Specifically, I am talking about using > the > "meta" method which returns a FullValueDescription. This has no > dependency on Java and does not need to be implemented multiple > times > for multiple languages. The information in the FullValueDescription > is sufficient to parse any IDL valuetype sent by any language. Simon, OK, please excuse my ignorance. But isn't this still a problem for programs that aren't prepared to act as servers? The program could provide a SendingContextCodeBase that points back to some other server, but then you have just invented an alternative IR with all the problems that entails. From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Wed, 29 Mar 2000 12:41:28 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: ,S_d9j16!!b3"!!i2j!! > From: Simon Nash [mailto:nash@hursley.ibm.com] > OK, but any proposed solution must take into account indirections to > previously marshaled valuetypes in the same message. I think > this means that > the receiver must fully parse the contents of the marshalled > "opaque" valuetype and any others to which indirections refer. It would be nice (for this) if indirections could not be buried inside of chunks, but had to be exposed (like a special kind of chunk). Paul Sender: jbiggar@corvette.floorboard.com Message-ID: <38E2ABA7.71702A5E@floorboard.com> Date: Wed, 29 Mar 2000 17:19:35 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.7 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD7E@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: ;6kd96HG!!]&(e9m?3e9 I've been following this discussion for a while, and I thought it might be useful to have a recap: The fundamental problem comes down to the desire that Services like the event service be able to pass through valuetypes that it does not have compile time knowledge of. The obvious solution is to maintain the valuetype in its CDR encoded form, but the valuetype CDR encoding is not self-identifying in several ways: 1. The non-chunked encoding does not indicate the end of a valuetype (or the existence and extent of valuetypes encoded in its state). 2. The chunked encoding solves this problem, but still is not adequate. The encoded form cannot be reliably remarshalled without knowing the valuetype's TypeCode, since the byte order of the encoding is not part of the encoding, and even the chunked form doesn't make it possible to find and patch valuetype indirections in a valuetype's state. The only solution that applies to GIOP 1.2 is for the receiver to find the TypeCode for the valuetype in question. It can find the TypeCode in two ways: 1. Using a CodeBase reference it receives from the sender. 2. Doing its own lookup in the IFR. This is not as reliable as option 1, since it can suffer from version skew problems. If these methods of finding the TypeCode fails, then the receiver has no choice but to raise the MARSHAL or NO_IMPLEMENT exceptions as detailed in Chapter 5. Looking forward, if the CDR encoding for valuetypes were modified in the following ways (in GIOP 1.3?), then valuetypes could be held in CDR encoded-form and remarshalled without having to know the TypeCode (or equivalent information): 1. Always require the chunked encoding, unless the valuetype is guaranteed to be known to the receiver (at minimum, the actual type matches the formal type.) 2. Use one of the four unused bits in the valuetype header to indicate the byte order used to encode the valuetype. This provides an equivalent to encapsulation for chunked valuetypes. Note that in this case, the use of the valuetype end-tag needs to be modified to not allow a single end-tag to terminate valuetypes with different byte order encodings. 3. Require that indirect valuetype encodings not nest in a parent valuetype's state. This can be accomplished by using another of the unused header bits to indicate that the valuetype is an indirection. It would probably be best to require this change for null valuetypes as well, just to simplify CDR processing. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Date: Thu, 30 Mar 2000 13:06:10 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Jonathan Biggar CC: Paul Kyzivat , "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD7E@bos1.noblenet.com> <38E2ABA7.71702A5E@floorboard.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: L)Od9p##!!k6M!!_QQd9 Jon, Jonathan Biggar wrote: > > I've been following this discussion for a while, and I thought it > might > be useful to have a recap: > > The fundamental problem comes down to the desire that Services like > the > event service be able to pass through valuetypes that it does not > have > compile time knowledge of. > > The obvious solution is to maintain the valuetype in its CDR encoded > form, but the valuetype CDR encoding is not self-identifying in > several > ways: > > 1. The non-chunked encoding does not indicate the end of a > valuetype > (or the existence and extent of valuetypes encoded in its state). > > 2. The chunked encoding solves this problem, but still is not > adequate. The encoded form cannot be reliably remarshalled without > knowing the valuetype's TypeCode, since the byte order of the > encoding > is not part of the encoding, and even the chunked form doesn't make > it > possible to find and patch valuetype indirections in a valuetype's > state. > Agreed. > The only solution that applies to GIOP 1.2 is for the receiver to find > the TypeCode for the valuetype in question. It can find the TypeCode in > two ways: > > 1. Using a CodeBase reference it receives from the sender. > > 2. Doing its own lookup in the IFR. This is not as reliable as option > 1, since it can suffer from version skew problems. > > If these methods of finding the TypeCode fails, then the receiver has no > choice but to raise the MARSHAL or NO_IMPLEMENT exceptions as detailed > in Chapter 5. > Agreed. > Looking forward, if the CDR encoding for valuetypes were modified in the > following ways (in GIOP 1.3?), then valuetypes could be held in CDR > encoded-form and remarshalled without having to know the TypeCode (or > equivalent information): > > 1. Always require the chunked encoding, unless the valuetype is > guaranteed to be known to the receiver (at minimum, the actual type > matches the formal type.) > This imposes a lot of redundant chunking in the many cases where the receiver does know the type definition, but the sender does not know that the receiver knows it. I am not keen to penalise the common case for the sake of the uncommon. I believe the forcing of chunking should be controlled by some explicit declaration (modifier?) in IDL. > 2. Use one of the four unused bits in the valuetype header to indicate > the byte order used to encode the valuetype. This provides an > equivalent to encapsulation for chunked valuetypes. Note that in this > case, the use of the valuetype end-tag needs to be modified to not allow > a single end-tag to terminate valuetypes with different byte order > encodings. > Since this requires both the marshaller and unmarshaller to change, why not go ahead and use encapsulations? I would prefer to use the bit in the valuetype header to say that the valuetype is enclosed in a (true) encapsulation. Also, I don't see why the restriction on end tags is needed. The byte order of an end tag should be the byte order specified in the highest- level header that the end tag matches. If lower-level headers use a different byte order, I don't think that causes any problems. > 3. Require that indirect valuetype encodings not nest in a parent > valuetype's state. This can be accomplished by using another of the > unused header bits to indicate that the valuetype is an > indirection. It > would probably be best to require this change for null valuetypes as > well, just to simplify CDR processing. > I think it's reasonable to move indirections outside of chunks. > However, this means that their encoding will have to change, since they are currently encoded as 0xffffffff followed by an offset. A value of > 0xffffffff following a chunk would currently be taken as an end tag. I assume > you are proposing that the 0xffffffff tag be replaced by a 0x7fffffxx tag with > one of the unused bits set. I don't think this change should be made for null values. There is some overhead in ending a chunk every time a null value is marshalled. Also, given that ORB implementers today have marshalling and unmarshalling code that handles null values inside chunks and other values outside chunks, it is likely to be less work to leave this as is rather than make a change. Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Thu, 30 Mar 2000 09:01:09 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: S`A!!e=id9iO1!!WD0!! First, I would like to thank Jon. This was an excellent rollup and clarification of the discussion to date, plus a concrete proposal. I fully agree. If we could agree on this, or some equivalent mutation of it, then I think we would have made a great improvement. I think there is still some work to do to realize the potential this would offer, but I think that is in other RTFs - the language bindings - so that a valuetype that is not understood can be held after it is received, and then passed on. I would hesitate to put a final stamp of approval on the interop changes until we can verify that the other changes are feasible, just to ensure we don't introduce extra bagage that can't be used. I think there is still something else to consider, that we have been pointedly ignoring until now (for good reason): other attributes that affect the way marshalling is done. The only one I know of right now is code set. This is a huge and ugly problem for this approach. But I think if we don't solve it we don't have a useful solution to the problem. About the only thing I can think of to do is break out each char or string as a separate, identifiable, chunk. But I doubt anyone (including me) would accept this. (I am hoping someone will come up with a reason why this isn't a problem, or is easily defined away.) One possibility would be to proactively send typecodes that the other end might not have, together with the mapping from the repository id to the typecode. There would be a lot of detail to this, but it might be made to work. It is contrary to the philosophy of the codebase, but could be more reliable. Comments to Simon's reply below. Paul > -----Original Message----- > From: Simon Nash [mailto:nash@hursley.ibm.com] > > Looking forward, if the CDR encoding for valuetypes were > modified in the > > following ways (in GIOP 1.3?), then valuetypes could be held in > CDR > > encoded-form and remarshalled without having to know the > TypeCode (or > > equivalent information): > > > > 1. Always require the chunked encoding, unless the valuetype is > > guaranteed to be known to the receiver (at minimum, the actual > type > > matches the formal type.) > > > This imposes a lot of redundant chunking in the many cases where the > receiver does know the type definition, but the sender does not know > that the receiver knows it. I am not keen to penalise the common > case for the sake of the uncommon. I believe the forcing of > chunking > should be controlled by some explicit declaration (modifier?) in > IDL. I strongly disagree about making this optional in the IDL. It is very unlikely that someone creating the IDL would be in a position to know all the places that valuetype might eventually be used. In the case of the java-to-idl mapping there wouldn't be any place to specify this, so presumably it would be all one way or the other. Some other ways to optimize this might be ok - for instance a policy on the sending side. But I am dubious of this - it complicates the marshalling process, and use of the policy would probably not be common. > > > 2. Use one of the four unused bits in the valuetype header > to indicate > > the byte order used to encode the valuetype. This provides an > > equivalent to encapsulation for chunked valuetypes. Note > that in this > > case, the use of the valuetype end-tag needs to be modified > to not allow > > a single end-tag to terminate valuetypes with different byte order > > encodings. > > > Since this requires both the marshaller and unmarshaller to change, > why not go ahead and use encapsulations? I would prefer to > use the bit > in the valuetype header to say that the valuetype is enclosed > in a (true) encapsulation. I agree that since the marshalling will change it would be possible to change the encoding in some other way to indicate this. But use of encapsulation to encode a full valuetype is not acceptable, because then there would be no way to control the size to fit in with fragmentation. If you only mean to use encapsulation for a chunk, then that might be ok. However, especially when used for small things, encapsulations are quite space inefficient - first you lay down an octet to indicate the byte order, and then quite often you waste 3 pad bytes before the first piece of the content. If you are concerned with the inefficiency of chunking, this would increase it. > > Also, I don't see why the restriction on end tags is needed. The > byte > order of an end tag should be the byte order specified in the > highest- > level header that the end tag matches. If lower-level headers use a > different byte order, I don't think that causes any problems. I had the same reaction, but haven't thought it through fully. Date: Thu, 30 Mar 2000 20:55:00 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD84@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: 0IHe9T\c!!p=V!!L\He9 Paul, See my responses below. Simon Paul Kyzivat wrote: > > First, I would like to thank Jon. This was an excellent rollup and > clarification of the discussion to date, plus a concrete proposal. > I fully agree. > > If we could agree on this, or some equivalent mutation of it, then I > think > we would have made a great improvement. I think there is still some > work to > do to realize the potential this would offer, but I think that is in > other > RTFs - the language bindings - so that a valuetype that is not > understood > can be held after it is received, and then passed on. I would > hesitate to > put a final stamp of approval on the interop changes until we can > verify > that the other changes are feasible, just to ensure we don't > introduce extra > bagage that can't be used. > > I think there is still something else to consider, that we have been > pointedly ignoring until now (for good reason): other attributes > that affect > the way marshalling is done. The only one I know of right now is > code set. > This is a huge and ugly problem for this approach. But I think if we > don't > solve it we don't have a useful solution to the problem. About the > only > thing I can think of to do is break out each char or string as a > separate, > identifiable, chunk. But I doubt anyone (including me) would accept > this. (I > am hoping someone will come up with a reason why this isn't a > problem, or is > easily defined away.) > I think it's a real problem. I've proposed a solution at the end of > this message. > One possibility would be to proactively send typecodes that the other end > might not have, together with the mapping from the repository id to the > typecode. There would be a lot of detail to this, but it might be made to > work. It is contrary to the philosophy of the codebase, but could be more > reliable. > I don't agree with this. The sender should not send extra data that the receiver may never need to use. What is unreliable about the use of SendingContextCodeBase and callbacks? > Comments to Simon's reply below. > > Paul > > > -----Original Message----- > > From: Simon Nash [mailto:nash@hursley.ibm.com] > > > > Looking forward, if the CDR encoding for valuetypes were > > modified in the > > > following ways (in GIOP 1.3?), then valuetypes could be held in > CDR > > > encoded-form and remarshalled without having to know the > > TypeCode (or > > > equivalent information): > > > > > > 1. Always require the chunked encoding, unless the valuetype is > > > guaranteed to be known to the receiver (at minimum, the actual > type > > > matches the formal type.) > > > > > This imposes a lot of redundant chunking in the many cases where > the > > receiver does know the type definition, but the sender does not > know > > that the receiver knows it. I am not keen to penalise the common > > case for the sake of the uncommon. I believe the forcing of > chunking > > should be controlled by some explicit declaration (modifier?) in > IDL. > > I strongly disagree about making this optional in the IDL. > It is very unlikely that someone creating the IDL would be in a > position to > know all the places that valuetype might eventually be used. > In the case of the java-to-idl mapping there wouldn't be any place > to > specify this, so presumably it would be all one way or the other. > This case is intended to cover an opaque value that the receiver does > not understand and cannot process itself, but wishes to be able to > unmarshal in a special form that allows remarshalling without loss of information. Clearly this is a different case from a value that the receiver needs to be able to process itself. That is why I suggested a different IDL declaration. For example, if the IDL definition of a method foo specifies a valuetype parameter of declared type Person, the receiving ORB needs to be able to unmarshal the received CDR encoding as either the Person type or some subtype of Person that it knows about. It also needs to be able to support Person semantics on the resulting unmarshalled language object. If a value of type Date is sent instead, the receiver must reject it. It cannot just unmarshal it into some generic holder for later remarshalling because this violates the type contract of the method which expects a Person. So the overhead of the special encoding that supports generic unmarshalling, should not apply in all cases. If the receiver is going to unmarshal into a generic holder and do nothing with this generic holder except pass it on for remarshalling, this should be declared in the IDL so that the unmarshaller can use generic unmarshalling (which accepts all possible values) instead of typed unmarshalling (which accepts only a subset of all possible values). This is close to declaring the parameter as type ValueBase, and we could say that all types declared as ValueBase are marshalled and unmarshalled in the special generic way. My suggestion of an extra modifier was to allow for the possibility that the receiver may receive an IDL ValueBase type and then downcast it to its actual runtime type based on other knowledge (e.g., other data passed on the method call). The generic holder form could not be downcast in this way. So an extra modifier to ValueBase could be helpful in allowing the "generic only" case to be distinguished. It would be up to the receiver to use the correct IDL declaration based on its intended use of the unmarshalled value. The Java to IDL mapping would never need to use the special IDL declaration, since it already has mechanisms that guarantee that the receiver will be able to fully unmarshal any value that can be sent using the Java to IDL mapping. > Some other ways to optimize this might be ok - for instance a policy on the > sending side. But I am dubious of this - it complicates the marshalling > process, and use of the policy would probably not be common. > It's definitely not something that should be controlled by the sending side. Whether or not it's needed depends on what the receiver intends to do with the received data. > > > > > 2. Use one of the four unused bits in the valuetype header > > to indicate > > > the byte order used to encode the valuetype. This provides an > > > equivalent to encapsulation for chunked valuetypes. Note > > that in this > > > case, the use of the valuetype end-tag needs to be modified > > to not allow > > > a single end-tag to terminate valuetypes with different byte > order > > > encodings. > > > > > Since this requires both the marshaller and unmarshaller to > change, > > why not go ahead and use encapsulations? I would prefer to > > use the bit > > in the valuetype header to say that the valuetype is enclosed > > in a (true) encapsulation. > > I agree that since the marshalling will change it would be possible > to > change the encoding in some other way to indicate this. > > But use of encapsulation to encode a full valuetype is not > acceptable, > because then there would be no way to control the size to fit in > with > fragmentation. If you only mean to use encapsulation for a chunk, > then that > might be ok. > > However, especially when used for small things, encapsulations are > quite > space inefficient - first you lay down an octet to indicate the byte > order, > and then quite often you waste 3 pad bytes before the first piece of > the > content. If you are concerned with the inefficiency of chunking, > this would > increase it. > It would not be necessary to use one encapsulation per chunk. One encapsulation could contain a number of chunks and intervening > non-chunked data such as valuetype headers, as long as they all have the same byte > order and they all fit into one fragment. So the space overhead should be > minimal. We could also extend the notion of encapsulation to include a definition of codeset as well as byte order, and use the same approach of putting mulitple chunks into one encapsulation if they were originally marshalled using the same codeset. This addresses the codeset problem that Paul described near the start of this message. > > > > Also, I don't see why the restriction on end tags is needed. The > byte > > order of an end tag should be the byte order specified in the > highest- > > level header that the end tag matches. If lower-level headers use > a > > different byte order, I don't think that causes any problems. > > I had the same reaction, but haven't thought it through fully. -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Date: Thu, 30 Mar 2000 21:46:34 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: interop@omg.org Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD7C@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: A^[d9K(+e9G > > From: Simon Nash [mailto:nash@hursley.ibm.com] > > > The SendingContextCodeBase described in chapter 5 of the core is a > > language neutral solution. If it weren't, I would not have > proposed > > it to solve this problem. Specifically, I am talking about using > the > > "meta" method which returns a FullValueDescription. This has no > > dependency on Java and does not need to be implemented multiple > times > > for multiple languages. The information in the > FullValueDescription > > is sufficient to parse any IDL valuetype sent by any language. > > Simon, > > OK, please excuse my ignorance. > > But isn't this still a problem for programs that aren't prepared to > act as > servers? The program could provide a SendingContextCodeBase that > points back > to some other server, but then you have just invented an alternative > IR with > all the problems that entails. > I suppose there are some "pure client" ORBs/applications that may not > be able to publish IORs. But what is the problem with inventing an > alternative IR? As long as a real IR exists on some server, the SendingContextCodeBase > only needs to be a trivial wrapper around this IR. Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Date: Thu, 30 Mar 2000 13:08:11 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Simon Nash CC: Paul Kyzivat , "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD84@bos1.noblenet.com> <38E3B114.594E8E78@hursley.ibm.com> Content-Type: multipart/mixed; boundary="------------A546C7252B21198902AEEEF5" X-UIDL: j%I!!`,"!!lK Paul, > See my responses below. > > Simon > > Paul Kyzivat wrote: > > One possibility would be to proactively send typecodes that the > other end > > might not have, together with the mapping from the repository id > to the > > typecode. There would be a lot of detail to this, but it might be > made to > > work. It is contrary to the philosophy of the codebase, but could > be more > > reliable. This was effectively the proposal I made. I agree that it is contrary to the codebase approach. However, it seems to me that it would not be very efficient for receivers to contact senders every time they received an Any for example, with different nested valuetypes. Imagine an event service working that way!! > I don't agree with this. The sender should not send extra data that the > receiver may never need to use. What is unreliable about the use of > SendingContextCodeBase and callbacks? It is not unreliable, it is very inefficient. (however, this is the reason I suggested this be done only inside anys, as anys should really be self contained in terms of having type information of the type they carry) > > Comments to Simon's reply below. > > > > Paul > > > > > -----Original Message----- > > > From: Simon Nash [mailto:nash@hursley.ibm.com] > > > > > > Looking forward, if the CDR encoding for valuetypes were > > > modified in the > > > > following ways (in GIOP 1.3?), then valuetypes could be held > in CDR > > > > encoded-form and remarshalled without having to know the > > > TypeCode (or > > > > equivalent information): > > > > > > > > 1. Always require the chunked encoding, unless the valuetype > is > > > > guaranteed to be known to the receiver (at minimum, the actual > type > > > > matches the formal type.) > > > > > > > This imposes a lot of redundant chunking in the many cases where > the > > > receiver does know the type definition, but the sender does not > know > > > that the receiver knows it. I am not keen to penalise the > common > > > case for the sake of the uncommon. I believe the forcing of > chunking > > > should be controlled by some explicit declaration (modifier?) in > IDL. > > > > I strongly disagree about making this optional in the IDL. > > It is very unlikely that someone creating the IDL would be in a > position to > > know all the places that valuetype might eventually be used. > > In the case of the java-to-idl mapping there wouldn't be any place > to > > specify this, so presumably it would be all one way or the other. I agree. Special IDL modifiers would help in certain cases, such as the one described by simon below. However, the problem would still exist if the signature is an Any and not the IDL modified ValueBase. Vijay > This case is intended to cover an opaque value that the receiver does not > understand and cannot process itself, but wishes to be able to unmarshal in > a special form that allows remarshalling without loss of information. > > Clearly this is a different case from a value that the receiver needs to > be able to process itself. That is why I suggested a different IDL > declaration. > > For example, if the IDL definition of a method foo specifies a valuetype > parameter of declared type Person, the receiving ORB needs to be able to > unmarshal the received CDR encoding as either the Person type or some subtype > of Person that it knows about. It also needs to be able to support Person > semantics on the resulting unmarshalled language object. If a value of type > Date is sent instead, the receiver must reject it. It cannot just unmarshal > it into some generic holder for later remarshalling because this violates > the type contract of the method which expects a Person. > > So the overhead of the special encoding that supports generic unmarshalling, > should not apply in all cases. If the receiver is going to unmarshal into > a generic holder and do nothing with this generic holder except pass it on > for remarshalling, this should be declared in the IDL so that the unmarshaller > can use generic unmarshalling (which accepts all possible values) instead of > typed unmarshalling (which accepts only a subset of all possible values). > > This is close to declaring the parameter as type ValueBase, and we could say > that all types declared as ValueBase are marshalled and unmarshalled in the > special generic way. My suggestion of an extra modifier was to allow for the > possibility that the receiver may receive an IDL ValueBase type and then > downcast it to its actual runtime type based on other knowledge (e.g., other > data passed on the method call). The generic holder form could not be downcast > in this way. So an extra modifier to ValueBase could be helpful in allowing > the "generic only" case to be distinguished. It would be up to the receiver > to use the correct IDL declaration based on its intended use of the > unmarshalled value. > > The Java to IDL mapping would never need to use the special IDL declaration, > since it already has mechanisms that guarantee that the receiver will be able > to fully unmarshal any value that can be sent using the Java to IDL mapping. > > > Some other ways to optimize this might be ok - for instance a policy on the > > sending side. But I am dubious of this - it complicates the marshalling > > process, and use of the policy would probably not be common. > > > It's definitely not something that should be controlled by the sending side. > Whether or not it's needed depends on what the receiver intends to do with > the received data. > > > > > > > > 2. Use one of the four unused bits in the valuetype header > > > to indicate > > > > the byte order used to encode the valuetype. This provides an > > > > equivalent to encapsulation for chunked valuetypes. Note > > > that in this > > > > case, the use of the valuetype end-tag needs to be modified > > > to not allow > > > > a single end-tag to terminate valuetypes with different byte order > > > > encodings. > > > > > > > Since this requires both the marshaller and unmarshaller to change, > > > why not go ahead and use encapsulations? I would prefer to > > > use the bit > > > in the valuetype header to say that the valuetype is enclosed > > > in a (true) encapsulation. > > > > I agree that since the marshalling will change it would be possible to > > change the encoding in some other way to indicate this. > > > > But use of encapsulation to encode a full valuetype is not acceptable, > > because then there would be no way to control the size to fit in with > > fragmentation. If you only mean to use encapsulation for a chunk, then that > > might be ok. > > > > However, especially when used for small things, encapsulations are quite > > space inefficient - first you lay down an octet to indicate the byte order, > > and then quite often you waste 3 pad bytes before the first piece of the > > content. If you are concerned with the inefficiency of chunking, this would > > increase it. > > > It would not be necessary to use one encapsulation per chunk. One > encapsulation could contain a number of chunks and intervening non-chunked > data such as valuetype headers, as long as they all have the same byte order > and they all fit into one fragment. So the space overhead should be minimal. > > We could also extend the notion of encapsulation to include a definition > of codeset as well as byte order, and use the same approach of putting > mulitple chunks into one encapsulation if they were originally marshalled > using the same codeset. This addresses the codeset problem that Paul > described near the start of this message. > > > > > > > Also, I don't see why the restriction on end tags is needed. The byte > > > order of an end tag should be the byte order specified in the highest- > > > level header that the end tag matches. If lower-level headers use a > > > different byte order, I don't think that causes any problems. > > > > I had the same reaction, but haven't thought it through fully. > > -- > Simon C Nash, Technology Architect, IBM Java Technology Centre > Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England > Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb [] vijayn.vcf From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Thu, 30 Mar 2000 16:30:15 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: @a^d9Y-4!!FRYd97pe!! Simon - comments below. Paul > From: Simon Nash [mailto:nash@hursley.ibm.com] > > One possibility would be to proactively send typecodes that > the other end > > might not have, together with the mapping from the > repository id to the > > typecode. There would be a lot of detail to this, but it > might be made to > > work. It is contrary to the philosophy of the codebase, but > could be more > > reliable. > > > I don't agree with this. The sender should not send extra > data that the > receiver may never need to use. What is unreliable about the use of > SendingContextCodeBase and callbacks? Maybe I wasn't clear before. If a client is sending a valuetype, it may not be prepared to service incoming calls. If so, it can't pass a SendingContextCodeBase that refers back into itself. In that case, what alternative does it have? At best, it could pass a reference to some other server that has the same information. Such a server would essentially be an IR with an different API. It would suffer from the same problems - it might not exist, might not be running, and might not be populated with information consistent with the client. These are all management nightmares that make IRs unappealing in general. While I am not wild about sending lots of extra information on the wire that might not be needed, I am not sure I see an acceptable alternative if we can't solve the codeset for encapsulation problem. If we can solve that then we don't need this. I think a sufficiently smart orb can probably avoid sending a lot of extra information - it need not send anything it knows the other end already has, and it has lots of hints about that. (When a particular operation is used, one can assume the other end knows all the types mentioned in the declaration of that operation, in other operations on the same interface, and in operations of ancestors of that interface.) But I admit it would be a pain to maintain this kind of information. > > I strongly disagree about making this optional in the IDL. > > It is very unlikely that someone creating the IDL would be > in a position to > > know all the places that valuetype might eventually be used. > > In the case of the java-to-idl mapping there wouldn't be > any place to > > specify this, so presumably it would be all one way or the other. > > > This case is intended to cover an opaque value that the > receiver does not > understand and cannot process itself, but wishes to be able > to unmarshal in > a special form that allows remarshalling without loss of > information. > > Clearly this is a different case from a value that the > receiver needs to > be able to process itself. That is why I suggested a different IDL > declaration. > > For example, if the IDL definition of a method foo specifies > a valuetype > parameter of declared type Person, the receiving ORB needs to > be able to > unmarshal the received CDR encoding as either the Person type > or some subtype > of Person that it knows about. It also needs to be able to > support Person > semantics on the resulting unmarshalled language object. If > a value of type > Date is sent instead, the receiver must reject it. It cannot > just unmarshal > it into some generic holder for later remarshalling because > this violates > the type contract of the method which expects a Person. > > So the overhead of the special encoding that supports generic > unmarshalling, > should not apply in all cases. If the receiver is going to > unmarshal into > a generic holder and do nothing with this generic holder > except pass it on > for remarshalling, this should be declared in the IDL so that > the unmarshaller > can use generic unmarshalling (which accepts all possible > values) instead of > typed unmarshalling (which accepts only a subset of all > possible values). > > This is close to declaring the parameter as type ValueBase, > and we could say > that all types declared as ValueBase are marshalled and > unmarshalled in the > special generic way. My suggestion of an extra modifier was > to allow for the > possibility that the receiver may receive an IDL ValueBase > type and then > downcast it to its actual runtime type based on other > knowledge (e.g., other > data passed on the method call). The generic holder form > could not be downcast > in this way. So an extra modifier to ValueBase could be > helpful in allowing > the "generic only" case to be distinguished. It would be up > to the receiver > to use the correct IDL declaration based on its intended use of the > unmarshalled value. Perhaps I don't understand where in IDL you would add this extra modifier. Do you mean it to be a property of the type, or a property of an argument? This need not be a pure ValueBase to need this treatment. It is entirely possible that the argument may be used by the recipient *and* passed on. The first recipient may not need any specialized behavior, but the ones later down the chain may. Also, usage may not be implicit in the *interface*, but only in the *implementation* of the interface. Some implementations may be consumers of the argument while others are intermediaries. We see exactly this kind of situation with the producer and consumer interfaces in the event service (currently using Any rather than ValueType). Another problem is that the valuetype might be inserted into an Any, or be a member of another type. I don't see where the proper choice could be declared in this case. > The Java to IDL mapping would never need to use the special > IDL declaration, > since it already has mechanisms that guarantee that the > receiver will be able > to fully unmarshal any value that can be sent using the Java > to IDL mapping. You seem to be assuming that the java to idl mapping is only used with java on both ends. An important use of this mapping is so that non-java programs can interact with existing java programs. In these cases those mechanisms do not apply. > It's definitely not something that should be controlled by > the sending side. > Whether or not it's needed depends on what the receiver > intends to do with the received data. Unfortunately, as noted above, what the receiver intends is an implementation decision, not an interface decision. > It would not be necessary to use one encapsulation per chunk. One > encapsulation could contain a number of chunks and > intervening non-chunked > data such as valuetype headers, as long as they all have the > same byte order > and they all fit into one fragment. So the space overhead > should be minimal. > > We could also extend the notion of encapsulation to include a > definition > of codeset as well as byte order, and use the same approach of > putting > mulitple chunks into one encapsulation if they were > originally marshalled > using the same codeset. This addresses the codeset problem that > Paul > described near the start of this message. We might be able to generalize encapsulation to solve the codeset problem *and* to supercede chunks as well. I think we may have more ways of doing the same thing than we need. But I don't think packing multiple chunks into an encapsulation is a good thing. I certainly don't want to be forced to double marshal - so the creation of these encapsulations and/or chunks needs to do doable on the fly. It is already excessively complicated to do this with chunking. Packing chunks into encapsulations would be considerably worse - there would be multiple lengths to fixup. Also, there would be multiple lengths on the wire - one for the encapsulation and another for each chunk in the encapsulation. So, I think we either need one more general purpose mechanism to replace both chunks and encapsulations, or else we may need to keep them both but add extra features to both. But I am not sure how we would go about adding codeset support to encapsulations, or even if it is possible to do so. It seems that at best we lose the ability to negotiate, so that a common transfer syntax must be supported by all the parties that share the encapsulation, and this will be determined by the first marshaller of the encapsulation. Date: Thu, 30 Mar 2000 22:54:42 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en MIME-Version: 1.0 To: Vijaykumar Natarajan CC: Paul Kyzivat , "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD84@bos1.noblenet.com> <38E3B114.594E8E78@hursley.ibm.com> <38E3C23B.B2103142@inprise.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: ,jdd9YmJe9g[*e9m$ed9 Vijay, Vijaykumar Natarajan wrote: > > Hi all, > > The original issue was to solve the basic problem of dealing with > Anys and with > DynValues w/o local knowledge of the type. > > We have now added to this the problem of loss of state information > in the case of > truncatable types and the extension of OBV to add the ability of > deal generically > with non truncatable valuetypes w/o type information. I think the > former is limited > in scope, while the latter is a much larger issue w/ the OBV > spec. Would it be > better to break these two into separate issues? > Sounds reasonable to me. > Comments below. > > Simon Nash wrote: > > > Paul, > > See my responses below. > > > > Simon > > > > Paul Kyzivat wrote: > > > One possibility would be to proactively send typecodes that the > other end > > > might not have, together with the mapping from the repository id > to the > > > typecode. There would be a lot of detail to this, but it might > be made to > > > work. It is contrary to the philosophy of the codebase, but > could be more > > > reliable. > > This was effectively the proposal I made. I agree that it is > contrary to the > codebase approach. However, it seems to me that it would not be very > efficient for > receivers to contact senders every time they received an Any for > example, with > different nested valuetypes. Imagine an event service working that > way!! > Not every time. Only each time they receive a new type that they have never seen before. > > I don't agree with this. The sender should not send extra data that the > > receiver may never need to use. What is unreliable about the use of > > SendingContextCodeBase and callbacks? > > It is not unreliable, it is very inefficient. (however, this is the reason I > suggested this be done only inside anys, as anys should really be self contained in > terms of having type information of the type they carry) > Not necessarily inefficient with a good caching algorithm. The other approach is much more inefficient as every any containing valuetype X must carry the runtime typecode for X, even if the same typecode has been sent many times before. > > > Comments to Simon's reply below. > > > > > > Paul > > > > > > > -----Original Message----- > > > > From: Simon Nash [mailto:nash@hursley.ibm.com] > > > > > > > > Looking forward, if the CDR encoding for valuetypes were > > > > modified in the > > > > > following ways (in GIOP 1.3?), then valuetypes could be held > in CDR > > > > > encoded-form and remarshalled without having to know the > > > > TypeCode (or > > > > > equivalent information): > > > > > > > > > > 1. Always require the chunked encoding, unless the > valuetype is > > > > > guaranteed to be known to the receiver (at minimum, the > actual type > > > > > matches the formal type.) > > > > > > > > > This imposes a lot of redundant chunking in the many cases > where the > > > > receiver does know the type definition, but the sender does > not know > > > > that the receiver knows it. I am not keen to penalise the > common > > > > case for the sake of the uncommon. I believe the forcing of > chunking > > > > should be controlled by some explicit declaration (modifier?) > in IDL. > > > > > > I strongly disagree about making this optional in the IDL. > > > It is very unlikely that someone creating the IDL would be in a > position to > > > know all the places that valuetype might eventually be used. > > > In the case of the java-to-idl mapping there wouldn't be any > place to > > > specify this, so presumably it would be all one way or the > other. > > I agree. Special IDL modifiers would help in certain cases, such as > the one > described by simon below. However, the problem would still exist if > the signature > is an Any and not the IDL modified ValueBase. > Then the Any type could imply this special marshalling. This still > makes the special marshalling a static decision based on declarations in > IDL. Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Sender: jbiggar@corvette.floorboard.com Message-ID: <38E3CFBC.C7916C84@floorboard.com> Date: Thu, 30 Mar 2000 14:05:48 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.7 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Simon Nash CC: Paul Kyzivat , "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD7E@bos1.noblenet.com> <38E2ABA7.71702A5E@floorboard.com> <38E34332.73E9EDE0@hursley.ibm.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: gQ\d9YE$"!~(Wd9,)1!! Simon Nash wrote: > > Looking forward, if the CDR encoding for valuetypes were modified > in the > > following ways (in GIOP 1.3?), then valuetypes could be held in > CDR > > encoded-form and remarshalled without having to know the TypeCode > (or > > equivalent information): > > > > 1. Always require the chunked encoding, unless the valuetype is > > guaranteed to be known to the receiver (at minimum, the actual > type > > matches the formal type.) > > > This imposes a lot of redundant chunking in the many cases where the > receiver does know the type definition, but the sender does not know > that the receiver knows it. I am not keen to penalise the common > case for the sake of the uncommon. I believe the forcing of > chunking > should be controlled by some explicit declaration (modifier?) in > IDL. I agree with Paul's comment that this requires a degree of omniscience on the part of the IDL writer that is impractical. Effectively every valuetype declaration would need this declaration, unless you are considering the lack of it to be equivalent to Java's "final". > > 2. Use one of the four unused bits in the valuetype header to indicate > > the byte order used to encode the valuetype. This provides an > > equivalent to encapsulation for chunked valuetypes. Note that in this > > case, the use of the valuetype end-tag needs to be modified to not allow > > a single end-tag to terminate valuetypes with different byte order > > encodings. > > > Since this requires both the marshaller and unmarshaller to change, > why not go ahead and use encapsulations? I would prefer to use the bit > in the valuetype header to say that the valuetype is enclosed in a (true) > encapsulation. Perhaps, but I think that adding the byte order bit would perturb existing marshalling engines less. > Also, I don't see why the restriction on end tags is needed. The byte > order of an end tag should be the byte order specified in the highest- > level header that the end tag matches. If lower-level headers use a > different byte order, I don't think that causes any problems. I still think there is a problem. However, requiring the end tag to be encoded using the byte order of the most deeply nested valuetype may take care of it. > > 3. Require that indirect valuetype encodings not nest in a parent > > valuetype's state. This can be accomplished by using another of > the > > unused header bits to indicate that the valuetype is an > indirection. It > > would probably be best to require this change for null valuetypes > as > > well, just to simplify CDR processing. > > > I think it's reasonable to move indirections outside of chunks. > However, > this means that their encoding will have to change, since they are > currently encoded as 0xffffffff followed by an offset. A value of > 0xffffffff > following a chunk would currently be taken as an end tag. I assume > you are > proposing that the 0xffffffff tag be replaced by a 0x7fffffxx tag > with one > of the unused bits set. Right. > I don't think this change should be made for null values. There is > some overhead in ending a chunk every time a null value is > marshalled. > Also, given that ORB implementers today have marshalling and > unmarshalling > code that handles null values inside chunks and other values outside > chunks, it is likely to be less work to leave this as is rather than > make a change. Perhaps so. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Thu, 30 Mar 2000 17:22:09 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: J1hd9!>7!!?_&!!j>2!! > From: Vijaykumar Natarajan [mailto:vnatarajan@inprise.com] > > > Hi all, > > The original issue was to solve the basic problem of dealing > with Anys and with > DynValues w/o local knowledge of the type. > > We have now added to this the problem of loss of state > information in the case of > truncatable types and the extension of OBV to add the ability > of deal generically > with non truncatable valuetypes w/o type information. I think > the former is limited > in scope, while the latter is a much larger issue w/ the OBV > spec. Would it be better to break these two into separate issues? If you consider DynValue to be the problem, and need sufficient typecode information with an Any to solve that, then perhaps it is a separate problem. But the other problem still exists, and clearly they are related. Most likely common techniques can and should be used to fix them. In any case, I am not sure you can fully separate them. Suppose I have an argument of type ValueBase. I receive some type I don't understand via this argument. I then insert it into an Any, convert it to a DynAny, narrow it to a DynValue, and try to explore it. What is going to happen? Shouldn't this do the right thing? Your bringing this up is still helpful. It says a solution to the unknown value problem that doesn't include access to the typecode isn't a complete solution. > This was effectively the proposal I made. Yes. I appologize for not crediting you. I didn't quite remember where it had originated. > I agree that it is contrary to the > codebase approach. However, it seems to me that it would not > be very efficient for > receivers to contact senders every time they received an Any > for example, with > different nested valuetypes. Imagine an event service working > that way!! I am not quite so concerned with that. It would only be a problem if the event service was constantly confronted with new clients sending more and more new types. Unless you have an automated client factory synthesizing new IDL, you will probably soon have learned all the types you need. But having a server available is a serious problem. > It is not unreliable, it is very inefficient. (however, this > is the reason I > suggested this be done only inside anys, as anys should > really be self contained in > terms of having type information of the type they carry) I gave examples elsewhere of why this can be a problem without Any. Paul Sender: jbiggar@corvette.floorboard.com Message-ID: <38E3D1F1.9FA7DF13@floorboard.com> Date: Thu, 30 Mar 2000 14:15:13 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.7 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD84@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: 'Ui!!QS=!!-22!!"2P!! Paul Kyzivat wrote: > > First, I would like to thank Jon. This was an excellent rollup and > clarification of the discussion to date, plus a concrete proposal. > I fully agree. > > If we could agree on this, or some equivalent mutation of it, then I > think > we would have made a great improvement. I think there is still some > work to > do to realize the potential this would offer, but I think that is in > other > RTFs - the language bindings - so that a valuetype that is not > understood > can be held after it is received, and then passed on. I would > hesitate to > put a final stamp of approval on the interop changes until we can > verify > that the other changes are feasible, just to ensure we don't > introduce extra > bagage that can't be used. > > I think there is still something else to consider, that we have been > pointedly ignoring until now (for good reason): other attributes > that affect > the way marshalling is done. The only one I know of right now is > code set. > This is a huge and ugly problem for this approach. But I think if we > don't > solve it we don't have a useful solution to the problem. About the > only > thing I can think of to do is break out each char or string as a > separate, > identifiable, chunk. But I doubt anyone (including me) would accept > this. (I > am hoping someone will come up with a reason why this isn't a > problem, or is > easily defined away.) Now you've done it! Just when I thought I had a workable proposal, you had to come along and wreck it. I'll bet your favorite pastime when you were a kid was knocking down other people's sandcastles. :-) Paul just identified a horrible problem that messes up everything. Embedded string data in an unknown valuetype prevents remarshalling it on a connection that has a different codeset. Without someway to identify and convert all of the string data, the whole idea falls apart. > One possibility would be to proactively send typecodes that the other end > might not have, together with the mapping from the repository id to the > typecode. There would be a lot of detail to this, but it might be made to > work. It is contrary to the philosophy of the codebase, but could be more > reliable. I was just thinking about the same idea. Instead of codebase, which is a reactive solution (but still necessary for Java to access implementations), have a proactive service context: typedef sequence TypeCodeSeq; struct ValueTypeInfo { TypeCodeSeq value_tc_data; }; This service context would carry the TypeCodes for all valuetypes that are marshalled in the request or response message. A valuetype's TypeCode would only have to be sent once, and could also be elided if the sender can guarantee that the receiver knows the type already (such as valuetype formal parameters.) The downside of this is that it would require a lot more state be kept on both sides of the connection. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org From: Paul Kyzivat To: interop@omg.org Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Thu, 30 Mar 2000 17:33:36 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: (] From: Simon Nash [mailto:nash@hursley.ibm.com] > I suppose there are some "pure client" ORBs/applications that > may not be able > to publish IORs. But what is the problem with inventing an > alternative IR? > As long as a real IR exists on some server, the > SendingContextCodeBase only > needs to be a trivial wrapper around this IR. There is no provision in the definition of the IR for federation, versioning of contents, alternative namespaces so unrelated users don't collide in use of names, etc. For all of these reasons, depending on an IR as an integral part of the runtime environment is a joke. The only IR that I would consider even vaguely reliable would be the anonymous one that implements an interface obtained by calling _get_interface. That one presumably has some relationship to the creator of the IOR it describes. Even that can only be presumed to be reliable if it is actually implemented as part of the server of the IOR itself. This situation doesn't hold in the case we are discussing, since a pure client can't act as its own server. In effect, this valuetype feature is effectively deprecating pure clients. Also note that callbacks may be impossible when going through firewalls. In fact the recipient of a valuetype may be prohibited from making calls to anything in the domain where the actual type of the object is known. Paul From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Thu, 30 Mar 2000 17:55:12 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: GU9!!L0bd9GEO!!i'Xd9 > From: Jonathan Biggar [mailto:jon@floorboard.com] > Now you've done it! Just when I thought I had a workable > proposal, you > had to come along and wreck it. I'll bet your favorite > pastime when you > were a kid was knocking down other people's sandcastles. :-) Were we neighbors? :-) This is the kind of problem that damned codeset stuff did to us! > I was just thinking about the same idea. Instead of > codebase, which is > a reactive solution (but still necessary for Java to access > implementations), have a proactive service context: > > typedef sequence TypeCodeSeq; > > struct ValueTypeInfo { > TypeCodeSeq value_tc_data; > }; > > This service context would carry the TypeCodes for all valuetypes > that > are marshalled in the request or response message. A valuetype's > TypeCode would only have to be sent once, and could also be elided > if > the sender can guarantee that the receiver knows the type > already (such > as valuetype formal parameters.) > > The downside of this is that it would require a lot more state be > kept > on both sides of the connection. The problem with this (for which I have no solution) is that the service context has to precede the content. I don't know about the orbs you are familiar with, but at the time the header is marshaled we don't know precisely what types are going to be marshalled as part of the data. It might be possible to deal with the declared types of the arguments at that time, but those are the ones we don't need to send. The important ones will be learned while descending through and marshalling a potentially complex graph. If these are to be sent, I think there must be a way to send them in a just-in-time way within the marshalling stream. Paul Sender: jbiggar@corvette.floorboard.com Message-ID: <38E3E804.CCD77624@floorboard.com> Date: Thu, 30 Mar 2000 15:49:24 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.7 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD89@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: 3e@e9lUHe9Yged9Wo[!! Paul Kyzivat wrote: > If you consider DynValue to be the problem, and need sufficient > typecode > information with an Any to solve that, then perhaps it is a separate > problem. > > But the other problem still exists, and clearly they are > related. Most > likely common techniques can and should be used to fix them. > > In any case, I am not sure you can fully separate them. > Suppose I have an argument of type ValueBase. I receive some type I > don't > understand via this argument. I then insert it into an Any, convert > it to a > DynAny, narrow it to a DynValue, and try to explore it. What is > going to > happen? Shouldn't this do the right thing? > > Your bringing this up is still helpful. It says a solution to the > unknown > value problem that doesn't include access to the typecode isn't a > complete > solution. Here's another complication: I've been trying to come up with a way that unknown valuetypes can be retained in marshalled form inside an Any and remarshalled again later. However, that brings up problems with the identity semantics of valuetypes: exactly how long is the identity of such a valuetype to be maintained? Forever, for the duration of an operation invocation? For the duration of the Any that contains it? Consider the following: //IDL interface I { void op_in(in any arg1, in any arg2); void op_out(out any arg1, out any arg2); }; >From my reading of the spec, if I insert the same valuetype into arg1 and arg2 and invoke op_in(), the receiver, when he extracts the valuetypes from arg1 and arg2, should get two references to the same valuetype. What happens if, instead of extracting the valuetypes, I just pass arg1 and arg2 off to another invocation of I::op_in()? I assume we expect that the third party will receive arg1 and arg2 preserving the valuetype identity semantics. Now what happens if the valuetype is unknown to the intermediate server? If the unknown valuetype is kept in marshalled form, do we expect the ORB to figure out the identity semantics corrrectly? What if, instead, the server stores the any values in memory, and then in a subsequent invocation of op_out() returns the anys. Do we expect the identity semantics to continue to hold? As we get farther and farther from the original invocation, the cost to the ORB implementation to preserve identity semantics appears to grow larger. Is there a point where preserving identity semantics is too great a cost? As a thought experiment, what if we required that a valuetype without registered factories be unmarshalled into a special stub valuetype, of the most derived base valuetype that is known by the receiver, that still carries around the extra unknown state. This stub valuetype would respond correctly to state member accessor and modifier invocations, but would raise NO_IMPLEMENT with an identified minor code for any operation invocations. The stub would also remarshal as the original valuetype, not the stub type. This would solve many of the issues with passing through valuetypes of derived types that have no factories in an intermediate server, and would remove the restriction that unknown valuetypes only be passed in an any. The valuetype's TypeCode would still be necessary to properly unmarshal into the special stub. What do people think? Is this too drastic an idea? -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Sender: jbiggar@corvette.floorboard.com Message-ID: <38E3E903.9FBC8D1B@floorboard.com> Date: Thu, 30 Mar 2000 15:53:39 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.7 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Paul Kyzivat CC: "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD8C@bos1.noblenet.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: D)Ce9M3L!!#Z=e9]+a!! Paul Kyzivat wrote: > The problem with this (for which I have no solution) is that the > service > context has to precede the content. I don't know about the orbs you > are > familiar with, but at the time the header is marshaled we don't know > precisely what types are going to be marshalled as part of the > data. It > might be possible to deal with the declared types of the arguments > at that > time, but those are the ones we don't need to send. The important > ones will > be learned while descending through and marshalling a potentially > complex > graph. > > If these are to be sent, I think there must be a way to send them in > a > just-in-time way within the marshalling stream. Yup, that is a problem. I guess we have to fall back to Vijay's proposal to have an alternative marshalled form for valuetypes that includes the TypeCode rather than just the RepositoryId information. Require that that form be used the first time a valuetype is transmitted on a connection (unless the sender knows the receiver knows it, blah, blah, blah). -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org From: Jeffrey Mischkinsky Message-Id: <200003310019.QAA01629@wheel.dcn.davis.ca.us> Subject: Re: Valuetype in anys. Unmarshaling problem? To: paulk@roguewave.com (Paul Kyzivat) Date: Thu, 30 Mar 2000 16:19:02 -0800 (PST) Cc: interop@omg.org ("Interop RTF (E-mail)") In-Reply-To: <9B164B713EE9D211B6DC0090273CEEA926BD87@bos1.noblenet.com> from "Paul Kyzivat" at Mar 30, 2000 04:30:15 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: E?#!!%;_d9I=O!!V`jd9 I've trimmed down the exchanges to just this one point. jeff 'Paul Kyzivat' writes: > > Simon - comments below. > > > 'Simon' wrote: > > the "generic only" case to be distinguished. It would be up > > to the receiver > > to use the correct IDL declaration based on its intended use of the > > unmarshalled value. > > Perhaps I don't understand where in IDL you would add this extra modifier. > Do you mean it to be a property of the type, or a property of an argument? I think this would be a property of an argument declaration as in: op(passalong in mytype parm) jeff > > This need not be a pure ValueBase to need this treatment. It is > entirely > possible that the argument may be used by the recipient *and* passed > on. The > first recipient may not need any specialized behavior, but the ones > later > down the chain may. Also, usage may not be implicit in the > *interface*, but > only in the *implementation* of the interface. Some implementations > may be > consumers of the argument while others are intermediaries. We see > exactly > this kind of situation with the producer and consumer interfaces in > the > event service (currently using Any rather than ValueType). > -- Jeff Mischkinsky jmischki@dcn.davis.ca.us +1 530-758-9850 jeff@persistence.com +1 650-372-3604 Date: Thu, 30 Mar 2000 17:34:21 -0800 From: "Vijaykumar Natarajan" X-Mailer: Mozilla 4.51 [en] (WinNT; I) X-Accept-Language: en MIME-Version: 1.0 To: Jonathan Biggar CC: Paul Kyzivat , "Interop RTF (E-mail)" Subject: Re: Valuetype in anys. Unmarshaling problem? References: <9B164B713EE9D211B6DC0090273CEEA926BD8C@bos1.noblenet.com> <38E3E903.9FBC8D1B@floorboard.com> Content-Type: multipart/mixed; boundary="------------C52E5E59165D6A19786D0D7A" X-UIDL: g^a!!NVgd9P=X!!"NR!! While we are on the topic of optimizing passing typecodes, does anybody know why we don't indirect typecodes in a request (Consider a sequence of 100 anys). Vijay Jonathan Biggar wrote: > Paul Kyzivat wrote: > > The problem with this (for which I have no solution) is that the > service > > context has to precede the content. I don't know about the orbs > you are > > familiar with, but at the time the header is marshaled we don't > know > > precisely what types are going to be marshalled as part of the > data. It > > might be possible to deal with the declared types of the arguments > at that > > time, but those are the ones we don't need to send. The important > ones will > > be learned while descending through and marshalling a potentially > complex > > graph. > > > > If these are to be sent, I think there must be a way to send them > in a > > just-in-time way within the marshalling stream. > > Yup, that is a problem. I guess we have to fall back to Vijay's > proposal to have an alternative marshalled form for valuetypes that > includes the TypeCode rather than just the RepositoryId information. > Require that that form be used the first time a valuetype is > transmitted > on a connection (unless the sender knows the receiver knows it, > blah, > blah, blah). > > -- > Jon Biggar > Floorboard Software > jon@floorboard.com > jon@biggar.org [] vijayn2.vcf From: Paul Kyzivat To: "Interop RTF (E-mail)" Subject: RE: Valuetype in anys. Unmarshaling problem? Date: Fri, 31 Mar 2000 09:32:06 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2448.0) Content-Type: text/plain; charset="iso-8859-1" X-UIDL: (-od94?]!!83%!!l$Ud9 > From: Jonathan Biggar [mailto:jon@floorboard.com] > Here's another complication: > > I've been trying to come up with a way that unknown valuetypes can > be > retained in marshalled form inside an Any and remarshalled > again later. > However, that brings up problems with the identity semantics of > valuetypes: exactly how long is the identity of such a > valuetype to be > maintained? Forever, for the duration of an operation > invocation? For > the duration of the Any that contains it? Consider the following: Sadly, I think we have now demonstrated that this technique can't work for other reasons, so there isn't much point in looking at other problems. Paul From: "Rutt, T E (Tom)" To: "'interop@omg.org'" Cc: "Rutt, T E (Tom)" Subject: wordsmithing issue 3434 Date: Tue, 7 Nov 2000 13:31:01 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain X-UIDL: @S[!!ZCO!!5O!!!]DA!! Please review this proposed resolution. If I receive no comments, it will go out as is for the next interop rtf vote. Tom Rutt Terutt@Lucent.com ------- Issue 3434: Valuetype in anys. Unmarshaling problem? (interop) Click here for this issue's archive. Source: Inprise Corporation (Mr. Vijaykumar Natarajan, vijayn@inprise.com) Nature: Uncategorized Issue Severity: Summary: There seems to be a problem with valuetypes contained in anys. Consider the following: valuetype A { A a; }; valuetype B : A { private long long x; }; If an instance w/ the following structure: A { a = {B { x = {}}}} is inserted into an any and the any is received by an event service. Given that the event service has no type information of the B encoded inside of A, the event service has no way of knowing how big A.a (an instance of B) is and how many bytes it needs to read (unless it contacts an IR) Am I missing something here? Discussion: The fundamental problem comes down to the desire that Services like the event service be able to pass through valuetypes that it does not have compile time knowledge of. The obvious solution is to maintain the valuetype in its CDR encoded form, but the valuetype CDR encoding is not self-identifying in several ways: 1. The non-chunked encoding does not indicate the end of a valuetype (or the existence and extent of valuetypes encoded in its state). 2. The chunked encoding solves this problem, but still is not adequate. The encoded form cannot be reliably remarshalled without knowing the valuetype's TypeCode, since the byte order of the encoding is not part of the encoding, and even the chunked form doesn't make it possible to find and patch valuetype indirections in a valuetype's state. The only solution that applies to GIOP 1.2 is for the receiver to find the TypeCode for the valuetype in question. It can find the TypeCode in two ways: 1. Using a CodeBase reference it receives from the sender. 2. Doing its own lookup in the IFR. This is not as reliable as option 1, since it can suffer from version skew problems. If these methods of finding the TypeCode fails, then the receiver has no choice but to raise the MARSHAL or NO_IMPLEMENT exceptions as detailed in Chapter 5. Looking forward, if a future version of CDR encoding for valuetypes were modified (in GIOP 1.3?), then valuetypes could be held in CDR encoded-form and remarshalled without having to know the TypeCode (or quivalent information). However such CDR modifications are beyond the scope of this RTF, and should be place on the GIOP wish list. Resolution: Close this issue and place it into the GIOP wish list for future enhancements to GIOP. Revised Text: None required for GIOP 1.2 Actions taken: March 20, 2000: received issue ter