Issue 4137: Implications of any/valuetype marshalling (corba-rtf) Source: Iconixx (Mr. Thomas S. Hawker, ) Nature: Uncategorized Issue Severity: Summary: RE: CCM chapters document [orbrev] 99-10-04, section 61.6.2, page 61-45. The document citation indicates that the integrity of the valuetype -- that is, the received marshalled state -- is to be preserved in an ORB-mediated operation, even if that valuetype cannot be unmarshalled, either partially (truncated) or at all. If this value is then passed to another operation, the original marshalled state is to be transmitted. This preserves the transmitted object in its entirety, regardless of local implementation concerns. This is obviously necessary for bridges or event processing, such as through the notification service. So the question arises, what happens if you have a partial (truncated) unmarshall and the recipient application changes the local state of the valuetype through its attributes or local operations? How can/will you even know the state was changed? Do you ignore the changes and send the originally received marshalled stream, send only the new valuetype even though it is a truncation of the original, or "merge" the new values for the unmarshalled part followed by the original appended data for the truncated part? Should this third option be possible through an explicit ORB call -- that is, the application is responsible to identify the change in state to the ORB? I assume that the semantics of "truncatable" must come to include the understanding that data in the truncatable portions may not be contextually dependent on the inherited parent of the valuetype. As a further question, is there a reason why this semantic interpretation should not be extended to be a general requirement rather than only with respect to transmission of anys? My experience has found that passing anys tends to be expensive and is avoided where it can be. A more general interpretation permits transmission of a comprehensive data structure among intermediate agents that only use (unmarshall) the information they need. Resolution: Revised Text: In ptc/99-10-04, remove section 61.2.2, “Integrity of value types contained in anys” on page 61-45. Actions taken: January 5, 2001: received issue May 13, 2002: closed issue June 17, 2002: issue re-opened as Core RTF issue April 11, 2012: Deferred Discussion: The integrity of value types contained in anys is a requirement on the ORB core, and an appropriate paragraph should be added to the Value Type Semantics chapter. This requirement is not limited to Components. The paragraph in the Components specification that mentions this requirement should be removed. This issue should then be moved to the Core RTF. End of Annotations:===== From: thomas.s.hawker@mail.sprint.com X-OpenMail-Hops: 1 Date: Fri, 5 Jan 2001 16:39:42 -0600 Message-Id: Subject: New CCM Issue: Implications of any/valuetype marshalling MIME-Version: 1.0 TO: issues@omg.org Content-Type: multipart/mixed; boundary="openmail-part-3898adb2-00000001" X-UIDL: ])N!!<8cd9hpj!!-a-!! RE: CCM chapters document [orbrev] 99-10-04, section 61.6.2, page 61-45.

The document citation indicates that the integrity of the valuetype -- that is, the received marshalled state -- is to be preserved in an ORB-mediated operation, even if that valuetype cannot be unmarshalled, either partially (truncated) or at all. If this value is then passed to another operation, the original marshalled state is to be transmitted. This preserves the transmitted object in its entirety, regardless of local implementation concerns. This is obviously necessary for bridges or event processing, such as through the notification service.

So the question arises, what happens if you have a partial (truncated) unmarshall and the recipient application changes the local state of the valuetype through its attributes or local operations? How can/will you even know the state was changed? Do you ignore the changes and send the originally received marshalled stream, send only the new valuetype even though it is a truncation of the original, or "merge" the new values for the unmarshalled part followed by the original appended data for the truncated part? Should this third option be possible through an explicit ORB call -- that is, the application is responsible to identify the change in state to the ORB? I assume that the semantics of "truncatable" must come to include the understanding that data in the truncatable portions may not be contextually dependent on the inherited parent of the valuetype.

As a further question, is there a reason why this semantic interpretation should not be extended to be a general requirement rather than only with respect to transmission of anys? My experience has found that passing anys tends to be expensive and is avoided where it can be. A more general interpretation permits transmission of a comprehensive data structure among intermediate agents that only use (unmarshall) the information they need.

Tom Hawker, Iconixx thawker@iconixx.com From: thomas.s.hawker@mail.sprint.com X-OpenMail-Hops: 1 Date: Thu, 25 Jan 2001 15:35:26 -0600 Message-Id: Subject: Follow up: Components Issue 4137 -- any/valuetype marshalling MIME-Version: 1.0 TO: components-ftf@emerald.omg.org, issues@emerald.omg.org Content-Type: multipart/mixed; boundary="openmail-part-3a48ba91-00000001" X-UIDL: mPGe950""!ibEe9jQI!! All, In reviewing some of the work my colleagues and I have done on the TINA standards now in the Telecom domain, I am convinced that this is a more difficult problem than first suggested. At the very minimum some additional clarification will be needed to the referenced paragraph with respect to nested or embedded valuetypes. The situation can be shown as follows: valuetype A { ... }; valuetype P { public A m; ... }; valuetype B : truncatable A { ... }; valuetype Q : truncatable P { ... }; Now, construct an instance of Q, and for member 'm', construct an instance of B. Transmit this to an active object through an 'any' that has only the knowledge of A and P, but must retransmit the member 'm' to yet another process. Although it is OK to unmarshall the member to A, on retransmission it is not clear that you should not be sending the entire object B. Obviously, retransmitting the P should retransmit the Q with the B. I wish to point out that this semantic ambiguity exists because the CCM wishes to do special handling when the type of a transmitted argument or member is 'any'. As pointed out before, I think this should be extended to include all valuetypes no matter how transmitted. This case merely falls into the gray area between truncation of the valuetype on one hand and retention of marshalled state on the other. Of course, it does nothing to address the original part of the issue, which is tracking changes to the internal state of what was transmitted. Changing the general interpretation to allow marshall stream retention may pose a significant drain on system resources. It seems to me that a user implementation would tend to know a priori whether such retention is needed. Adding a policy could control such behavior. I would also add one or more methods to determine whether a valuetype was truncated and, if so, whether the marshalled state was retained. As for the original problem, I would also add methods to set the desired behavior for retransmission (remarshall and truncate, remarshall and merge, or use retained state) and to indicate whether the object was changed. I am not expert here, so I do not know what IDL to propose with respect to the above. In an application that has multiple ORBs running simultaneously, if the valuetype did not know from which ORB it was unmarshalled, it would not be able to support the inquiry calls directly in ValueBase. Any ideas? Tom Hawker, Iconixx thawker@iconixx.com (913) 534-6569 CST/CDT USA Sender: jon@corvette.floorboard.com Message-ID: <3A74E0F5.BA968E03@floorboard.com> Date: Sun, 28 Jan 2001 19:18:13 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.76 [en] (X11; U; SunOS 5.7 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: thomas.s.hawker@mail.sprint.com CC: components-ftf@emerald.omg.org, orb_revision@omg.org, interop@omg.org Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: Ja]d9^TAe9RIe!!Z6@!! thomas.s.hawker@mail.sprint.com wrote: > In reviewing some of the work my colleagues and I have done on the TINA > standards now in the Telecom domain, I am convinced that this is a more > difficult problem than first suggested. At the very minimum some > additional clarification will be needed to the referenced paragraph with > respect to nested or embedded valuetypes. The situation can be shown as > follows: > > valuetype A { ... }; > valuetype P { public A m; ... }; > > valuetype B : truncatable A { ... }; > valuetype Q : truncatable P { ... }; > > Now, construct an instance of Q, and for member 'm', construct an > instance of B. Transmit this to an active object through an 'any' that > has only the knowledge of A and P, but must retransmit the member 'm' to > yet another process. Although it is OK to unmarshall the member to A, > on retransmission it is not clear that you should not be sending the > entire object B. Obviously, retransmitting the P should retransmit the > Q with the B. > > I wish to point out that this semantic ambiguity exists because the CCM > wishes to do special handling when the type of a transmitted argument or > member is 'any'. As pointed out before, I think this should be extended > to include all valuetypes no matter how transmitted. This case merely > falls into the gray area between truncation of the valuetype on one hand > and retention of marshalled state on the other. Of course, it does > nothing to address the original part of the issue, which is tracking > changes to the internal state of what was transmitted. I've been aware of this problem for quite some time as a member of the Core & Interop RTFs. Essentially, there is a conflict between the following requirements: 1. Valuetypes that are not understood by the receiver can be truncated (if the valuetype allows it). 2. Valuetype graph relationships must be preserved across invocations. 3. A valuetype stored in an any must be transmitted unchanged by a third party application that doesn't understand the valuetype and doesn't unwrap the valuetype from the any. You can implement any of the 2 rules above without much overhead, but not all three. This example shows the problem completely: valuetype A { ... }; valuetype B : truncatable A { ... }; interface I { void op(in A vt, in any a); }; Now call op with the same B valuetype instance for both arguments on a server that does not understand the B valuetype. Now think about what happens if this server then tries to call op on another server that *does* understand the B valuetype. It gets messy real quick, doesn't it? > Changing the general interpretation to allow marshall stream retention > may pose a significant drain on system resources. It seems to me that a > user implementation would tend to know a priori whether such retention > is needed. Adding a policy could control such behavior. I would also > add one or more methods to determine whether a valuetype was truncated > and, if so, whether the marshalled state was retained. As for the > original problem, I would also add methods to set the desired behavior > for retransmission (remarshall and truncate, remarshall and merge, or > use retained state) and to indicate whether the object was changed. Unfortunately, the problem is compounded by a problem in GIOP CDR. The problem is that the marshalled stream of a valuetype cannot be reliably remarshalled without knowing the valuetype's complete TypeCode (and not at all for custom valuetypes) because the valuetype header encoding does not specify the byte order of the marshalled valuetype data. The upshot of this is that your suggestions to preserve the valuetype marshal stream doesn't work in almost all circumstances. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org From: thomas.s.hawker@mail.sprint.com X-OpenMail-Hops: 1 Date: Mon, 29 Jan 2001 09:19:42 -0600 Message-Id: Subject: RE: Follow up: Components Issue 4137 -- any/valuetype marshalling MIME-Version: 1.0 TO: jon@floorboard.com CC: components-ftf@emerald.omg.org, interop@omg.org, orb_revision@omg.org Content-Type: multipart/mixed; boundary="openmail-part-3a7b924b-00000001" X-UIDL: aL]!!/3,!!jJi!!%4)e9 Jon, Thank you for your informative reply. I'm not willing to concede defeat yet, however, as I think you may be throwing the baby out with the bathwater. I will admit that you cannot remarshall custom-marshalled valuetypes, thus either changes must be abandoned to retransmit the original value or the extensions must be discarded to marshall the truncated value. OK, I don't really have a problem with that. I guess this is "caveat emptor" if you're going to specify custom marshalling. You make the claim below, however, that other forms of remarshalling won't work because you need to know the actual TypeCode as well as the byte-encoding order. But wasn't this information available when the original value was demarshalled? The byte order was determined when the stream was extracted. I thought the value's TypeCode was included at the beginning. I'm not saying this would be easy, but if you retain the data during the unmarshall, the ORB would then be able to "substitute" or use the saved data where required. After all, we are assuming the ORB knows that this is remarshalling of a previously received value. I'm not wedded to the idea that the ORB should remarshall at all. I find this a particularly difficult semantic issue, because quite often [at least in my experience] changing data in the unmarshalled truncated portion will affect the interpretation of the data that could not be extracted. I'm not sure I want to introduce the possibility for such subtle programming defects. I'm not sure from your reply, but is the embedded valuetype concern still an independent but related problem? The retransmission semantics of the embedded values, when truncated or extracted from truncated enclosures, is unclearly specified. Tom Hawker, Iconixx thawker@iconixx.com > -----Original Message----- > From: jon@corvette.floorboard.com > [mailto:jon@corvette.floorboard.com]On > Behalf Of Jonathan Biggar > Sent: Sunday, January 28, 2001 9:18 PM > To: thomas.s.hawker@mail.sprint.com > Cc: components-ftf@emerald.omg.org; orb_revision@omg.org; > interop@omg.org > Subject: Re: Follow up: Components Issue 4137 -- any/valuetype > marshalling > > > thomas.s.hawker@mail.sprint.com wrote: > > In reviewing some of the work my colleagues and I have done on the TINA > > standards now in the Telecom domain, I am convinced that this is a more > > difficult problem than first suggested. At the very minimum some > > additional clarification will be needed to the referenced paragraph with > > respect to nested or embedded valuetypes. The situation can be shown as > > follows: > > > > valuetype A { ... }; > > valuetype P { public A m; ... }; > > > > valuetype B : truncatable A { ... }; > > valuetype Q : truncatable P { ... }; > > > > Now, construct an instance of Q, and for member 'm', construct an > > instance of B. Transmit this to an active object through an 'any' that > > has only the knowledge of A and P, but must retransmit the member 'm' to > > yet another process. Although it is OK to unmarshall the member to A, > > on retransmission it is not clear that you should not be sending the > > entire object B. Obviously, retransmitting the P should retransmit the > > Q with the B. > > > > I wish to point out that this semantic ambiguity exists because the CCM > > wishes to do special handling when the type of a transmitted argument or > > member is 'any'. As pointed out before, I think this should be extended > > to include all valuetypes no matter how transmitted. This case merely > > falls into the gray area between truncation of the valuetype on one hand > > and retention of marshalled state on the other. Of course, it does > > nothing to address the original part of the issue, which is tracking > > changes to the internal state of what was transmitted. > > I've been aware of this problem for quite some time as a member of the > Core & Interop RTFs. Essentially, there is a conflict between the > following requirements: > > 1. Valuetypes that are not understood by the receiver can be truncated > (if the valuetype allows it). > > 2. Valuetype graph relationships must be preserved across invocations. > > 3. A valuetype stored in an any must be transmitted unchanged by a > third party application that doesn't understand the valuetype and > doesn't unwrap the valuetype from the any. > > You can implement any of the 2 rules above without much overhead, but > not all three. > > This example shows the problem completely: > > valuetype A { ... }; > valuetype B : truncatable A { ... }; > > interface I { > void op(in A vt, in any a); > }; > > Now call op with the same B valuetype instance for both arguments on a > server that does not understand the B valuetype. Now think about what > happens if this server then tries to call op on another server that > *does* understand the B valuetype. It gets messy real quick, doesn't > it? > > > Changing the general interpretation to allow marshall stream retention > > may pose a significant drain on system resources. It seems to me that a > > user implementation would tend to know a priori whether such retention > > is needed. Adding a policy could control such behavior. I would also > > add one or more methods to determine whether a valuetype was truncated > > and, if so, whether the marshalled state was retained. As for the > > original problem, I would also add methods to set the desired behavior > > for retransmission (remarshall and truncate, remarshall and merge, or > > use retained state) and to indicate whether the object was changed. > > Unfortunately, the problem is compounded by a problem in GIOP CDR. The > problem is that the marshalled stream of a valuetype cannot be reliably > remarshalled without knowing the valuetype's complete TypeCode (and not > at all for custom valuetypes) because the valuetype header encoding does > not specify the byte order of the marshalled valuetype data. > > The upshot of this is that your suggestions to preserve the valuetype > marshal stream doesn't work in almost all circumstances. > > -- > Jon Biggar > Floorboard Software > jon@floorboard.com > jon@biggar.org Date: Mon, 29 Jan 2001 18:28:18 +0100 (MET) Message-Id: <200101291728.SAA19712@pandora.informatik.hu-berlin.de> X-Authentication-Warning: pandora.informatik.hu-berlin.de: loewis set sender to loewis@informatik.hu-berlin.de using -f From: Martin von Loewis To: thomas.s.hawker@mail.sprint.com CC: jon@floorboard.com, components-ftf@emerald.omg.org, interop@omg.org, orb_revision@omg.org In-reply-to: (thomas.s.hawker@mail.sprint.com) Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: User-Agent: SEMI/1.13.7 (Awazu) FLIM/1.13.2 (Kasanui) Emacs/20.6 (sparc-sun-solaris2.6) MULE/4.0 (HANANOEN) MIME-Version: 1.0 (generated by SEMI 1.13.7 - "Awazu") Content-Type: text/plain; charset=US-ASCII X-UIDL: T^>e9!5'e9641!!n8>e9 > You make the claim below, however, that other forms of remarshalling > won't work because you need to know the actual TypeCode as well as > the > byte-encoding order. But wasn't this information available when the > original value was demarshalled? Sure, but you don't know which bytes in the extension are subject to byte order. > The byte order was determined when the stream was extracted. I > thought the value's TypeCode was included at the beginning. It isn't. GIOP is not self-describing (nor do I think it should be). The only information that is transmitted is what the truncatable-to base types are. > I'm not wedded to the idea that the ORB should remarshall at all. Unfortunately, it has to. Two different cases: valuetype Base{}; valuetype Derived:truncatable Base{ public string foo; public long bar; }; Suppose a little-endian ORB receives two instances of Derived , but only knows about Base. Further assume that it got one value from a little-endian, the other from a big-endian ORB, and now wants to transmit both of them in a single message. The length of the string has to be byte-swapped, and the long value; the bytes of the string must be left alone. To know that this must happen, the ORB would need to know that there is a long following a string in the extension. Another problem is assignment. Suppose you have valuetype Base{}; valuetype Derived:truncatable Base{ public long l; public double d; }; Depending on the exact position of the Base start in the stream, there sometimes is padding between the l and the d, and sometimes isn't. So when retransmitting the value, the ORB may need to remove padding from the middle of the message. That is impossible without type information. I Regards, Martin From: thomas.s.hawker@mail.sprint.com X-OpenMail-Hops: 1 Date: Mon, 29 Jan 2001 12:34:19 -0600 Message-Id: Subject: RE: Follow up: Components Issue 4137 -- any/valuetype marshalling MIME-Version: 1.0 TO: loewis@informatik.hu-berlin.de CC: components-ftf@emerald.omg.org, interop@omg.org, jon@floorboard.com, orb_revision@omg.org Content-Type: multipart/mixed; boundary="openmail-part-3a850b01-00000001" X-UIDL: Pe)!!,]_!!LcFe9AfC!! Martin, Jon, Alright, remarshalling for changed values of the unmarshalled portion of a valuetype is a bad idea, or at least unworkable. What about the other facets of this problem? Also, see other comments below. Tom > -----Original Message----- > From: loewis [mailto:loewis@informatik.hu-berlin.de] > Sent: Monday, January 29, 2001 11:28 AM > To: thomas.s.hawker > Cc: loewis; jon; components-ftf; interop; orb.revision > Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling > > > [example snipped] > > Another problem is assignment. Suppose you have > > valuetype Base{}; > valuetype Derived:truncatable Base{ > public long l; > public double d; > }; > > Depending on the exact position of the Base start in the stream, there > sometimes is padding between the l and the d, and sometimes isn't. So > when retransmitting the value, the ORB may need to remove padding from > the middle of the message. That is impossible without type > information. Yes, I can see this gets worse. Put a string in Base, then change the value of the string, etc. However, this seems to imply that retransmission in general is impossible without some rethinking here. Since padding is inserted as needed, unless one always aligns valuetypes to the maximum alignment (this doesn't seem unreasonable), one can never safely retransmit any truncated value, as one will never be able to deduce the padding previously inserted, as per your example. Would encapsulating valuetypes (marshalling separately and transmitting with byte-order) work, even though it would be run-time expensive because of all the temporary strings involved? I'm unclear as to what effect this would have on Jon's case of dual references (sending 'B' for both arguments) or other forms of graphs and cycles: valuetype A { ... }; valuetype B : truncatable A { ... }; interface I { void op(in A vt, in any a); }; Come to think of it, I'm not sure one could take an embedded member and reconstruct it either through the current techniques or independent encapsulation, simply because of the complexity of tracking down references to previously encoded valuetypes that aren't contained directly. Given: valuetype Node { Node left; Node right; }; valuetype Tree : truncatable Node { long type; string id; ... }; and the tree of instances (A(left->B, right->C), B(left->D), C(left->D, right->B), D), if I remember the section on marshalling correctly, the actual sequence will be A(left => B (left => D (left, right, type, id), right, type, id, right => C (left => D/ref, right => B/ref, type, id), type, id), at least by members. C's references to both B and D use the "already seen" shortcut encoding. If I turn around and try to transmit C to another process, I would have to remarshall C using previously encoded snippets of B and D (a simple string extraction wouldn't work). This, of course, is recursive in large graphs. I can see at least two ways to handle this -- self-descriptive valuetypes and encapsulated valuetypes with reference tables (I'll explain in detail if desired) -- but both would require a change in the GIOP encoding format, and of course this wouldn't deal with backward compatibility issues caused by component unaware ORBs. It seems nothing is going to handle those. This whole thing is beginning to sound like an "impossible" requirement. My head hurts. I think it's time for lunch. Cheers. Date: Tue, 30 Jan 2001 10:36:59 +0100 (MET) Message-Id: <200101300936.KAA25364@pandora.informatik.hu-berlin.de> X-Authentication-Warning: pandora.informatik.hu-berlin.de: loewis set sender to loewis@informatik.hu-berlin.de using -f From: Martin von Loewis To: thomas.s.hawker@mail.sprint.com CC: components-ftf@emerald.omg.org, interop@omg.org, jon@floorboard.com, orb_revision@omg.org In-reply-to: (thomas.s.hawker@mail.sprint.com) Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: User-Agent: SEMI/1.13.7 (Awazu) FLIM/1.13.2 (Kasanui) Emacs/20.6 (sparc-sun-solaris2.6) MULE/4.0 (HANANOEN) MIME-Version: 1.0 (generated by SEMI 1.13.7 - "Awazu") Content-Type: text/plain; charset=US-ASCII X-UIDL: Z_4e93Yg!!e3`d9o(Ae9 > However, this seems to imply that retransmission in general is > impossible without some rethinking here. Since padding is inserted > as > needed, unless one always aligns valuetypes to the maximum alignment > (this doesn't seem unreasonable), one can never safely retransmit > any > truncated value, as one will never be able to deduce the padding > previously inserted, as per your example. It is certainly possible to retransmit the base valuetype part of it, as you know its structure and can remarshal it. It is impossible (given the current inter-orb protocol) to retransmit any truncated-away derived valuetype, indeed. > Would encapsulating valuetypes (marshalling separately and transmitting > with byte-order) work, even though it would be run-time expensive > because of all the temporary strings involved? I'm unclear as to what > effect this would have on Jon's case of dual references (sending 'B' for > both arguments) or other forms of graphs and cycles That would be one problem, yes: Valuetype semantics requires sharing across arguments, which would not be possible if each valuetype was transmitted as its own encapsulation. The bigger problem (as you point out) is that such a change would break emerging interoperability of valuetypes, so this should be part of the next GIOP revision (in which I'd personally like to see padding and bi-endianness removed). Regards, Martin Date: Tue, 30 Jan 2001 08:58:25 +0000 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.72 [en] (Windows NT 5.0; I) X-Accept-Language: en MIME-Version: 1.0 To: Jonathan Biggar CC: thomas.s.hawker@mail.sprint.com, components-ftf@emerald.omg.org, orb_revision@omg.org, interop@omg.org Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: <3A74E0F5.BA968E03@floorboard.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: f(Ie9N;5e93M"e9Xhcd9 Jon, Jonathan Biggar wrote: > > This example shows the problem completely: > > valuetype A { ... }; > valuetype B : truncatable A { ... }; > > interface I { > void op(in A vt, in any a); > }; > > Now call op with the same B valuetype instance for both arguments on a > server that does not understand the B valuetype. Now think about what > happens if this server then tries to call op on another server that > *does* understand the B valuetype. It gets messy real quick, doesn't > it? > I don't see any problem or messiness here. When the B instance is truncated to an A instance by the first server, this creates a new valuetype instance with a distinct identity. This new instance is not shared with any other members of the graph that was transmitted from the client to the first server. Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Sender: jon@corvette.floorboard.com Message-ID: <3A76E8CE.B5BF68B8@floorboard.com> Date: Tue, 30 Jan 2001 08:16:14 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.76 [en] (X11; U; SunOS 5.7 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Simon Nash CC: thomas.s.hawker@mail.sprint.com, components-ftf@emerald.omg.org, orb_revision@omg.org, interop@omg.org Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: <3A74E0F5.BA968E03@floorboard.com> <3A768231.31EAC023@hursley.ibm.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: 0~Oe9)MX!!B7?!!dh;e9 Simon Nash wrote: > > This example shows the problem completely: > > > > valuetype A { ... }; > > valuetype B : truncatable A { ... }; > > > > interface I { > > void op(in A vt, in any a); > > }; > > > > Now call op with the same B valuetype instance for both arguments on a > > server that does not understand the B valuetype. Now think about what > > happens if this server then tries to call op on another server that > > *does* understand the B valuetype. It gets messy real quick, doesn't > > it? > > > I don't see any problem or messiness here. When the B instance is truncated > to an A instance by the first server, this creates a new valuetype instance > with a distinct identity. This new instance is not shared with any other > members of the graph that was transmitted from the client to the first server. There certainly is a problem, as the components spec requires the first server to preserve the valuetype untruncated inside the any, yet it must truncate it when it receives it via the first argument. While I'm always a fan of the Schroedinger's Cat and quantum physics, I don't think it ought to apply to software. :) Again, we've got three rules that when put together are contradictory: 1. Preserve the valuetype inside the any. (This is important so that valuetypes transmitted via the event, notification, or trader services work well.) 2. Preserve valuetype identity in valuetype graphs. (This is pretty fundamental. If you drop this one, I'll bet I can construct a case where breaking a valuetype relationship loop results in infinite recursion.) 3. Allowing truncation so that servers and clients can deal with the parts of the valuetype that they understand. (Pretty fundamental, otherwise valuetype inheritence is too risky to use in software that must evolve gradually.) The only solution I see at this point is to require truncated valuetypes to carry along the untruncated information somehow, so it can be restored when the truncated valuetype is retransmitted. This will only work if we fix the CDR rules for valuetype encoding problems that prevent remarshalling of valuetype data where we don't have the TypeCode, or if we force all clients and servers to support the GIOP SendingContextRunTime service context so that intermediary parties can always discover the TypeCode of valuetypes. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Date: Tue, 30 Jan 2001 17:16:34 +0100 (MET) Message-Id: <200101301616.RAA09852@pandora.informatik.hu-berlin.de> X-Authentication-Warning: pandora.informatik.hu-berlin.de: loewis set sender to loewis@informatik.hu-berlin.de using -f From: Martin von Loewis To: nash@hursley.ibm.com CC: jon@floorboard.com, thomas.s.hawker@mail.sprint.com, components-ftf@emerald.omg.org, orb_revision@omg.org, interop@omg.org In-reply-to: <3A768231.31EAC023@hursley.ibm.com> (message from Simon Nash on Tue, 30 Jan 2001 08:58:25 +0000) Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: <3A74E0F5.BA968E03@floorboard.com> <3A768231.31EAC023@hursley.ibm.com> User-Agent: SEMI/1.13.7 (Awazu) FLIM/1.13.2 (Kasanui) Emacs/20.6 (sparc-sun-solaris2.6) MULE/4.0 (HANANOEN) MIME-Version: 1.0 (generated by SEMI 1.13.7 - "Awazu") Content-Type: text/plain; charset=US-ASCII X-UIDL: pZcd9W0U!!eN]d9T47e9 > > valuetype A { ... }; > > valuetype B : truncatable A { ... }; > > > > interface I { > > void op(in A vt, in any a); > > }; [...] > I don't see any problem or messiness here. When the B instance is > truncated to an A instance by the first server, this creates a new > valuetype instance with a distinct identity. This new instance is > not shared with any other members of the graph that was transmitted > from the client to the first server. Where exactly is this property specified? I assume you are thinking of the following scenario valuetype C{ public A a1; public A a2; }; Now, somebody creates a C instance, and puts the same B instance into both a1 and a2. It then transmits the C instance, so that the contained B instance gets tuncated. Now, you are saying that the truncation yields an A instance that is not shared with any other members of the graph. So the A instance is not in the c.a1? If so, what else is in the c.a1? I'd expect to c.a1 to refer to the same valuetype as c.a2, which is an instance of an A implementation (since the original B value got truncated). Is that not the case? Regards, Martin Date: Tue, 30 Jan 2001 21:58:33 +0000 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.72 [en] (Windows NT 5.0; I) X-Accept-Language: en MIME-Version: 1.0 To: Jonathan Biggar CC: thomas.s.hawker@mail.sprint.com, components-ftf@emerald.omg.org, orb_revision@omg.org, interop@omg.org Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: <3A74E0F5.BA968E03@floorboard.com> <3A768231.31EAC023@hursley.ibm.com> <3A76E8CE.B5BF68B8@floorboard.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: ^VIe9#c$!!p;fd9ATld9 Jonathan, When a B is truncated to an A, it isn't a B any more. So it can't have the same identity as a B within an any. If this is a problem, either don't use truncatable valuetypes or else make all arguments containing truncatable valuetypes into anys. Simon Jonathan Biggar wrote: > > Simon Nash wrote: > > > This example shows the problem completely: > > > > > > valuetype A { ... }; > > > valuetype B : truncatable A { ... }; > > > > > > interface I { > > > void op(in A vt, in any a); > > > }; > > > > > > Now call op with the same B valuetype instance for both arguments on a > > > server that does not understand the B valuetype. Now think about what > > > happens if this server then tries to call op on another server that > > > *does* understand the B valuetype. It gets messy real quick, doesn't > > > it? > > > > > I don't see any problem or messiness here. When the B instance is truncated > > to an A instance by the first server, this creates a new valuetype instance > > with a distinct identity. This new instance is not shared with any other > > members of the graph that was transmitted from the client to the first server. > > There certainly is a problem, as the components spec requires the first > server to preserve the valuetype untruncated inside the any, yet it must > truncate it when it receives it via the first argument. While I'm > always a fan of the Schroedinger's Cat and quantum physics, I don't > think it ought to apply to software. :) > > Again, we've got three rules that when put together are contradictory: > > 1. Preserve the valuetype inside the any. (This is important so that > valuetypes transmitted via the event, notification, or trader services > work well.) > > 2. Preserve valuetype identity in valuetype graphs. (This is pretty > fundamental. If you drop this one, I'll bet I can construct a case > where breaking a valuetype relationship loop results in infinite > recursion.) > > 3. Allowing truncation so that servers and clients can deal with the > parts of the valuetype that they understand. (Pretty fundamental, > otherwise valuetype inheritence is too risky to use in software that > must evolve gradually.) > > The only solution I see at this point is to require truncated valuetypes > to carry along the untruncated information somehow, so it can be > restored when the truncated valuetype is retransmitted. This will only > work if we fix the CDR rules for valuetype encoding problems that > prevent remarshalling of valuetype data where we don't have the > TypeCode, or if we force all clients and servers to support the GIOP > SendingContextRunTime service context so that intermediary parties can > always discover the TypeCode of valuetypes. > > -- > Jon Biggar > Floorboard Software > jon@floorboard.com > jon@biggar.org -- Simon C Nash, Chief Technical Officer, IBM Java Technology Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Date: Tue, 30 Jan 2001 22:19:28 +0000 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.72 [en] (Windows NT 5.0; I) X-Accept-Language: en MIME-Version: 1.0 To: Martin von Loewis CC: jon@floorboard.com, thomas.s.hawker@mail.sprint.com, components-ftf@emerald.omg.org, orb_revision@omg.org, interop@omg.org Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: <3A74E0F5.BA968E03@floorboard.com> <3A768231.31EAC023@hursley.ibm.com> <200101301616.RAA09852@pandora.informatik.hu-berlin.de> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: LU7!!d?$e9_S"e9aeYd9 Martin, No truncation occurs when a B instance is assigned to c.a1 or c.a2, or when c is marshalled. It only occurs when c.a1 and c.a2 are unmarshalled by a server that understands type A but not type B. In this case (unlike Jon's example), since both c.a1 and c.a2 are getting truncated from type B to type A, it should be possible for the unmarshaller to use the same A instance for both. However, I don't think the spec says anywhere that preserving instance sharing during truncation is required. Simon Martin von Loewis wrote: > > > > valuetype A { ... }; > > > valuetype B : truncatable A { ... }; > > > > > > interface I { > > > void op(in A vt, in any a); > > > }; > > [...] > > I don't see any problem or messiness here. When the B instance is > > truncated to an A instance by the first server, this creates a new > > valuetype instance with a distinct identity. This new instance is > > not shared with any other members of the graph that was transmitted > > from the client to the first server. > > Where exactly is this property specified? I assume you are thinking of > the following scenario > > valuetype C{ > public A a1; > public A a2; > }; > > Now, somebody creates a C instance, and puts the same B instance into > both a1 and a2. It then transmits the C instance, so that the > contained B instance gets tuncated. > > Now, you are saying that the truncation yields an A instance that is > not shared with any other members of the graph. So the A instance is > not in the c.a1? If so, what else is in the c.a1? I'd expect to > c.a1 to refer to the same valuetype as c.a2, which is an instance of > an A implementation (since the original B value got truncated). > > Is that not the case? > > Regards, > Martin -- Simon C Nash, Chief Technical Officer, IBM Java Technology Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Sender: jbiggar@corvette.floorboard.com Message-ID: <3A77408D.2698FEE@floorboard.com> Date: Tue, 30 Jan 2001 14:30:37 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.76 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Simon Nash CC: Martin von Loewis , thomas.s.hawker@mail.sprint.com, components-ftf@emerald.omg.org, orb_revision@omg.org, interop@omg.org Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: <3A74E0F5.BA968E03@floorboard.com> <3A768231.31EAC023@hursley.ibm.com> <200101301616.RAA09852@pandora.informatik.hu-berlin.de> <3A773DF0.5CB56D9D@hursley.ibm.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: ==[!!,OG!!@U"e99(Me9 Simon Nash wrote: > > Martin, > No truncation occurs when a B instance is assigned to c.a1 or c.a2, or when > c is marshalled. It only occurs when c.a1 and c.a2 are unmarshalled by a > server that understands type A but not type B. > > In this case (unlike Jon's example), since both c.a1 and c.a2 are getting > truncated from type B to type A, it should be possible for the unmarshaller > to use the same A instance for both. However, I don't think the spec says > anywhere that preserving instance sharing during truncation is required. Simon, if instance sharing isn't preserved via truncation, then it is trivial to construct an example where truncation causes an infinite expansion of the valuetype graph: valuetype A { public A partner; }; valuetype B : truncatable A { }; Create two B instances, make each the partner of the other, and send it to a server that only understands valuetype A. If the server doesn't preserve instances when truncating, it will loop forever creating an infinitely long list of partners. Ergo, truncation must still preserve instance identity. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org From: thomas.s.hawker@mail.sprint.com X-OpenMail-Hops: 1 Date: Tue, 30 Jan 2001 16:47:55 -0600 Message-Id: Subject: RE: Follow up: Components Issue 4137 -- any/valuetype marshalling MIME-Version: 1.0 TO: loewis@informatik.hu-berlin.de, nash@hursley.ibm.com CC: components-ftf@emerald.omg.org, interop@omg.org, jon@floorboard.com, orb_revision@omg.org Content-Type: multipart/mixed; boundary="openmail-part-3ab1b21c-00000001" X-UIDL: ;fj!!%#pd9BGjd99HF!! Simon, > -----Original Message----- > From: nash [mailto:nash@hursley.ibm.com] > Sent: Tuesday, January 30, 2001 4:19 PM > To: loewis > Cc: nash; jon; thomas.s.hawker; components-ftf; orb.revision; interop > Subject: Re: Follow up: Components Issue 4137 -- any/valuetype > marshalling > > > Martin, > No truncation occurs when a B instance is assigned to c.a1 or c.a2, or when > c is marshalled. It only occurs when c.a1 and c.a2 are unmarshalled by a > server that understands type A but not type B. > > In this case (unlike Jon's example), since both c.a1 and c.a2 are getting > truncated from type B to type A, it should be possible for the unmarshaller > to use the same A instance for both. However, I don't think the spec says > anywhere that preserving instance sharing during truncation is required. > > Simon See sections 5.2.4.2 [sharing semantics] and 5.2.5.2 [value instance -> value type, under substitutablility], item 2. My interpretation is that identity must be preserved because by definition the base type subsumes the derived types. Thus, on truncation all references must be to the same truncated object. In Jon's example, both argument 'vt' and the any 'a' must resolve to the same object. This breaks on custom marshalling, but again by definition (or explicit restriction in section 3.8.5), custom marshalled valuetypes may not be truncatable. I will admit that Jon's infinite recursion case is sweeter. Tom Date: Tue, 30 Jan 2001 18:16:40 -0500 From: Paul Kyzivat X-Mailer: Mozilla 4.73 [en]C-CCK-MCD (WinNT; U) X-Accept-Language: en MIME-Version: 1.0 CC: components-ftf@emerald.omg.org, orb_revision@omg.org, interop@omg.org Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: <3A74E0F5.BA968E03@floorboard.com> <3A768231.31EAC023@hursley.ibm.com> <3A76E8CE.B5BF68B8@floorboard.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: B/He9UD;!!W!>!!e:/e9 Jon, Jeff M and I had a long discussion of this over dinner once. (I forget when or where.) I was advocating exactly what you are proposing. Jeff came up with some strong reasons why this couldn't work, but I can't recall them any longer. Maybe Jeff can reconstruct them. Personally I think it is going to require real changes to the way valuetypes are marshalled to fix this. One that I had in mind was to make a valuetype more like an encapsulation, by including the byte order with it. This could make remarshalling easier. (It would probably require a byte order with every chunk.) But I think Jeff found a problem even with this. Paul Jonathan Biggar wrote: [snip] > The only solution I see at this point is to require truncated valuetypes > to carry along the untruncated information somehow, so it can be > restored when the truncated valuetype is retransmitted. This will only > work if we fix the CDR rules for valuetype encoding problems that > prevent remarshalling of valuetype data where we don't have the > TypeCode, or if we force all clients and servers to support the GIOP > SendingContextRunTime service context so that intermediary parties can > always discover the TypeCode of valuetypes. Date: Wed, 31 Jan 2001 09:16:11 +0100 (MET) Message-Id: <200101310816.JAA19646@pandora.informatik.hu-berlin.de> X-Authentication-Warning: pandora.informatik.hu-berlin.de: loewis set sender to loewis@informatik.hu-berlin.de using -f From: Martin von Loewis To: nash@hursley.ibm.com CC: jon@floorboard.com, thomas.s.hawker@mail.sprint.com, components-ftf@emerald.omg.org, orb_revision@omg.org, interop@omg.org In-reply-to: <3A773DF0.5CB56D9D@hursley.ibm.com> (message from Simon Nash on Tue, 30 Jan 2001 22:19:28 +0000) Subject: Re: Follow up: Components Issue 4137 -- any/valuetype marshalling References: <3A74E0F5.BA968E03@floorboard.com> <3A768231.31EAC023@hursley.ibm.com> <200101301616.RAA09852@pandora.informatik.hu-berlin.de> <3A773DF0.5CB56D9D@hursley.ibm.com> User-Agent: SEMI/1.13.7 (Awazu) FLIM/1.13.2 (Kasanui) Emacs/20.6 (sparc-sun-solaris2.6) MULE/4.0 (HANANOEN) MIME-Version: 1.0 (generated by SEMI 1.13.7 - "Awazu") Content-Type: text/plain; charset=US-ASCII X-UIDL: 4$$!!C25e9meO!!V9Ld9 > In this case (unlike Jon's example), since both c.a1 and c.a2 are > getting truncated from type B to type A, it should be possible for > the unmarshaller to use the same A instance for both. However, I > don't think the spec says anywhere that preserving instance sharing > during truncation is required. I think it should. Is there any reason for not preserving sharing in that case? Likewise, if a value that marshalled as part of the then-truncated part of a derived valuetype, if there are multiple references to that value, I think sharing should be preserved also. Regards, Martin Sender: jon@floorboard.com Message-ID: <3C3DD798.8B54C479@floorboard.com> Date: Thu, 10 Jan 2002 10:04:08 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.77 [en] (X11; U; SunOS 5.7 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Simon Nash CC: Jishnu Mukerji , orb_revision@omg.org, components-ftf@omg.org, interop@omg.org, java2idl-rtf@omg.org Subject: Re: GIOP 1.3 and CCM (was: Proposed resolution for 4337) References: <3C224A22.83406C0@hp.com> <3C225717.9F2F9625@hursley.ibm.com> <3C2260A1.FDF8E8FB@hp.com> <3C25F451.28FCCB1B@hursley.ibm.com> <3C263E63.1BBA2008@floorboard.com> <3C3D8B85.93B4DBAE@hursley.ibm.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: T!>!!i&bd9*e1!!5Ph!! Simon Nash wrote: > > Jon, > I did a little research on this, and according to the final report of the > components FTF, this issue (4137) was resolved by removing the paragraph > in the components spec to which you are referring. I see, they punted... It's not a good idea to leave it that way, since it leaves things like the event service hopelessly broken. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org