Issue 5879: Custom Marshaling Format Clarification (java2idl-rtf) Source: (, ) Nature: Uncategorized Issue Severity: Summary: Consider the following Java classes: public class A implements java.io.Serializable { // This is a non custom class // A's data -- omitted // A's constructors - omitted // A's methods -- omitted } public class B extends A { // B's data -- omitted here // B's cobstructors -- omitted here // This is a custom class private void writeObject(java.io.ObjectOutputStream s) { .... } // Other methods omitted } public class C extends B { // This is a non-custom class // C's data -- omitted // C's constructors - omitted // C's methods -- omitted } public class D extends C { // D's data -- omitted here // D's cobstructors -- omitted here // This is a custom class private void writeObject(java.io.ObjectOutputStream s) { .... } // Other methods omitted } Here we have a class inheritance hierarchy as D:C:B:A with A and C as non-custom classes, B and D as custom classes. What is the rmi-iiop data layout on the wire when an instance of D is marshaled in terms of customer header and data w.r.t. to each of the super classes? Which one of the following layout is correct? ----------------------------------------------------------------------------------------- a. | A-Data | B-FV B-DWO B-Data | CData | FV-D DWO-D DData | ----------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------- b. | A-FV A-DWO A-Data | B-FV B-DWO B-Data | C-FV C-DWO C-Data | D-FV D- DWO DData | ------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------- c. | A-FV A-DWO A-Data | B-Data | C-Data | DData | --------------------------------------------------------------------------- Terms used for illustration purpose here: A-FV ------ means the Format version octet for class A A-DWO ------ means the defaultWriteObject boolean flag for class A A-Data ----- means the data fields of class A. B-FV ------ means the Format version octet for class B B-DWO ------ means the defaultWriteObject boolean flag for class B B-Data ----- means the data fields of class B C-FV ------ means the Format version octet for class C C-DWO ------ means the defaultWriteObject boolean flag for class C C-Data ----- means the data fields of class C. D-FV ------ means the Format version octet for class D D-DWO ------ means the defaultWriteObject boolean flag for class D D-Data ----- means the data fields of class D. Resolution: Revised Text: Actions taken: March 7, 2003: received issue Discussion: End of Annotations:===== VVVVV Date: Fri, 07 Mar 2003 16:44:29 -0800 From: "Tracy (Cuie) Zhao" Subject: Custom Marshaling Format Clarification To: java2idl-rtf@omg.org X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 Consider the following Java classes: public class A implements java.io.Serializable { // This is a non custom class // A's data -- omitted // A's constructors - omitted // A's methods -- omitted } public class B extends A { // B's data -- omitted here // B's cobstructors -- omitted here // This is a custom class private void writeObject(java.io.ObjectOutputStream s) { .... } // Other methods omitted } public class C extends B { // This is a non-custom class // C's data -- omitted // C's constructors - omitted // C's methods -- omitted } public class D extends C { // D's data -- omitted here // D's cobstructors -- omitted here // This is a custom class private void writeObject(java.io.ObjectOutputStream s) { .... } // Other methods omitted } Here we have a class inheritance hierarchy as D:C:B:A with A and C as non-custom classes, B and D as custom classes. What is the rmi-iiop data layout on the wire when an instance of D is marshaled in terms of customer header and data w.r.t. to each of the super classes? Which one of the following layout is correct? ----------------------------------------------------------------------------------------- a. | A-Data | B-FV B-DWO B-Data | CData | FV-D DWO-D DData | ----------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------- b. | A-FV A-DWO A-Data | B-FV B-DWO B-Data | C-FV C-DWO C-Data | D-FV D- DWO DData | ------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------- c. | A-FV A-DWO A-Data | B-Data | C-Data | DData | --------------------------------------------------------------------------- Terms used for illustration purpose here: A-FV ------ means the Format version octet for class A A-DWO ------ means the defaultWriteObject boolean flag for class A A-Data ----- means the data fields of class A. B-FV ------ means the Format version octet for class B B-DWO ------ means the defaultWriteObject boolean flag for class B B-Data ----- means the data fields of class B C-FV ------ means the Format version octet for class C C-DWO ------ means the defaultWriteObject boolean flag for class C C-Data ----- means the data fields of class C. D-FV ------ means the Format version octet for class D D-DWO ------ means the defaultWriteObject boolean flag for class D D-Data ----- means the data fields of class D. Cheers Tracy Date: Sat, 08 Mar 2003 18:31:05 +0000 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en To: "Tracy (Cuie) Zhao" CC: java2idl-rtf@omg.org, tim_baldwin@uk.ibm.com, caroline_maynard@uk.ibm.com Subject: Re: Custom Marshaling Format Clarification Tracy, I have asked Juergen to raise an issue for this. This is an important issue since it impacts interoperability. We need to clarify the spec on this since I don't believe it's clear as it currently stands. I believe either a. or d. (d. is my addition, see below) are the two approaches that would be consistent with the current spec. c. is definitely not correct since the DWO flag needs to appear once for each class in the hierarchy that has a writeObject method. b. is also not quite correct since there is no reason to put the FV and DWO flags on the non-custom class A, and doing this would require A to have different FullValueDescriptions depending on whether it is sent as part of a B instance or as part of an A instance (or a non-custom subclass of A). So I believe the choices come down to: ----------------------------------------------------------------------------------------- a. | A-Data | B-FV B-DWO B-Data | CData | FV-D DWO-D DData | ----------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------- d. | A-Data | B-FV B-DWO B-Data | C-FV C-DWO C-Data | D-FV D- DWO DData | ------------------------------------------------------------------------------------------------------------------------------- Option a. seems to have all the on-the-wire information that is needed. Specifically, it does include a separate DWO flag for each class with a writeObject method. Also, it is likely to be the format currently used by most ORBs, since until quite recently class C was considered to be non-custom. (Issue 4497 changed this.) The only problem with this approach is that in order to unmarshal object D successfully from a FullValueDescription, it is necessary for the is_custom flag in FVDs to be set to true if and only if the FVD's class has a directly defined writeObject method. So in this example, the FVDs for classes D and B must say isCustom=true, and the FVDs for C and A must say isCustom=false, even though C is actually custom from an IDL and chunking perspective. Effectively, the isCustom flag in the FVD has become a "has writeObject" flag. This works but doesn't really seem correct, so let's consider option d. This allows the is_custom flag in the FVD to truly mean "is custom", which seems cleaner. The downside with this approach is that it will not interoperate with older ORBs (pre 4497) unless some versioning rules are applied. Perhaps we could tie this change to the move from FV=1 to FV=2 and say that FV=1 implies the use of option a. but FV=2 implies the use of option d. Tying these together does not seem quite right but it may be a pragmatic solution depending on what ORB vendors have done so far in their released products. Could RTF members please state a preference between options a. and d. as described above, or make other proposals. Since this affects interoperability, I think it should be treated as an urgent issue. If other RTF members agree with this then I will ask Andrew to classify it as urgent. Simon > "Tracy (Cuie) Zhao" wrote: > > Consider the following Java classes: > > public class A implements java.io.Serializable { > // This is a non custom class > // A's data -- omitted > // A's constructors - omitted > // A's methods -- omitted > } > > public class B extends A { > // B's data -- omitted here > // B's cobstructors -- omitted here > > // This is a custom class > private void writeObject(java.io.ObjectOutputStream s) { .... } > > // Other methods omitted > } > > public class C extends B { > // This is a non-custom class > // C's data -- omitted > // C's constructors - omitted > // C's methods -- omitted > } > > public class D extends C { > // D's data -- omitted here > // D's cobstructors -- omitted here > > // This is a custom class > private void writeObject(java.io.ObjectOutputStream s) { .... } > > // Other methods omitted > } > > Here we have a class inheritance hierarchy as D:C:B:A with A and C as non-custom classes, B and D as custom classes. What is the rmi-iiop data layout on the wire when an instance of D is marshaled in terms of customer header and data w.r.t. to each of > the super classes? Which one of the following layout is correct? > > ----------------------------------------------------------------------------------------- > a. | A-Data | B-FV B-DWO B-Data | CData | FV-D DWO-D DData | > ----------------------------------------------------------------------------------------- > > -------------------------------------------------------------------------------------------------------------------------------- > b. | A-FV A-DWO A-Data | B-FV B-DWO B-Data | C-FV C-DWO C-Data | D-FV D- DWO DData | > ------------------------------------------------------------------------------------------------------------------------------- > > ---------------------------------------------------------------------------- > c. | A-FV A-DWO A-Data | B-Data | C-Data | DData | > --------------------------------------------------------------------------- > > Terms used for illustration purpose here: > > A-FV ------ means the Format version octet for class A > A-DWO ------ means the defaultWriteObject boolean flag for class A > A-Data ----- means the data fields of class A. > > B-FV ------ means the Format version octet for class B > B-DWO ------ means the defaultWriteObject boolean flag for class B > B-Data ----- means the data fields of class B > > C-FV ------ means the Format version octet for class C > C-DWO ------ means the defaultWriteObject boolean flag for class C > C-Data ----- means the data fields of class C. > > D-FV ------ means the Format version octet for class D > D-DWO ------ means the defaultWriteObject boolean flag for class D > D-Data ----- means the data fields of class D. > > Cheers > Tracy -- Simon C Nash, Chief Technical Officer, IBM Java Technology Hursley Park, Winchester, UK nash@hursley.ibm.com Tel. +44-1962-815156 Fax +44-1962-818999 Date: Mon, 10 Mar 2003 12:34:54 -0500 (EST) From: Bob Scheifler - SMI Software Development Reply-To: Bob Scheifler - SMI Software Development Subject: Re: Custom Marshaling Format Clarification To: czhao@borland.com Cc: java2idl-rtf@omg.org X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc > Here we have a class inheritance hierarchy as D:C:B:A with A and C as > non-custom classes, B and D as custom classes. C is a custom valuetype according to 1.3.5.6, right? "If the class does not implement java.io.Externalizable but does have a writeObject method, or extends such a class directly or indirectly, then it is mapped to an OMG IDL custom valuetype using the rules for mapping data members specified below." - Bob X-Sender: andyp@san-francisco.beasys.com X-Mailer: QUALCOMM Windows Eudora Version 5.0 Date: Tue, 18 Mar 2003 13:40:00 -0800 To: "Tracy (Cuie) Zhao" From: Andy Piper Subject: Re: Custom Marshaling Format Clarification Cc: Simon Nash , java2idl-rtf@omg.org, tim_baldwin@uk.ibm.com, caroline_maynard@uk.ibm.com At 01:28 PM 3/18/2003 -0800, Tracy (Cuie) Zhao wrote: This works but doesn't really seem correct, so let's consider option d. This allows the is_custom flag in the FVD to truly mean "is custom", which seems cleaner. The downside with this approach is that it will not interoperate with older ORBs (pre 4497) unless some versioning rules are applied. Perhaps we could tie this change to the move from FV=1 to FV=2 and say that FV=1 implies the use of option a. but FV=2 implies the use of option d. Tying these together does not seem quite right but it may be a pragmatic solution depending on what ORB vendors have done so far in their released products. Yes and yes (I like (d) and enabled on FV=2) andy Date: Tue, 18 Mar 2003 13:28:56 -0800 From: "Tracy (Cuie) Zhao" Subject: Re: Custom Marshaling Format Clarification To: Simon Nash Cc: java2idl-rtf@omg.org, tim_baldwin@uk.ibm.com, caroline_maynard@uk.ibm.com X-Accept-Language: en-us, en User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0 Simon/Juergen, I am wondering what is the current status on this issue as I am not seeing any further discussions/concerns on this. I agree with Simon that option a. does not seem correct post issue 4497, we'd prefer to have option d. for both FV=1 and FV=2. Meanwhile, we'd really like to raise it as an urgent issue to be resolved as it is impacting our coming product release. Cheers Tracy Simon Nash wrote: Tracy, I have asked Juergen to raise an issue for this. This is an important issue since it impacts interoperability. We need to clarify the spec on this since I don't believe it's clear as it currently stands. I believe either a. or d. (d. is my addition, see below) are the two approaches that would be consistent with the current spec. c. is definitely not correct since the DWO flag needs to appear once for each class in the hierarchy that has a writeObject method. b. is also not quite correct since there is no reason to put the FV and DWO flags on the non-custom class A, and doing this would require A to have different FullValueDescriptions depending on whether it is sent as part of a B instance or as part of an A instance (or a non-custom subclass of A). So I believe the choices come down to: ----------------------------------------------------------------------------------------- a. | A-Data | B-FV B-DWO B-Data | CData | FV-D DWO-D DData | ----------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------- d. | A-Data | B-FV B-DWO B-Data | C-FV C-DWO C-Data | D-FV D- DWO DData | ------------------------------------------------------------------------------------------------------------------------------- Option a. seems to have all the on-the-wire information that is needed. Specifically, it does include a separate DWO flag for each class with a writeObject method. Also, it is likely to be the format currently used by most ORBs, since until quite recently class C was considered to be non-custom. (Issue 4497 changed this.) The only problem with this approach is that in order to unmarshal object D successfully from a FullValueDescription, it is necessary for the is_custom flag in FVDs to be set to true if and only if the FVD's class has a directly defined writeObject method. So in this example, the FVDs for classes D and B must say isCustom=true, and the FVDs for C and A must say isCustom=false, even though C is actually custom from an IDL and chunking perspective. Effectively, the isCustom flag in the FVD has become a "has writeObject" flag. This works but doesn't really seem correct, so let's consider option d. This allows the is_custom flag in the FVD to truly mean "is custom", which seems cleaner. The downside with this approach is that it will not interoperate with older ORBs (pre 4497) unless some versioning rules are applied. Perhaps we could tie this change to the move from FV=1 to FV=2 and say that FV=1 implies the use of option a. but FV=2 implies the use of option d. Tying these together does not seem quite right but it may be a pragmatic solution depending on what ORB vendors have done so far in their released products. Could RTF members please state a preference between options a. and d. as described above, or make other proposals. Since this affects interoperability, I think it should be treated as an urgent issue. If other RTF members agree with this then I will ask Andrew to classify it as urgent. Simon "Tracy (Cuie) Zhao" wrote: Consider the following Java classes: public class A implements java.io.Serializable { // This is a non custom class // A's data -- omitted // A's constructors - omitted // A's methods -- omitted } public class B extends A { // B's data -- omitted here // B's cobstructors -- omitted here // This is a custom class private void writeObject(java.io.ObjectOutputStream s) { .... } // Other methods omitted } public class C extends B { // This is a non-custom class // C's data -- omitted // C's constructors - omitted // C's methods -- omitted } public class D extends C { // D's data -- omitted here // D's cobstructors -- omitted here // This is a custom class private void writeObject(java.io.ObjectOutputStream s) { .... } // Other methods omitted } Here we have a class inheritance hierarchy as D:C:B:A with A and C as non-custom classes, B and D as custom classes. What is the rmi-iiop data layout on the wire when an instance of D is marshaled in terms of customer header and data w.r.t. to each of the super classes? Which one of the following layout is correct? ----------------------------------------------------------------------------------------- a. | A-Data | B-FV B-DWO B-Data | CData | FV-D DWO-D DData | ----------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------- b. | A-FV A-DWO A-Data | B-FV B-DWO B-Data | C-FV C-DWO C-Data | D-FV D- DWO DData | ------------------------------------------------------------------------------------------------------------------------------- ---------------------------------------------------------------------------- c. | A-FV A-DWO A-Data | B-Data | C-Data | DData | --------------------------------------------------------------------------- Terms used for illustration purpose here: A-FV ------ means the Format version octet for class A A-DWO ------ means the defaultWriteObject boolean flag for class A A-Data ----- means the data fields of class A. B-FV ------ means the Format version octet for class B B-DWO ------ means the defaultWriteObject boolean flag for class B B-Data ----- means the data fields of class B C-FV ------ means the Format version octet for class C C-DWO ------ means the defaultWriteObject boolean flag for class C C-Data ----- means the data fields of class C. D-FV ------ means the Format version octet for class D D-DWO ------ means the defaultWriteObject boolean flag for class D D-Data ----- means the data fields of class D. Cheers Tracy Date: Mon, 18 Aug 2003 23:21:48 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en To: Vijaykumar Natarajan CC: java2idl-rtf@omg.org Subject: Re: issue 5879 Vijay, My apologies for not progressing the resolution of this. Your proposal seems logical but I am concerned that this would break interoperability for the much simpler case of object B in Tracy's example (a custom class extending a non-custom class). It is my belief (though I am open to correction on this) that most ORBs currently format Object B as: --------------------------------------------- | A-Data | B-FV B-DWO B-Data | --------------------------------------------- Your proposed change would require sending this as -------------------------------------------- | FV | A-Data | B-DWO B-Data | -------------------------------------------- which is an incompatible change. It has also been pointed out to me that my suggestion of making the choice between format a and d dependent on the FV value does not work when the receiver ORB needs to FVD back to the sender ORB for the definition of class C. The sender ORB may have sent C-type instances on different occasions to different receiver ORBs as FV=1 and FV=2, so when it is asked for the FVD of class C it will not be easy for it to make sure that it sets the is_custom flag in the FVD to a value consistent with the FV level of C-type data that it has previously sent to the particular receiver ORB making the FVD request. This is further complicated by the possibility that some of the IORs for a given receiver ORB could use the TAG_RMI_CUSTOM_MAX_STREAM_FORMAT TaggedComponent to change the allowable FV level for data sent on requests to that IOR, which could mean that transmission formats for C-type objects might not be consistent even between the same pair of sender and receiver ORBs. All this seems to lead to the conclusion that the best option technically is d, consistently for both FV=1 and FV=2. The downside of this is the interoperability impact that it may have on ORB vendors. Option a, while less clean, may be a more pragmatic solution for backwards interoperabilty. I would like to hear the views of RTF members on this. Would all RTF members please indicate their preference for 1. option d, consistently for both FV=1 and FV=2 2. option a 3. neither of these (please say why, or indicate what other solution you would prefer) If there is consensus support for either 1 or 2 above, I will write up a formal resolution for voting. Simon > Vijaykumar Natarajan wrote: > > Simon, > > This issue appears to not have been resolved. What's the status on this? > > Also, it seems to me that in the cases below, the FV should not be per level in the state as it describes how we are marshaling the entire state. i.e I think there is an option e: > > -------------------------------------------------------------------------------------------------------------------------------- > d. FV | A-Data | B-DWO B-Data | C-DWO C-Data | D- DWO DData | > ------------------------------------------------------------------------------------------------------------------------------- > Vijay > > Simon Nash wrote: > > Tracy, > I have asked Juergen to raise an issue for this. This is an important > issue since it impacts interoperability. We need to clarify the spec > on this since I don't believe it's clear as it currently stands. > > I believe either a. or d. (d. is my addition, see below) are the two > approaches that would be consistent with the current spec. c. is > definitely not correct since the DWO flag needs to appear once for > each class in the hierarchy that has a writeObject method. b. is > also not quite correct since there is no reason to put the FV and DWO > flags on the non-custom class A, and doing this would require A to > have different FullValueDescriptions depending on whether it is sent > as part of a B instance or as part of an A instance (or a non-custom > subclass of A). So I believe the choices come down to: > > ----------------------------------------------------------------------------------------- > a. | A-Data | B-FV B-DWO B-Data | CData | FV-D DWO-D DData | > ----------------------------------------------------------------------------------------- > > -------------------------------------------------------------------------------------------------------------------------------- > d. | A-Data | B-FV B-DWO B-Data | C-FV C-DWO C-Data | D-FV D- DWO DData | > ------------------------------------------------------------------------------------------------------------------------------- > > Option a. seems to have all the on-the-wire information that is > needed. Specifically, it does include a separate DWO flag for each > class with a writeObject method. Also, it is likely to be the format > currently used by most ORBs, since until quite recently class C was > considered to be non-custom. (Issue 4497 changed this.) The only > problem with this approach is that in order to unmarshal object D > successfully from a FullValueDescription, it is necessary for the > is_custom flag in FVDs to be set to true if and only if the FVD's > class has a directly defined writeObject method. So in this example, > the FVDs for classes D and B must say isCustom=true, and the FVDs for > C and A must say isCustom=false, even though C is actually custom > from an IDL and chunking perspective. Effectively, the isCustom flag > in the FVD has become a "has writeObject" flag. > > This works but doesn't really seem correct, so let's consider option d. > This allows the is_custom flag in the FVD to truly mean "is custom", > which seems cleaner. The downside with this approach is that it will > not interoperate with older ORBs (pre 4497) unless some versioning > rules are applied. Perhaps we could tie this change to the move from > FV=1 to FV=2 and say that FV=1 implies the use of option a. but FV=2 > implies the use of option d. Tying these together does not seem quite > right but it may be a pragmatic solution depending on what ORB vendors > have done so far in their released products. > > Could RTF members please state a preference between options a. and d. > as described above, or make other proposals. Since this affects > interoperability, I think it should be treated as an urgent issue. > If other RTF members agree with this then I will ask Andrew to > classify it as urgent. -- Simon C Nash IBM Distinguished Engineer Hursley Park, Winchester, UK nash@hursley.ibm.com Tel. +44-1962-815156 Fax +44-1962-818999 X-Sender: andyp:san-francisco.beasys.com@127.0.0.1 X-Mailer: QUALCOMM Windows Eudora Version 5.0 Date: Mon, 18 Aug 2003 15:32:38 -0700 To: Simon Nash , Vijaykumar Natarajan From: Andy Piper Subject: Re: issue 5879 Cc: java2idl-rtf@omg.org Simon, Option (a). The interoperability aspects of this are the most important to us. Option (d) effectively breaks the whole reason for support FVD and CodeBase which only really get used in an interop setting. My only other thought is would it be possible to alter the hash in the repository ID to reflect version 1 or 2? That way CodeBase could cope. Another alternative would be to version the CodeBase API somehow. andy At 11:21 PM 8/18/2003 +0100, Simon Nash wrote: Vijay, My apologies for not progressing the resolution of this. Your proposal seems logical but I am concerned that this would break interoperability for the much simpler case of object B in Tracy's example (a custom class extending a non-custom class). It is my belief (though I am open to correction on this) that most ORBs currently format Object B as: --------------------------------------------- | A-Data | B-FV B-DWO B-Data | --------------------------------------------- Your proposed change would require sending this as -------------------------------------------- | FV | A-Data | B-DWO B-Data | -------------------------------------------- which is an incompatible change. It has also been pointed out to me that my suggestion of making the choice between format a and d dependent on the FV value does not work when the receiver ORB needs to FVD back to the sender ORB for the definition of class C. The sender ORB may have sent C-type instances on different occasions to different receiver ORBs as FV=1 and FV=2, so when it is asked for the FVD of class C it will not be easy for it to make sure that it sets the is_custom flag in the FVD to a value consistent with the FV level of C-type data that it has previously sent to the particular receiver ORB making the FVD request. This is further complicated by the possibility that some of the IORs for a given receiver ORB could use the TAG_RMI_CUSTOM_MAX_STREAM_FORMAT TaggedComponent to change the allowable FV level for data sent on requests to that IOR, which could mean that transmission formats for C-type objects might not be consistent even between the same pair of sender and receiver ORBs. All this seems to lead to the conclusion that the best option technically is d, consistently for both FV=1 and FV=2. The downside of this is the interoperability impact that it may have on ORB vendors. Option a, while less clean, may be a more pragmatic solution for backwards interoperabilty. I would like to hear the views of RTF members on this. Would all RTF members please indicate their preference for 1. option d, consistently for both FV=1 and FV=2 2. option a 3. neither of these (please say why, or indicate what other solution you would prefer) If there is consensus support for either 1 or 2 above, I will write up a formal resolution for voting. Simon > Vijaykumar Natarajan wrote: > > Simon, > > This issue appears to not have been resolved. What's the status on this? > > Also, it seems to me that in the cases below, the FV should not be per level in the state as it describes how we are marshaling the entire state. i.e I think there is an option e: > > -------------------------------------------------------------------------------------------------------------------------------- > d. FV | A-Data | B-DWO B-Data | C-DWO C-Data | D- DWO DData | > ------------------------------------------------------------------------------------------------------------------------------- > Vijay > > Simon Nash wrote: > > Tracy, > I have asked Juergen to raise an issue for this. This is an important > issue since it impacts interoperability. We need to clarify the spec > on this since I don't believe it's clear as it currently stands. > > I believe either a. or d. (d. is my addition, see below) are the two > approaches that would be consistent with the current spec. c. is > definitely not correct since the DWO flag needs to appear once for > each class in the hierarchy that has a writeObject method. b. is > also not quite correct since there is no reason to put the FV and DWO > flags on the non-custom class A, and doing this would require A to > have different FullValueDescriptions depending on whether it is sent > as part of a B instance or as part of an A instance (or a non-custom > subclass of A). So I believe the choices come down to: > > ----------------------------------------------------------------------------------------- > a. | A-Data | B-FV B-DWO B-Data | CData | FV-D DWO-D DData | > ----------------------------------------------------------------------------------------- > > -------------------------------------------------------------------------------------------------------------------------------- > d. | A-Data | B-FV B-DWO B-Data | C-FV C-DWO C-Data | D-FV D- DWO DData | > ------------------------------------------------------------------------------------------------------------------------------- > > Option a. seems to have all the on-the-wire information that is > needed. Specifically, it does include a separate DWO flag for each > class with a writeObject method. Also, it is likely to be the format > currently used by most ORBs, since until quite recently class C was > considered to be non-custom. (Issue 4497 changed this.) The only > problem with this approach is that in order to unmarshal object D > successfully from a FullValueDescription, it is necessary for the > is_custom flag in FVDs to be set to true if and only if the FVD's > class has a directly defined writeObject method. So in this example, > the FVDs for classes D and B must say isCustom=true, and the FVDs for > C and A must say isCustom=false, even though C is actually custom > from an IDL and chunking perspective. Effectively, the isCustom flag > in the FVD has become a "has writeObject" flag. > > This works but doesn't really seem correct, so let's consider option d. > This allows the is_custom flag in the FVD to truly mean "is custom", > which seems cleaner. The downside with this approach is that it will > not interoperate with older ORBs (pre 4497) unless some versioning > rules are applied. Perhaps we could tie this change to the move from > FV=1 to FV=2 and say that FV=1 implies the use of option a. but FV=2 > implies the use of option d. Tying these together does not seem quite > right but it may be a pragmatic solution depending on what ORB vendors > have done so far in their released products. > > Could RTF members please state a preference between options a. and d. > as described above, or make other proposals. Since this affects > interoperability, I think it should be treated as an urgent issue. > If other RTF members agree with this then I will ask Andrew to > classify it as urgent. -- Simon C Nash IBM Distinguished Engineer Hursley Park, Winchester, UK nash@hursley.ibm.com Tel. +44-1962-815156 Fax +44-1962-818999 X-Sender: andyp:san-francisco.beasys.com@127.0.0.1 X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Tue, 19 Aug 2003 08:47:03 -0700 To: Simon Nash , java2idl-rtf@omg.org From: Andy Piper Subject: Re: [Fwd: FW: issue 5879] My guess is that there are 3 main valuehandlers in existence currently - IBM's, Sun's and Borland's. We probably need Sun to weigh in here since they will be the ones with the most severe interop issues. andy At 11:39 AM 8/19/2003 +0100, Simon Nash wrote: -- Simon C Nash IBM Distinguished Engineer Hursley Park, Winchester, UK nash@hursley.ibm.com Tel. +44-1962-815156 Fax +44-1962-818999Received: from mail-gw2.hursley.ibm.com (mail-gw2.hursley.ibm.com [9.20.56.36]) by sp15en17.hursley.ibm.com (AIX5.1/8.11.6p2/8.11.0) with ESMTP id h7J38AB67508 for ; Tue, 19 Aug 2003 04:08:10 +0100 Received: from localhost.localdomain ([127.0.0.1] helo=mail-gw2.hursley.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 19owqv-00016i-00 for nash@hursley.ibm.com; Tue, 19 Aug 2003 04:08:09 +0100 Received: from [9.166.84.167] (helo=d06relay03.portsmouth.uk.ibm.com) by mail-gw2.hursley.ibm.com with esmtp (Exim 4.12) id 19owqv-00016d-00 for nash@hursley.ibm.com; Tue, 19 Aug 2003 04:08:09 +0100 Received: from d06as04.portsmouth.uk.ibm.com (d06as04.portsmouth.uk.ibm.com [9.149.37.247]) by d06relay03.portsmouth.uk.ibm.com (8.12.9/NCO/VER6.5) with ESMTP id h7J3830J156610 for ; Tue, 19 Aug 2003 04:08:04 +0100 Received: from d06lmsgate-6.emea.ibm.com ([9.149.37.213]) by d06as04.portsmouth.uk.ibm.com with Microsoft SMTPSVC(5.0.2195.5329); Tue, 19 Aug 2003 04:08:14 +0100 Received: from ussvs-mta02.amer.borl.net (host143186050011.borland.com [143.186.50.11]) by d06lmsgate-6.emea.ibm.com (8.12.9/8.12.8) with SMTP id h7J386s2059632 for ; Tue, 19 Aug 2003 04:08:07 +0100 Received: from USSVS-MAIL01 ([10.186.99.32]) by ussvs-mta02.amer.borl.net with Microsoft SMTPSVC(5.0.2195.6713); Mon, 18 Aug 2003 20:08:04 -0700 Received: from USCUS-MAIL01.amer.borl.net ([143.186.79.38]) by USSVS-MAIL01 with Microsoft SMTPSVC(5.0.2195.6713); Mon, 18 Aug 2003 20:08:04 -0700 X-MimeOLE: Produced By Microsoft Exchange V6.0.6375.0 content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Subject: FW: issue 5879 Date: Mon, 18 Aug 2003 20:08:04 -0700 Message-ID: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: issue 5879 Thread-Index: AcNl1zszApm1EtJERXSMS2ggztiYXwAADVggAAnlAHA= From: "Vijaykumar Natarajan" To: "Simon Nash" X-OriginalArrivalTime: 19 Aug 2003 03:08:04.0947 (UTC) FILETIME=[1B880230:01C365FF] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by sp15en17.hursley.ibm.com id h7J38AB67508 X-Mozilla-Status2: 00000000 Simon, For some reason, my mails to java2idl-rtf bounce back. Could you forward please. I'll get it fixed in the meantime. Vijay -----Original Message----- From: Vijaykumar Natarajan Sent: Monday, August 18, 2003 4:01 PM To: 'Simon Nash' Cc: java2idl-rtf@omg.org Subject: RE: issue 5879 Simon, I believe the custom format version is to identify the format that is used to marshal a custom valuetype, a FV for each level in the hierarchy just doesn't make sense. To highlight this, consider the case of D: Does it at all make sense that the FV for B is 1 and FV for D is 2. Obviously (& I hope you agree) that makes no sense whatsoever. But this format opens up the possibility of doing so. The CFV is to identify the format of the entire state of that instance and should not be applied to a level in the hierarchy. Further, if FV is for one level in the hierarchy, what is the difference between v1 and v2, because all that follows is the DWO flag and then the state! Also, with regards to interop, do most of the appservers use this format that you describe below: --------------------------------------------- | A-Data | B-FV B-DWO B-Data | --------------------------------------------- Note that such types are quite rare. The usual case is custom at the first level: -------------------------- B-FV B-DWO B-Data | --------------------------- Which looks no different w/ either proposal. Borland, for example, does the latter format. W.r.t the DWO flag, I am open to either a or d versions ie. C is custom or C is not custom, whichever makes it more convenient to the other issues. Vijay -----Original Message----- From: Simon Nash [mailto:nash@hursley.ibm.com] Sent: Monday, August 18, 2003 3:22 PM To: Vijaykumar Natarajan Cc: java2idl-rtf@omg.org Subject: Re: issue 5879 Vijay, My apologies for not progressing the resolution of this. Your proposal seems logical but I am concerned that this would break interoperability for the much simpler case of object B in Tracy's example (a custom class extending a non-custom class). It is my belief (though I am open to correction on this) that most ORBs currently format Object B as: --------------------------------------------- | A-Data | B-FV B-DWO B-Data | --------------------------------------------- Your proposed change would require sending this as -------------------------------------------- | FV | A-Data | B-DWO B-Data | -------------------------------------------- which is an incompatible change. It has also been pointed out to me that my suggestion of making the choice between format a and d dependent on the FV value does not work when the receiver ORB needs to FVD back to the sender ORB for the definition of class C. The sender ORB may have sent C-type instances on different occasions to different receiver ORBs as FV=1 and FV=2, so when it is asked for the FVD of class C it will not be easy for it to make sure that it sets the is_custom flag in the FVD to a value consistent with the FV level of C-type data that it has previously sent to the particular receiver ORB making the FVD request. This is further complicated by the possibility that some of the IORs for a given receiver ORB could use the TAG_RMI_CUSTOM_MAX_STREAM_FORMAT TaggedComponent to change the allowable FV level for data sent on requests to that IOR, which could mean that transmission formats for C-type objects might not be consistent even between the same pair of sender and receiver ORBs. All this seems to lead to the conclusion that the best option technically is d, consistently for both FV=1 and FV=2. The downside of this is the interoperability impact that it may have on ORB vendors. Option a, while less clean, may be a more pragmatic solution for backwards interoperabilty. I would like to hear the views of RTF members on this. Would all RTF members please indicate their preference for 1. option d, consistently for both FV=1 and FV=2 2. option a 3. neither of these (please say why, or indicate what other solution you would prefer) If there is consensus support for either 1 or 2 above, I will write up a formal resolution for voting. Simon > Vijaykumar Natarajan wrote: > > Simon, > > This issue appears to not have been resolved. What's the status on > this? > > Also, it seems to me that in the cases below, the FV should not be per > level in the state as it describes how we are marshaling the entire > state. i.e I think there is an option e: > > ------------------------------------------------------------------------ -------------------------------------------------------- > d. FV | A-Data | B-DWO B-Data | C-DWO C-Data | D- DWO DData | > > ---------------------------------------------------------------------- > --------------------------------------------------------- > Vijay > > Simon Nash wrote: > > Tracy, > I have asked Juergen to raise an issue for this. This is an important > issue since it impacts interoperability. We need to clarify the spec > on this since I don't believe it's clear as it currently stands. > > I believe either a. or d. (d. is my addition, see below) are the two > approaches that would be consistent with the current spec. c. is > definitely not correct since the DWO flag needs to appear once for > each class in the hierarchy that has a writeObject method. b. is also > not quite correct since there is no reason to put the FV and DWO flags > on the non-custom class A, and doing this would require A to have > different FullValueDescriptions depending on whether it is sent as > part of a B instance or as part of an A instance (or a non-custom > subclass of A). So I believe the choices come down to: > > ------------------------------------------------------------------------ ----------------- > a. | A-Data | B-FV B-DWO B-Data | CData | FV-D DWO-D DData | > > ---------------------------------------------------------------------- > ------------------- > > ------------------------------------------------------------------------ -------------------------------------------------------- > d. | A-Data | B-FV B-DWO B-Data | C-FV C-DWO C-Data | D-FV D- DWO DData | > > ---------------------------------------------------------------------- > --------------------------------------------------------- > > Option a. seems to have all the on-the-wire information that is > needed. Specifically, it does include a separate DWO flag for each > class with a writeObject method. Also, it is likely to be the format > currently used by most ORBs, since until quite recently class C was > considered to be non-custom. (Issue 4497 changed this.) The only > problem with this approach is that in order to unmarshal object D > successfully from a FullValueDescription, it is necessary for the > is_custom flag in FVDs to be set to true if and only if the FVD's > class has a directly defined writeObject method. So in this example, > the FVDs for classes D and B must say isCustom=true, and the FVDs for > C and A must say isCustom=false, even though C is actually custom from > an IDL and chunking perspective. Effectively, the isCustom flag in > the FVD has become a "has writeObject" flag. > > This works but doesn't really seem correct, so let's consider option > d. This allows the is_custom flag in the FVD to truly mean "is > custom", which seems cleaner. The downside with this approach is that > it will not interoperate with older ORBs (pre 4497) unless some > versioning rules are applied. Perhaps we could tie this change to the > move from FV=1 to FV=2 and say that FV=1 implies the use of option a. > but FV=2 implies the use of option d. Tying these together does not > seem quite right but it may be a pragmatic solution depending on what > ORB vendors have done so far in their released products. > > Could RTF members please state a preference between options a. and d. > as described above, or make other proposals. Since this affects > interoperability, I think it should be treated as an urgent issue. If > other RTF members agree with this then I will ask Andrew to classify > it as urgent. -- Simon C Nash IBM Distinguished Engineer Hursley Park, Winchester, UK nash@hursley.ibm.com Tel. +44-1962-815156 Fax +44-1962-818999 ubject: RE: FW: [Fwd: FW: issue 5879] Date: Wed, 20 Aug 2003 13:08:22 -0700 X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: FW: [Fwd: FW: issue 5879] Thread-Index: AcNnVCIrjMqqowz2SIK5eE8MuywtiQAAp/MA From: "Vijaykumar Natarajan" To: "Juergen Boldt" X-OriginalArrivalTime: 20 Aug 2003 20:08:22.0048 (UTC) FILETIME=[CE2E1200:01C36756] Hey, Good to hear from you too! Don't bother about forwarding, I am having others do it already. Thanks, Vijay -----Original Message----- From: Juergen Boldt [mailto:juergen@omg.org] Sent: Wednesday, August 20, 2003 1:02 PM To: Vijaykumar Natarajan Subject: Re: FW: [Fwd: FW: issue 5879] Hi Vijay, didn't hear from you in a long time !! Looks like your email has changed and the anti-spam box that protects the OMG mailing list doesn't show much merci. I'll change your email information in the db and you should be all set by tomorrow morning. In the meantime, want me to forward mail to the list>? -Juergen At 12:18 PM 8/20/03 -0700, you wrote: Juergen, My emails to this rtf are bouncing. Is there a filter or something? Vijay -----Original Message----- From: System Administrator Sent: Wednesday, August 20, 2003 12:14 PM To: java2idl-rtf@omg.org Subject: Undeliverable: RE: [Fwd: FW: issue 5879] Your message did not reach some or all of the intended recipients. Subject: RE: [Fwd: FW: issue 5879] Sent: 8/20/2003 12:14 PM The following recipient(s) could not be reached: java2idl-rtf@omg.org on 8/20/2003 12:14 PM There was a SMTP communication problem with the recipient's email server. Please contact your system administrator. unknown> ================================= Jürgen Boldt Director, Member Services Object Management Group 250 First Avenue, Suite 100 Needham, MA 02494 Tel. +1 781 444 0404 ext. 132 Fax: +1 781 444 0320 email: juergen@omg.org www www.omg.org ================================ Date: Wed, 20 Aug 2003 18:29:03 -0400 (EDT) From: Bob Scheifler - SMI Software Development Reply-To: Bob Scheifler - SMI Software Development Subject: Re: [Fwd: FW: issue 5879] To: Harold.Carr@sun.com, andyp@bea.com Cc: java2idl-rtf@omg.org X-Mailer: dtmail 1.3.0 @(#)CDE Version 1.4.6_06 SunOS 5.8 sun4u sparc > Right, but what precisely does the Sun ORB do now As near as I can tell, by looking at both source and an actual binary stream (but someone else ought to verify), Sun's 1.4.2 implementation does option a with format 1. - Bob Date: Thu, 21 Aug 2003 10:48:23 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en To: Andy Piper CC: Vijaykumar Natarajan , java2idl-rtf@omg.org Subject: Re: issue 5879 Andy, Versioning the repository ID is an excellent suggestion and I believe it is the solution to the interoperability problem with option d. Suppose we adopt option d and version the hash code. Old ORBs (pre 4497 and 5879) will marshal class C as non-custom and return false for C-FVD.is_custom(). New ORBs (post 4497 and 5879) will marshal class C as custom and return true for C-FVD.is_custom(), and will use a different hash code for C's repid. This should not cause any problems with interoperability between old and new ORBs. When a new ORB receives an old C, the hash code mismatch in C's repid will trigger the FVD mechanism. The new ORB will get the old (non-custom) C-FVD, and will correctly read the non-custom value because C-FVD.is_custom() will return false. Similarly if an old ORB receives a new C, it will get the new (custom) C-FVD, and it will correctly read the custom value because C-FVD.is_custom() will return true. I believe this leaves the way clear for us to adopt option d, coupled with a change to the hash code algorithm that factors in the 4479 resolution. For this I would suggest changing step 2 in the 10.7.2 hash code algorithm from the following: 2. The value 1 if the class has no writeObject method, or the value 2 if the class has a writeObject method, written as a 32-bit integer. to the following: 2. The value 1 if the class and all its superclasses have no writeObject method, or the value 2 if the class has a writeObject method, or the value 3 if the class has no writeObject method but at least one of its superclasses has a writeObject method, written as a 32-bit integer. What do people think of this approach? If there is agreement on this solution to the subclassing problem raised by Tracy, then we will also need to agree on whether there should be a single FV for the whole valuetype or a separate FV for each class within the valuetype. This latter discussion is a broader one than the original scope of issue 5879 since it impacts every custom valuetype that extends a non-custom valuetype. Specifically, with Tracy's A and B classes, we need to decide whether a B instance should be marshalled as (i) --------------------------------------------- | A-Data | B-FV B-DWO B-Data | --------------------------------------------- or as (ii) -------------------------------------------- | FV | A-Data | B-DWO B-Data | -------------------------------------------- We clearly have an interoperability problem here, given that the IBM and Sun valuehandlers have implemented (i) and the Borland valuehandler has implemented (ii). I don't see any way for an ORB to interoperate with both formats, since the very first byte of the valuetype needs to be interpreted differently. So I think we will need to pick one of these by RTF vote and move on. Are there any other suggestions for how to solve this problem? Simon Andy Piper wrote: > > Simon, > > Option (a). The interoperability aspects of this are the most important to > us. Option (d) effectively breaks the whole reason for support FVD and > CodeBase which only really get used in an interop setting. > > My only other thought is would it be possible to alter the hash in the > repository ID to reflect version 1 or 2? That way CodeBase could cope. > Another alternative would be to version the CodeBase API somehow. > > andy > > At 11:21 PM 8/18/2003 +0100, Simon Nash wrote: > >Vijay, > >My apologies for not progressing the resolution of this. Your > >proposal seems logical but I am concerned that this would break > >interoperability for the much simpler case of object B in > >Tracy's example (a custom class extending a non-custom class). > > > >It is my belief (though I am open to correction on this) that > >most ORBs currently format Object B as: > > > >--------------------------------------------- > >| A-Data | B-FV B-DWO B-Data | > >--------------------------------------------- > > > >Your proposed change would require sending this as > > > >-------------------------------------------- > >| FV | A-Data | B-DWO B-Data | > >-------------------------------------------- > > > >which is an incompatible change. > > > >It has also been pointed out to me that my suggestion of making > >the choice between format a and d dependent on the FV value > >does not work when the receiver ORB needs to FVD back to the > >sender ORB for the definition of class C. The sender ORB may > >have sent C-type instances on different occasions to different > >receiver ORBs as FV=1 and FV=2, so when it is asked for the > >FVD of class C it will not be easy for it to make sure that it > >sets the is_custom flag in the FVD to a value consistent with > >the FV level of C-type data that it has previously sent to the > >particular receiver ORB making the FVD request. > > > >This is further complicated by the possibility that some of > >the IORs for a given receiver ORB could use the > >TAG_RMI_CUSTOM_MAX_STREAM_FORMAT TaggedComponent to change the > >allowable FV level for data sent on requests to that IOR, > >which could mean that transmission formats for C-type objects > >might not be consistent even between the same pair of sender > >and receiver ORBs. > > > >All this seems to lead to the conclusion that the best option > >technically is d, consistently for both FV=1 and FV=2. The > >downside of this is the interoperability impact that it may > >have on ORB vendors. Option a, while less clean, may be a > >more pragmatic solution for backwards interoperabilty. I would > >like to hear the views of RTF members on this. Would all > >RTF members please indicate their preference for > > 1. option d, consistently for both FV=1 and FV=2 > > 2. option a > > 3. neither of these (please say why, or indicate what other > > solution you would prefer) > > > >If there is consensus support for either 1 or 2 above, I will > >write up a formal resolution for voting. > > > > Simon > > > > > Vijaykumar Natarajan wrote: > > > > > > Simon, > > > > > > This issue appears to not have been resolved. What's the status on this? > > > > > > Also, it seems to me that in the cases below, the FV should not be per > > level in the state as it describes how we are marshaling the entire > > state. i.e I think there is an option e: > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------- > > > d. > > FV | A-Data | B-DWO B-Data | C-DWO C-Data | D- > > DWO DData | > > > > > ------------------------------------------------------------------------------------------------------------------------------- > > > Vijay > > > > > > Simon Nash wrote: > > > > > > Tracy, > > > I have asked Juergen to raise an issue for this. This is an important > > > issue since it impacts interoperability. We need to clarify the spec > > > on this since I don't believe it's clear as it currently stands. > > > > > > I believe either a. or d. (d. is my addition, see below) are the two > > > approaches that would be consistent with the current spec. c. is > > > definitely not correct since the DWO flag needs to appear once for > > > each class in the hierarchy that has a writeObject method. b. is > > > also not quite correct since there is no reason to put the FV and DWO > > > flags on the non-custom class A, and doing this would require A to > > > have different FullValueDescriptions depending on whether it is sent > > > as part of a B instance or as part of an A instance (or a non-custom > > > subclass of A). So I believe the choices come down to: > > > > > > > > ----------------------------------------------------------------------------------------- > > > > > a. | A-Data | B-FV B-DWO B-Data | CData | FV-D > > DWO-D DData | > > > > > ----------------------------------------------------------------------------------------- > > > > > > > > -------------------------------------------------------------------------------------------------------------------------------- > > > > > d. | A-Data | B-FV B-DWO B-Data | C-FV C-DWO > > C-Data | D-FV D- DWO DData | > > > > > ------------------------------------------------------------------------------------------------------------------------------- > > > > > > Option a. seems to have all the on-the-wire information that is > > > needed. Specifically, it does include a separate DWO flag for each > > > class with a writeObject method. Also, it is likely to be the format > > > currently used by most ORBs, since until quite recently class C was > > > considered to be non-custom. (Issue 4497 changed this.) The only > > > problem with this approach is that in order to unmarshal object D > > > successfully from a FullValueDescription, it is necessary for the > > > is_custom flag in FVDs to be set to true if and only if the FVD's > > > class has a directly defined writeObject method. So in this example, > > > the FVDs for classes D and B must say isCustom=true, and the FVDs for > > > C and A must say isCustom=false, even though C is actually custom > > > from an IDL and chunking perspective. Effectively, the isCustom flag > > > in the FVD has become a "has writeObject" flag. > > > > > > This works but doesn't really seem correct, so let's consider option d. > > > This allows the is_custom flag in the FVD to truly mean "is custom", > > > which seems cleaner. The downside with this approach is that it will > > > not interoperate with older ORBs (pre 4497) unless some versioning > > > rules are applied. Perhaps we could tie this change to the move from > > > FV=1 to FV=2 and say that FV=1 implies the use of option a. but FV=2 > > > implies the use of option d. Tying these together does not seem quite > > > right but it may be a pragmatic solution depending on what ORB vendors > > > have done so far in their released products. > > > > > > Could RTF members please state a preference between options a. and d. > > > as described above, or make other proposals. Since this affects > > > interoperability, I think it should be treated as an urgent issue. > > > If other RTF members agree with this then I will ask Andrew to > > > classify it as urgent. > > > >-- > >Simon C Nash IBM Distinguished Engineer > >Hursley Park, Winchester, UK nash@hursley.ibm.com > >Tel. +44-1962-815156 Fax +44-1962-818999 -- Simon C Nash IBM Distinguished Engineer Hursley Park, Winchester, UK nash@hursley.ibm.com Tel. +44-1962-815156 Fax +44-1962-818999 X-Sender: andyp:san-francisco.beasys.com@127.0.0.1 X-Mailer: QUALCOMM Windows Eudora Version 5.0 Date: Thu, 21 Aug 2003 09:49:02 -0700 To: Simon Nash From: Andy Piper Subject: Re: issue 5879 Cc: Vijaykumar Natarajan , java2idl-rtf@omg.org At 10:48 AM 8/21/2003 +0100, Simon Nash wrote: What do people think of this approach? It gets my vote. I especially like this because for uncomplicated classes the FVD mechanism would not be invoked. We clearly have an interoperability problem here, given that the IBM and Sun valuehandlers have implemented (i) and the Borland valuehandler has implemented (ii). I don't see any way for an ORB to interoperate with both formats, since the very first byte of the valuetype needs to be interpreted differently. So I think we will need to pick one of these by RTF vote and move on. Are there any other suggestions for how to solve this problem? BEA would vote for (i), this is unfortunate for Borland, although I'm guessing they would still have an interop problem with older versions of their valuehandler. andy Date: Fri, 22 Aug 2003 10:51:58 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.79 [en] (Windows NT 5.0; U) X-Accept-Language: en To: Vijaykumar Natarajan CC: Andy Piper , java2idl-rtf@omg.org Subject: Re: issue 5879 Vijay, The difference between FV=1 and FV=2 is the format of the class state data that follows the DWO flag. From section 1.4.10: For format version 1, this data is optional and if present must be written "as is". For format version 2, if optional data is present then it must be enclosed within a CDR custom valuetype with no codebase and repid "RMI:org.omg.custom." where is the fully-qualified name of the class whose writeObject method is being invoked. For format version 2, if optional data is not present then a null valuetype (0x00000000) must be written to indicate the absence of optional data. So it would be legal (for example) to encode the valuetype C as ---------------------------------------------------------- | A-Data | FV=1 B-DWO B-Data | FV=2 C-DWO C-Data | ---------------------------------------------------------- and this would mean that B-data is written "as is" but C-data is enclosed within a CDR custom valuetype. Although this would be a legal format on the wire, I am hard pressed to think of a case where an ORB would actually mix the formats in this way. Nevertheless I believe the semantics of such a mixture are clearly defined and do not impose any more burden on a receiver ORB than the current requirement of needing to be able to deal with the cases of both ------------------------------------------------------------------ | A-Data | FV=1 B-DWO B-Data-asis | FV=1 C-DWO C-Data-asis | ------------------------------------------------------------------ and ------------------------------------------------------------------ | A-Data | FV=2 B-DWO B-Data-wrapped | FV=2 C-DWO C-Data-wrapped | ------------------------------------------------------------------ I would also suggest that the extra flexibility provided by option (i) might be useful in some future evolution of the protocol. (Will there ever be a format 3?) The interoperabilty problems I am most concerned about if we were to choose option (ii) for the FV flag are not between different levels of a vendor product (where prorietary mechanisms can be used) but those involving combinations of ORB vendors, such as WebLogic, WebSphere, and Sun ONE. Choosing option (ii) would break interoperability for these valuetypes between current WebLogic servers and future WebSphere or Sun ONE servers, and between current WebSphere servers and future WebLogic or Sun ONE servers. This would be a serious problem for customers who are using these mixed environments, especially after all the work that was done in J2EE 1.3 to get these different application servers to interoperate. Interoperability between current levels of WebLogic/WebSphere/Sun ONE and Borland Application Server is apparently currently broken for these valuetypes, so this issue of breaking a currently working combination does not arise with customers who are using Borland servers in combination with WebSphere, WebLogic or Sun ONE servers. Choosing option (i) would allow Borland customers with mixed environments to move up to a new level of Borland Application Server that would be able to interoperate with older Borland levels as well as with the other vendors' servers. So for all these reasons, I think the best choice is (i). Simon Vijaykumar Natarajan wrote: > > Simon, > > Thanks for the excellent summary of these issues. > > I like the Repository ID versioning solution. Very neat, andy. > > W.r.t to the latter issue of CFV. Bottomline, interoperability is more > important. We can handle either approach, but it seems better to fix the > problem correctly rather than perpetrate a bad format. > > If we use (i) then we will (I believe) have to specify what happens when > CFV for B is 1 and CFV for D is 2. Also as I pointed out in my earlier > mail, what exactly is the difference between CFV1 and CFV2? > > Borland can easily handle the interop problem w/ our earlier versions > and so we'd like to clean this up correctly. > > However, I agree that we should just vote and move on. > > Thanks, > Vijay > > -----Original Message----- > From: Simon Nash [mailto:nash@hursley.ibm.com] > Sent: Thursday, August 21, 2003 2:48 AM > To: Andy Piper > Cc: Vijaykumar Natarajan; java2idl-rtf@omg.org > Subject: Re: issue 5879 > > Andy, > Versioning the repository ID is an excellent suggestion and > I believe it is the solution to the interoperability problem with option > d. > > Suppose we adopt option d and version the hash code. > Old ORBs (pre 4497 and 5879) will marshal class C as > non-custom and return false for C-FVD.is_custom(). New ORBs (post 4497 > and 5879) will marshal class C as custom and return true for > C-FVD.is_custom(), and will use a different hash code for C's repid. > > This should not cause any problems with interoperability between old and > new ORBs. When a new ORB receives an old C, the hash code mismatch in > C's repid will trigger the FVD mechanism. The new ORB will get the old > (non-custom) C-FVD, and will correctly read the non-custom value because > C-FVD.is_custom() will return false. Similarly if an old > ORB receives a new C, it will get the new (custom) C-FVD, > and it will correctly read the custom value because > C-FVD.is_custom() will return true. > > I believe this leaves the way clear for us to adopt option d, coupled > with a change to the hash code algorithm that factors in the 4479 > resolution. For this I would suggest changing step 2 in the 10.7.2 hash > code algorithm from the following: > > 2. The value 1 if the class has no writeObject method, or > the value 2 if the class has a writeObject method, > written as a 32-bit integer. > > to the following: > > 2. The value 1 if the class and all its superclasses have > no writeObject method, or the value 2 if the class has a > writeObject method, or the value 3 if the class has no > writeObject method but at least one of its superclasses > has a writeObject method, written as a 32-bit integer. > > What do people think of this approach? > > If there is agreement on this solution to the subclassing problem raised > by Tracy, then we will also need to agree on whether there should be a > single FV for the whole valuetype or a separate FV for each class within > the valuetype. This latter discussion is a broader one than the > original scope of issue 5879 since it impacts every custom valuetype > that extends a non-custom valuetype. > > Specifically, with Tracy's A and B classes, we need to decide whether a > B instance should be marshalled as > (i) > --------------------------------------------- > | A-Data | B-FV B-DWO B-Data | > --------------------------------------------- > > or as > (ii) > -------------------------------------------- > | FV | A-Data | B-DWO B-Data | > -------------------------------------------- > > We clearly have an interoperability problem here, given that the IBM and > Sun valuehandlers have implemented (i) and the Borland valuehandler has > implemented (ii). I don't see any way for an ORB to interoperate with > both formats, since the very first byte of the valuetype needs to be > interpreted differently. So I think we will need to pick one of these by > RTF vote and move on. Are there any other suggestions for how to solve > this problem? > > Simon > > Andy Piper wrote: > > > > Simon, > > > > Option (a). The interoperability aspects of this are the most > > important to us. Option (d) effectively breaks the whole reason for > > support FVD and CodeBase which only really get used in an interop > > setting. > > > > My only other thought is would it be possible to alter the hash in the > > > repository ID to reflect version 1 or 2? That way CodeBase could cope. > > > Another alternative would be to version the CodeBase API somehow. > > > > andy > > > > At 11:21 PM 8/18/2003 +0100, Simon Nash wrote: > > >Vijay, > > >My apologies for not progressing the resolution of this. Your > > >proposal seems logical but I am concerned that this would break > > >interoperability for the much simpler case of object B in Tracy's > > >example (a custom class extending a non-custom class). > > > > > >It is my belief (though I am open to correction on this) that most > > >ORBs currently format Object B as: > > > > > >--------------------------------------------- > > >| A-Data | B-FV B-DWO B-Data | > > >--------------------------------------------- > > > > > >Your proposed change would require sending this as > > > > > >-------------------------------------------- > > >| FV | A-Data | B-DWO B-Data | > > >-------------------------------------------- > > > > > >which is an incompatible change. > > > > > >It has also been pointed out to me that my suggestion of making the > > >choice between format a and d dependent on the FV value does not work > > > >when the receiver ORB needs to FVD back to the sender ORB for the > > >definition of class C. The sender ORB may have sent C-type instances > > > >on different occasions to different receiver ORBs as FV=1 and FV=2, > > >so when it is asked for the FVD of class C it will not be easy for it > > > >to make sure that it sets the is_custom flag in the FVD to a value > > >consistent with the FV level of C-type data that it has previously > > >sent to the particular receiver ORB making the FVD request. > > > > > >This is further complicated by the possibility that some of the IORs > > >for a given receiver ORB could use the > > >TAG_RMI_CUSTOM_MAX_STREAM_FORMAT TaggedComponent to change the > > >allowable FV level for data sent on requests to that IOR, which could > > > >mean that transmission formats for C-type objects might not be > > >consistent even between the same pair of sender and receiver ORBs. > > > > > >All this seems to lead to the conclusion that the best option > > >technically is d, consistently for both FV=1 and FV=2. The downside > > >of this is the interoperability impact that it may have on ORB > > >vendors. Option a, while less clean, may be a more pragmatic > > >solution for backwards interoperabilty. I would like to hear the > > >views of RTF members on this. Would all RTF members please indicate > > >their preference for > > > 1. option d, consistently for both FV=1 and FV=2 > > > 2. option a > > > 3. neither of these (please say why, or indicate what other > > > solution you would prefer) > > > > > >If there is consensus support for either 1 or 2 above, I will write > > >up a formal resolution for voting. > > > > > > Simon > > > > > > > Vijaykumar Natarajan wrote: > > > > > > > > Simon, > > > > > > > > This issue appears to not have been resolved. What's the status on > > > > > this? > > > > > > > > Also, it seems to me that in the cases below, the FV should not be > > > > > per > > > level in the state as it describes how we are marshaling the entire > > > state. i.e I think there is an option e: > > > > > > > > > > > -------------------------------------------------------------------- > > > ------------------------------------------------------------ > > > > d. > > > FV | A-Data | B-DWO B-Data | C-DWO C-Data | > D- > > > DWO DData | > > > > > > > -------------------------------------------------------------------- > > > ----------------------------------------------------------- > > > > Vijay > > > > > > > > Simon Nash wrote: > > > > > > > > Tracy, > > > > I have asked Juergen to raise an issue for this. This is an > > > > important issue since it impacts interoperability. We need to > > > > clarify the spec on this since I don't believe it's clear as it > > > > currently stands. > > > > > > > > I believe either a. or d. (d. is my addition, see below) are the > > > > two approaches that would be consistent with the current spec. c. > > > > > is definitely not correct since the DWO flag needs to appear once > > > > for each class in the hierarchy that has a writeObject method. b. > > > > > is also not quite correct since there is no reason to put the FV > > > > and DWO flags on the non-custom class A, and doing this would > > > > require A to have different FullValueDescriptions depending on > > > > whether it is sent as part of a B instance or as part of an A > > > > instance (or a non-custom subclass of A). So I believe the > > > > choices come down to: > > > > > > > > > > > -------------------------------------------------------------------- > > > --------------------- > > > > > > > a. | A-Data | B-FV B-DWO B-Data | CData | > FV-D > > > DWO-D DData | > > > > > > > -------------------------------------------------------------------- > > > --------------------- > > > > > > > > > > > -------------------------------------------------------------------- > > > ------------------------------------------------------------ > > > > > > > d. | A-Data | B-FV B-DWO B-Data | C-FV C-DWO > > > C-Data | D-FV D- DWO DData | > > > > > > > -------------------------------------------------------------------- > > > ----------------------------------------------------------- > > > > > > > > Option a. seems to have all the on-the-wire information that is > > > > needed. Specifically, it does include a separate DWO flag for > > > > each class with a writeObject method. Also, it is likely to be > > > > the format currently used by most ORBs, since until quite recently > > > > > class C was considered to be non-custom. (Issue 4497 changed > > > > this.) The only problem with this approach is that in order to > > > > unmarshal object D successfully from a FullValueDescription, it is > > > > > necessary for the is_custom flag in FVDs to be set to true if and > > > > only if the FVD's class has a directly defined writeObject method. > > > > > So in this example, the FVDs for classes D and B must say > > > > isCustom=true, and the FVDs for C and A must say isCustom=false, > > > > even though C is actually custom from an IDL and chunking > > > > perspective. Effectively, the isCustom flag in the FVD has become > > > > > a "has writeObject" flag. > > > > > > > > This works but doesn't really seem correct, so let's consider > > > > option d. This allows the is_custom flag in the FVD to truly mean > > > > "is custom", which seems cleaner. The downside with this approach > > > > > is that it will not interoperate with older ORBs (pre 4497) unless > > > > > some versioning rules are applied. Perhaps we could tie this > > > > change to the move from FV=1 to FV=2 and say that FV=1 implies the > > > > > use of option a. but FV=2 implies the use of option d. Tying > > > > these together does not seem quite right but it may be a pragmatic > > > > > solution depending on what ORB vendors have done so far in their > > > > released products. > > > > > > > > Could RTF members please state a preference between options a. and > > > > > d. as described above, or make other proposals. Since this > > > > affects interoperability, I think it should be treated as an > > > > urgent issue. If other RTF members agree with this then I will ask > > > > > Andrew to classify it as urgent. > > > > > >-- > > >Simon C Nash IBM Distinguished Engineer > > >Hursley Park, Winchester, UK nash@hursley.ibm.com > > >Tel. +44-1962-815156 Fax +44-1962-818999 > > -- > Simon C Nash IBM Distinguished Engineer > Hursley Park, Winchester, UK nash@hursley.ibm.com > Tel. +44-1962-815156 Fax +44-1962-818999 -- Simon C Nash IBM Distinguished Engineer Hursley Park, Winchester, UK nash@hursley.ibm.com Tel. +44-1962-815156 Fax +44-1962-818999