Issue 3995: Copying boxed valuetypes on local calls (java-rtf) Source: (Mr. Simon C. Nash, ) Nature: Uncategorized Issue Severity: Summary: n section 1.12.2 of the IDL to Java mapping spec, the second bullet should be clarified to state that the copying it describes applies only to non-boxed valuetypes and not to boxed valuetypes. Valuetypes need to be copied (unlike structs, which aren't copied) because the methods of a valuetype can modify its state and it is not reasonable to require the callee to know whether or not calling a valuetype method has resulted in a change to the valuetype's state. This does not apply to boxed valuetypes, which cannot have methods. Therefore, with respect to this copying, they should be treated like IDL constructed types, which are not copied. Proposed resolution: In section 1.12.2, second bullet, change "For IDL in parameters that are valuetypes" to "For IDL in parameters that are non-boxed valuetypes:" In section 1.12.2, second bullet, change "With the exception of value types" to "With the exception of non-boxed value types". In section 1.12.2, add new third bullet: • For IDL in parameters that are boxed valuetypes: • Java objects passed for boxed valuetype IDL in parameters are created and owned by the caller. The callee must not modify boxed valuetype in parameters or retain a reference to the in parameter beyond the duration of the call. Violation of these rules can result in unpredictable behavior. In section 1.21.5.1, second paragraph, change "the stub copies all valuetypes" to "the stub copies all non-boxed valuetypes". Resolution: close no change Revised Text: Actions taken: October 25, 2000: received issue April 28, 2003: closed issue Discussion: As was noted in the discussion 2 years ago on this issue, the situation is not quite as simple as the issue originally supposed. For IDL, copying comes done to the following simple idea: - If the only way that an IDL entity can be modified is by an explicit write accessor method of some sort, or the object is immutable, no copy is required. - If an IDL entity can have methods with unpredictable side effects on the state of the entity, it must be copied when marshalled. In addition, all aliasing must be preserved, both within a single value, and between different arguments to the same invocation. In situations where a boxed value type can indirectly reference a value that is passed in the same call in a different argument, the boxed value type must be copied so that it refers to the same copy of the value that is used elsewhere in the call. However, I am not sure that this is the only problem here. Simon makes reference to the idea that any type that refers to a non-boxed valuetype must be copied when passed as an in parameter, so for example a struct S containing a reference to a valuetype V must also be copied. But this directly directly contradicts the first bullet in section 1.12.2, which implies that non-valuetypes (like S) are not copied. Clearly if an instance s1 of S that refers to an instance v1 of V is created, and s1 and v1 are both passed in the same call, we would expect that sharing would be preserved. But that would require copying s1. Given this problem, which is independent of whether we are considering boxed valuetypes or not, and the fact that no one has worked on this issue for over 2 years, I would like to close this issue in the current RTF. I would rather see an issue created to deal with defects in the copying of parameters than just address the narrow boxed valuetype question. End of Annotations:===== Date: Wed, 25 Oct 2000 08:52:48 +0100 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.72 [en] (Windows NT 5.0; I) X-Accept-Language: en MIME-Version: 1.0 To: issues@omg.org CC: java-rtf@omg.org Subject: Copying boxed valuetypes on local calls Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=iso-8859-1 X-UIDL: o\O!!@,U!!aG-e9!$4e9 In section 1.12.2 of the IDL to Java mapping spec, the second bullet should be clarified to state that the copying it describes applies only to non-boxed valuetypes and not to boxed valuetypes. Valuetypes need to be copied (unlike structs, which aren't copied) because the methods of a valuetype can modify its state and it is not reasonable to require the callee to know whether or not calling a valuetype method has resulted in a change to the valuetype's state. This does not apply to boxed valuetypes, which cannot have methods. Therefore, with respect to this copying, they should be treated like IDL constructed types, which are not copied. Proposed resolution: In section 1.12.2, second bullet, change "For IDL in parameters that are valuetypes" to "For IDL in parameters that are non-boxed valuetypes:" In section 1.12.2, second bullet, change "With the exception of value types" to "With the exception of non-boxed value types". In section 1.12.2, add new third bullet: For IDL in parameters that are boxed valuetypes: Java objects passed for boxed valuetype IDL in parameters are created and owned by the caller. The callee must not modify boxed valuetype in parameters or retain a reference to the in parameter beyond the duration of the call. Violation of these rules can result in unpredictable behavior. In section 1.21.5.1, second paragraph, change "the stub copies all valuetypes" to "the stub copies all non-boxed valuetypes". Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Sender: jbiggar@corvette.floorboard.com Message-ID: <39FDE99E.B3BD3E51@floorboard.com> Date: Mon, 30 Oct 2000 13:35:26 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.76 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Simon Nash CC: orb_revision@omg.org, java-rtf@omg.org Subject: Re: Issues 3364 and 3995 References: <39F9ECF8.99231F2B@fpk.hp.com> <39FA030E.D38A598F@inprise.com> <39FC8356.99506818@hursley.ibm.com> <39FCB08A.1D8C2B4A@hursley.ibm.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: <8'e9"QE!!V\Zd9;cK!! Simon Nash wrote: > > Core RTF members might care to look at and comment on issue 3995 which I > raised in the Java RTF. It might be more appropriate for this to be > considered by the core RTF since valuetype copying semantics are also > described in section 5.2.4.3 of the core spec. To summarize the issue, > if the main reason that valuetypes are copied is that calling their methods > can unknowingly modify their state, then it should not be necessary to copy > boxed valuetypes, which don't have methods. > > On reflection, there appears to be a problem with this proposal when regular > valuetypes and boxed valuetypes are mixed on a call. If a regular valuetype > parameter V contains a boxed valuetype B, and B is also passed as a parameter, > then copying V (and its contained B) without copying the top-level B is a > problem, since the identity/sharing semantics of B would not be preserved > correctly. This doesn't matter for structs but it does matter for boxed > valuetypes. You have identified one problem with the proposal, but there is an even more serious one. If I have: // IDL valuetype V { }; struct S { V a_v; }; valuetype B struct S; If I don't copy instances of B, then I don't copy instances of V and I have broken invocation transparency. Valueboxes are really nothing but a shorthand for a valuetype with a single public writable attribute, and need to be treated the same way. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Date: Tue, 31 Oct 2000 13:42:24 +0000 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.72 [en] (Windows NT 5.0; I) X-Accept-Language: en MIME-Version: 1.0 To: Jonathan Biggar CC: orb_revision@omg.org, java-rtf@omg.org Subject: Re: Issues 3364 and 3995 References: <39F9ECF8.99231F2B@fpk.hp.com> <39FA030E.D38A598F@inprise.com> <39FC8356.99506818@hursley.ibm.com> <39FCB08A.1D8C2B4A@hursley.ibm.com> <39FDE99E.B3BD3E51@floorboard.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: Hl*!!cC$!!C;,!!R7Ud9 Jon, Jonathan Biggar wrote: > > Simon Nash wrote: > > > > Core RTF members might care to look at and comment on issue 3995 which I > > raised in the Java RTF. It might be more appropriate for this to be > > considered by the core RTF since valuetype copying semantics are also > > described in section 5.2.4.3 of the core spec. To summarize the issue, > > if the main reason that valuetypes are copied is that calling their methods > > can unknowingly modify their state, then it should not be necessary to copy > > boxed valuetypes, which don't have methods. > > > > On reflection, there appears to be a problem with this proposal when regular > > valuetypes and boxed valuetypes are mixed on a call. If a regular valuetype > > parameter V contains a boxed valuetype B, and B is also passed as a parameter, > > then copying V (and its contained B) without copying the top-level B is a > > problem, since the identity/sharing semantics of B would not be preserved > > correctly. This doesn't matter for structs but it does matter for boxed > > valuetypes. > > You have identified one problem with the proposal, but there is an even > more serious one. If I have: > > // IDL > valuetype V { }; > > struct S { > V a_v; > }; > > valuetype B struct S; > > If I don't copy instances of B, then I don't copy instances of V and I > have broken invocation transparency. > This case would be covered by the proviso that (non-boxed) valuetypes that are part of other IDL constructed types must be copied. Following the containment hierachy outwards from V requires copying of S, which in turn requires copying of B. > Valueboxes are really nothing but a shorthand for a valuetype with a > single public writable attribute, and need to be treated the same > way. > This isn't quite correct. They are limited in various other ways > (e.g., no inheritance, no value factory) and the intent of these limitations was > to permit optimized handling in the language bindings. So if we can find > a way to support remote/local transparent semantics without always > copying these, I think this would be worthwhile. So coming back to my broken scenario above, the code to handle the local case would have to detect that B was copied as part of V and therefore all other types containing B (or B itself at the top-level) also need to be copied. Even if this decision were made at compile time on a per-type basis rather than at runtime on a per-instance basis (i.e., there are two Bs being passed, and they could be the same instance so copy them just in case), this would still be far less expensive than always copying boxed valuetypes willy-nilly. I think we should clearly state the guarantees for remote/local transparency (e.g., that regular valuetypes are always copied but boxed valuetypes need only be copied when necessary to preserve correct sharing semantics) and let the ORB vendors decide how aggressive they want to be in optimizing out the copying where they can detect that it isn't needed. Simon -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Sender: jon@corvette.floorboard.com Message-ID: <39FEE6F7.F9051195@floorboard.com> Date: Tue, 31 Oct 2000 07:36:23 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.76 [en] (X11; U; SunOS 5.7 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Simon Nash CC: orb_revision@omg.org, java-rtf@omg.org Subject: Re: Issues 3364 and 3995 References: <39F9ECF8.99231F2B@fpk.hp.com> <39FA030E.D38A598F@inprise.com> <39FC8356.99506818@hursley.ibm.com> <39FCB08A.1D8C2B4A@hursley.ibm.com> <39FDE99E.B3BD3E51@floorboard.com> <39FECC40.83A2ACD3@hursley.ibm.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: A+h!!ni~!!iLOe9'kG!! Simon Nash wrote: > I think we should clearly state the guarantees for remote/local transparency > (e.g., that regular valuetypes are always copied but boxed valuetypes need > only be copied when necessary to preserve correct sharing semantics) and > let the ORB vendors decide how aggressive they want to be in optimizing > out the copying where they can detect that it isn't needed. THat's workable. Never hurts to have a "just as if" clause to allow extra possibility for optimization. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Date: Thu, 02 Nov 2000 01:07:23 +0000 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.72 [en] (Windows NT 5.0; I) X-Accept-Language: en MIME-Version: 1.0 To: Jonathan Biggar CC: orb_revision@omg.org, java-rtf@omg.org Subject: Re: Issues 3364 and 3995 References: <39F9ECF8.99231F2B@fpk.hp.com> <39FA030E.D38A598F@inprise.com> <39FC8356.99506818@hursley.ibm.com> <39FCB08A.1D8C2B4A@hursley.ibm.com> <39FDE99E.B3BD3E51@floorboard.com> <39FECC40.83A2ACD3@hursley.ibm.com> <39FEE6F7.F9051195@floorboard.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: 5PBe9>LXd9P,Oe9e-V!! Jon, Just to clarify, I'm not saying that copy semantics should be specified for boxed valuetypes and implementations can optimize out the copying. I'm saying that copy semantics should be specified for regular valuetypes but not for boxed valuetypes, that sharing semantics should be specified for both, and that implementations can optimize within these constraints. Simon Jonathan Biggar wrote: > > Simon Nash wrote: > > I think we should clearly state the guarantees for remote/local transparency > > (e.g., that regular valuetypes are always copied but boxed valuetypes need > > only be copied when necessary to preserve correct sharing semantics) and > > let the ORB vendors decide how aggressive they want to be in optimizing > > out the copying where they can detect that it isn't needed. > > THat's workable. Never hurts to have a "just as if" clause to allow > extra possibility for optimization. > > -- > Jon Biggar > Floorboard Software > jon@floorboard.com > jon@biggar.org -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Sender: jbiggar@corvette.floorboard.com Message-ID: <3A01F67E.8826C303@floorboard.com> Date: Thu, 02 Nov 2000 15:19:26 -0800 From: Jonathan Biggar X-Mailer: Mozilla 4.76 [en] (X11; U; SunOS 5.6 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: orb_revision@omg.org Subject: Re: Issues 3364 and 3995 References: <39F9ECF8.99231F2B@fpk.hp.com> <39FA030E.D38A598F@inprise.com> <39FC8356.99506818@hursley.ibm.com> <39FCB08A.1D8C2B4A@hursley.ibm.com> <39FDE99E.B3BD3E51@floorboard.com> <39FECC40.83A2ACD3@hursley.ibm.com> <39FEE6F7.F9051195@floorboard.com> <3A00BE4B.AC5B13A2@hursley.ibm.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: )EF!!>%'!!]cj!!!! Simon Nash wrote: > > Jon, > Just to clarify, I'm not saying that copy semantics should be specified for > boxed valuetypes and implementations can optimize out the copying. I'm saying > that copy semantics should be specified for regular valuetypes but not for > boxed valuetypes, that sharing semantics should be specified for both, and > that implementations can optimize within these constraints. I still don't like this much. A dumb ORB implementation can easily meet the semantic requirements by always copying a valuebox. It seems better to me to make that the baseline requirement but allow the avoidance of copying as an optimization. -- Jon Biggar Floorboard Software jon@floorboard.com jon@biggar.org Date: Fri, 03 Nov 2000 16:57:49 +0000 From: Simon Nash Organization: IBM X-Mailer: Mozilla 4.72 [en] (Windows NT 5.0; I) X-Accept-Language: en MIME-Version: 1.0 To: Jonathan Biggar CC: orb_revision@omg.org Subject: Re: Issues 3364 and 3995 References: <39F9ECF8.99231F2B@fpk.hp.com> <39FA030E.D38A598F@inprise.com> <39FC8356.99506818@hursley.ibm.com> <39FCB08A.1D8C2B4A@hursley.ibm.com> <39FDE99E.B3BD3E51@floorboard.com> <39FECC40.83A2ACD3@hursley.ibm.com> <39FEE6F7.F9051195@floorboard.com> <3A00BE4B.AC5B13A2@hursley.ibm.com> <3A01E8C0.E6A645C8@floorboard.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii X-UIDL: #eZd9m^>!!N(;!!`\Pd9 Jon, The issue is what information is needed to enable implementation of the optimization. If the semantic requirement is to support referential integrity of regular and boxed valuetypes passed on the call, then the optimization can be based on what references to regular valuetypes and boxed valuetypes appear in the GIOP stream. I don't think this is very hard to implement. However, if the semantic requirement is to make callee modification of the boxed valuetype transparent to the caller, then the optimization has to be based on whether the callee method contains any code that could make such a modification. I don't see any reasonable way to implement this. So I am proposing that the callee method can always assume referential integrity, but cannot assume that it is free to modify the contents of a boxed valuetype without such changes being seen by the caller. This latter restriction is consistent with the rules for other IDL types such as structs. Simon Jonathan Biggar wrote: > > Simon Nash wrote: > > > > Jon, > > Just to clarify, I'm not saying that copy semantics should be specified for > > boxed valuetypes and implementations can optimize out the copying. I'm saying > > that copy semantics should be specified for regular valuetypes but not for > > boxed valuetypes, that sharing semantics should be specified for both, and > > that implementations can optimize within these constraints. > > I still don't like this much. A dumb ORB implementation can easily meet > the semantic requirements by always copying a valuebox. It seems better > to me to make that the baseline requirement but allow the avoidance of > copying as an optimization. > > -- > Jon Biggar > Floorboard Software > jon@floorboard.com > jon@biggar.org -- Simon C Nash, Technology Architect, IBM Java Technology Centre Tel. +44-1962-815156 Fax +44-1962-818999 Hursley, England Internet: nash@hursley.ibm.com Lotus Notes: Simon Nash@ibmgb Issue 3995: Copying boxed valuetypes on local calls ------------------------------------------------------------------------------------------- Summary: In section 1.12.2 of the IDL to Java mapping spec, the second bullet should be clarified to state that the copying it describes applies only to non-boxed valuetypes and not to boxed valuetypes. Valuetypes need to be copied (unlike structs, which aren't copied) because the methods of a valuetype can modify its state and it is not reasonable to require the callee to know whether or not calling a valuetype method has resulted in a change to the valuetype's state. This does not apply to boxed valuetypes, which cannot have methods. Therefore, with respect to this copying, they should be treated like IDL constructed types, which are not copied. Discussion: As was noted in the discussion 2 years ago on this issue, the situation is not quite as simple as the issue originally supposed. For IDL, copying comes done to the following simple idea: - If the only way that an IDL entity can be modified is by an explicit write accessor method of some sort, or the object is immutable, no copy is required. - If an IDL entity can have methods with unpredictable side effects on the state of the entity, it must be copied when marshalled. In addition, all aliasing must be preserved, both within a single value, and between different arguments to the same invocation. In situations where a boxed value type can indirectly reference a value that is passed in the same call in a different argument, the boxed value type must be copied so that it refers to the same copy of the value that is used elsewhere in the call. However, I am not sure that this is the only problem here. Simon makes reference to the idea that any type that refers to a non-boxed valuetype must be copied when passed as an in parameter, so for example a struct S containing a reference to a valuetype V must also be copied. But this directly directly contradicts the first bullet in section 1.12.2, which implies that non-valuetypes (like S) are not copied. Clearly if an instance s1 of S that refers to an instance v1 of V is created, and s1 and v1 are both passed in the same call, we would expect that sharing would be preserved. But that would require copying s1. Given this problem, which is independent of whether we are considering boxed valuetypes or not, and the fact that no one has worked on this issue for over 2 years, I would like to close this issue in the current RTF. I would rather see an issue created to deal with defects in the copying of parameters than just address the narrow boxed valuetype question. Proposed Resolution: close no change.