Issue 5885: issue with section 2.4.2 retrieveById (lwlog-ftf) Source: PrismTech (Mr. Vincent J. Kovarik, Jr., Ph.D., gs(at)prismtech.com) Nature: Uncategorized Issue Severity: Summary: We have implemented the above service as part of our SCA Core Framework implementation. However, we have an issue with section 2.4.2 retrieveById of the document. This section states (we have numbered the sentences in question): S1: "The log will update howMany to indicate the number of records returned and will set currentId to the id of the record following the last retrieved record. S2: "If there are no further records available, currentId will be set to zero." S3: "If the record specified by the currentId does not exist, or if the Log is empty, the retrieveById operation returns an empty list of LogRecords, and sets both, currentId and howMany to zero." This behavior can create a problem as follows. 1) A log is operational with 10 records, Id 1 through 10. 2) An application issues a retrievById request for 5 logs starting at ID 6. 3) The log retrieves records 6 through 10, updates the howMany to 5, and sets the currentId to 11, per S1 above NOTE: At this point it is unclear if S2 applies. Certainly, at this point in time, no more records are available but setting currentId to zero is in conflict with S1 and doesn't appear to make sense. The following steps assume that S1 is followed and S2 is ignored. 4) The log is still operational but has not written any additional records when the same application issues a retrievById for 10 records starting at record id 11 (what it believes to be the next record). 5) The currentId of 11 does not exist in the log, so S3 applies. Consequently, the log returns an empty set of LogRecords and sets both howMany and currentId to zero. 6) The application now believes the currentId in the log has been reset to zero. 7) The log writes records 11 through 14. At this point the application has lost a valid reference into the log to obtain the next record by Id. We would like to recommend the following changes to the retrieveById . S1: No Change S2: Strike from the paragraph. S3: Strike as worded and add the following conditions: S3.A) If the record specified by currentId does not exist AND the currentId provided by the call is greater than the Id of the last record written by the log, the retrieveById operation returns an empty list of LogRecords, sets howMany to zero, and leaves the currentId unchanged. S3.B) If the record specified by currentId does not exist AND the currentId provided by the call is zero OR less than the first Id of the log, the retrieveById operation returns an list of LogRecords starting with the first available records, sets howMany to the number of records retrieved, and sets the currentId to one past the last record retrieved. S3.C) If the record specified by currentId does not exist and the Log is empty, the retrieveById operation returns an empty list of LogRecords, sets hoMany to zero, and sets the curentId to zero. Change S3.A allows a client to request log records and leave the currentId pointing the next log record in the eent that at the time of the retrievById operation, there were no log records available but the log is operational and just had not written any additional records since the applications last request. Change S3.B addresses the scenarios where the client may have a Id reference that has been written and cleared since it's last retrieveId request and the first available log Id is greater than the Id requested. It also provides the mechanism for allowing a client to issue a retrieveById without knowing the current state of the log and obtain the first set of available log records. Change S3.C identifies the empty log condition to the application as opposed to simply no further records available at the time of the call. Resolution: Revised Text: Actions taken: March 19, 2003: received issue Discussion: End of Annotations:===== From: "Kovarik, Vincent" To: "'issues@omg.org'" Cc: "'lwlog-ftf@omg.org'" Subject: FW: Issue with Lightweight Logging Service Date: Wed, 19 Mar 2003 10:00:36 -0500 X-Mailer: Internet Mail Service (5.5.2653.19) Lightweight Log FTF, We have implemented the above service as part of our SCA Core Framework implementation. However, we have an issue with section 2.4.2 retrieveById of the document. This section states (we have numbered the sentences in question): S1: "The log will update howMany to indicate the number of records returned and will set currentId to the id of the record following the last retrieved record. S2: "If there are no further records available, currentId will be set to zero." S3: "If the record specified by the currentId does not exist, or if the Log is empty, the retrieveById operation returns an empty list of LogRecords, and sets both, currentId and howMany to zero." This behavior can create a problem as follows. 1) A log is operational with 10 records, Id 1 through 10. 2) An application issues a retrievById request for 5 logs starting at ID 6. 3) The log retrieves records 6 through 10, updates the howMany to 5, and sets the currentId to 11, per S1 above NOTE: At this point it is unclear if S2 applies. Certainly, at this point in time, no more records are available but setting currentId to zero is in conflict with S1 and doesn't appear to make sense. The following steps assume that S1 is followed and S2 is ignored. 4) The log is still operational but has not written any additional records when the same application issues a retrievById for 10 records starting at record id 11 (what it believes to be the next record). 5) The currentId of 11 does not exist in the log, so S3 applies. Consequently, the log returns an empty set of LogRecords and sets both howMany and currentId to zero. 6) The application now believes the currentId in the log has been reset to zero. 7) The log writes records 11 through 14. At this point the application has lost a valid reference into the log to obtain the next record by Id. We would like to recommend the following changes to the retrieveById . S1: No Change S2: Strike from the paragraph. S3: Strike as worded and add the following conditions: S3.A) If the record specified by currentId does not exist AND the currentId provided by the call is greater than the Id of the last record written by the log, the retrieveById operation returns an empty list of LogRecords, sets howMany to zero, and leaves the currentId unchanged. S3.B) If the record specified by currentId does not exist AND the currentId provided by the call is zero OR less than the first Id of the log, the retrieveById operation returns an list of LogRecords starting with the first available records, sets howMany to the number of records retrieved, and sets the currentId to one past the last record retrieved. S3.C) If the record specified by currentId does not exist and the Log is empty, the retrieveById operation returns an empty list of LogRecords, sets hoMany to zero, and sets the curentId to zero. Change S3.A allows a client to request log records and leave the currentId pointing the next log record in the eent that at the time of the retrievById operation, there were no log records available but the log is operational and just had not written any additional records since the applications last request. Change S3.B addresses the scenarios where the client may have a Id reference that has been written and cleared since it's last retrieveId request and the first available log Id is greater than the Id requested. It also provides the mechanism for allowing a client to issue a retrieveById without knowing the current state of the log and obtain the first set of available log records. Change S3.C identifies the empty log condition to the application as opposed to simply no further records available at the time of the call. Thanks for considering our request. Vince Kovarik, Eric Held, Lance Starr --------------------------------------- Vincent J. Kovarik, Jr., Ph.D. Harris Corporation PO Box 37 Melbourne, FL 32902 321-984-5631 From: "Olynick, Jeffrey L." To: "'Kovarik, Vincent'" Cc: "'lwlog-ftf@omg.org'" , "'issues@omg.org'" Subject: RE: Issue with Lightweight Logging Service Date: Thu, 20 Mar 2003 11:00:19 -0500 X-Mailer: Internet Mail Service (5.5.2653.19) Thanks for reviewing the specification and providing input. Please find some comments below. The analysis seems to assume that the record ids are consecutive and monotonically increasing, which is not required. The ids could be, for example, pointers in a linked list. In fact I believe that there are some implementations already that do not use consecutive and monotonically increasing record ids. Therefore the concept of a currentId being "greater than" the Id of the last record written or "less than" the first ID in the log has no defined meaning. I was one of the architects of the original Log Service, and I can tell you that the original idea was that when the application got a return value for currentId of zero, for whatever reason, the application would call getRecordIdFromTime to get started again with another record id. To avoid the problems described in the example given below, the time that is provided in the call would be the time of the last record received, which in the example given would be the Log Time that was retrieved with record 10. This always allows the application to get the next record after a given time, and has the same effect as leaving the currentId unchanged in the case that the record doesn't exist. In the example, the application would try to get id 11 from the log, would get a currentId return of zero, would then call getRecordIdFromTime giving the Log Time of record 10 (previously retrieved), and get a return value of 11 (whether a record has been written to id 11 or not). I would not object to changing the operation of the retrieveById to keep the currentId the same (instead of setting it to zero) if the currentId provided in the call to retrieveById is the id of the next record to be written (in fact, I think that this is a good idea). That would enable the application to avoid calling getRecordIdFromTime in the specific example given. That would be in place of the "change S3A" suggested below. More consideration would have to be given to this in order to make sure it does not cause some problem that I don't see at the moment. I agree that S2 is ambiguous. If we decide to change the operation of retrieveById to keep the currentID the same if it is the id of the next record to be written, it makes it unnecessary anyway. For "change S3B", since the ids are not required to be consecutive or monotonically increasing, it is not possible to tell by looking at an id by itself that does not exist in the log whether it was meant to refer to an id that is earlier chronologically than the first record in the log. The application can always find the id of the first available set of records in the log (or the id of the first record yet to be written, if the log is empty) by first calling getRecordIdFromTime with a time of zero. So, even though I do not believe that the spec is broken, if we take the spirit of the suggested change, I believe that the description could be rewritten as follows: S1: The log will update howMany to indicate the number of records returned and will set currentId to the id of the record following the last retrieved record. [This is the same as in the spec now.] S2: [Replace with the following.] If the record specified by currentId does not exist, but corresponds to the next record that will be recorded in the future, the retrieveById operation returns an empty list of LogRecords, sets howMany to zero, and leaves the value of currentId unchanged. S3: If the record specified by currentId does not exist, and does not correspond to the next record that will be recorded in the future, or if the Log is empty, the retrieveById operation returns an empty list of LogRecords, and sets both currentId and howMany to zero. [This is the same as in the spec now, but excludes the case where the currentId corresponds to the next record that will be recorded in the future.] To distinguish an empty log from other conditions, the application, after getting a return value of currentId of zero from retrieveById, could then call getRecordIdFromTime with a Log Time of zero, then call retrieveById again with the value returned by getRecordIdFromTime. If this second call to retrieveById returns with howMany set to zero, the log is empty. Could you and your team please comment on this suggestion? Thanks. - Jeff O BAE SYSTEMS Jeffrey L. Olynick Technical Staff, System Products & Processes Communication, Navigation, Identification, and Reconnaissance Mail Stop 18A41 150 Parish Drive Wayne, NJ 07474-0932 Telephone (973) 305-2030 E-mail jeffrey.olynick@baesystems.com -----Original Message----- From: Kovarik, Vincent [mailto:vkovarik@harris.com] Sent: Wednesday, March 19, 2003 10:01 AM To: 'issues@omg.org' Cc: 'lwlog-ftf@omg.org' Subject: FW: Issue with Lightweight Logging Service Lightweight Log FTF, We have implemented the above service as part of our SCA Core Framework implementation. However, we have an issue with section 2.4.2 retrieveById of the document. This section states (we have numbered the sentences in question): S1: "The log will update howMany to indicate the number of records returned and will set currentId to the id of the record following the last retrieved record. S2: "If there are no further records available, currentId will be set to zero." S3: "If the record specified by the currentId does not exist, or if the Log is empty, the retrieveById operation returns an empty list of LogRecords, and sets both, currentId and howMany to zero." This behavior can create a problem as follows. 1) A log is operational with 10 records, Id 1 through 10. 2) An application issues a retrievById request for 5 logs starting at ID 6. 3) The log retrieves records 6 through 10, updates the howMany to 5, and sets the currentId to 11, per S1 above NOTE: At this point it is unclear if S2 applies. Certainly, at this point in time, no more records are available but setting currentId to zero is in conflict with S1 and doesn't appear to make sense. The following steps assume that S1 is followed and S2 is ignored. 4) The log is still operational but has not written any additional records when the same application issues a retrievById for 10 records starting at record id 11 (what it believes to be the next record). 5) The currentId of 11 does not exist in the log, so S3 applies. Consequently, the log returns an empty set of LogRecords and sets both howMany and currentId to zero. 6) The application now believes the currentId in the log has been reset to zero. 7) The log writes records 11 through 14. At this point the application has lost a valid reference into the log to obtain the next record by Id. We would like to recommend the following changes to the retrieveById . S1: No Change S2: Strike from the paragraph. S3: Strike as worded and add the following conditions: S3.A) If the record specified by currentId does not exist AND the currentId provided by the call is greater than the Id of the last record written by the log, the retrieveById operation returns an empty list of LogRecords, sets howMany to zero, and leaves the currentId unchanged. S3.B) If the record specified by currentId does not exist AND the currentId provided by the call is zero OR less than the first Id of the log, the retrieveById operation returns an list of LogRecords starting with the first available records, sets howMany to the number of records retrieved, and sets the currentId to one past the last record retrieved. S3.C) If the record specified by currentId does not exist and the Log is empty, the retrieveById operation returns an empty list of LogRecords, sets hoMany to zero, and sets the curentId to zero. Change S3.A allows a client to request log records and leave the currentId pointing the next log record in the eent that at the time of the retrievById operation, there were no log records available but the log is operational and just had not written any additional records since the applications last request. Change S3.B addresses the scenarios where the client may have a Id reference that has been written and cleared since it's last retrieveId request and the first available log Id is greater than the Id requested. It also provides the mechanism for allowing a client to issue a retrieveById without knowing the current state of the log and obtain the first set of available log records. Change S3.C identifies the empty log condition to the application as opposed to simply no further records available at the time of the call. Thanks for considering our request. Vince Kovarik, Eric Held, Lance Starr --------------------------------------- Vincent J. Kovarik, Jr., Ph.D. Harris Corporation PO Box 37 Melbourne, FL 32902 321-984-5631 Issue 5885: issue with section 2.4.2 retrieveById (lwlog-ftf) Click here for this issue's archive. Source: Harris (Mr. Vincent J. Kovarik, Jr., Ph.D., vkovarik@harris.com) Nature: Uncategorized Issue Severity: Summary: We have implemented the above service as part of our SCA Core Framework implementation. However, we have an issue with section 2.4.2 retrieveById of the document. This section states (we have numbered the sentences in question): S1: "The log will update howMany to indicate the number of records returned and will set currentId to the id of the record following the last retrieved record. S2: "If there are no further records available, currentId will be set to zero." S3: "If the record specified by the currentId does not exist, or if the Log is empty, the retrieveById operation returns an empty list of LogRecords, and sets both, currentId and howMany to zero." This behavior can create a problem as follows. 1) A log is operational with 10 records, Id 1 through 10. 2) An application issues a retrievById request for 5 logs starting at ID 6. 3) The log retrieves records 6 through 10, updates the howMany to 5, and sets the currentId to 11, per S1 above NOTE: At this point it is unclear if S2 applies. Certainly, at this point in time, no more records are available but setting currentId to zero is in conflict with S1 and doesn't appear to make sense. The following steps assume that S1 is followed and S2 is ignored. 4) The log is still operational but has not written any additional records when the same application issues a retrievById for 10 records starting at record id 11 (what it believes to be the next record). 5) The currentId of 11 does not exist in the log, so S3 applies. Consequently, the log returns an empty set of LogRecords and sets both howMany and currentId to zero. 6) The application now believes the currentId in the log has been reset to zero. 7) The log writes records 11 through 14. At this point the application has lost a valid reference into the log to obtain the next record by Id. We would like to recommend the following changes to the retrieveById . S1: No Change S2: Strike from the paragraph. S3: Strike as worded and add the following conditions: S3.A) If the record specified by currentId does not exist AND the currentId provided by the call is greater than the Id of the last record written by the log, the retrieveById operation returns an empty list of LogRecords, sets howMany to zero, and leaves the currentId unchanged. S3.B) If the record specified by currentId does not exist AND the currentId provided by the call is zero OR less than the first Id of the log, the retrieveById operation returns an list of LogRecords starting with the first available records, sets howMany to the number of records retrieved, and sets the currentId to one past the last record retrieved. S3.C) If the record specified by currentId does not exist and the Log is empty, the retrieveById operation returns an empty list of LogRecords, sets hoMany to zero, and sets the curentId to zero. Change S3.A allows a client to request log records and leave the currentId pointing the next log record in the eent that at the time of the retrievById operation, there were no log records available but the log is operational and just had not written any additional records since the applications last request. Change S3.B addresses the scenarios where the client may have a Id reference that has been written and cleared since it's last retrieveId request and the first available log Id is greater than the Id requested. It also provides the mechanism for allowing a client to issue a retrieveById without knowing the current state of the log and obtain the first set of available log records. Change S3.C identifies the empty log condition to the application as opposed to simply no further records available at the time of the call. Resolution: Revised Text: Actions taken: March 19, 2003: received issue Proposed Resolution - S2: [Replace with the following.] If the record specified by currentId does not exist, but corresponds to the next record that will be recorded in the future, the retrieveById operation returns an empty list of LogRecords, sets howMany to zero, and leaves the value of currentId unchanged. S3: If the record specified by currentId does not exist and does not correspond to the next record that will be recorded in the future, or if the Log is empty, the retrieveById operation returns an empty list of LogRecords, and sets both currentId and howMany to zero. [This is the same as in the spec now, but excludes the case where the currentId corresponds to the next record that will be recorded in the future.] Discussion (From Jeff O) Thanks for reviewing the specification and providing input. Please find some comments below. The analysis seems to assume that the record ids are consecutive and monotonically increasing, which is not required. The ids could be, for example, pointers in a linked list. In fact I believe that there are some implementations already that do not use consecutive and monotonically increasing record ids. Therefore the concept of a currentId being "greater than" the Id of the last record written or "less than" the first ID in the log has no defined meaning. I was one of the architects of the original Log Service, and I can tell you that the original idea was that when the application got a return value for currentId of zero, for whatever reason, the application would call getRecordIdFromTime to get started again with another record id. To avoid the problems described in the example given below, the time that is provided in the call would be the time of the last record received, which in the example given would be the Log Time that was retrieved with record 10. This always allows the application to get the next record after a given time, and has the same effect as leaving the currentId unchanged in the case that the record doesn't exist. In the example, the application would try to get id 11 from the log, would get a currentId return of zero, would then call getRecordIdFromTime giving the Log Time of record 10 (previously retrieved), and get a return value of 11 (whether a record has been written to id 11 or not). I would not object to changing the operation of the retrieveById to keep the currentId the same (instead of setting it to zero) if the currentId provided in the call to retrieveById is the id of the next record to be written (in fact, I think that this is a good idea). That would enable the application to avoid calling getRecordIdFromTime in the specific example given. That would be in place of the "change S3A" suggested below. More consideration would have to be given to this in order to make sure it does not cause some problem that I don't see at the moment. I agree that S2 is ambiguous. If we decide to change the operation of retrieveById to keep the currentID the same if it is the id of the next record to be written, it makes it unnecessary anyway. For "change S3B", since the ids are not required to be consecutive or monotonically increasing, it is not possible to tell by looking at an id by itself that does not exist in the log whether it was meant to refer to an id that is earlier chronologically than the first record in the log. The application can always find the id of the first available set of records in the log (or the id of the first record yet to be written, if the log is empty) by first calling getRecordIdFromTime with a time of zero. So, even though I do not believe that the spec is broken, if we take the spirit of the suggested change, I believe that the description could be rewritten as follows: S1: The log will update howMany to indicate the number of records returned and will set currentId to the id of the record following the last retrieved record. [This is the same as in the spec now.] S2: [Replace with the following.] If the record specified by currentId does not exist, but corresponds to the next record that will be recorded in the future, the retrieveById operation returns an empty list of LogRecords, sets howMany to zero, and leaves the value of currentId unchanged. S3: If the record specified by currentId does not exist, and does not correspond to the next record that will be recorded in the future, or if the Log is empty, the retrieveById operation returns an empty list of LogRecords, and sets both currentId and howMany to zero. [This is the same as in the spec now, but excludes the case where the currentId corresponds to the next record that will be recorded in the future.] To distinguish an empty log from other conditions, the application, after getting a return value of currentId of zero from retrieveById, could then call getRecordIdFromTime with a Log Time of zero, then call retrieveById again with the value returned by getRecordIdFromTime. If this second call to retrieveById returns with howMany set to zero, the log is empty. Could you and your team please comment on this suggestion? Thanks. - Jeff O Issue 5885 ---------- I support Jeff O's suggestion. Although I'm worried about the "magicness" of a zero return value. So I'd like to amend his suggestion by throwing an InvalidParam exception in the S3 case. Note that the S3 case can never occur due to S1, so it indicates an error on behalf of the client. Issue 5888 - Agreed Issue 5889 - Agreed (in the tables, you can just remove the "void" rows) Issue 5890 - Agreed X-Sender: giddiv@postel X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Tue, 22 Apr 2003 11:02:23 -0400 To: Jeff Smith From: Victor Giddings Subject: Issue 5885 discusion question Cc: lwlog-ftf@omg.org In the discussion of 5885, you said: This always allows the application to get the next record after a given time, and has the same effect as leaving the currentId unchanged in the case that the record doesn't exist. In the example, the application would try to get id 11 from the log, would get a currentId return of zero, would then call getRecordIdFromTime giving the Log Time of record 10 (previously retrieved), and get a return value of 11 (whether a record has been written to id 11 or not). I think this is true only record 11 had a differernt (later) timestamp than record 10. Otherwise, if it had the same timestamp then it (and possibly others) would have been skipped by the getRecordIdFromTime call. I realize that the timestamp has nanosecond precision, so the probability might seem negligible that record 11 would be assigned the same time stamp. But there is no explicit requirement that the implementation actually have a nanosecond clock available, and such a requirement would be contrary to the "resource constrained" environment cited as the target. Remember, for example, that the only portable clock supported by vxWorks is derived from the 60Hz or 50Hz power supply. So I think I disagree with your contention that the "spec is not broken", and I think the proposed change is necessary. Victor Giddings mailto:victor.giddings@ois.com Senior Product Engineer +1 703 295 6500 Objective Interface Systems Fax: +1 703 295 6501 Subject: RE: LW Log Issue Resolution Vote Date: Tue, 22 Apr 2003 12:15:13 -0400 Thread-Topic: LW Log Issue Resolution Vote Thread-Index: AcMID9RPLMorw3P1EdeMjACgyTbmFwA1pHfA From: "Pilhofer, Frank" To: Cc: X-MIME-Autoconverted: from quoted-printable to 8bit by amethyst.omg.org id h3MGFgsd018690 > > Enclosed is a list of the LW Log issues with proposed resolutions in > bold. Please respond with your vote on the resolutions by Tues. 29 > April. I've received a few new items that I may post on the > issues list > this week and there was also a question regarding the writeRecords > operation that inadvertently was left off the list. > Hi, here's my suggestions: Issue 5767 ---------- I concur with Virginie that a client should not need to download the full log if it is interested in some detail only, e.g. if it wants to find out whether a FAILURE_ALARM occured somewhere. The Log server already has do "parsing" as part as get_record_id_ from_time; I propose to add functionality comparative to this ope- ration, i.e. RecordId get_record_id_with_level (in RecordId currentId, in LogLevel level); RecordId get_record_id_with_producer_id (in RecordId currentId, in string producerId); RecordId get_record_id_with_producer_name (in RecordId currentId, in string producerName); The currentId parameter would give the RecordId where the server starts looking (inclusive). The operations would return the Record Id of the first record with a log that satisfies the condition. The client can then use retrieve_by_id to retrieve the record. This way, the implementation of retrieve_by_id can continue to be as efficient as before. Issue 5884 ---------- Agreed. While you're at it, don't forget to adjust the name of LogProducerRecordSequence. Issue 5885 ---------- I support Jeff O's suggestion. Although I'm worried about the "magicness" of a zero return value. So I'd like to amend his suggestion by throwing an InvalidParam exception in the S3 case. Note that the S3 case can never occur due to S1, so it indicates an error on behalf of the client. Issue 5888 - Agreed Issue 5889 - Agreed (in the tables, you can just remove the "void" rows) Issue 5890 - Agreed Subject: RE: LW Log Issue Resolution Vote Date: Fri, 25 Apr 2003 16:54:23 -0400 Thread-Topic: LW Log Issue Resolution Vote Thread-Index: AcMLXvQzi7/rS3dEEdeMjACgyTbmFwADUBqA From: "Pilhofer, Frank" To: Cc: "Kevin Richardson" X-MIME-Autoconverted: from quoted-printable to 8bit by amethyst.omg.org id h3PKtFuO017207 > > Issue 5885: No (see my proposal in Tuesday's mail) > Kevin, can you please change my vote on 5885 to YES. My mistake. I did not realize that the proposed resolution was based on Jeff's suggestion rather than on Vincent's original proposal. I would prefer an exception in the S3 case, but overall I agree to get it fixed first, and then maybe get it right some other time. Frank