Issue 15772: Merging distributed semantical descriptions of elements (uml-testing-profile-rtf) Source: Fraunhofer FOKUS (Mr. Marc-Florian Wendland, marc-florian.wendland(at)fokus.fraunhofer.de) Nature: Revision Severity: Minor Summary: The subsection 6.3.1 ­ 6.3.4 define the profiles concepts, semantics and descriptions. Those descriptions are the main section of the whole specification, though it must be ideally free from ambiguities, redundancies and impreciseness. Before introducing the profile’s elements, an introduction section is given. Unfortunately, those introductions have been used to describe also detailed descriptions of an element’s objective and semantic. Even if an introduction into the following problem space is needful and helpful for the reader, the way how it was done in UTP is confusing, since the semantical descriptions are scattered over several subsections and sometimes even redundant. This affects the following elements and their descriptions: - 6.3.1: Arbiter, Scheduler, SUT - 6.3.2: Verdict, Default, FinishAction, LogAction, determAlt, TestLog - 6.3.3: Data Pool, Data Partition, Data Selector, Coding Rules - 6.3.4: Timer, TimeOut, Timezone In order to keep the specification clean and easy to read and maintain, it should be precisely identified, what parts of the description should rather be positioned in the element’s subsections and what could remain in the introduction. Resolution: The resolution of this issue will be done by a complete editorial review of the profile section, where the introductory sections of UTP concepts will be merged with the stereotype sections. Complete redundant information will be removed. Furthermore, the stereotype sections will be checked for redundancy as well. If necessary, the redundant information will be either merged or removed Revised Text: Remove Text Remove the following text of section 6.1 (notice, numbering already removed by resolution of issue 15770) to avoid redundancy. All information are present in section 1: This section provides an overview and introduction to the UML Testing Profile. The UML Testing Profile defines a language for designing, visualizing, specifying, analyzing, constructing, and documenting the artifacts of test systems. It is a test modeling language that can be used with all major object and component technologies and applied to testing systems in various application domains. The UML Testing Profile can be used stand alone for the handling of test artifacts or in an integrated manner with UML for a handling of system and test artifacts together. The UML Testing Profile is defined by using the metamodeling approach of UML. It has been architected with the following design principles in mind: • UML integration: as a real UML profile, the UML Testing Profile is defined on the basis of the metamodel provided in the UML superstructure volume and follows the principles of UML profiles as defined in the UML infrastructure volume of UML 2.0. • Reuse and minimality: wherever possible, the UML Testing Profile makes direct use of the UML concepts and extends them and adds new concepts only where needed. Only those concepts are extended/added to UML, which have been demonstrated in the software, hardware, and protocol testing area to be of central relevance to the definition of test artifacts and are not part of UML. Remove Text Remove section title 6.2 “Structure of the UML Testing Profile” (numbering already removed by resolution of issue 15770) and merge its contents with Overview subsection Remove Text Remove introductory subsection Arbiter of section 6.3.1 (section already re-numbered to 7.1 by resolution of issue 15770) to avoid redundancy Update Text Change subsection Semantics of stereotype section Arbiter from The verdict setting semantics is defined by the classifier realizing the arbiter interface. One example of how to implement the setVerdict operation is the following: • If a verdict is pass, it can be changed to inconclusive, fail, or error only. • If a verdict is inconclusive, it can be changed to fail or error only. If a verdict is fail, it can be changed to error only. • If a verdict is error, it cannot be changed. The verdict setting semantics is defined by the classifier realizing the arbiter interface. One example of how to implement To Arbiter is a predefined interface provided with the UML Testing Profile. The purpose of an arbiter implementation is to determine the final verdict for a test case. This determination is done according to a particular arbitration strategy, which is provided in the implementation of the arbiter interface. The arbiter interface provides two operations for use in verdict setting: getVerdict and setVerdict. The setVerdict operation is used to provide updated information to the arbiter by a test component regarding the current status of the test case in which it is participating. Every validation action causes the setVerdict operation on the arbiter implementation to be invoked (see Section 7.2, “Test Behavior,” on page 17 for more information on validation actions.) A test case or a test context uses the arbiter to evaluate test results and to assign the overall verdict of a test case or test context respectively. There is a default arbitration algorithm based on functional, conformance testing, which generates Pass, Fail, Inconclusive, and Error as verdict, where the precedence of these verdicts are defined as Pass < Inconclusive < Fail < Error. The arbitration algorithm can be user-defined, if needed. Remove Text Remove introductory subsection Scheduler of section 6.3.1 (section already re-numbered to 7.1 by resolution of issue 15770) since Scheduler is no longer pat of the normative part of the profile. Remove Text Remove introductory subsection SUT of section 6.3.1 (section already re-numbered to 7.1 by resolution of issue 15770) to avoid redundancy. The content of SUT will be addressed by resolution for issue 16105. Remove Text Remove introductory subsection TestElements of section 6.3.1 (section already re-numbered to 7.1 by resolution of issue 15770) to avoid confusion of the readers. TestElement is neither an established term of testing or UTP, nor part of UTP. Update Text Shift the following parts of Description section of stereotype section TestComponent to the very beginning of the Semantics section: - A test component is commonly an active class with a set of ports and interfaces. - The classifier behavior of a test component can be used to specify low level test behavior, such as test scripts, or it can be automatically generated by deriving the behavior from all test cases in which the component takes part. Resulting text: Semantics A test component is a structured classifier participating in test behaviors. The classifier behavior of a test component can be used to specify low level test behavior, such as test scripts, or it can be automatically generated by deriving the behavior from all test cases in which the component takes part. The zone attribute is available at run-time, and GetTimezoneAction and SetTimezoneAction are used to get and set the timezone in run-time. The run-time representation of the timezone is not specified. The initial timezone of the component is specified in the model as a tagged value on the zone attribute. Remove Text Remove the introductory subsection Default from Section 6.3.2 (section already re-numbered to 7.2 by resolution of issue 15770) UpdateText Change the Description section of stereotype section Default from A default is a behavior used to handle unexpected or exceptional messages or events on one test component. To A system specification expressed in UML is not necessarily complete. To be complete in this sense means that it specifies every possible trace of execution. In particular if Interactions are used to specify the behavior, the normal situation is that the specification is partial only specifying in detail those scenarios that are of particular importance. In a testing context, however, there is a need to have complete definitions such that the number of erroneous test case executions can be kept to a minimum. The default specifications are units of specification defined in the UML Testing Profile as a means to make partial definitions of test components complete in a compact, yet flexible way. The UML Testing Profile defines mechanisms for defaults on Interactions as well as State Machines. The general idea about defaults is the following. A test behavior specification typically describes the normative or expected behaviors for the SUT. However, if during test execution an unexpected behavior is observed, then a default handler is applied. We have included default behavior definitions on several different levels. If the innermost default fail to recognize the observed behavior, the default of the next level is tried. The reason for designing with defaults rather than making sure that the main description is complete, is to separate the most common and normal situations from the more esoteric and exceptional. The distinction between the main part and the default is up to the designer and the test strategies. The UML Testing Profile has chosen to associate the default applications to static behavioral structures. In Interactions we may apply defaults to interaction fragments, in State Machines to StateMachines, States or Regions, and in Activities to Actions and Activities. Since each default in an Interaction applies only to one test component, we attach the defaults on interaction fragments to the intersection between the fragment and the test component. We said above that default behavior is invoked when the main description cannot describe the observed behavior. More precisely the default mechanism is invoked when a trigger in a State Machine or message reception in an Interaction or an action in an Activity is not defined by the main description or an explicit runtime constraint is violated. The point of default invocation will be well-defined as an event occurrence in Interactions or a State (-stack) in State Machines or an accept event action in Activities. Whenever a default behavior has executed, there is the question of where the main behavior should resume. There are several options, and the UML Testing Profile has chosen to distinguish between these different situations: • The default execution is considered an interrupt, and the main behavior should resume exactly at the trigger that led to the default interruption. This situation is called repeat. • The default execution is considered a limited disruption such that the resumption should be after the unit to which the executed default was attached. This situation is called continue. • The default execution is considered to conclude the test case execution. The test component should not return to its main description for this test case. This situation can be considered a special case of the continue-situation provided that there exists an action called «FinishAction», which ensures the completion of the test case execution locally for this test component. We acknowledge that defaults are often described in the notation that the main description is made in. If the main specification is written with Interactions, the defaults will normally be Interactions. But it is possible with hybrid descriptions where, for example, some defaults to a main description written in Interactions are in fact State Machines. Remove Text Remove introductory subsection Verdict from Section 6.3.2 (section already re-numbered to 7.2 by resolution of issue 15770) UpdateText Change the Description section of stereotype section Verdict from A verdict is a predefined enumeration specifying the set of possible evaluations of a test. Four literals are defined: pass, fail, inconclusive, error. To A verdict is a predefined enumeration specifying the set of possible evaluations of a test case. Five enumeration literals are defined: none, pass, fail, inconclusive, error. UpdateText Add an enumeration literal none into the list of verdicts in the Description section of stereotype section Verdict • none The test case has not been executed yet. Update Text Change the Semantics section of stereotype section Verdict from The Verdict has no semantics other than that of a plain enumeration. The order of the predefined literals are: error, fail, inconclusive, pass. To The verdict is a predefined enumeration datatype that contains at least the values none, fail, inconclusive, pass, error indicating how this test case execution has performed. When a test case is not executed yet or just invoked, its verdict is none. None indicates that no communication between test components and the SUT has been carried out yet. None is the weakest verdict. It is never set by the tester directly. A pass indicates that the test case is successful and that the SUT has behaved according to what should be expected. A fail on the other hand shows that the SUT is not behaving according to the specification. An inconclusive means that the test execution cannot determine whether the SUT performs well or not. An error tells that the test system itself and not the SUT fails. The precedence rules for the predefined literals are: none < pass < inconclusive < fail < error. This means whenever a test component submits a local verdict fail, the final test case verdict will never result with a pass, nor inconclusive. The final verdict of a test case is determined by an arbiter. UpdateText Change Notation section of Enumeration section Verdict from The predefined literals pass, inconclusive, fail, and error are shown as keywords (normally in bold face). To The predefined literals none, pass, inconclusive, fail, and error are shown as keywords (normally in bold face). Replace Figure Replace Figure 6.4 (notice this figure has been renumbered to Figure 7.4 by resolution of issue 15770 and was already replaced by resolution of issue 16563) with Remove Text Remove introductory sections Utilities from Section 6.3.2 (section already re-numbered to 7.2 by resolution of issue 15770) to avoid redundancy. This includes the paragraphs for FinishAction, LogAction, determAlt and TestLog. Remove Text Remove introductory section Data Pool from Section 6.3.3 (section already re-numbered to 7.3 by resolution of issue 15770) to avoid redundancy Update Text Change the Semantics section of stereotype section DataPool from The semantics of a data pool are given by the classifier, and contained data partitions, that realizes it. To Test cases are often executed repeatedly with different data values to stimulate the SUT in various ways. Also when observing data, abstract equivalence classes are used to define sets of allowable values. Typically these values are taken from data partitions, or lists of explicit values. For this purpose a data pool provides a means for associating data sets with test contexts and test cases. A data pool is a classifier containing either data partitions (equivalence classes), or explicit values; and can only be associated with either a test context or test components. Remove Text Remove introductory section Data Partition from Section 6.3.3 (section already re-numbered to 7.3 by resolution of issue 15770) to avoid redundancy Update Text Change the Semantics section of stereotype section DataPartition from The semantics of a data partition are given by the classifier that realizes it. To A data partition is used to define an equivalence class for a given type (e.g., “ValidUserNames” etc.). By denoting the partitioning of data explicitly we provide a more visible differentiation of data. A data partition is a stereotyped classifier that must be associated with a data pool. Remove Text Remove introductory section Data Selector from Section 6.3.3 (section already re-numbered to 7.3 by resolution of issue 15770) to avoid redundancy Update Text Change the Semantics section of stereotype section DataSelector from The semantics of a data selector are given by the classifier that realizes it. To To facilitate the different data selection strategies and data checking one or more data selectors can be associated with either a data pool or data partition. Typically, these data selectors are operations that operate over the contained values or value sets. Remove Text Remove introductory section Coding Rules from Section 6.3.3 (section already re-numbered to 7.3 by resolution of issue 15770) to avoid redundancy Remove Text Remove second paragraph from section 6.3.4 (section already re-numbered to 7.4 by resolution of issue 15770). Removed text: Timers are used as a means to manipulate and control test behavior, or to assure that a test case terminates properly. Timezones are used to group and synchronize test components within a distributed test system. Remove Text Remove introductory section Timer and Timeout from Section 6.3.4 (section already re-numbered to 7.4 by resolution of issue 15770) to avoid redundancy Update Text Change the Description section of stereotype section Timer from A predefined interface specifying features needed to specify a timer. A timer is owned by an active class and is started with a predefined time of expiration. A timeout is generated automatically when a timer expires and is sent to the timer owner. A timer can either be private or public. If private, the timer can only be accessed by its owned active class. If public, anyone with sufficient visibility may access and manipulate the timer. To A predefined interface specifying features needed to specify a timer. A timeout is generated automatically when a timer expires and is sent to the timer owner. Update Text Change the Semantics section of stereotype section Timer from Timers can be started, stopped, and checked by StartTimerAction, StopTimerAction, TimerRunningAction, and ReadTimerAction. A timeout is generated when the timer expires. A public timer is allowed to be started or stopped by any other active class. Start an active timer means a restart of the timer. To A timer is owned by an active class, commonly participating in test cases, and is started with a predefined time of expiration. A timer can either be private or public. If private, the time can only be accessed by its owning active class. If public, anyone with sufficient visibility may access and manipulate the timer. When a timer expires after its predefined time, a timeout is generated automatically. It is sent immediately to the active class that owns the timer. What timeout representation is used for the timeout depends on the behavioral description in that the timeout is supposed to occur. Timers can be started, stopped, and checked by a start timer action (calling start()), a stop timer action (calling stop()), a timer running action (reading property isRunning), and a read timer action (calling read()). By means of the start() operation, a timer may be started with a certain time value. The predefined time value of a timer is always positive. For example, “start Timer1(now+2.0)” means to start a timer and to stop it at latest in 2 time units, otherwise it expires. With the stop() operation, an active timer can be stopped. The expiration time of an active timer can be retrieved by the read() operation. The timer attribute isRunning is a boolean value and indicates whether the timer is still active or not. Remove Text Remove introductory section Timezone from Section 6.3.4 (section already re-numbered to 7.4 by resolution of issue 15770) to avoid redundancy Update Text Append the following text to the subsection Semantics of primitive type section Timezone: Comparing time-critical events within the same timezone is allowed. Comparing time-critical events of different timezones is a matter of semantic variation point and should be decided by the tool vendor. By default, comparison between events in two different timezones is illegal Update Text Change text of section Description of stereotype section TimeOut from A TimeOut is generated by a timer when it expires and may trigger an associated behavior (e.g., a transition in a statemachine). To A timeout event represents the timeout mechanism for usage in StateMachines. Update Text Change text of section Description of stereotype section TimeOutMessage from A TimeOutMessage is generated by a timer when it expires. The timeout message is sent to the active class that owns the timer. To A timeout message represents the timeout mechanism for usage in Interactions. Update Text Change text of section Description of stereotype section TimeOutAction from A timeout is enabled by a timer when it expires. An activity having the TimeOutAction as input condition occurs, when the timeout is enabled (and when all further input conditions are satisfied). To A timeout action represents the timeout mechanism for usage in Activities. Update Text Shift the following part from Description section to the end of Semantics section of stereotype section TimerRunningAction: Returns a boolean value, true if the timer is running, false otherwise. Actions taken: October 22, 2010: received issue October 21, 2011: closed issue Discussion: see figure on page 68 of ptc/2011-07-18 End of Annotations:===== m: webmaster@omg.org Date: 22 Oct 2010 03:31:32 -0400 To: Subject: Issue/Bug Report ******************************************************************************* Name: Marc-Florian Wendland Employer: Fraunhofer Institut FOKUS mailFrom: marc-florian.wendland@fokus.fraunhofer.de Terms_Agreement: I agree Specification: UML Testing Profile Section: 6.3 FormalNumber: formal/05-07-07 Version: 1.0 Doc_Year: 2005 Doc_Month: July Doc_Day: 07 Page: 10-36 Title: Merging distributed semantical descriptions of elements Nature: Revision Severity: Minor CODE: 3TMw8 B1: Report Issue Description: Description ########################################## The subsection 6.3.1 ­ 6.3.4 define the profiles concepts, semantics and descriptions. Those descriptions are the main section of the whole specification, though it must be ideally free from ambiguities, redundancies and impreciseness. Before introducing the profile.s elements, an introduction section is given. Unfortunately, those introductions have been used to describe also detailed descriptions of an element.s objective and semantic. Even if an introduction into the following problem space is needful and helpful for the reader, the way how it was done in UTP is confusing, since the semantical descriptions are scattered over several subsections and sometimes even redundant. This affects the following elements and their descriptions: - 6.3.1: Arbiter, Scheduler, SUT - 6.3.2: Verdict, Default, FinishAction, LogAction, determAlt, TestLog - 6.3.3: Data Pool, Data Partition, Data Selector, Coding Rules - 6.3.4: Timer, TimeOut, Timezone In order to keep the specification clean and easy to read and maintain, it should be precisely identified, what parts of the description should rather be positioned in the element.s subsections and what could remain in the introduction.