Issues for Mailing list of the combined CORBA/ZIOP Revision task Force
To comment on any of these issues, send email to corba-ziop-rtf@omg.org. (Please include the issue number in the Subject: header, thusly: [Issue ###].) To submit a new issue, send email to issues@omg.org.
List of issues (green=resolved, yellow=pending Board vote, red=unresolved)
Issue 16922: ZIOP has to be part of the core CORBA specification
Issue 16922: ZIOP has to be part of the core CORBA specification (corba-ziop-rtf)
Click here for this issue's archive.
Source: Remedy IT (Mr. Johnny Willemsen, jwillemsen(at)remedy.nl)
Nature: Uncategorized Issue
Severity:
Summary:
The ZIOP specification is an addition to CORBA and should be merged into the main CORBA specification instead of being a stand alone specification
Resolution: Move the ZIOP Compression module to CORBA Part 1, section 18
Move the ZIOP Module to CORBA Part 2, section 12
Revised Text: Add to the copyright of CORBA Part 1 Interfaces and Part 2 Interoperability documents:
the following additional copyrights
Copyright © 2007, IONA Technologies, PLC
Copyright © 2011-2012 Object Management Group
Copyright © 2007-2012 Remedy IT
Copyright © 2008-2009, Telefónica Investigación y Desarrollo S.A.Unipersonal
Add to CORBA Part 1 Interfaces, the new section 18 with the text:
18 Compression
The Compression module provides a set of interfaces to create and register entities that provide compression and
decompression functionalities. These features may be used in stand-alone mode to obtain compressed and
decompressed CORBA octet sequences, or internally by the ORB to compress GIOP messages when the ZIOP
protocol is enabled.
The Compressor interface is an abstraction that provides the basic mechanism to compress and decompress
CORBA octet sequences. The compressor collects statistical information about its compression. A specific
compressor is identified by its CompressorId. CompressorIds are maintained by the OMG, vendors and users
must request specific CompressorIds for their own compressors.
The CompressorFactory interface is a factory to create different compressors using a particular algorithm
depending on its compression level.
The CompressionManager interface is an ORB initial reference for register CompressorFactories depending on
its compression algorithm.
March 12, 2012 9
All these entities, Compressor, CompressorFactory, and CompressionManager are local CORBA interfaces.
The Compression module provides the way to easily create custom compressors. The procedure involves two
steps.
• First, the user provides an implementation of CompressorFactory and Compressor interfaces.
• Second, this new custom CompressorFactory must be registered in the CompressionManager to make it
accessible through the ORB services.
The zlib compressor must be provided by default and may be used easily as another CORBA feature. Also it
must be possible to implement a new custom compressor by implementing the Compressor interface.
18.1 Compressor Interface
This interface is an abstraction of a specific algorithm for compression and decompression. All different
algorithm implementations will support this common interface.
// IDL
module Compression {
exception CompressionException {
long reason;
string description;
};
typedef CORBA::OctetSeq Buffer;
typedef unsigned short CompressionLevel;
typedef float CompressionRatio;
local interface Compressor {
void compress(
in Buffer source,
inout Buffer target)
raises (CompressionException);
void decompress(
in Buffer source,
inout Buffer target)
raises (CompressionException);
readonly attribute CompressorFactory compressor_factory;
readonly attribute CompressionLevel compression_level;
readonly attribute unsigned long long compressed_bytes;
readonly attribute unsigned long long uncompressed_bytes;
readonly attribute CompressionRatio compression_ratio;
};
};
18.1.1 CompressionException
This exception is thrown when compress or decompress fails. The reason can be used by the concrete compressor
implementation to give some feedback on the technical reason of the failure. This could be used by application
mode when they are aware of the concrete compressor and their list of possible reasons. Because there are a lot
of different compression algorithms with different possible error reasons we don’t want to attempt to list all
possible error reasons. When the underlying compression algorithm has the possibility to retrieve an error string
this will be put in the description field.
March 12, 2012 10
18.1.2 compress
This operation compresses the data contained in a source buffer into the target buffer. If an error occurs during
the compression, it throws a CompressionException. The buffer is an octet sequence that could be extended with
ORB specific operations.
18.1.3 decompress
This operation decompresses the data contained in the source buffer into the target buffer. If an error occurs
during the decompression, it throws a CompressionException. The buffer is an octet sequence that could be
extended with ORB specific operations.
18.1.4 compressor_factory
This attribute represents the object reference to CompressorFactory that created this Compressor.
18.1.5 compression_level
This attribute represents, for the specific algorithm, the compression level that will be applied using this
Compressor. For ZIOP we define that 0 means no compression, 1 low compression, 9 the highest compression
available.
18.1.6 compressed_bytes
This attribute represents the total number of compressed bytes written by this compressor during compression
(i.e., the “target” argument of Compressor::compress). This information could be useful for statistical purposes.
18.1.7 uncompressed_bytes
This attribute represents the total number of uncompressed bytes read by this compressor during compression
(i.e., the “source” argument of Compressor::compress). This information could be useful for statistical purposes.
18.1.8 compression_ratio
This attribute represents the compression ratio achieved by this compressor. The ratio must be obtained with the
following formula: compressed_bytes / uncompressed_bytes.
18.2 CompressorFactory Interface
The CompressorFactory interface allows the retrieval of a Compressor with a particular algorithm
implementation. Compressors are retrieved for a specific compression level.
// IDL
local interface CompressorFactory {
readonly attribute CompressorId compressor_id;
Compressor get_compressor(in CompressionLevel compression_level);
};
March 12, 2012 11
18.2.1 compressor_id
This attribute represents the specific compression algorithm associated with this CompressorFactory. All
Compressors retrieved from this factory use this algorithm.
18.2.2 get_compressor
This operation retrieves a Compressor instance with the given compression level. Calling this operation multiple
times with the same compression level should return the same instance. The CompressorFactory is responsible
for managing the lifetime of the Compressors. If a compression level > 9 is passed, a BAD_PARAM exception
with minor code 44 is raised.
18.3 CompressionManager Interface
This is the interface to register and unregister CompressorFactories objects with an ORB. It is obtained by
resolving initial references: “CompressionManager.”
// IDL
exception FactoryAlreadyRegistered {
};
exception UnknownCompressorId {
};
local interface CompressionManager {
void register_factory(
in CompressorFactory compressor_factory)
raises (FactoryAlreadyRegistered);
void unregister_factory(
in CompressorId compressor_id)
raises (UnknownCompressorId);
CompressorFactory get_factory(
in CompressorId compressor_id)
raises (UnknownCompressorId);
Compressor get_compressor(
in CompressorId compressor_id,
in CompressorLevel compression_level)
raises (UnknownCompressorId);
CompressorFactorySeq get_factories();
};
18.3.1 register_factory
This operation registers a new CompressorFactory.
18.3.2 unregister_factory
This operation unregisters a CompressorFactory with the given CompressorId from the CompressionManager.
18.3.3 get_factory
This operation retrieves a CompressorFactory with the given CompressorId from the CompressionManager.
March 12, 2012 12
18.3.4 get_compressor
This operation retrieves a Compressor with the given compression_level from the CompressorFactory with the
given
CompressorId. Calling this operation multiple times with the same compressor id and compression level should
return the same instance. If a compression level > 9 is passed a BAD_PARAM exception with minor code 44 is
raised.
18.3.5 get_factories
This operation lists all registered CompressorFactories in the CompressionManager.
18.4 Consolodated IDL
#pragma prefix "omg.org"
module Compression {
exception CompressionException {
long reason;
string description;
};
exception FactoryAlreadyRegistered { };
exception UnknownCompressorId { };
typedef unsigned short CompressorId { };
const CompressorId COMPRESSORID_NONE = 0;
const CompressorId COMPRESSORID_GZIP = 1;
const CompressorId COMPRESSORID_PKZIP = 2;
const CompressorId COMPRESSORID_BZIP2 = 3;
const CompressorId COMPRESSORID_ZLIB = 4;
const CompressorId COMPRESSORID_LZMA = 5;
const CompressorId COMPRESSORID_LZO = 6;
const CompressorId COMPRESSORID_RZIP = 7;
const CompressorId COMPRESSORID_7X = 8;
const CompressorId COMPRESSORID_XAR = 9;
typedef unsigned short CompressionLevel;
typedef float CompressionRatio;
struct CompressorIdLevel {
CompressorId compressor_id;
CompressionLevel compression_level;
} typedef sequence <
CompressorIdLevel> CompressorIdLevelList;
typedef CORBA::OctetSeq Buffer;
local interface Compressor {
void compress(
in Buffer source,
inout Buffer target)
raises (CompressionException);
void decompress(
in Buffer source,
March 12, 2012 13
inout Buffer target)
raises (CompressionException);
readonly attribute CompressorFactory compressor_factory;
readonly attribute CompressionLevel compression_level;
readonly attribute unsigned long long compressed_bytes;
readonly attribute unsigned long long uncompressed_bytes;
readonly attribute CompressionRatio compression_ratio;
};
local interface CompressorFactory {
readonly attribute CompressorId compressor_id;
Compressor get_compressor(in CompressionLevel compression_level);
};
typedef sequence<CompressorFactory> CompressorFactorySeq;
local interface CompressionManager {
void register_factory(
in CompressorFactory compressor_factory)
raises (FactoryAlreadyRegistered);
void unregister_factory(
in CompressorId compressor_id)
raises (UnknownCompressorId);
CompressorFactory get_factory(
in CompressorId compressor_id)
raises (UnknownCompressorId);
Compressor get_compressor(
in CompressorId compressor_id,
in CompressorLevel compression_level)
raises (UnknownCompressorId);
CompressorFactorySeq get_factories();
};
};
To CORBA Part 2, Interoperability, add to bulleted list as part of the Scope, Section 1 the
following bullet as new last bullet:
• The Zipped Inter-ORB Protocol (ZIOP) which defines a compression mechanism for the CORBA GIOP
protocol.
To CORBA Part 2 Interoperability, Add a new Section 2.2 that defines the GIOP Compression conformance as:
2.2 GIOP Compression
GIOP Compression is defined as an optional CORBA conformance point. In order to claim GIOP Compression
compliance an ORB implementation must support the following conformance point:
• ZIOP - The ORB implements the ZIOP wire protocol and the ZIOP module policy interfaces for controlling it,
with support for at least the zlib algorithm.
When an ORB claims GIOP compression compliance it optionally can claim the following GIOP compression
compliance point:
• Pluggable compression - The ORB implements the Compression module interfaces, and the registered
CompressorFactory instances are available for use by ZIOP.
March 12, 2012 14
To CORBA Part 2 Interoperability add a new Section 12:
12 ZIOP Protocol
The ZIOP protocol applies compression to GIOP, it is the same as GIOP Compression. ZIOP is the way to
introduce compression between CORBA parties with the aim to reduce the amount of data to be transmitted on
the wire. In a CORBA communication that uses ZIOP protocol, the GIOP message is compressed using a specific
compression algorithm. For this purpose a compressed message is defined as ZIOP message.
The compression features will be provided to ZIOP protocol by some entities. The Compressor will be in charge
of basic compression and decompression operations. The CompressorFactory will create Compressors and then
CompressorFactory will be registered by the CompressionManager interface.
ORB vendors may deliver ZIOP through pluggable compressors or support a standard and well known
compression algorithm.
12.1 ZIOP Messages
A ZIOP message is a GIOP message that has ZIOP as first four magical bytes instead of the regular GIOP
magical bytes. GIOP compression can be applied to send or receive GIOP 1.2 and higher messages and includes
fragmented messages.
// PIDL: ZIOP body in ZIOP Message
module ZIOP {
struct CompressionData {
Compression::CompressorId compressor;
unsigned long original_length;
Compression::Buffer data;
};
};
A ZIOP message defines how the application data of the GIOP Messages is compressed: when the magic bytes
are ZIOP then the data after the GIOP MessageHeader is replaced by the CompressionData structure, which
contains the following items encoded in this order:
1. compressor: contains the identifier that indicates the compressor used for the current ZIOP message.
2. original_length: contains an unsigned long value that represents the GIOP body length of the current GIOP
message without applying any compression.
3. data: is an octet sequence that contains the compressed message.
The length in the GIOP Header is updated to reflect the new message length, the other fields are unchanged as it
is described below.
To allow interoperability between a ZIOP and a non ZIOP party the client that supports ZIOP will send only
ZIOP messages to servers that have been declared to accept ZIOP messages.
At message level, the sequence of message exchange is as follows:
1. When client and server ORB support a compatible compression algorithm, and if the message fulfills the
compression policies (for example message size threshold), the message is compressed and the four magic
start bytes are changed to ZIOP. The length field in the GIOP MessageHeader is updated, all other fields are
unchanged.
March 12, 2012 15
2. The server ORB reads the ZIOP header. It then takes the CompressionData struct and uncompresses the
data. The other fields of the header and the uncompressed data can then be used as a regular GIOP message.
3. In the server side, if the GIOPReply message fulfills the compression policies, a compressor object is
retrieved and server ORB will generate a compressed GIOP Reply and will send it to client where the magic
bytes in the header are set to ZIOP.
4. The client ORB side will read the ZIOP magic bytes message and then will continue reading the compressed
GIOP Reply and decompressing the GIOPBody.
Both client and server only send ZIOP messages when it knows that the remote ORB supports ZIOP and it has a
compatible compressor implementation, as is described in the following section.
12.2 ZIOP Message use
Client and server ORBs interchange available compression details through a set of new ZIOP CORBA Policies.
These policies must be propagated as standard CORBA Policies in a ServiceContext into a GIOP Request and
GIOP Reply messages. They may also be propagated into an IOR by using the Messaging propagation of QoS.
Policies which values are transferred to the remote ORB are called ‘client-exposed’ policies. The Messaging
propagation mechanism is described in detail in section 17.3 of the CORBA 3.3 specification (Part 1 Interfaces:
formal/2011-11--1).
ORB server side applications may set available compression algorithms via appointing ZIOP Policies list to the
POA that will create object references that embed these policies into the IOR component. The client side ORB
could send ZIOP messages defining similar Policies using PolicyContext interfaces, at ORB, thread or reference
level.
As previously described servers and clients must agree on which compression algorithm will be used. To allow
this, each party must know if the other party supports ZIOP and its preferences about compression before
sending to it a ZIOP message.
The server must register the CORBA object in a POA that was created with ZIOP Policies. These ZIOP Policies
will be transmitted as part of the IOR through the Messaging QoS Profile Component. The client may indicate
through 'set_policies_overrides' over the remote CORBA object reference the ZIOP Polices which it has as
preferences.
The client-side ORB will decide the compatible ZIOP Policies list which the ORB must use to send a
GIOPRequest to the server. For this, the client-side ORB will extract the compression server preferences (ZIOP
Policies) from a TaggedComponent of an IOR if it is present. The client will select a compression algorithm and
send the application data compressed to the server. The client-side ORB will also create a Policy list with its
compression policies and send them in the Request as a Messaging ServiceContext.
The server-side ORB will reply to the request taking into account the ZIOP Policies that it found in the
ServiceContext of the ZIOP messaging and compare it with the ZIOP Policies of the POA object.
If the server does not allow receipt of compressed GIOP Requests, then the client-side ORB should not send any
GIOP compressed messages. Instead, the client-side ORB will only send the ZIOP Policies values that the client
supports in Messaging ServiceContext. In a similar way a server may not respond to a client with a compressed
GIOP Reply if the client does not support GIOP compression.
In this way, a client and server may decide independently if compression could be used or not. There is no
necessity to exchange CORBA messages between client and server to obtain the best set of ZIOP Policies to be
applied in communication to get the optimal performance.
March 12, 2012 16
12.3 ZIOP Compression Policies
This module ZIOP provides all necessary elements to allow interchange of compressed GIOP messages between
client and servers using mechanisms defined in Compression module. If a specific policy is not supplied, then an
ORB default is used. The following interfaces are the ZIOP policies.
12.3.1 CompressionEnablingPolicy interface
This interface represents the ZIOP policy CompressionEnablingPolicy that has a boolean attribute indicating if
compression is enabled or not by the tier. Only when this policy has been set to true ZIOP may be used by the
ORB. This policy is client-exposed and both client and server must have set this policy to TRUE in order to
enable ZIOP.
12.3.2 CompressorIdLevelListPolicy interface
This interface represents the ZIOP policy CompressorIdLevelListPolicy. It has a list of CompressorId/
CompressionLevel attributes indicating the compression algorithms with their respective levels that may be used.
The CompressorIdLevelListPolicy contains a sequence of structures and this sequence is ordered by preference
priority. This policy is client-exposed, the client/server will take its own sequence and search for the first
CompressorId that is also supported by the other tier. For this Compressor then the lowest CompressionLevel is
selected.
12.3.3 CompressionLowValuePolicy interface
This interface represents the ZIOP policy CompressionLowValuePolicy. It has an unsigned long attribute
indicating the minimum size of application data that has to be sent before the ORB will consider this as a ZIOP
message. This policy is not client exposed.
12.3.4 CompressionMinRatioPolicy interface
This interface represents the ZIOP policy CompressionMinRatioPolicy. It has a float attribute indicating the
minimum compression ratio that must be obtained at compression time to send with a compressed GIOP
message. This policy tries to prevent the sending of compressed messages with few improvements about the
original size in order to not overload the server with a useless decompression process. The ratio must be obtained
with the following formula: compressed_length / original_length. This policy is not client exposed.
12.4 Propagation of ZIOP Compression Policies
ZIOP Compression policies are transferred using the Messaging QoS Profile Component that is defined in
section 17.3 of the CORBA 3.3 specification (Part 1 Interfaces: formal/2012-01-01). That section also describes
the concept of client-exposed policies.
12.5 Consolidated IDL
#pragma prefix “omg.org”
module ZIOP {
struct CompressedData {
Compression::CompressorId compressorid;
unsigned long original_length;
Compression::Buffer data;
March 12, 2012 17
};
typedef boolean CompressionEnablingPolicyValue;
const CORBA::PolicyType COMPRESSION_ENABLING_POLICY_ID = 64;
local interface CompressionEnablingPolicy: CORBA::Policy
{
readonly attribute CompressionEnablingPolicyValue compression_enabled;
};
const CORBA::PolicyType COMPRESSOR_ID_LEVEL_LIST_POLICY_ID = 65;
local interface CompressionIdLevelListPolicy: CORBA::Policy
{
readonly attribute Compression::CompressorIdLevelList compressor_ids;
};
typedef unsigned long CompressionLowValuePolicyValue;
const CORBA::PolicyType COMPRESSION_LOW_VALUE_POLICY_ID = 66;
local interface CompressionLowValuePolicy: CORBA::Policy
{
readonly attribute CompressionLowValuePolicyValue low_value;
};
const CORBA::PolicyType COMPRESSION_MIN_RATIO_POLICY_ID = 67;
local interface CompressionMinRatioPolicy: CORBA::Policy
{
readonly attribute Compression::CompressionRatio ratio;
};
};
To Part 1 Interfaces and Part 2 Operability, to Section A.4 Policy Type Tags add the following policies to the table at the bottom
Actions taken:
December 20, 2011: received issue