XCON WG C. Jennings Internet-Draft Cisco Systems, Inc. Expires: September 20, 2004 March 22, 2004 Conference State Markup Language draft-jennings-mixer-control-00.txt Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on September 20, 2004. Copyright Notice Copyright (C) The Internet Society (2004). All Rights Reserved. Abstract This draft is in a very early stage and has many known mistakes. Media mixers are capable of a limited number of transformations and combinations of various media streams. This draft describes an XML document that can be used to control the state of a mixer that performs some common media manipulations. This work is related to the work of the XCON working group but is not part of the scope of that working group. It is being discussed on the xcon@ietf.org mailing list. Jennings Expires September 20, 2004 [Page 1] Internet-Draft CSML March 2004 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Conventions and Definitions . . . . . . . . . . . . . . . . . 3 3. Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 3 4. Describing Mixer State . . . . . . . . . . . . . . . . . . . . 4 4.1 General Conference Parameters . . . . . . . . . . . . . . . . 4 4.2 General Stream Parameters . . . . . . . . . . . . . . . . . . 4 4.3 Cascading . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4.4 Audio . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4.5 Video . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4.6 Text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 5. Manipulating State . . . . . . . . . . . . . . . . . . . . . . 6 6. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 6.1 Create an Audio Conference with One Participant . . . . . . . 7 6.2 Adding a Stream . . . . . . . . . . . . . . . . . . . . . . . 8 6.3 Deleting a Stream . . . . . . . . . . . . . . . . . . . . . . 9 6.4 Creating a Side Bar . . . . . . . . . . . . . . . . . . . . . 9 6.5 Adding 5+1 Video Layout . . . . . . . . . . . . . . . . . . . 10 6.6 Changing a video layout . . . . . . . . . . . . . . . . . . . 10 6.7 Controlling a multicast video switcher . . . . . . . . . . . . 10 6.8 Setting up a Customer Agent Supervisor call . . . . . . . . . 11 7. Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 8. IANA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 9. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 10. To Do . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 15 Normative References . . . . . . . . . . . . . . . . . . . . . 15 Informative References . . . . . . . . . . . . . . . . . . . . 16 Author's Address . . . . . . . . . . . . . . . . . . . . . . . 16 Intellectual Property and Copyright Statements . . . . . . . . 17 Jennings Expires September 20, 2004 [Page 2] Internet-Draft CSML March 2004 1. Introduction Conference mixers have many controls that change how the media is combined for each media stream in the conference. Applications need to be able to control this state on a mixer. The description of this control needs to be rich enough to allow common operations in an interoperable way; extensible, since there will always be new operations; yet explicit and clear to the user what media flow is required, so that a highly optimized implementation is possible. The general approach is to model the various parameters that can be controlled to set up the media flow in the mixer. It is assumed that some other protocol can be used to play announcements, record, or be notified about DTMF from any stream or conference using the identifier for that stream on conference. This work only describes how to control the media mixers. Floor control is done at a higher layer than this media control but a floor control command may cause the application controlling the mixer to send new state to the mixer indicating the change in which streams are allowed to contribute media to the mix. 2. Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [1]. Definitions: 'Stream' A flow of media, such as RTP, to or from an end client. 'Conference' A set of related streams and media transformations that result in one virtual result of a single media type. This may be slightly different from the overall meeting, which may contain several media types and some sidebar mixes. This overall meeting would result in several conferences on various mixers that each took care of producing one stream. 'RosterId' An identifier used to correlate media streams that are somehow grouped to a common "user" or "participant" in a confernce. 3. Requirements This work needs to be capable of describing the state for the media transformation required by the use cases in [3]. Modification to the state needs to be idempotent. It needs to be able to fully describe the state of a mixer other than the state associated with the current media or the signaling to set up the streams. Jennings Expires September 20, 2004 [Page 3] Internet-Draft CSML March 2004 4. Describing Mixer State The conference consists of some parameters that affect the whole conference and a set of input and output streams of various media types. For example, the XML fragment below describes a simple audio conference with streams two two audio endpoints. 4.1 General Conference Parameters expectedNumStreams - This sets the number of input streams eventually expected for each media type. In some cases, such as when this is one or two, this allows the mixer to pick a substantially different optimization approach. TODO - Is there any need for this or is it just an optimization hint. 4.2 General Stream Parameters Each stream has a unique identifier that is chosen by the application controlling the mixer. It also has a direction and a media type. rosterId - This identifier is used to identify the "user" or "participant" that is creating the media in an input stream and is used to correlate different streams usually of different media types. Jennings Expires September 20, 2004 [Page 4] Internet-Draft CSML March 2004 When the video is supposed to follow the active speaker, this provides a way to correlate which video stream corresponds to which speaker. source - Identifies another conference as the source of this media stream. rtp - Provides the port and IP of an RTP session that forms the stream. TODO - need to generalize source in port, ip, other ways to identify a stream such as fid or even a PSTN circuit. url - Provides the URL for non-RTP media sessions that form a stream. priority - Provides a relative priority of this stream for being included in the mix. The mix SHOULD not add any streams to the mix that are of a lower priority than some stream it has chosen to ignore from the mix. A priority of 1.0 means the mixer should make all efforts to include this media while a priority of 0.0 indicates that the media should only be included as a last resort. 4.3 Cascading A conference can specify that one of it's input streams actually comes from some named output stream of another conference. TODO - deal with issues of "gain" for audio, video, text, application. 4.4 Audio gain - this applies a gain to the stream as it comes in or out of the mix. The gains are specified in dB; a value of -1000 is used to mean the same as negative infinity. muteOnDtmf - Indicates that the stream should be muted on any input DTMF to the participant of this stream. automaticGainControl - Automatic gain control :-) echoCancelation - you guessed it annoucementGain - specifies a gain for announcements played to the particular stream or played to the whole conference that this stream participates in. supppresionInAnouncement - specifies a gain in dB for how much the Jennings Expires September 20, 2004 [Page 5] Internet-Draft CSML March 2004 main conference mix is suppressed during an announcement. A suppression of -1000dB would effectively mute the conference mix. 4.5 Video freeze - indicates the image should be frozen at the current image blank - indicates that a black image or logo should replace the image overlay - specifies some text to render over the video image. layout - specifies a grid for layout of a video composition output stream. Contains several cell elements. Each cell covers some rectangular region in the grid and contains one of the video input streams. This allows only fairly simple layout control. If more complex control is required, an approach such as SMIL [4] could be used. Inside of each layout are cell tags. Each cell identifies a retangular space in the grid and the video stream inside it. The streams to be placed in a cell can be statically identified by the stream label or indirectly identified by referring to a location from which to receive stream labels. Two common forms of this are "active speaker" and "previous speaker" which use the audio to pick the corresponding video stream. 4.6 Text 5. Manipulating State State is manipulated over some transport protocol that can send XML documents and receive a response of whether that worked or not. The response may also include an XML document. The protocol may also support an asynchronous send of an XML document from the mixer to an application that had previously sent it a document. This can be mapped to a SIP PUBLISH to send and change information and use of a SUB/NOT to querry state and find out about asyncronous changes. New conferences and streams are added by sending a document that has a conference ID that is new. Conferences and streams are deleted by sending an XML tag identifying that object and having it not contain any child elements. An item can be changed by sending just the portion of the XML document that contains all the changed elements. There are two ways updating state can not work. One is that the Jennings Expires September 20, 2004 [Page 6] Internet-Draft CSML March 2004 request may be malformed or unauthorized - this is referred to as failure and the protocol used to transport the request needs to indicate a failure in the response to the request. The other way something can not work is that the the mixer may not have exactly matched the requested operation and may have approximated it in some way. For example, an audio gain of 3.2 dB may have been requested but the mixer may have only been able to do multiples of 3 dB and may have therefore rounded this to 3.0 dB. In this case the actual state of the confernce is returned in an XML body in the response. The returned XML contains the actual state acheived (3.0 in this case) instead of the state requested (3.3 in this case). 6. Examples 6.1 Create an Audio Conference with One Participant Jennings Expires September 20, 2004 [Page 7] Internet-Draft CSML March 2004 6.2 Adding a Stream This example adds one more input and one more output stream. Jennings Expires September 20, 2004 [Page 8] Internet-Draft CSML March 2004 6.3 Deleting a Stream 6.4 Creating a Side Bar Jennings Expires September 20, 2004 [Page 9] Internet-Draft CSML March 2004 6.5 Adding 5+1 Video Layout Bob in Atlanta 6.6 Changing a video layout 6.7 Controlling a multicast video switcher Jennings Expires September 20, 2004 [Page 10] Internet-Draft CSML March 2004 6.8 Setting up a Customer Agent Supervisor call 7. Syntax TODO - several types in here could be improved. Jennings Expires September 20, 2004 [Page 11] Internet-Draft CSML March 2004 Jennings Expires September 20, 2004 [Page 12] Internet-Draft CSML March 2004 Jennings Expires September 20, 2004 [Page 13] Internet-Draft CSML March 2004 Jennings Expires September 20, 2004 [Page 14] Internet-Draft CSML March 2004 8. IANA 9. Security 10. To Do Fix the names spaces and add XML extensibility. 11. Acknowledgments Normative References [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [2] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996. Jennings Expires September 20, 2004 [Page 15] Internet-Draft CSML March 2004 Informative References [3] Even, R., "Conferencing Scenarios", draft-ietf-xcon-conference-scenarios-00 (work in progress), December 2003. [4] Ossenbruggen, J., Rutledge, L., Saccocio, B., Schmitz, P., Kate, W., Ayars, J., Bulterman, D., Cohen, A., Day, K., Hodge, E., Hoschka, P., Hyche, E., Jourdan, M., Kubota, K., Lanphier, R., Layaïda, N., Michel, T. and D. Newman, "Synchronized Multimedia Integration Language (SMIL 2.0) Specification", W3C REC REC-smil20-20010807, August 2001. [5] Dyke, J., Burger, E. and A. Spitzer, "Media Server Control Markup Language (MSCML) and Protocol", draft-vandyke-mscml-03 (work in progress), July 2003. [6] Melanchuk, T. and G. Sharratt, "Media Sessions Markup Language (MSML)", draft-melanchuk-sipping-msml-01 (work in progress), October 2003. [7] Melanchuk, T. and G. Sharratt, "Media Objects Markup Language (MOML)", draft-melanchuk-sipping-moml-01 (work in progress), October 2003. Author's Address Cullen Jennings Cisco Systems, Inc. 170 West Tasman Dr. MS: SJC-21/3 San Jose, CA 95134 USA Phone: +1 408 527-9132 EMail: fluffy@cisco.com Jennings Expires September 20, 2004 [Page 16] Internet-Draft CSML March 2004 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. Full Copyright Statement Copyright (C) The Internet Society (2004). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION Jennings Expires September 20, 2004 [Page 17] Internet-Draft CSML March 2004 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Jennings Expires September 20, 2004 [Page 18]