Skip to content

Positional Audio

Note

This feature is also called 'audio panning'.

You can configure MT Canvus to support positional audio. This means that canvas sounds (for example, a movie or video stream) are directed to the nearest speaker. To set up positional audio, you must create an audio configuration file and pass this file to MT Canvus at runtime.

Audio Sources

MT Canvus can play sounds from several audio sources:

  • Movie files: If a user drags an MP4 file onto the canvas, it automatically displays in a movie widget. This widget is a simple video player that allows users to play, pause, and mute the movie, but it has no volume control; volume levels for the movie's audio track are controlled using your speakers or application computer.
  • Video streams: Users can stream video (and accompanying audio) on the canvas, including webcam streams and videos running on an external computer. The video stream widget has no volume control; volume levels are controlled using your speakers or application computer.
  • Shared screens (and audio sharing): Users can display the screen of their laptop on the canvas. When a user shares their screen, audio output is shared along with video output. This is important for remote users sharing their screen, allowing them to attend online meetings or video conferences, with their contributions displayed and heard on the MT Canvus video wall. The shared screen widget has no volume control; volume levels are controlled using your speakers or application computer.
  • Browsers: Browser sounds include any audio output generated by the browser content. This typically means the audio track for a video running in the browser. The browser widget has no volume control; volume levels are controlled using your speakers or application computer.

For details about which sound sources support positional audio, see audio-sources-support

About Positional Audio

By default, MT Canvus directs sounds equally to all speakers attached to your video wall, even if the sound source is not visible on-screen. This can happen if, for example, the viewport is zoomed in on a corner of the total canvas.

But you can configure MT Canvus to support positional audio. When positional audio is enabled on your application computer, sounds are directed to the nearest speaker. This is useful on large video walls. If the sound source moves away from one speaker towards an adjacent speaker (for example, if a user drags a movie widget), audio output smoothly shifts from the first speaker to the second.

For example, consider a long video wall with five evenly spaced speakers. A video playing in the center of the wall outputs audio to the central speaker, but a video playing on the left side of the wall outputs audio to the leftmost speaker. Similarly, if a user drags a video across the screen from left to right, its audio output follows the video, panning across the speakers from left to right.

For positional audio setup instructions, see positional

Which Audio Sources Support Positional Audio?

Movie widgets, video stream widgets, and shared screen widgets all support positional audio if it is enabled on your application computer.

Browser widgets do not support positional audio. Even if positional audio is enabled, browser sounds are always directed to all available speakers, regardless of the browser widget's screen position.

Audio Requirements

Sound Card

We do not currently recommend any specific sound cards. To set up audio panning, your application computer simply needs a multi-channel sound card that supports speaker configurations such as 5.1 surround sound.

Channel Index Numbers

You need to know the channel index for each speaker. You will reference these index numbers when you create an audio configuration file. For advice on how to obtain channel index numbers, see setup-positional

For example, a 5.1 surround sound system typically uses these channel index numbers:

  • Front Left: 0
  • Front Right: 1
  • Front Center: 2
  • Rear Left: 3
  • Rear Right: 4

Note

Left and Right mean the user's left and right when they face the screen. The subwoofer has no channel index and can be ignored when setting up positional audio for MT Canvus.

Audio Configuration Files

You configure positional audio in an XML audio configuration file that is passed to MT Canvus at runtime. The audio configuration defines:

  • Speaker zones on your video screen. Each zone is rectangular and has two audio channels, on its left and right edges. The size, location, and number of speaker zones match the position and number of speakers attached to your video wall.
  • Fade borders at the edges of each speaker zone. If the sound source (such as a video widget) moves out of a speaker zone and into a fade border, audio output smoothly attenuates to zero.
  • Stereo panning within each speaker zone. This controls audio attenuation on a zone's left and right channels. For example, as the sound source moves away from the left speaker towards the right speaker, audio output falls on the left channel and rises on the right channel.

An audio configuration contains the following elements:

Where:

  • mode: Specifies the type of positional audio. Always set this to 1 for rectangular speaker zones.
  • SoundRectangle: Defines a single rectangular speaker zone. Its child elements define the zone's size and location, and the speakers assigned to the zone's left and right channels. You must add a separate SoundRectangle element for each speaker zone in your audio configuration. Typically, you need s-1 speaker zones, where s is the number of speakers attached to your video wall.
  • left-channel: Sets the channel index for the speaker on the left edge of the speaker zone; n is the channel index.
  • right-channel: Sets the channel index for the speaker on the right edge of the speaker zone; n is the channel index.
  • location: Defines the screen location of the speaker zone; x y define the horizontal and vertical coordinates of the zone's top-left corner, in pixels.
  • size: Defines the size of the speaker zone; w h define the width and height of the zone, in pixels.
  • stereo-pan: Controls audio attenuation on the zone's left and right channels. Audio output changes on each channel dynamically as the sound source moves. Permitted values range from 0 to 1. The default is 0.3, meaning that when the sound source is fully on the right, audio output on the left channel falls by 30%. (That is, when the right channel audio is 100%, the left channel is 70%. Likewise, when the left channel audio is 100%, the right channel is 70%.)
  • fade-width: Defines the width, in pixels, of the fade borders. These lie outside the left and right edges of the speaker zone. Within the fade border, audio output falls to 0% or rises to 100%, depending on whether the sound source is leaving or entering a speaker zone.

Where Do I Save My Audio Configuration?

We recommend you save your audio configuration file in the same folder as the MultiTaction Cornerstone configuration files, screen.xml and config.txt. By default, these files are saved in the following locations:

  • Ubuntu application computers: The files are in the ~/.MultiTouch folder, where ~/ refers to the home folder of the MT Canvus runtime user. If the user logged on while MT Canvus runs is 'multi', the expanded path is: /home/multi/.MultiTouch/.
  • Windows application computers: The files are saved in the MT Canvus runtime user's profile: %APPDATA%\MultiTouch. If the user logged on while MT Canvus runs is 'multi', the expanded path is: C:\Users\multi\AppData\Roaming\MultiTouch\screen.xml.

Instructions for passing the audio configuration to MT Canvus are in setup-positional

Stereo Panning

Stereo panning is best illustrated in a simple two-speaker stereo setup. In the example below, a video wall has two speakers, with a single speaker zone extending across the entire screen. The left and right edges of the speaker zone correspond to the left and right audio channels. As the sound source (such as a movie widget) is moved left to right across the screen, audio output also pans left to right, fading from the left channel and gaining on the right channel.

You use the stereo-pan setting to control the level of attenuation. Permitted values range from 0 to 1. The default is 0.3, meaning that when the sound source is fully on the right, audio output on the left channel falls by 30%. (That is, when the right channel audio is 100%, the left channel is 70%. Likewise, when the left channel audio is 100%, the right channel is 70%.)

Audio attenuation is linear. For example, if stereo-pan is 0.3, then output on both the left and right channels falls by 15% when the sound source is in the center of the speaker zone i.e., midway between the left and right edges.

Fade Borders

Fade borders enable you to configure smooth audio transitions from one speaker to the next. Fade borders are vertical strips of screen space either side of a speaker zone.

When a sound source (such as a movie widget) approaches a speaker zone, it first passes through the entry fade border and its audio output rises from zero to 100%. When it leaves a speaker zone, it passes through the exit fade border and its audio output fades from 100% to zero.

By default, fade borders are 100 pixels wide and automatically configured outside the left and right edges of a speaker zone. For smoother audio transitions, try experimenting with wider fade borders (say, 200 or 300 pixels).

Set Up Positional Audio

Follow these steps:

  1. Confirm that the speakers are connected to the application computer and recognized by the operating system. Also, note the channel index for each speaker.
    • Ubuntu application computers: Run the pavucontrol command to launch the PulseAudio Volume Control app. In the app's Configuration tab, select the setup that most closely matches the speaker configuration on the application computer. Then obtain the channel index numbers from the Playback tab; this tab lists the satellite speakers in speaker index order. So the first speaker is channel 0, the second is channel 1, and so on. (PulseAudio is a Linux sound server included with Ubuntu distributions.)
    • Windows application computers: Use the Sound applet to select your speakers from the list of playback devices. Then use the Speaker Setup wizard (launched from the Sounds applet) to select and test the setup that most closely matches the speaker configuration on the application computer. When you test the setup, the Speaker Setup wizard test plays output on each satellite speaker in index order. So the first speaker is channel 0, the second is channel 1, and so on.
  2. Using your preferred XML editor, create an audio configuration file. For example, MyAudioSetup.xml.
    • For syntax details, see section 9.4.
    • For examples, see section 9.8.
  3. Run one of the following commands to pass the audio configuration file to the MT Canvus client at runtime:
    • Ubuntu application computers: $ mt-canvus.sh --audio-config <config file>
    • Windows application computers: mt-canvus.bat --audio-config <config file>
    • Where <config file> is the XML file you created in step 2.

For more about starting the MT Canvus client, see section 6.1.

Example Audio Configuration Files

Two Speakers and a Single Speaker Zone

Here, the video wall comprises three MultiTaction Cells in portrait mode with two speakers (one at each end of the video wall). The wall is approximately 2m wide, with a total display area of 3260 x 1080 pixels, where 3260 is three Cell widths (3 x 1080) plus two 10-pixel bezels.

The audio configuration defines a single speaker zone, with a left and right channel. Default stereo panning is enabled, and fade borders are not relevant (because only one speaker zone is defined). The left channel index is 0 and the right channel index is 1.

<!DOCTYPE mtdoc>
<pan2d>
  <mode>1</mode>
  <rectangles>
    <SoundRectangle>
      <left-channel>0</left-channel>
      <right-channel>1</right-channel>
      <location>0 0</location>
      <size>3260 1080</size>
      <stereo-pan>0.3</stereo-pan>
      <!-- fade-width is not relevant in this setup -->
    </SoundRectangle>
  </rectangles>
</pan2d>

Five Speakers and Four Speaker Zones

Here, the video wall comprises 4x3 MultiTaction Cells in landscape with five speakers arranged evenly along the wall. The wall is approximately 4.8m wide, with a total display area of 7710 x 3260 pixels, where 7710 is four Cell widths (4 x 1920) plus three 10-pixel bezels.

The audio configuration defines four speaker zones, each with a left and right channel. Adjoining zones 'share' a speaker. Default stereo panning and default fade borders are both enabled. From left to right, channel index numbers are 0 through 4.

<!DOCTYPE mtdoc>
<pan2d>
  <mode>1</mode>
  <rectangles>
    <!-- Speaker Zone 1 -->
    <SoundRectangle>
      <left-channel>0</left-channel>
      <right-channel>1</right-channel>
      <location>0 0</location>
      <size>1930 3260</size>
      <stereo-pan>0.3</stereo-pan>
      <fade-width>100</fade-width>
    </SoundRectangle>
    <!-- Speaker Zone 2 -->
    <SoundRectangle>
      <left-channel>1</left-channel>
      <right-channel>2</right-channel>
      <location>1930 0</location>
      <size>1925 3260</size>
      <stereo-pan>0.3</stereo-pan>
      <fade-width>100</fade-width>
    </SoundRectangle>
    <!-- Speaker Zone 3 -->
    <SoundRectangle>
      <left-channel>2</left-channel>
      <right-channel>3</right-channel>
      <location>3855 0</location>
      <size>1925 3260</size>
      <stereo-pan>0.3</stereo-pan>
      <fade-width>100</fade-width>
    </SoundRectangle>
    <!-- Speaker Zone 4 -->
    <SoundRectangle>
      <left-channel>3</left-channel>
      <right-channel>4</right-channel>
      <location>5780 0</location>
      <size>1930 3260</size>
      <stereo-pan>0.3</stereo-pan>
      <fade-width>100</fade-width>
    </SoundRectangle>
  </rectangles>
</pan2d>