Getting Started

Getting Started with Eclipsa Audio Creation

The Eclipsa Audio Plugins consist of two plugins: the Eclipsa Audio Element Plugin and the Eclipsa Audio Renderer Plugin. These plugins work together to allow you to position, monitor, and render your audio in 3D space.

Note: To ensure proper setup, instantiate the Eclipsa Audio Renderer Plugin first and create an audio element, as the Eclipsa Audio Element Plugin requires this configuration to produce output.

Understanding How Eclipsa Audio Records and Plays Spatialized Audio

To produce Eclipsa Audio spatialized audio and video we need to first record audio in the IAMF file format. This format is used to store the data required to play spatialized audio on Eclipsa Audio compatible devices.

IAMF files are made up of a series of audio channels where each audio channel is associated with a speaker. These channels are grouped into audio elements which define the speaker layouts of the audio channels. These audio elements also serve as the building blocks for mix presentations. During playback the listener selects a mix presentation to be listened to. The mix presentations constituent audio elements, and the audio associated with them, is then played.

• Audio Channels: Each channel corresponds to a specific speaker (e.g., left, right, center).

• Audio Elements: Collections of audio channels that define a specific spatial speaker layout. For example:

  • A stereo audio element contains two channels for left and right positioning.

  • A 5.1 audio element includes six channels: front left, front right, center, subwoofer, surround left, and surround right.

• Mix Presentations: Combine multiple audio elements into unique configurations for playback, such as different languages, formats, or speaker layouts. An audio element may belong to multiple mix presentations.

Overview of Playback Flow

The Eclipsa Audio Plugins consist of two plugins, the Eclipsa Audio Renderer Plugin and the Eclipsa Audio Element Plugin. The Eclipsa Audio Renderer Plugin serves as the central hub for managing the various components of an IAMF file and the Eclipsa Audio Element plugin serves to assign audio elements to spatialized audio tracks.

One instance of the Eclipsa Audio Renderer Plugin must be added to a projects mix bus, and all of a projects audio must pass through the Eclipsa Audio Renderer Plugin instance. The Eclipsa Audio Renderer Plugin is used to configure audio elements which define the various spatialized speaker layouts for recording. It also defines the Mix Presentations. These can be configured to customize different playback scenarios, such as different speaker setups or languages. Finally, the Eclipsa Audio Renderer Plugin handles the rendering and playback of the Eclipsa audio, as well as creates the IAMF file.

Once the Eclipsa Audio Renderer Plugin is in place, each spatialized audio track must have an Eclipsa Audio Element Plugin instance added to it. This plugin is used to assign an audio element and it's constituent channels to the audio track. The plugin also connects to the Eclipsa Audio Renderer Plugin to ensure accurate spatial mixing by separating audio from different audio elements onto different channels. Audio must pass through an Eclipsa Audio Element Plugin to be included in the 3D mix. This setup ensures all spatialized audio is organized and rendered correctly in the final mix.

Note: Due to DAW and plugin library limitations, the DAW Mix Bus may not be exactly 28 channels. For example, in Avid ProTools a 5th order ambisonics bus is used as it is the smallest bus which contains 28 channels. Similarly, in Adobe Premiere Pro, the DAW Mix Bus is only 16 channels, as this is the maximum number of channels supported by the DAW.

The Eclipsa Audio Renderer Plugin

The Eclipsa Audio Renderer Plugin manages the overall 3D audio mix, enabling you to configure speaker setups, monitor your mix, and export the final mix to an IAMF file or Eclipsa Audio compatible MP4 file. Additionally, it’s used to create audio elements and configure mix presentations, both of which are required for playback.

How to Use the Eclipsa Audio Renderer Plugin

1. Add the Eclipsa Audio Renderer Plugin to the Mix Bus

Avid ProTools

REAPER

Adobe Premiere Pro

1. Widen the mix bus to 5th Order Ambisonics

• Navigate to your Mix Bus track or create a new Aux Input track to act as your master bus.

• Set the track width to 5th Order Ambisonics to support the plugin as shown below.

2. Insert the Eclipsa Audio Renderer Plugin:

• Click on an empty Insert slot in the Mix Bus track.

• Choose Multi-Channel Plug-In and select Eclipsa Audio Renderer from the Instrument plugin list.

• Select the instance of the Eclipsa Audio Renderer with the highest speaker layout you wish to play back on from within ProTools.

1. Add the Eclipsa Audio Renderer Plugin to the Master Track

• Find the Master track on the REAPER Mixer tab

• Select the "FX" button, then select "Add" if other plugins have already been added to this track

• Under "All Plugins" --> "VST3" select "VST3: Eclipsa Audio Renderer (Eclipsa Project)

• Select "Add" to add the plugin to the Master track

2. Use the Plug-In Pin Connector to connect the plugin

• On the "Eclipsa Audio Renderer" UI, change the "Speaker Setup" to match the speaker layout you wish to play back audio locally on

• In the Plug-In UI window, select the Pin Connector in the top right corner to open the Pin Connector UI

• Under "Track Channels" select a value larger than 28, typically 32

• Under "VST3 bus size" select a value matching the number of speakers in the speaker layout you wish to play back audio locally on

1. Create a new sequence with 16 channels

• In Adobe Premiere Pro select "File" --> "New" --> "Sequence..."

• Select "Tracks", then under "Audio" change the "Mix" to "Multichannel"

• Change the "Number of Channels" to 16

• Add or remove tracks as needed for your project. Ensure the "Track Type" of all tracks is "Adaptive". A minimum of 1 track is required.

2. Refresh the Audio Plugins

• This step needs to be done once after installation

• If it's not visible already, open the "Audio Track Mixer" window by selecting "Window" --> "Audio Track Mixer" --> Your Sequence Name

• Select the hamburger button in the top left corner and select "Audio Plug-In Manager"

• In the Audio Plug-In Manager select "Scan for Plug-Ins" and verify the Eclipsa Audio plugins are present

3. Add the Eclipsa Audio Renderer Plugin to the Mix Track

• In the "Audio Track Mixer" window, select the fx for the mix track, which is typically the rightmost track. It should be identifiable by the selectable channel count box showing 16.

  • If the fx table is not shown, it may be hidden and can be unhidden by clicking on the leftmost side of the Audio Track Mixer

• Click one of the fx slots and select "AU" --> "Eclipsa Project" --> "Eclipsa Audio Renderer"

---

Note: Adobe Premiere Pro only supports up to 16 channels for AU plugins, limiting it to the "Simple" routing profile only.

---

2. Configure the Plugin and Add an Audio Element:

• Once the Eclipsa Audio Renderer Plugin interface opens, you can configure the monitored speaker layout by selecting the "Speaker Setup" in the top left corner.

• To enable playback and monitoring, add a stereo audio element to your mix presentation:

  • Click the re-route button (bottom right of the interface) to access the audio element creation screen.

  • Select Add Audio Element and choose the type (e.g., stereo).

  • Confirm the audio element creation to ensure your panned audio can be routed here for playback.

3. Set Up Mix Presentations:

• In the Eclipsa Audio Renderer Plugin, configure a mix presentation by clicking the "Edit" button

• Rename the default presentation and assign it a language if desired.

• Add the previously created audio element to the mix presentation by selecting "Add Audio Element".

The Eclipsa Audio Renderer Plugin is now set up to handle monitoring, playback, and mix presentation configurations, allowing you to route all panned elements here for a complete 3D audio experience.

The Eclipsa Audio Element Plugin

The Eclipsa Audio Element Plugin is used to place and manipulate individual audio tracks within the 3D space, adding spatial information to each track and routing it to an audio element within the Eclipsa Audio Renderer Plugin.

How to Use the Eclipsa Audio Element Plugin

1. Add the Eclipsa Audio Element Plugin to a track

Avid ProTools

REAPER

Adobe Premiere Pro

1. Create a new audio track

• Click Track > New and choose Mono or Stereo Audio Track based on your needs.

• Add audio to the new track, typically by dragging and dropping an external sample audio file onto the new track.

  

2. Insert the Eclipsa Audio Element Plugin:

• Locate an empty Insert slot on your audio track (in the Mix or Edit window).

• Click the Insert slot, navigate to Multi-Channel Plug-In, and select Eclipsa Audio Element from the list.

  

2. Configure a send from the track the Eclipsa Audio Element Plugin is on to the track the Eclipsa Audio Renderer Plugin is on.

• Select "Sends" --> "track" and then selecting the track hosting the Eclipsa Audio Renderer Plugin, typically the "Mix Bus" track.

1. Create a new audio track

• Right Click on the track view and select "Insert new track"

• Add audio to the new track, typically by dragging and dropping an external sample audio file onto the new track.

2. Add the plugin to the new audio track

• Select "FX" under the new track

• Under "VST3" select the "VST3: Eclipsa Audio Element Plugin"

3. Configure the pin connectors for the plugin

• In the top right corner, select the pin connector button

• Change the "Track channels" and "VST3 bus size" to 32 as 32 is the smallest available bus size for the required 28 channels.

1. Add the plugin to an existing audio track in the sequence

• In the "Audio Track Mixer" window select the track to add the Eclipsa Audio Element Plugin to.

• In the fx list for the track select "AU" --> "Eclipsa Project" --> "Eclipsa Audio Element Plugin".

2. Ensure the track sends audio to the mix

• By default, in Adobe Premiere Pro, audio channels are automatically forwarded from individual tracks to the Mix track.

• Ensure audio is forwarded correctly by verifying that the bus target for tracks with the Eclipsa Audio Element Plugin is "Mix"

2. Configure the Audio Element Plugin and Choose an Audio Element:

• In the Audio Element Plugin interface, attach the track’s output to an audio element created in the Eclipsa Audio Renderer Plugin.

• Select the audio element in the "Audio Element" drop down.

  

3. Playback audio

• Configuration should now be completed

• Playback audio and check the channel monitoring in both the Eclipsa Audio Element Plugin and Eclipsa Audio Renderer Plugin to ensure audio is reaching both plugins

• If audio appears in the Eclipsa Audio Element Plugin but not the Eclipsa Audio Renderer Plugin, ensure the audio track is correctly connected to the mix bus.

Additional Tips

Ensure Audio Elements and Mix Presentations Are Correctly Configured

Audio elements and mix presentations in the Eclipsa Audio Renderer Plugin are essential for proper 3D playback and monitoring. During playback, only audio associated with audio elements assigned to the currently playing mix presentation will be played.

Each Audio Element Plugin instance must be configured with an audio element to ensure output during playback.

Spatialized Audio Must Pass Through an Audio Element Plugin

Insert an Audio Element Plugin on every track you want to spatialize. This setup ensures each sound source is routed to the correct audio element and fully integrated into the 3D mix. To reduce the number of Audio Element Plugins needed you can also use busses to route multiple tracks through a single Audio Element Plugin instance before routing the audio to the Eclipsa Audio Renderer Plugin. For more information see Audio Element Plugin Routing.