Clip Editor
Last updated
Last updated
The Clip Editor is one of the most important parts of LipSync Pro. This is where you will do most of your work in creating your dialogue animations. It has a number of features for adding and editing phonemes, emotions and gestures, and is the home of AutoSync.
The main body of the Clip Editor is made up of the audio clip waveform and the timeline. The waveform will show up as soon as an AudioClip is selected in the field near the top of the Window. The timeline is immediately above the waveform and contains all the markers added to the clip. Above that is the tab control: this lets you chose between the Phoneme, Emotion & Gesture marker editing modes.
The playhead can be moved around by dragging along the waveform, also referred to as "scrubbing". You'll hear a short preview of the audio at that moment as the playhead is moved.
Below the waveform is a scrollbar that can be used to zoom in/out and move around on the timeline, and a lower toolbar containing the time display, play/pause, stop and loop buttons and the Add Marker button.
In the top toolbar, you'll find all the common functions in the Clip Editor.
The File menu contains all the I/O features including Save, Load etc. The XML Import & XML Export options let you save your clip as a text-based XML file instead of the standard LipSyncData .asset format. This can be useful if you want to use these files outside of Unity, for example for further processing. While they can be played back on a LipSync Component, we generally don't recommend it as they can't store a reference to the audio clip. This means you'll need to keep track of the correct audio clip yourself. The standard Export option creates a .unitypackage containing your clip as a LipSyncData .asset file, along with the audio clip used. This can then be easily imported into another Unity project without losing any references.
The Edit menu is again fairly standard, containing Copy/Paste, Select All/None etc. Additionally the Set Intensity from Volume option can be used in phoneme mode to set the intensity of all Phoneme Markers according to the volume of the audio clip at the time of the marker. The Reset Intensities option will set all markers back to full intensity.
Use of the Set Intensity from Volume option is no-longer encouraged, and the feature is no longer being updated. Instead, we recommend using the AutoSync module of the same name, as it supports more advanced features and can be automated.
The Default Marker Settings option brings up the Default Marker Settings window, which can be used to set default settings that will be applied to new markers when they are created. See the Marker Settings Window for more info. The Clip Settings option simply brings up the Clip Settings window, where settings relating to the current clip can be found.
The AutoSync menu is used for accessing AutoSync features: Setup will show the AutoSync Setup Wizard, as described in the AutoSync Guide. This guide can also be accessed from this menu, by clicking the View Guide option. The next option down is Run Default Preset (the displayed name will change depending on your chosen default preset). This is a quickstart option that will immediately run the preset on the contents of the Clip Editor. If you want to run a different preset on the Clip Editor's contents, the Presets submenu immediate below it contains all the AutoSync Presets in your project. The Modules submenu then contains all the modules present in the project. Selecting any of these will open the AutoSync Window and add an instance of the chosen module to get you started. You can also choose the Open AutoSync Window option to simply open the window without adding any modules to it - for example if you want to load a preset and then customise it.
Finally, at the right-hand side of the menu are the settings & preview buttons. The Settings button will open the Settings Screen, while clicking the Preview button will bring up a drop-down menu listing all the available LipSync characters in the currently opened scene. Picking one of these characters will enabled Preview Mode, where the contents of the Clip Editor will be displayed on your chosen character in real time when playing or scrubbing along the timeline.
The Marker Mode Tabs can be used to switch between Phoneme, Emotion & Gesture Marker modes. The Clip Editor UI changes in a number of ways depending on which mode you have selected - including the options available in the top and bottom toolbars and, most importantly, the markers displayed along the Timeline.
In this mode, Phoneme Markers are shown, and only options relating to Phonemes are visible, including the Set Intensity From Volume option in the Edit menu, the Add Phoneme button and the Filters setting. Additionally, Default Marker Settings will show settings for Phoneme markers.
Phonemes can be moved around freely by dragging them along the timeline. Each marker also has a right-click menu which includes Copy and Delete options, a Change option for changing the phoneme a marker represents, Sustain Marker, which toggles sustain mode between two phoneme markers of the same type, and Marker Settings, which opens the Marker Settings Window.
In this mode, Emotion Markers are shown, and only options relating to Emotions are visible, including the Remove Missing Emotions option in the Edit menu and the Add Emotion button. Additionally, Default Marker Settings will show settings for Emotion markers.
Emotions can be moved around the timeline by dragging the main body of the marker, resized by dragging the grey handles at the start or end of the marker, or have their blend-in and blend-out times changed by dragging the blend handles inside the marker. Blend handles are invisible by default, and appear when hovered over:
If the start or end of a marker is dragged into another marker, the two markers will blend into each other, instead of blending back to a neutral pose. In this case, the blend is set automatically based on the overlap between the two markers.
Each marker also has a right-click menu which includes Copy and Delete options, a Change option for changing the emotion a marker represents (including the ability to convert it to an Emotion Mixer), and Marker Settings, which opens the Marker Settings Window.
In this mode, Gesture Markers are shown, and only options relating to Gestures are visible, including the Remove Missing Gestures option in the Edit menu and the Add Gesture button. The Default Marker Settings option is not available, as Gesture markers have no additional settings.
Gestures can be moved around freely by dragging them along the timeline. Each marker also has a right-click menu which includes Copy and Delete options, and a Change option for changing the gesture a marker represents.
In any mode, markers can be added to the timeline in one of two ways: the Add [Marker Type] button in the lower toolbar, which will add a marker at the current position of the playhead, and the Add Marker Here option found when right-clicking on the waveform, which will add the chosen marker at the position you right-clicked.
The Settings Screen houses various global options relating to LipSync Pro and the Clip Editor. Unlike the Project Settings asset, options here are stored in Unity's EditorPrefs and so are shared for all projects using the same Unity version. It is sorted into three tabs:
The General tab contains general Clip Editor settings
Continuous Update - improves the responsiveness of the Clip Editor by updating it every frame. On low-powered systems such as a netbook, you may want to disable this.
Load Transcripts from .Txt - if checked, the clip editor will automatically attempt to load in a transcript when the audio clip is changed or opened, by looking for a .txt file with the same name as the audio file.
Set Viewport on File Load - sets the size of the Clip Editor's viewport to show the entire clip when one is loaded.
Show Time Markers - toggles the time ruler display along the top of the waveform, just below the timeline.
Scrubbing Preview Length - controls how much of the audio clip is played (in seconds) when scrubbing (dragging the playhead along the waveform).
Preview Volume - controls the volume of audio played in the Clip Editor, including playback and scrubbing preview.
Show Extensions Window - whether or not to automatically open and dock the Extensions Window alongside the Clip Editor when it's opened.
Emotion Snapping - enables the "snapping" effect when Emotion markers are dragged or resized - making placing markers next to each other or the start/end of the clip easier.
Snapping Distance - controls the distance from other markers and the start/end of the clip that snapping will take place.
Use Emotion Colors - Whether or not to color Emotion markers according to their defined color in the Project Settings.
The AutoSync tab contains all options relating to AutoSync:
Default Preset - the Preset that is shown for quick access in the top level of the Clip Editor's AutoSync menu in the top bar.
SoX Sound Exchange Path - the path to the SoX Sound Exchange application used by AutoSync for converting, trimming and merging audio clips as required.
Additionally, each AutoSync Module can define its own settings which will appear here. For the Montreal Forced Aligner Module for instance, an Application Path option is exposed, which stores the path to the MFA application, the core component of the module.
Finally, the Keyboard Shortcuts tab allows for the viewing, creation and modification of the Clip Editor's various keyboard shortcuts. There is a default list which includes Copy/Paste, Save, Select All, etc. You can change the command and the keypress + modifier keys (Control, Shift, Alt) for any shortcut, and additionally add any additional ones from this screen.
