Multitrack Productions and Presets

Auphonic is optimized to process stereo or mono audio files of a final mix. However, many recordings consist of multiple tracks: speaker tracks from multiple microphones, music tracks, remote speakers via skype, mumble, etc. Until now, it was necessary to mixdown all tracks before processing the final mix with Auphonic, otherwise the results could be problematic.

With our special multitrack algorithms, it’s possible to upload all individual tracks of your recordings. Auphonic will process all tracks and creates the final mixdown automatically. Using knowledge from signals of all individual tracks allows us to create a new class of multitrack audio algorithms and improves our results.

See Multitrack Productions and Multitrack Post Production Algorithms for more details.
This page describes how to use the multitrack algorithms with our API.

Multitrack Productions

Multitrack Productions can be created in the same way as any other productions. The only difference is, that you have to specify multiple audio input files (multiple tracks) by using the parameter multi_input_files (or the corresponding separate request) as introduced in Adding Intro and Outro.

The following command creates a multitrack production with 2 tracks:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/productions.json \
    -u username:password \
    -d '{
            "multi_input_files": [
                {"type": "multitrack", "id": "speech track"},
                {"type": "multitrack", "id": "music track"}
            ],
            "metadata": { "title": "My first Multitrack Production" },
            "output_files": [{"format": "mp3"}],
            "is_multitrack": true
        }'

Parameters:

• is_multitrack: must be set to true for multitrack productions
• multi_input_files:

• type: every track of a multitrack production must set the type to multitrack (other possible types are intro or outro, see Adding Intro and Outro)
• id: a readable identifier for the current track

Uploading Audio Files

After the production is created, you can upload audio files for each track. This uses the same request as in Start a Production and Upload a File, but the the track id is used as the key for each file:

curl -X POST https://auphonic.com/api/production/{uuid}/upload.json \
    -u username:password \
    -F "speech track=@/home/user/Desktop/file-for-track1.wav" \
    -F "music track=@/home/user/Desktop/file-for-track2.wav"

This uploads the local file file-for-track1.wav to "speech track" and the file file-for-track2.wav to "music track" of our previous request.

Note

For uploading files, it’s necessary to encode your POST request in multipart/form-data, not JSON!

You can now start your production:

curl -X POST https://auphonic.com/api/production/{uuid}/start.json -u username:password

Uploading Files from External Services

Now we add 3 more tracks to our production: one from an External Service, one from an HTTP source and also an intro (type is set to intro):

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/production/{uuid}/multi_input_files.json \
    -u username:password \
    -d '[
            {
                "service": "pmefeNCzkyT4TbRbDmoCDf",
                "input_file": "my_dropbox_file.mp3",
                "type": "multitrack",
                "id": "speech track 2"
            },
            {
                "input_file": "http://your_server.com/speech_fritz.mp3",
                "type": "multitrack",
                "id": "speech track 3"
            },
            {
                "service": "pmefeNCzkyT4TbRbDmoCDf",
                "input_file": "intro_from_soundcloud.m4a",
                "type": "intro"
            }
        ]'

The synthax is the same as in Adding Intro and Outro.

Multitrack Audio Algorithm Settings

Audio algorithm settings can be specified for each track and for the master output file in the same way as it is done in the web interface - see Audio Tracks and Algorithms and Master Audio Algorithms.
If some parameters are not set, the default values are used.

Here is an example with audio algorithm settings for each track and for the master track:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/productions.json \
    -u username:password \
    -d '{
            "metadata": { "title": "Multitrack Production with Algorithms" },
            "output_files": [{"format": "mp3"}],
            "multi_input_files": [
                {
                    "type": "multitrack", "id": "speech track",
                    "algorithms": {"denoise": true, "denoiseamount": 12}
                },
                {
                    "type": "multitrack", "id": "music track",
                    "algorithms": {"hipfilter": false, "backforeground": "background"}
                }
            ],
            "algorithms":
                {
                    "loudnesstarget": -23,
                    "leveler": true,
                    "gate": true,
                    "crossgate": true
                }
        }'

Algorithms per track:

• hipfilter: Filters unnecessary and disturbing low frequencies depending on the context (speech, music, noise). Default is true.
• denoise: Automatic noise reduction in current track. Default is false.
• denoiseamount: Maximum noise reduction amount, higher values remove more noise. Default is 0 (= automatic).
• backforeground: You can force one track to be foreground, background (e.g. background music) or ducking (track will be reduced in volume if other speakers are active). If nothing is set, Auphonic will decide automatically which parts of your track should be in foreground or background (this should work most of the time).

Master algorithms:

• loudnesstarget: Loudness target in LUFS for Loudness Normalization, higher values result in louder audio outputs. Default is -16 LUFS.
• leveler: The Multitrack Adaptive Leveler corrects level differences between and within tracks. Default is true.
• gate: The Adaptive Noise Gate decreases background noise if a speaker is not active. Default is true.
• crossgate: Analyzes which speaker is active and decreases crosstalk/spill, ambience, reverb and noise recorded from other tracks. Default is true.

For details about our multitrack algorithms see Multitrack Post Production Algorithms.
See also (Single Track) Audio Algorithms API.

Getting back the Processed Tracks

By default, a multitrack production will create a final mixdown of all your input files. If you want to get back the individual, processed tracks, you have to add an additional tracks output format:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/productions.json \
    -u username:password \
    -d '{
            "metadata": { "title": "Multitrack Tracks Export" },
            "multi_input_files": [
                {
                    "type": "multitrack", "id": "speech track",
                    "algorithms": {"denoise": true, "denoiseamount": 12}
                },
                {
                    "type": "multitrack", "id": "music track",
                    "algorithms": {"hipfilter": false, "backforeground": "background"}
                }
            ],
            ...
            "output_files": [
                {"format": "tracks"},
                {"format": "tracks", "ending": "flac"}
            ]
        }'

Individual tracks can be exported as WAV (default) or as FLAC files. All tracks are zipped into one single file to download.

Adding offsets to Tracks

Each track in a multitrack production can have an offset, which determines when the track starts: an offset of 3.5 would start the track 3.5 seconds after the first track started.

Add the parameter offset, which must be a positive number and in seconds, to your tracks in the following way:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/production/{uuid}/multi_input_files.json \
    -u username:password \
    -d '[
            {
                "service": "pmefeNCzkyT4TbRbDmoCDf",
                "input_file": "my_dropbox_file.mp3",
                "type": "multitrack",
                "id": "speech track 2"
                "offset": 1.25
            },
            {
                "input_file": "http://your_server.com/speech_fritz.mp3",
                "type": "multitrack",
                "id": "speech track 3"
                "offset": 5.1
            }
            ,
            ...
        ]'

Multitrack Presets

The creation of multitrack presets is mostly equivalent to the creation of multitrack productions, with the exception that it’s not possible to add an input file for each track:

curl -X POST -H "Content-Type: application/json" \
    https://auphonic.com/api/presets.json \
    -u username:password \
    -d '{
            "preset_name": "My Multitrack Preset",
            "output_files": [{"format": "mp3"}],
            "multi_input_files": [
                {
                    "type": "multitrack", "id": "speech track",
                    "algorithms": {"denoise": true, "denoiseamount": 12}
                },
                {
                    "type": "multitrack", "id": "music track",
                    "algorithms": {"hipfilter": false}
                },
                {
                    "input_file": "http://your_server.com/intro.mp3",
                    "type": "intro", "id": ""
                },
            ],
            "algorithms":  {
                "loudnesstarget": -23,
                "leveler": true,
                "gate": false,
                "crossgate": true
            },
            "is_multitrack": true
        }'

See also Creation of Presets.

API Response of Multitrack Productions

curl https://auphonic.com/api/production/{uuid}.json -u username:password

returns the following response (some details are stripped):

{
    "status_code": 200,
    "form_errors": {},
    "error_code": null,
    "error_message": "",
    "data": {
        "status": 3,
        "status_string": "Done",
        "is_multitrack": true,
        "image": null,
        "creation_time": "2014-08-29T10:07:43.009Z",
        "chapters": [ ... ],
        "edit_page": "https://auphonic.com/engine/multitrack/edit/WMLwjD4sHJMJV5zEg6oaMh",
        "output_files": [ ... ],
        "uuid": "WMLwjD4sHJMJV5zEg6oaMh",
        "error_status": null,
        "thumbnail": null,
        "metadata": { ... },
        "warning_status": null,
        "webhook": null,
        "status_page": "https://auphonic.com/engine/status/WMLwjD4sHJMJV5zEg6oaMh",
        "length": 118.975986394558,
        "length_timestring": "00:01:58.975",
        "output_basename": "multitrack_test_production",
        "outgoing_services": [],
        "change_time": "2014-08-29T10:12:19.104Z",
        "error_message": "",
        "warning_message": "",
        "speech_recognition": null,
        "waveform_image": "https://auphonic.com/api/download/audio-result/WMLwjD4sHJMJV5zEg6oaMh/waveform.png",
        "multi_input_files": [
            {
                "input_channels": 1,
                "service": null,
                "input_file": "file_1.m4a",
                "input_samplerate": 48000,
                "algorithms": {
                    "hipfilter": true,
                    "denoise": false,
                    "denoiseamount": 0,
                    "backforeground": "auto"
                },
                "input_bitrate": 82.701,
                "input_filetype": "aac",
                "type": "multitrack",
                "id": "track 1",
                "input_length": 118.975986394558,
                "offset": 0
            },
            {
                "input_channels": 2,
                "service": null,
                "input_file": "File 2.mp3",
                "input_samplerate": 44100,
                "algorithms": {
                    "hipfilter": true,
                    "denoise": false,
                    "denoiseamount": 0,
                    "backforeground": "background"
                },
                "input_bitrate": 128.0,
                "input_filetype": "mp3",
                "type": "multitrack",
                "id": "track 2",
                "input_length": 91.6908616780045,
                "offset": 3.5
            }
        ],
        "statistics": {
            "tracks": [
                {
                    "music_speech": "speech",
                    "levels": {
                        "lra": [9.94, "LU"],
                        "gain_mean": [-25.23, "dB"],
                        "noise_level": [-76.85, "dB"],
                        "max_momentary": [-27.23, "LUFS"],
                        "signal_level": [-32.52, "dB"],
                        "gain_min": [-98.86, "dB"],
                        "snr": [44.33, "dB"],
                        "max_shortterm": [-31.27, "LUFS"],
                        "loudness": [-34.12, "LUFS"],
                        "gain_max": [17.01, "dB"]
                    },
                    "identifier": "track 2",
                    "activity": [[3.32, 7.73], [12.08, 20.67], [22.22, 28.87], ...]
                },
                {
                    "music_speech": "speech",
                    "levels": {
                        "lra": [3.52, "LU"],
                        "gain_mean": [-1.52, "dB"],
                        "noise_level": [-49.72, "dB"],
                        "max_momentary": [-10.58, "LUFS"],
                        "signal_level": [-16.2, "dB"],
                        "gain_min": [-61.3, "dB"],
                        "snr": [33.52, "dB"],
                        "max_shortterm": [-15.51, "LUFS"],
                        "loudness": [-17.64, "LUFS"],
                        "gain_max": [2.07, "dB"]
                    },
                    "identifier": "track 1",
                    "activity": [[0.11, 3.78], [7.93, 15.23], ...]
                }
            ],
            "master": {
                "levels": {
                    "lra": [3.04, "LU"],
                    "max_momentary": [-11.44, "LUFS"],
                    "peak": [-1.0, "dBTP"],
                    "max_shortterm": [-14.19, "LUFS"],
                    "loudness": [-16.0, "LUFS"]
                }
            }
        },
        "algorithms": {
            "leveler": true,
            "loudnesstarget": -16,
            "crossgate": true,
            "gate": true
        },
        "start_allowed": false,
        "change_allowed": true
    }
}

Multitrack specific parameters:

• is_multitrack: true for multitrack production
• length: output audio length in seconds (longest track with intro/outro) - IMPORTANT: if you have intro or outro files, they are included in this length!
• length_timestring: parameter length in HH:MM:SS.mmm
• multi_input_files: audio file details, identifier and algorithms for each track
• statistics: level, activity, music-speech, noise reduction (and more) statistics per track and for the master track of multitrack productions

For more information about all the other parameters please see Details about a Production.