Welcome to the updated Overtone documentation for Unreal Engine 5 (UE5)! This guide will walk you through setting up and utilizing the Overtone plugin in UE5 to bring text-to-speech capabilities to your projects. We’ll cover the core features, setup process, and provide examples on how to use the different components to generate speech from text.

Overview

Overtone for UE5 introduces a powerful text-to-speech solution, supporting over 15 languages and offering multiple quality levels (X-LOW, LOW, MEDIUM, HIGH) to suit various application needs. With more than 900 unique voices available in the default English model (LibriTTS), Overtone is equipped to provide high-quality voice outputs for a wide range of uses, from gaming to mobile applications.

Downloading Voice Models

In UE5, you can download voice models directly through the Overtone plugin. This functionality is accessible via the Overtone Download Manager, which can be found under Window > Overtone > Download Manager. This interface allows for easy management and downloading of voice models to use within your projects.

Overtone Window

You can open the Overtone windows by going to Window > Overtone.

Overtone Download Manager

The Overtone Download Manager provides a convenient interface for downloading and removing voice models. You can see all the available models and download them. The interface is constantly updated with new models, so be sure to check back regularly for the latest additions.

API

Overtone provides a simple to use C++ or Blueprints API to generate text from speech.

UUndertoneRuntimeSubsystem

The UUndertoneRuntimeSubsystem serves as the central point for managing speech-to-text operations, including generating text from speech and managing models.

Reference

UFUNCTION(BlueprintCallable, Category = "Undertone")
UNeuralModelWrapper* LoadModelFor(const FModelMetadata& ModelIdentity) const;

UFUNCTION(BlueprintCallable, Category = "Undertone")
TArray<FModelMetadata> GetAllModelMetadatas() const;

UFUNCTION(BlueprintCallable, Category = "Undertone")
UNeuralModelWrapper* LoadModelByName(const FString& Name) const;

UFUNCTION(BlueprintCallable, Category = "Undertone")
FUndertoneResult UndertoneSpeechToText(const UNeuralModelWrapper* Model, const TArray<float>& Samples, const FString& Language, bool TranslateToEnglish, const FString& InitialPrompt);

Example usage: Getting all the models downloaded

UUndertoneRuntimeSubsystem* UndertoneRuntimeSubsystem = GetWorld()->GetGameInstance()->GetSubsystem<UUndertoneRuntimeSubsystem>();
/* Array of model metadatas */
TArray<FModelMetadata> AvailableModels = UndertoneRuntimeSubsystem->GetAllModelMetadatas();
/* Load the first model */
auto Model = UndertoneRuntimeSubsystem->LoadModelFor(AvailableModels[0]);

URealtimeSpeechTranscriberComponent

The URealtimeSpeechTranscriberComponent is a component that can be added to any actor to enable realtime transcription functionality. It provides a simple interface for generating text from speech.

Reference

UFUNCTION(BlueprintCallable, Category = "Undertone")
void SetModel(UNeuralModelWrapper* Model);

UFUNCTION(BlueprintCallable, Category = "Undertone")
UNeuralModelWrapper* GetModel() const;

UPROPERTY(BlueprintReadWrite, EditAnywhere, Category = "Undertone")
FString Language;

UPROPERTY(BlueprintReadWrite, EditAnywhere, Category = "Undertone")
bool TranslateToEnglish;

UFUNCTION(BlueprintCallable, Category = "Undertone")
void StartListening();

UFUNCTION(BlueprintCallable, Category = "Undertone")
void StopListening();

UPROPERTY(BlueprintAssignable, Category = "Undertone")
FOnTextTranscribed OnTextTranscribed;

UPROPERTY(BlueprintReadOnly, Category = "Undertone")
bool IsListening;

URecordedSpeechTranscriberComponent

Reference

UFUNCTION(BlueprintCallable, Category = "Undertone")
void SetModel(UNeuralModelWrapper* Model);

UFUNCTION(BlueprintCallable, Category = "Undertone")
UNeuralModelWrapper* GetModel() const;

UPROPERTY(BlueprintReadWrite, EditAnywhere, Category = "Undertone")
FString Language;

UPROPERTY(BlueprintReadWrite, EditAnywhere, Category = "Undertone")
bool TranslateToEnglish;

UPROPERTY(BlueprintReadOnly, EditAnywhere, Category = "Undertone")
bool IsRecording;
	
UFUNCTION(BlueprintCallable, Category = "Undertone")
void StartRecording();

UFUNCTION(BlueprintCallable, Category = "Undertone")
FText StopRecording();

Example usage: Transcribing speech

Include the following headers

#include "Subsystems/OvertoneRuntimeSubsystem.h"
#include "Components/OvertoneComponent.h"
#include "Structs/VoiceModelInformation.h"

Get all voice models and select the first one

UOvertoneRuntimeSubsystem* OvertoneRuntimeSubsystem = GetWorld()->GetGameInstance()->GetSubsystem<UOvertoneRuntimeSubsystem>();

TArray<FVoiceMetadata> AvailableVoices = OvertoneRuntimeSubsystem->GetAllVoiceMetadatas();
FVoiceMetadata SelectedVoiceMetadata = AvailableVoices[0];

Run text to speech and play the audio

UOvertoneComponent* OvertoneComponent = ...; // Get the component from the actor
OvertoneComponent->SetVoice(SelectedVoiceMetadata);
// UOvertoneComponent inherits an UAudioComponent
OvertoneComponent->TextToSpeech(TEXT("Hello, Unreal Engine 5!"));

Demos

The plugin contains a demo to demonstrate the text to speech functionality. You can input text, select a downloaded voice in the dropdown and listen to the output sound.

Supported Platforms

Overtone supports the following platforms:

If interested in any other platforms, please reach out.

Supported Languages

Troubleshooting

For any questions, issues or feature requests don’t hesitate to email us at help@leastsquares.io or join the discord. Very are happy to help and aim to have very fast response times :)

About us

We are a small company focused on building tools for game developers. Send us an email to careers@leastsquares.io if interested in working with us. For any other inquiries, feel free to contact us at hello@leastsquares.io or contact us on the discord