Building a UE4 Plugin FMOD STUDIO Case Study

Motivation – Why make a plugin? Its easy Reuse common components Marketplace $$$

FMOD Studio FMOD Studio provides rich and powerful tools for the sound designer. This is the event view. FMOD Events can be a single sound, or they can be a like this music track with transitions quantized to the beat. Events can have multiple tracks, each with their own DSP effects. They have parameters that can trigger sounds or automate volume, pitch or other values.

FMOD Studio FMOD Studio has a mixer hierarchy of buses. Buses can have effects, and can send signal to other buses in the hierarchy. FMOD Studio has mixer snapshots which can affect the mix. A simple example is ducking the volume of a bus, say when entering the main menu. Or snapshots could modify DSP parameters.

FMOD Studio FMOD Studio supports connecting to the game as it is running and performing live editing. FMOD can capture profiler sessions which can be used to see what is going on at a particular part of the game.

FMOD Studio Industry leading audio middleware Easy and powerful authoring tools Free for indies ($100K or less) And it is free for indies!

Unreal Integration Previous UE3 Integration UE4 Integration Plan Deep integration with engine code Long time to development UE4 Integration Plan Basic version and iterate on it Estimate 2 weeks for first version Open questions Can it be done as a pure plugin? Content window/asset integration? FMOD has had previous experience integrating with Unreal. FMOD Designer was integrated with UE3. That took a long while to release and changed lots of engine code. That meant it was also a support burden as the code had to be kept up to date with Unreal. The goals for the UE4 integration was a quick turnaround time to iterate on the plugin, and to do it as a pure plugin if possible. There was an question of how much could be done as a plugin, particularly when it comes to the content window.

Plugin Basics FMODStudio plugin FMOD Studio module FMOD Studio Editor YourGame module Unreal 4 Engine UE4 Android module UE4 Core module UE4 Slate module UE4 XmlParser module A plugin is essentially one or more modules. In Unreal, everything is a module. If you go into Unreal and create a new code project, it is setting up the game as a module. And the engine itself is a series of modules. The Unreal engine has over 300 modules. For FMOD Studio we have two modules: FMODStudio and FMODStudioEditor. We follow the same pattern as Paper2D and other plugins where they put their editor code in a separate module. When you compile for the editor, it will use both modules. When you compile for the game, it will just use the first module and not the editor one. This is cleaner than using #ifdef WITH_EDITOR everywhere.

Plugin Basics Plugins get built by the Unreal Build Tool, also called UBT. It looks for plugins under every folder of the engine plugins, and also in the game plugins directory. Some plugins recommend installing the engine, others the game. We chose to tell users to install under engine since we figure that they will want to use FMOD Studio across multiple games for UE4.

Plugin Basics The .uplugin file is a simple text file, in the JSON format. It just has the name of the plugin, some extra description, and what modules the plugin has.

Plugin Basics If you have used UE4, then you will have seen the typical folder structure of modules with Classes, Private, Public, and a .Build.cs file. UBT is a C# app that loads up the Build.cs file and dynamically instantiates a class. It has rules for building the module. In our case we have bFasterWithoutUnity. That is turning off the Unity builds, also called Uber build, where several .cpp files are combined together. That can speed up compilation but it can hide errors, and we want to know if we’ve missed an include file. The file also says where the include files are for the module and what inputs the module depends on.

Plugin Basics Once you’ve got the plugin files in there, if you build and then run UE4 editor then the plugin will appear.

Module Class Macro to register module Virtual functions when module loaded/unloaded Each plugin needs a class that derives from IModuleInterface. There are virtual functions for Startup and Shutdown module, and that macro which has some extra magic to get the module working.

Basic Asset Class This is a basic class which should be familiar to anybody who has programmed with UE4. It derives from UObject and there are several macros for UCLASS, GENERATED_UCLASS_BODY, and UPROPERTY. One thing to note is the FMODSTUDIO_API. That means this class exports its symbols out of the module so they can be used by other modules. The uses the name of the module. So if you had a plugin with a module called MySuperGui, then there would be a define you can use called MYSUPERGUI_API.

Settings Class This is a class that holds settings. UE4 like UE3 has settings that come out of an ini file but it has been wrapped in a really nice way. This class again derived from UObject. In the UCLASS macro there is “config = Engine, defaultconfig”. That tells Unreal that the settings will be stored in the engine ini file and will be per project, not per user. If you had, say, an editor extension module then you might want the settings per user in which case you wouldn’t have the defaultconfig part. It has some fields with UPROPERTY and it specifies Category = Basic, which means they’ll all get grouped into a “Basic” section. For the 4th parameter we really just need a string. FDirectoryPath is a struct that has a string in it. Slate has been customized to draw that type slightly differently to a normal string. Furthermore that customization is itself customized by that meta field in the UPROPERTY that tells the customization where the path should be based on.

Settings Class The settings class then has to be registered with the Settings module. Here is the settings view in the editor. It already has typical defaults so they user doesn’t have to set anything up to get FMOD running with UE4. At runtime to get the ini settings you just get the default instance of the class and it will have the settings already set up. So there is no explicit reading or writing of ini file settings, it is all done for you.

Blueprints Its worth mentioning how simple Blueprints are to hook up. We originally thought that we’d leave blueprints for subsequent version since we expected it may take a while to get them integrated. But its actually just a one liner. You just add BlueprintCallable and the function is exposed to blueprint. Blueprint knows how to deal with the different types such as an asset pointer, or a bool or float. It works if you create a struct, you get a new pin and you can drag off and get new blueprint actions to set each field of the struct up. It also works with opaque pointers to classes, even ones Blueprint doesn’t know about. If you return a pointer to a opaque class you get a pin that you can use as an argument for a subsequent function. Its all really easy to get up and running.

Blueprints Here is the LoadBank function. If you hover, its pulled the C++ comment out and is displaying that for extra information.

Sound Banks .xml files FMOD Studio .bank files .uasset files UE4 Cooked .pak We ran into some complications with the FMOD Studio plugin. The biggest conceptual one is working with banks. When using FMOD Studio, it is reading and writing to project xml files. Then when you export, it packs a number of events into a binary file called a bank. UE4 works by importing individual assets and storing them on disk as a folder of uassets. This is similar to how Unity works. When you cook, it takes those individual assets and exports them for the platform, and usually ends up bundling everything together in a pak file.

Sound Banks Choices Bank with uasset and GUID references Dummy uassets on disk Dummy uassets in memory Explode bank into individual files and import We had several choices when it comes to dealing with assets. One way is to import the bank as the real asset type and then use GUID references everywhere for events. But the interface for GUIDs is not very good and slate customization can only do so much. Another way is to store lots of dummy uassets on disk, each one just having a reference to an event. This is what the FMOD plugin does for Unity, and it works fine. It does mean there are lots of extra files on disk though which have to be added/removed from source control whenever the banks change. A third way is to have those dummy assets created purely in memory, and not have any extra files on disk. We found we could do this, so that’s what we do for UE4. A last way is to change Studio exporting not to write out bank files any more, and instead export lots of individual assets that get imported into UE4. We didn’t go down that road for a few reasons. One thing is that would slow down the UE4 plugin because we’d have to look into a whole different way of exporting from Studio. Another is that we’d have quite a complex graph of assets. We have events that feed into buses, buses that feed into other buses, mixer snapshots that operate on a bunch of things. If it was imported into UE4 it would be a fragile network of assets with references that might run into problems especially next time they export from Studio if they made any changes.

Sound Banks Create dummy assets on demand in memory The good: The bad: Assets as first class citizens They appear in the content window Blueprints just work The bad: Not the recommended approach! Serialized by name not Guid Not read-only in content window, but they should be So we create assets on demand in memory. This isn’t the typical workflow for a plugin, so its not something you are likely to do. It also doesn’t solve the issue of the fact that assets in Unreal are serialized by name not GUID. That means if the user has to be careful not to rename their events once they start getting used in game. There doesn’t appear to be any way to solve that as a plugin. There is a LazyObjectPtr type that comes close but it doesn’t do what we want. Also these dummy assets aren’t read only but it doesn’t make sense for the user to move them around or rename them. It would be good if they were read-only but we can’t see a way of doing that as a plugin.

Sound Banks This is the end result. As far as the user is concerned it’s a really good workflow. You add events, re-export your banks and they will appear in the content window automatically. They can be dragged and dropped into the 3d window or the blueprint view and does what you would expect.

Deployment Building from GitHub/P4 Downloaded UE4 binary Code project DLL deployment issues Wrong static library name Plugin version must match Epic’s Blueprint-only project Plugin won’t link There are some issues you might run into with deployment. The first issue is if your plugin itself needs to load a .dll file (or .dylib or .so). I’ll talk about that in a bit. You get extra problems if the user has downloaded the UE4 binary rather than building it from github. The library name isn’t what it expects, and you’ll have issues with the version. If the user has downloaded the UE4 binary and is doing a blueprint-only project then there are all the above issues but also plugins just don’t link. There isn’t anything we can do about this but we can workaround the others. Epic is aware of these issues and they should be fixed fairly soon, but they aren’t fixed in 4.5 and may not be fixed in 4.6 either.

Deployment - names Win64 plugin outputs: Name is expected to be: UE4Editor-FMODStudio.dll UE4Editor-FMODStudioEditor.dll UE4Game-FMODStudio-Static.lib Name is expected to be: FMODStudio-Static.lib Here are the output files on Win64. For editor builds there are the two dlls, and for the game there is just the single module as a static library. But the downloaded version of UE4 builds and expects plugins of the shorter name. It seems to be a “rocket” command line setting that determines this. So we just rename our library.

Deployment - plugin version Plugin version number Baked into DLL Must match exactly Defines BUILT_FROM_CHANGELIST MODULE_API_VERSION Each plugin DLL has a version number that has to match the engine version number. If you build UE4 and plugins from github, then you’ll get version number 0. But users who download UE4 get some large number instead. What happens is Epic has a rolling BUILT_FROM_CHANGELIST number that then gets set into MODULE_API_VERSION, which is then baked onto the DLL. So you need to modify your ModuleVersion.h and set the version. You’ll see Epic do the same thing for their patch releases.

Deployment – DLL deployment External DLLs Third party directory Packaging Copies third party DLLs into destination directory Android has additional requirements .jar file needs to be linked in .so files need to be loaded by main java Activity How does it work for plugins? It doesn’t The biggest issue we have is with the FMOD DLLs. There is a third party directory for DLLs and we put our FMOD ones there too. When you launch the build or package it, those DLLs need to come along for the ride. But there is a hard-coded list of steamworks, vorbis, PhysX etc. There isn’t any way for a plugin to hook into them. Android is even worse since we have a .jar file and the .so files need to be explicitly loaded by a java activity. So we have a workaround per platform. Mac and IOS just work. For Windows you need to copy the DLLs manually. For Android you’ll need a small engine change or to add the .jar and .so stuff into the game yourself – we can provide details per support case but so far nobody has asked yet.

Marketplace plugin support Coming soon*! * Not sure what soon means So the marketplace doesn’t support plugins yet but it will at some point. But there are already lots of cool plugins out there, you can check out the UE4 forums for them. Examples are the Kinect Integration, the RadiantUI plugin, a C# plugin which has just been released, the Razer Hydra plugin (which also needs its own DLL).

Demo/Questions/Comments FMOD Tutorial videos (YouTube FMODTV) FMOD Q/A (www.fmod.com/questions) FMOD Support (support@fmod.com) Questions? We’ve got a series on using FMOD and UE4 up on youtube, which content and walkthrough by professional sound designer Sally-anne Kellaway. And we’ve got our Q/A database and private support email too.