1. Building a UE4 Plugin
FMOD STUDIO Case Study

2. Motivation – Why make a plugin?
Its easy
Reuse common components
Marketplace
$$$

3. FMOD Studio
FMOD Studio provides rich and powerful tools for the sound designer. This is the event view. FMOD Events can be a single sound, or they can be a like this music track with transitions quantized to the beat. Events can have multiple tracks, each with their own DSP effects. They have parameters that can trigger sounds or automate volume, pitch or other values.

4. FMOD Studio
FMOD Studio has a mixer hierarchy of buses. Buses can have effects, and can send signal to other buses in the hierarchy. FMOD Studio has mixer snapshots which can affect the mix. A simple example is ducking the volume of a bus, say when entering the main menu. Or snapshots could modify DSP parameters.

5. FMOD Studio
FMOD Studio supports connecting to the game as it is running and performing live editing. FMOD can capture profiler sessions which can be used to see what is going on at a particular part of the game.

6. FMOD Studio Industry leading audio middleware
Easy and powerful authoring tools
Free for indies ($100K or less)
And it is free for indies!

7. Unreal Integration Previous UE3 Integration UE4 Integration Plan
Deep integration with engine code
Long time to development
UE4 Integration Plan
Basic version and iterate on it
Estimate 2 weeks for first version
Open questions
Can it be done as a pure plugin?
Content window/asset integration?
FMOD has had previous experience integrating with Unreal. FMOD Designer was integrated with UE3. That took a long while to release and changed lots of engine code. That meant it was also a support burden as the code had to be kept up to date with Unreal. The goals for the UE4 integration was a quick turnaround time to iterate on the plugin, and to do it as a pure plugin if possible.
There was an question of how much could be done as a plugin, particularly when it comes to the content window.

8. Plugin Basics FMODStudio plugin FMOD Studio module FMOD Studio Editor
YourGame module
Unreal 4 Engine
UE4 Android module
UE4
Core module
UE4
Slate module
UE4
XmlParser module
A plugin is essentially one or more modules. In Unreal, everything is a module. If you go into Unreal and create a new code project, it is setting up the game as a module. And the engine itself is a series of modules. The Unreal engine has over 300 modules.
For FMOD Studio we have two modules: FMODStudio and FMODStudioEditor. We follow the same pattern as Paper2D and other plugins where they put their editor code in a separate module. When you compile for the editor, it will use both modules. When you compile for the game, it will just use the first module and not the editor one. This is cleaner than using #ifdef WITH_EDITOR everywhere.

9. Plugin Basics
Plugins get built by the Unreal Build Tool, also called UBT. It looks for plugins under every folder of the engine plugins, and also in the game plugins directory. Some plugins recommend installing the engine, others the game. We chose to tell users to install under engine since we figure that they will want to use FMOD Studio across multiple games for UE4.

10. Plugin Basics
The .uplugin file is a simple text file, in the JSON format. It just has the name of the plugin, some extra description, and what modules the plugin has.

11. Plugin Basics
If you have used UE4, then you will have seen the typical folder structure of modules with Classes, Private, Public, and a .Build.cs file. UBT is a C# app that loads up the Build.cs file and dynamically instantiates a class.
It has rules for building the module. In our case we have bFasterWithoutUnity. That is turning off the Unity builds, also called Uber build, where several .cpp files are combined together. That can speed up compilation but it can hide errors, and we want to know if we’ve missed an include file.
The file also says where the include files are for the module and what inputs the module depends on.

12. Plugin Basics
Once you’ve got the plugin files in there, if you build and then run UE4 editor then the plugin will appear.

13. Module Class Macro to register module
Virtual functions when module loaded/unloaded
Each plugin needs a class that derives from IModuleInterface. There are virtual functions for Startup and Shutdown module, and that macro which has some extra magic to get the module working.

14. Basic Asset Class
This is a basic class which should be familiar to anybody who has programmed with UE4. It derives from UObject and there are several macros for UCLASS, GENERATED_UCLASS_BODY, and UPROPERTY.
One thing to note is the FMODSTUDIO_API. That means this class exports its symbols out of the module so they can be used by other modules. The uses the name of the module. So if you had a plugin with a module called MySuperGui, then there would be a define you can use called MYSUPERGUI_API.

15. Settings Class
This is a class that holds settings. UE4 like UE3 has settings that come out of an ini file but it has been wrapped in a really nice way. This class again derived from UObject. In the UCLASS macro there is “config = Engine, defaultconfig”. That tells Unreal that the settings will be stored in the engine ini file and will be per project, not per user. If you had, say, an editor extension module then you might want the settings per user in which case you wouldn’t have the defaultconfig part.
It has some fields with UPROPERTY and it specifies Category = Basic, which means they’ll all get grouped into a “Basic” section. For the 4th parameter we really just need a string. FDirectoryPath is a struct that has a string in it. Slate has been customized to draw that type slightly differently to a normal string. Furthermore that customization is itself customized by that meta field in the UPROPERTY that tells the customization where the path should be based on.

16. Settings Class
The settings class then has to be registered with the Settings module. Here is the settings view in the editor. It already has typical defaults so they user doesn’t have to set anything up to get FMOD running with UE4.
At runtime to get the ini settings you just get the default instance of the class and it will have the settings already set up. So there is no explicit reading or writing of ini file settings, it is all done for you.

17. Blueprints
Its worth mentioning how simple Blueprints are to hook up. We originally thought that we’d leave blueprints for subsequent version since we expected it may take a while to get them integrated.
But its actually just a one liner. You just add BlueprintCallable and the function is exposed to blueprint. Blueprint knows how to deal with the different types such as an asset pointer, or a bool or float. It works if you create a struct, you get a new pin and you can drag off and get new blueprint actions to set each field of the struct up.
It also works with opaque pointers to classes, even ones Blueprint doesn’t know about. If you return a pointer to a opaque class you get a pin that you can use as an argument for a subsequent function.
Its all really easy to get up and running.

18. Blueprints
Here is the LoadBank function. If you hover, its pulled the C++ comment out and is displaying that for extra information.

19. Sound Banks .xml files FMOD Studio .bank files .uasset files UE4
Cooked .pak
We ran into some complications with the FMOD Studio plugin. The biggest conceptual one is working with banks. When using FMOD Studio, it is reading and writing to project xml files. Then when you export, it packs a number of events into a binary file called a bank.
UE4 works by importing individual assets and storing them on disk as a folder of uassets. This is similar to how Unity works. When you cook, it takes those individual assets and exports them for the platform, and usually ends up bundling everything together in a pak file.

20. Sound Banks Choices Bank with uasset and GUID references
Dummy uassets on disk
Dummy uassets in memory
Explode bank into individual files and import
We had several choices when it comes to dealing with assets. One way is to import the bank as the real asset type and then use GUID references everywhere for events. But the interface for GUIDs is not very good and slate customization can only do so much.
Another way is to store lots of dummy uassets on disk, each one just having a reference to an event. This is what the FMOD plugin does for Unity, and it works fine. It does mean there are lots of extra files on disk though which have to be added/removed from source control whenever the banks change.
A third way is to have those dummy assets created purely in memory, and not have any extra files on disk. We found we could do this, so that’s what we do for UE4.
A last way is to change Studio exporting not to write out bank files any more, and instead export lots of individual assets that get imported into UE4. We didn’t go down that road for a few reasons. One thing is that would slow down the UE4 plugin because we’d have to look into a whole different way of exporting from Studio. Another is that we’d have quite a complex graph of assets. We have events that feed into buses, buses that feed into other buses, mixer snapshots that operate on a bunch of things. If it was imported into UE4 it would be a fragile network of assets with references that might run into problems especially next time they export from Studio if they made any changes.

21. Sound Banks Create dummy assets on demand in memory The good: The bad:
Assets as first class citizens
They appear in the content window
Blueprints just work
The bad:
Not the recommended approach!
Serialized by name not Guid
Not read-only in content window, but they should be
So we create assets on demand in memory. This isn’t the typical workflow for a plugin, so its not something you are likely to do. It also doesn’t solve the issue of the fact that assets in Unreal are serialized by name not GUID. That means if the user has to be careful not to rename their events once they start getting used in game. There doesn’t appear to be any way to solve that as a plugin. There is a LazyObjectPtr type that comes close but it doesn’t do what we want.
Also these dummy assets aren’t read only but it doesn’t make sense for the user to move them around or rename them. It would be good if they were read-only but we can’t see a way of doing that as a plugin.

22. Sound Banks
This is the end result. As far as the user is concerned it’s a really good workflow. You add events, re-export your banks and they will appear in the content window automatically. They can be dragged and dropped into the 3d window or the blueprint view and does what you would expect.

23. Deployment Building from GitHub/P4 Downloaded UE4 binary Code project
DLL deployment issues
Wrong static library name
Plugin version must match Epic’s
Blueprint-only project
Plugin won’t link
There are some issues you might run into with deployment. The first issue is if your plugin itself needs to load a .dll file (or .dylib or .so). I’ll talk about that in a bit.
You get extra problems if the user has downloaded the UE4 binary rather than building it from github. The library name isn’t what it expects, and you’ll have issues with the version.
If the user has downloaded the UE4 binary and is doing a blueprint-only project then there are all the above issues but also plugins just don’t link. There isn’t anything we can do about this but we can workaround the others.
Epic is aware of these issues and they should be fixed fairly soon, but they aren’t fixed in 4.5 and may not be fixed in 4.6 either.

24. Deployment - names Win64 plugin outputs: Name is expected to be:
UE4Editor-FMODStudio.dll
UE4Editor-FMODStudioEditor.dll
UE4Game-FMODStudio-Static.lib
Name is expected to be:
FMODStudio-Static.lib
Here are the output files on Win64. For editor builds there are the two dlls, and for the game there is just the single module as a static library. But the downloaded version of UE4 builds and expects plugins of the shorter name. It seems to be a “rocket” command line setting that determines this. So we just rename our library.

25. Deployment - plugin version
Plugin version number
Baked into DLL
Must match exactly
Defines
BUILT_FROM_CHANGELIST
MODULE_API_VERSION
Each plugin DLL has a version number that has to match the engine version number. If you build UE4 and plugins from github, then you’ll get version number 0. But users who download UE4 get some large number instead.
What happens is Epic has a rolling BUILT_FROM_CHANGELIST number that then gets set into MODULE_API_VERSION, which is then baked onto the DLL. So you need to modify your ModuleVersion.h and set the version. You’ll see Epic do the same thing for their patch releases.

26. Deployment – DLL deployment
External DLLs
Third party directory
Packaging
Copies third party DLLs into destination directory
Android has additional requirements
.jar file needs to be linked in
.so files need to be loaded by main java Activity
How does it work for plugins?
It doesn’t
The biggest issue we have is with the FMOD DLLs. There is a third party directory for DLLs and we put our FMOD ones there too. When you launch the build or package it, those DLLs need to come along for the ride. But there is a hard-coded list of steamworks, vorbis, PhysX etc. There isn’t any way for a plugin to hook into them.
Android is even worse since we have a .jar file and the .so files need to be explicitly loaded by a java activity.
So we have a workaround per platform. Mac and IOS just work. For Windows you need to copy the DLLs manually. For Android you’ll need a small engine change or to add the .jar and .so stuff into the game yourself – we can provide details per support case but so far nobody has asked yet.

27. Marketplace plugin support
Coming soon*!
* Not sure what soon means
So the marketplace doesn’t support plugins yet but it will at some point. But there are already lots of cool plugins out there, you can check out the UE4 forums for them. Examples are the Kinect Integration, the RadiantUI plugin, a C# plugin which has just been released, the Razer Hydra plugin (which also needs its own DLL).

28. Demo/Questions/Comments
FMOD Tutorial videos (YouTube FMODTV)
FMOD Q/A (
FMOD Support
Questions?
We’ve got a series on using FMOD and UE4 up on youtube, which content and walkthrough by professional sound designer Sally-anne Kellaway. And we’ve got our Q/A database and private support too.