2. Ten Years of Keeping People Working [At Bungie]
Mat Noguchi
Bungie Studios

3. Who am I? Halo: Combat Evolved Halo 2 Halo 3 Halo 3: ODST Halo: Reach
I’m Mat Noguchi, senior programmer at Bungie studios
Past ten years, I’ve worked on a few games you may have heard of.
And what is my job? Making sure all the work everyone does ends up in a disk on your box.
Why am I always so angry? Or at the very least annoyed?

4. Why so angry? It’s my job Halo is big and awesome
(I’m not actually that angry. Maybe stressed.)
Halo is big and awesome
Halo is literally big and awesome
Halo: Reach == 30 Halo: CEs!
Still has to fit on a DVD
And we never actually planned for this
It’s my job to take all the content that gets produced and to make sure it gets on a disk in your hands that you can pop in and play
Big and awesome: millions of copies sold and millions of bungie fans
Big and awesome: asset size (huge monstrosity!) [expand out] It evolved over ten years without a long term plan. No one know knew we would make Halo for ten years until ten years later.
And the size of disk that it ships on?
We never planned for the sequel to be at least twice as big as the previous game.
Even the hub level in ODST dwarfed anything we had built for Halo 3.
Painful to scale up the same engine over ten years.
[fixed platform, existing engine]

5. Well known bottlenecks
Disk/storage footprint
DVD or BluRay or downloadable package
Memory
512 MB for current generation
I/O
1 or 2 devices, limited bandwidth
Internet (much more limited bandwidth)
Scaling content (runtime) is a big deal; The secret is know your bottlenecks and execute accordingly!
These are the known bottlenecks

6. It’s also about the people!
Someone has to make all that content
Someone has to program the game to use that content
And everyone has to make it awesome

7. It starts with programmers
Programmers should only work with content in a simple way
Not as raw memory
Not as files
Not as I/O
Otherwise, they will mess it up.
Even me. (DOH!)
And it starts with programmers. It has to, since programmers write code for the features that use content.
Ultimately, what we want to do is commoditize content for programmers. In other words, if we can get programmers to think and write code in terms of content rather than in terms of specific aspects of content, like files and memory and I/O, we can move the responsibility for managing all those bottlenecks from a majority of the programmers to a few (or one) programmer. And if we can centralize content management, we can actually have a reasonable chance of tacking those bottlenecks.
Case in point, I forgot the right way to do asynchronous I/O on windows between Halo 1 and Halo 3. [Insert funny story about mat being an idiot here.]

8. content == tags Programmers think in terms of tags
Containers, not memory
Dependencies, not files
Don’t think about I/O at all!
Programmers think about content in terms of tags, not memory or file footprint or I/O.
Easier to think in terms of tags than to manage all those systems separately. Plus, I work very hard to prevent that anyway.

9. Tag system: Halo 1 Features API Tag build (development)
Hot loading for all tags
Cache build (ship)
Optimized memory layout
Monolithic file
long tag_load( const char *name, tag group_tag, dword flags);
void *tag_get( long tag_index);
In halo 1, load tag into handle, use tag_get, “
The api is stable [blow some smoke]

10. Tag system: Halo 2 More Features API Tag build Cache build
Append only versioning
Cache build
Automatic cross-map sharing
Automatic immutable data sharing
long tag_load( const char *name, tag group_tag, dword flags);
void *tag_get( long tag_index);
In halo 1, load tag into handle, use tag_get, “
The api is stable [blow some smoke]

11. Tag system: Halo 3 Even More Features API Tag build Cache build
Automatic and explicit tag versioning
Arbitrary paging support
Monolithic files on devkit
Cache build
Whole disk optimizations
Can run without HDD
long tag_load( const char *name, tag group_tag, dword flags);
void *tag_get( long tag_index);
In halo 1, load tag into handle, use tag_get, “
The api is stable [blow some smoke]

12. More technical details
The Technology of Halo 2 (2004) – Chris Butcher
Content Management for Halo 2 and Beyond (2005) – Mat Noguchi
New Dog, Old Tricks: Running Halo 3 without a Hard Drive (2008) – Mat Noguchi
I could go into some depth about how it works, but there’s already a bunch of info out there that does a much better job explaining this.
These are older papers about the tag system from Halo 2, but they are still relevant; [the underlying issues of managing content haven’t really changed that much, they’ve just gotten bigger.]

13. It’s simple! Every in-game feature is driven by tags!
Encourages communication!
It’s expected that features can be driven by tags. In fact, at Bungie, we don’t talk about content or assets, we talk about tags.
Because we only have one way of loading content (through the tag system), when a programmer (or anyone for that matter) wants to do something with content that isn’t immediately obvious, it forces a conversation to figure out what to do: in most cases the conversation is simple: either how to do it, or why not to do it.

14. PORKCHOP SANDWICHES!
But occasionally you get the conversations that give you headaches and bad dreams. These are the conversations you really care about, because they give you tremendous insights into your content pipeline. At the cost of a few or many gray hairs.

15. Shaders tags, my nemesis
19 shader types
The HLSL is tag driven (woo )
310,718,936 permutations
Only use 1,659 (boo )
Can generate on-demand
Usually several seconds to a minute
But we can’t cache them on the devkit
More than 10 and artists get mad. Really mad.
Build shaders periodically on our build farm
Life on the Bungie Farm … - Luis Villegas and Sean Shypula)
Let’s not have to do that next time.
One particular pain in my ass has been our shader system. We can’t magically get rid of compiling shaders, so we manually build all the shaders referenced by a shippable level on our build farm. We have to run this on a semi regular basis or whenever a programmer made changes that invalidate shaders so that our load times for our development build won’t suffer.
Even then, we still can have longer load times any time an artist makes a shader that has never been compiled. That tends to happen a lot when an artist is experimenting.
Obviously, in the future, we should have better support for data generated from tags.

16. Personal growth From 2002-ish to 2007 Level editor Tag system
Tag editor
Audio
From about a third of the way through Halo 2 up until the end of Halo 3, I was responsible for not just for the level editor, for which I was originally hired, but also the tag system, the tag editor, and audio. Clearly this is way too much for a single person to handle, so we hired another programmer to help me out with our editors, but by the time we finished Halo 3, I was crumbling under the weight of all these systems. Physically and psychologically.
And while I was having this existential crisis, something a little bit more important happened
As teams grow, shouldn’t own so many systems,
Life changes

17. But then…
In early 2007, I fell asleep. And someone put this baby in my arms.

18. And now…
This is my son today; this is when he found out that I had to crunch that day.
And just so they aren’t left out, this is my wife and brand new baby daughter. Watching fireworks.

19. Today I own Other people own Tools team Tag system Level editor Audio
Tag editor
So at the end of Halo 3, while it was still possible to maintain all these systems by myself, it was also obvious that we shouldn’t do that anymore; not just for my personal sanity but so that we could actually have individuals and teams fully focused on all these areas.
And one team that really stepped up to the plate is our tools team.
Initially, I balked at the idea of handing our tag editor over to the tools team. [insert long diatribe]
Because programmers write code in terms of tags, we can use the tag system as a backend for our editors.
And our tools team has taken that idea to a whole new level.

20. Tag editor evolved: Bonobo
This is the new hotness for editing tags. Aside from the spiffy WPF driven GUI, the tools team has added some pretty nifty features that leverage the power of the tag system.

21. Content explorer
This is the tag relationship viewer; it shows the ancestry above and below a given tag by iterating over all the references with a tag recursively. This is extremely useful for determining how content can get into a particular level.

22. Quick preview
And while I can’t show this particular feature in action, it’s one of the features that is just so fucking cool. The quick preview button does a very fast tag synchronization with the devkit; it’s basically an uber-fast file sync, so turnaround can be usually within 1-5 seconds or so. The Use Live Mode button does something similar, but instead of syncing the entire file, it synchronizes individual changes live (duh), so instead of 1-5 second turnarounds we get down to however fast we can send data over the network and process it on the devkit. It’s fast enough to be interactive. And it’s based on our tag system, it can be used for pretty much any tag we have [, except for tags that represent bulk binary data like sound and animations].

23. Mantini

24. Why this matters It’s not just about the code
all content code has the same structure

25. Content code structure
Boost
Tag system (pseudocode)
template<class Archive> void serialize( Archive & ar, gps_position & g, const unsigned int version) { ar & g.degrees; ar & g.minutes; ar & g.seconds; }
TAG_GROUP( gps_position) { {_field_long_integer, “degrees”}, {_field_long_integer, “minutes”}, {_field_real, “seconds”}, {_field_terminator}, };

26. Why this matters It’s not just about the code
All content code has the same structure
More about what you can’t do

27. Why this matters It’s not just about the abstractions
It’s important to have them
Ideal if they map to a decent implementation
Even better if you don’t have to ship them

28. Why this matters
It’s about the culture Bungie built around the tag system
“You forgot the part where you need to stand behind the rest of the team with a baseball bat.“ – Andy Firth
The bat- If it’s not a tag, it’s not in the game
But that’s okay, because it’s simple to use!

29. Per audacia ad astra!
Thank you