game programming – Random Musings of Miller

Writing games and writing platforms..

Much like my love of fiction books, I have been a gamer for basically as long as I’ve been able to be. I have no idea where it started, but the first time I really I remember playing games was at a pizza place back in Saint Louis called Pantera’s. I played all kinds of games back there, although I remember wasting way too many quarters in Rastan. Before that there was Spy Hunter and if we go back far enough, even Pole Position. I’m sure there was probably others before that, but we’re edging back to the limits of my memory, I’m not *that* old after all.

At the time I was playing these though, it hadn’t yet occurred to me that there were people who actually created these wonderful things. Sure, it seems obvious in retrospect, but at the time I was a young kid, I probably would have believed that unicorns and fairies made the games I loved. I knew there was some kind of a computer in there, and that’s what I wanted, but we were quite poor at the time and couldn’t afford one.

Luckily, my mother saved up for a year and she was quite excited to finally give me a computer for Christmas one year, and boy was I excited to get it. It was an amazing piece of hardware called the TRS-80 (that has a nickname now of “trash 80”, but I didn’t care, it was a real computer). Looking back at that now, it’s kind of funny, the very first thing it said when you turned it on was Microsoft (MS did the BASIC for the system). Who would have thought that 15 years later I would actually be *working* at Microsoft, the computer had given me an inspiration I didn’t even realize.

The system came with the BASIC mentioned above and it was fascinating to me. I could type things and stuff would happen. I devoured the few “tutorials” in the book that came with the system and was anxious to find anything more I could. I started out probably the way many kids back then did, by copying huge chunks of text out of a magazine, not necessarily understanding what was going on, but being amazed by the results. Well, after running it the first time, realizing it was broken and then spending hours painstakingly trying to find the single spot I had messed up typing.

With that I knew what I wanted to do when I grew up. I would write games, and despite not even being a teenager yet, I was going to write the best game the world had ever seen. It was here that I learned a very powerful lesson; that being that writing a game was hard. I wasn’t even remotely qualified to do this.

I spent the next several years learning how to be a developer in general, always thinking of games in the background, but learning the basics. I learned new languages, being fascinated by the power of Pascal before migrating to C, and then discovering Visual Basic and realizing that as a development language, this was the beginning of a boon to developers everywhere. It had a low barrier to entry and could explode at any time.

Now, I still wanted to be a game developer, and I still didn’t think I was qualified to be one, so it was then that I decided what better way to become qualified than by opening up game development to the masses and help *everyone* become qualified. By now, I was already working at Microsoft and migrated over to the DirectX team and began a project that intended to bring the power of DirectX to Visual Basic. I’ve told this story many times, so I’m sure everyone knows by now that DirectX for Visual Basic turned into DirectX.NET which turned into Managed DirectX which turned into XNA. By the time I looked up I realized that I had been writing platforms for an entire decade, and still not writing games.

That’s when I switched. I moved over to Microsoft Studios (then Microsoft Games Studios) and started making games. So here we are now, a year and a half later. What have I discovered?

Despite both of them being writing code, and actually, even somewhat related code, the two activities are so very far apart. There are so many things you need to take care of for a platform (such as robust parameter validation) that you simply don’t need to worry about in a game. On the flip side, writing games has extra constraints you just don’t see on the platform side such as game designers. Ok, sure, you could argue that program managers help define the “mechanics” of APIs much like a game designer helps define the “mechanics” of the game, but once the program manager has defined the mechanics, she is done. The game designer must also make those mechanics fun.

There’s so much more creativity required when writing games as well. Sure, there is some creativity in designing API’s and functionality and feature set on a platform, but it’s not nearly the same magnitude. Actually, I suppose that isn’t fair, there is a lot of creativity in developing a platform, just not the type of creativity that excites me the way writing a game does; coming up with narrative, etc.

It’s also taught me that despite spending a decade making game development easier to do, I still have so much to learn. I would feel comfortable developing a platform completely on my own (hell, I have done it before), while I’m not sure I could say the same thing about writing a game. The work game designers do (and the way in which they think about things) is a skill I need to learn, and I’m thankful I have the opportunity to learn from a group of people who are amazing at what they do. Some of my coworkers have done design work on some of my favorite games of all time such as Command and Conquer; Gears of War; Alan Wake; Dungeon Siege; the list goes on and on.

That isn’t to say that I worked with less amazing people before (I’m sure everyone knows who Shawn is, which would disprove that), it’s just a different type of awesome. Developing a platform, the cool things we normally get to see could best be described as tech demos, and while they’re amazing in their own right, you normally have to wait for your customers to write something awesome to fully realize your initial vision.

If there was one thing I missed about being on the platform team it was the intimate knowledge of what was coming next. I just don’t have that anymore, and it is frustrating. I wish someone would say something about XNA and Windows 8 just like everyone else does. However, for where I was in my career and my life, making the transition to making games was definitely the right move for me. I can flex my creative juices and expand in the areas I am weak in.

It’s nice to realize that your dream job at eleven can still be your dream job at thirty six.

Garbage detector from my Gamefest Slides

One of the last things in my slides for Gamefest was a simple little object I had written to help us detect when our code was allocating memory we weren’t expecting. Due to the size limitations of slides I had attempted to to shorten the object to the minimalist version, and in doing so lost a few things. Here is the object I used in its entirety:

Not very much different than what’s in the slides, but a few subtle differences. The most important being whether or not ‘true’ or “false” is passed in during the two calls to GetTotalMemory. What does this code actually do though?

I wrote it basically to allow myself to “wrap” a section of code and measure any allocations from it. At the constructor of the helper object I measure the total memory in use and store how much allocation (in bytes) I expect the code block to take. Then at the end I measure the memory once more and if more memory was allocated than I was expecting I break into the debugger.

A few things to note. First, when you pass in ‘true’ to GetTotalMemory a full collection will occur, so if you are doing this often you will dramatically slow down your application.

Second, you’ll notice that the slides pass in ‘true’ in both spots which is probably not the behavior you want (you’ll notice in the code above that ‘false’ is passed at the end in the second constructor). If you pass in true in both spots, you will only catch allocations that were not garbage because any garbage would have been collected due to the ‘true’ being passed in!

Now, there are times when you do want to know this, so having the option is certainly fine, but the majority of time you want to know about allocations that create garbage, in which case you need to pass in false to the second call.

Another point I brought up during the talk that isn’t obvious is that this code completely ignores other threads. Which means if you have another thread allocating memory in between the two calls, you will get “false positives” that can prove difficult to track down. When I used this method I kept the code I wrapped in this object small to help eliminate them.

To see an example of using this helper in code see below:

Some thoughts on Azure…

With all of the spare time that I don’t seem to have, i’ve been working with Azure a bit for a personal project that I’m not going to go into much detail about (yet). Well actually, I suppose that isn’t fair. Initially I started simply using the hosting I have on this site to do some simple stuff before I quickly got annoyed with it. I didn’t need a web host, I needed some real cloud computing. I briefly looked at Google’s App Engine and Amazon’s AWS before figuring since I work at Microsoft I should probably just dive in with Azure, so that’s what I did!

Once I got over the initial system shock of “what on earth is all of this stuff” and followed through a few examples, I was actually pretty pleased at how simple getting something up and running was. I had downloaded and tried the Azure SDK quite a while back in beta (before we announced it) and it was hard to use, the docs were non-existent, and no samples existed at all. I was completely lost and gave up on it until recently. While I can’t speak for the SDKs for Google or Amazon, the Azure SDK has improved leaps and bounds now and it finally started making sense!

It took me basically a week to write the underlying infrastructure for what my side project wanted when I was using this hosting environment, and a large portion of that was trying to remember SQL, and writing stored procs. I made the equivalent Azure worker role in about a day and didn’t have to write a single SQL statement (unless you count some LINQ in my role, which I don’t) or stored procedure! I also didn’t have to worry about transactions or any of that painful stuff!

One thing that I guess makes me naive is that I was surprised at the potential cost for even the simplest of things! For example, a “small core” on Azure costs $0.12 per CPU hour (and CPU hour is measured by the amount of time the application is deployed). This means for a single core, for an average month you’re looking at $0.12 * 30 days * 24 hours = ~$86. Worse, it’s recommended you run at least two instances, and you have to if you want the “guaranteed” 99.99% uptime. Since each instance is another core, basically double that cost so $172 per month to run your application in the cloud. You can get 3+ years of web hosting for less than that. That’s just the CPU cost though, you end up paying for storage as well!

At first, storage “sounds” pretty cheap. You pay $0.01 per 10,000 transactions which sounds super cheap, but that adds up as well. In most of the samples for Azure you’ll notice that in the run method it will essentially check for messages in a never-ending loop to see if there is work to do, and will normally check these messages about once every five seconds. The act of checking to see if a message exists is a storage transaction though! Checking every five seconds is 12 times per minute or 720 per hour or 17,280 per day or 518,400 per month or an extra 51.84 pennies per month. Ok, so that isn’t all that expensive I guess. It does start to add up though, and there is an extra $0.15 per gigabyte of data you’re storing.. So a million transactions a month storing 2gigs of data is an extra $1.30, not terrible.

Now, I’m sure that large companies needing cloud computing find these prices to be quite acceptable (the Azure prices seem to be similar to the prices offered by Google and Amazon), and I actually don’t mind them too much, but they do seem to be a decent chunk of change for a developer just starting out. I guess I was just expecting prices to be similar to hosting prices so was surprised by the costs I saw.

I’m no where near an expert on the subject yet, but so far working with Azure and “in the cloud” has been a pleasure. Now if I could just get off my lazy butt and finish this thing…

Game development is still hard…

It’s hard to believe that over ten years ago I joined the DirectX team with a goal of making game development easier. Back then, being a game developer was a relatively “exclusive” club. The barrier to entry for developers who wanted to make games was very high in all aspects.

My first goal was to make the act of writing the code itself to be simpler. DirectX is very powerful, but was quite esoteric if you didn’t use it every day. At the time, it was C/C++ only as well, and while these are great languages the fact was that Visual Basic developers (C# didn’t exist at the time) outnumbered them five to one (at least). Over the next few years many VB (and some C/C++) developers migrated to C#, DXVB became Managed DirectX, which in turn morphed into XNA Game Studio. Through each of these iterations the act of writing the code for a game became easier and easier.

If my only goal was to make writing code easier, then by any measure I can think of I accomplished this goal, but it wasn’t. I wanted to expand beyond the PC as well, I wanted to allow every day folks to write code for their home consoles, and in 2006 (with the first version of XNA Game Studio) that goal was accomplished as well (followed by the ability to publish and make money a year later). People don’t think much about it now, but that was huge back then. Nowadays it seems every device has a public marketplace and an ability for anyone to write apps and make money from them (and you would dismiss them if they didn’t), but it wasn’t the landscape in 2006.

It was all of these “successes” that prompted my exodus away from the XNA team and into a game studio in 2010. I felt I had accomplished all of the goals I had in lowering the barrier to entry for game development and wanted to move on to my passion of actually creating them. The release of Beards and Beaks has given me a bit of time to be introspective somewhat (which isn’t to say I’m not busy, but still).

For all the successes that we’ve had though, the simple fact is that today, halfway through 2011, game development is still hard. In some ways, I would argue that it is actually even harder today than it was when I started, albeit for completely different reasons.

While we certainly helped remove some of the barriers to entry over the last ten years, we didn’t remove them all, and we’ve actually added a few new ones. For example, it’s still extremely difficult for an up and coming game developer to hook up with an artist, just as it’s hard for that artist looking for things to stick in their portfolio to find a developer. People are still confused about what a good “game idea” is.

Actually, that’s a good enough side track that I need to break off for a moment for a mini-rant. I swear if I hear one more person tell me that they have a great idea for a game, and then begin describe some story I’m going to scream. In the majority of cases, the story is not what is going to make or break your game (and invariably when someone is describing a “game idea” to me in this fasion, they are not in the minority where it matters). You need to know what game mechanic makes your game interesting. Even in story heavy games, the game mechanic is what makes your game most times. If your game idea is essentially “all the game mechanics of Gears of War with a different story”, this isn’t a good idea.

Sorry, with that rant out of the way, I’ll continue. I believe the barrier to entry for a budding game developer is so low now that the real problems are no longer “how do I make games”, but instead “how do I stand out in the crowd”. The barrier is no longer to entry, the barrier is now to recognition. How can your shining gem of a game rise above the cess pool of thousands of terrible games?

If you ask me, this is an even harder problem to solve, and despite rambling on for all this time so far, I’m not about to say some magic incantation that solves it because truthfully, I don’t know how. Sure, I have some ideas which I’ll go into presently, but I have no delusions that they are a sure-fire way to success.

First, as I alluded to above, you need to have a good mechanic (and a fun game). If you’re trying to make a rip-off of Gears of War, well stop it because you’re not helping anyone and just adding to problem. Now, that isn’t to say that taking a formula that works and improving it is a bad thing, but don’t try to copy something whole-sale unless you can do it better. You want to make a game that’s like Gears of War but includes the ability to turn into a dinosaur and eat folks? Awesome! You want to make a copy of Gears of War with a budget of $30 and a diet coke? Get out of here.

When I say taking a copy of a game and making it better, what I really mean (most times) is “polishing” the game. The term is a little weird to begin with, but basically what it means is that the game looks and feels professional. It’s hard to describe, but you can see it when it’s there and will miss it when it’s not.

People think that polish doesn’t matter as much, but in reality it matters probably more than anything. Consumers can tell. Angry Birds is like a printing press that spits out money, but it isn’t an original idea or anything. The exact same game came out years before it, but that game was unpolished and forgotten to the sands of time. Rovio took the basic idea, polished the hell out of it and have made more money than they know what to do with.

Of course, once your amazing game is out, you then need to market yourself. I feel a bit sorry for the single guys (or small teams) in this regard. At Microsoft, we have entire marketting teams, and it’s still a difficult proposition! I hope one day soon to talk more about this and some ideas I have, but right now this post is already way longer than I was intending it to be.

As the title says, game development is still hard.. I suppose I have a lot more work to do..

Visual Basic for XNA?

So it seems that it was annouced today that XNA is coming to Visual Basic! I have to admit the announcement caught me a bit by surprise. As one of the people who has had to investigate (at various times) what it would take to make this happen, I have no idea where the team has gotten the time to do this! As the article says though, it was one of the most popular feature requests I can remember though so it’s good that it’s finally happening.

Wish there was more information though. For example, when is it coming? Is it coming on all the platforms, particularly Xbox? The picture in the article implies it will, but that doesn’t necessarily mean anything. I know they said to wait a few months, but given people have been waiting for literally years at all, I wish they would have waited until they could give us all the information rather than teasing it early when they cannot.

Although, I suppose I could be in the minority in thinking that. Some folks just like be teased and waiting a few months.

Loading… I hate loading…

I guess I’m impatient, because there are very few things that annoy me more than loading screens. In a world of instant gratification, waiting for any significant amount of time is unacceptable. Hell, I’ve only just started typing this and I’m already annoyed that I’m not finished yet. Loading just bugs me, and I see it creep a lot on Windows Phone 7 games. Perhaps I should ramble on about that for a while.

The first goal of the game should be to show me something on screen as soon as possible. While you can “somewhat” use the splashscreen.jpg trick to show something quick, in XNA applications, it doesn’t stay on screen long enough and looks awkward (in my opinion). I personally would rather show my splash screen on the first render. However, that leads me to another common mistake I see people making, and that’s loading a bunch of content (or all!) in the LoadContent method. This is certainly a bad idea if you want to show something on screen quickly. This is essentially the order you will see things happen at startup:

Game’s Constructor is called
Initialize is called
LoadContent is called
Update is called
Draw is called
Goto 4.

While there are certainly nuances here, that’s the basic order, and notice that Draw isn’t called until way at the end. I’ve seen people decide that the best way to fix this would be to draw something in LoadContent, and then they implement that, see that it works and call it a success. If only it were so easy. If your application is starting up once more after being tombstoned, essentially the same sequence of events happens, however there is a caveat this time. While the “startup sequence” is happening, the phone will be displaying the “Resuming…” screen (my how I dislike that screen), and will not show anything from your application until the “startup sequence” has finished. In this case, the sequence doesn’t finish until *after* LoadContent has finished. So if you showed something on screen early in your LoadContent method, and then loaded a bunch of content, it would work in the initial startup scenario, and potentially fail in the restore from tombstone scenarios.

The easy way to get around these issues is to do all of your content loading (aside from the first thing you want to see on screen, such as a splash screen) on the second update call. This allows you to show something on screen quickly in both the initial startup and coming back from tombstoning cases, and if you don’t have much content, should be sufficient enough. Your splash screen image will stay static on screen (and your application is essentially non-interactive) while the content is loading. If it takes more than a few seconds to get to this state though (ie, you have too much content), you have more work to do.

The most common thing I see happening here is the game will load only the content needed for the main UI portions, and will force you behind a loading screen after you choose the level or what have you. Again, this works perfectly fine, but we can do so much better! Why can’t you load everything you need while I’m playing around with the UI? For ease of explanation, let’s say that you are making a 2D game, and you have a large list of strings that are all the sprite textures you will need to load. What if you had a class such as this:

So long as you called BackgroundLoad.Go(this); sometime after you started up, your game would dutifully start loading all of your assets behind the scenes in a background thread. This isn’t a foolproof solution though. For one, you have very little control over background threads on Windows Phone 7. You cannot change priority, and you have no control over how much they will execute (although on average for every 4ms your main thread is executing your background thread will run for 2ms). In this example, you are also using the graphics device on another thread, which will take out a lock to avoid multiple threads manipulating it at the same time. In the larger scheme of things though, it’s still probably faster in this mechanism than without.

You will notice two potentially useless pieces of code here as well. First, a property to dictate when the loading has been complete, and a second in which the actual asset load is happening within an action rather than directly in code. The property is self explanatory if you think about it, you still need to know when all of your content has been loaded so you can allow the user to start the game! This just gives you easy access to it. The action delegate is slightly less obvious though.

What should you do when the user has gone through the UI and is ready to play the game but the content isn’t done loading? As was mentioned earlier, you don’t have any control over the background threads execution time, and because it is running half the speed as the main thread, as the code is written now, your loading time could potentially be *increased*! This is most definitely not what we wanted. What you really want is a mechanism where you would load things in the background if the user was still messing around with the UI, but will switch to loading in the foreground if the user was waiting on loading, and that is what the action delegate is for. If you replaced the code in the thread method with the following, you will get this behavior:

Now your code is loading in the background while the user is messing around with the UI and will switch to the foreground once they’re done. In the degenerate case, this is still slightly slower than simply forcing them to wait at a loading screen, but given the degenerate case is almost certain to never happen, a mechanism such as this will (for most intents and purposes) always be faster. It certainly took me long enough to exlain all this, I should have had a loading screen for this post…

As a final caveat, this was an extremely simple example of multithreading. If you do anything more complex and don’t have knowledge of multithreading (or what the ‘lock’ keyword does), you should probably read up a bit on that!

A new book on Game Studio 4.0

It has been about five years since my last book came out. During that time, a lot has happened. Heck, that book was about Managed DirectX, and at the time it came out XNA Game Studio didn’t even exist as an idea, much less a product. Since then, it’s not only been released, but we’ve announced the *fourth* version of the product! We’ve added support for Windows, Xbox, Zune, and soon, Windows Phone 7.

So much has changed, so much time has passed, it was time to do a new book. This new book will cover Game Studio 4.0 in its entirety, including Windows Phone 7, the new features, Xbox, everything. While we haven’t announced any release dates for Game Studio, I’m spending my spare time (what little I have) to make sure the book will come out as close as possible to the release of the product.

I’m also having some help with this one, and will have a co-author, so that is a bit exciting too! Writing has begun so if you have suggestions, now is the time!

GDC? Books? A web site? A blog post?

I never seem to find time to write posts here anymore. Well, to be more accurate, I get frustrated with the number of things I get as direct responses to posts (or even just random mails from people from this blog). Honestly the top two questions I get are people wanting me to send them copies of my books and people asking me for help with their homework. These are normally followed up shortly by those same people letting me know how much of a jerk I am for not relenting to their request. It doesn’t seem to matter if I ignore it or respond politely, the same result occurs either way. It’s almost enough to make me wonder why I even write anything (and I’m fully convinced it will be worse at least for a short time after this post).

Anyway, enough random complaints from me, and onto what I originally was going to write about..

After the two topics above, the next most asked about topic is about books. When is my next book coming out, when am I updating one of my older books, etc. So let me ask you what you’d like. If I were to write a third book, what would you like to see? I’m guessing not many people will suggest science fiction or an in depth discussion of the civil war.

If you haven’t heard recently (sometimes I am a little slow on the announcements) we have launched a new creators club web site. It has a new starter kit for you to download and enjoy, a number of samples to look at, and even new forums. Naturally, there is more there as well, so go check it out!

We also have our Dream. Build. Play. contest going on, and we announced some of the prizes which are simply amazing in my opinion ($10,000 cash, a new computer, a chance to have your game published on Xbox Live Arcade and more?). There was also a ‘warm up’ contest using the Spacewars starter kit that had some amazing entries. I’m excited to see the things that will come out of the real contest now. Like I’ve mentioned before, I’ve always wanted to run a contest for something like this.

As part of the GDC list of announcements, we also have a lot of great information. Such as Creators Club members getting a license to the Torque X engine. None of this even hints at some of the things we are hoping to have done in the not too distant future. For anyone who thought we were going to be resting on our laurels as they say, rest assured we are doing nothing of the sort.

Now if I could just get myself to be more like Shawn and actually write some technical posts here once in a while…

Why are these textures taking up so much memory..

Someone on the newsgroups was recently asking why loading his simple 800×600 texture was taking up so much memory (more than 5 megs). He was using code something like the following:

Texture myTexture = TextureLoader.FromFile(myDevice, somePath);

The quest seemed interesting enough for me to post my response here as well as in the newsgroup since this reinforces my belief that you should always know what the methods you’re calling ‘cost’. It wouldn’t surprise me to find out that most people don’t realize all the things that go in with that simple line above.

So here was my response:

First, if you’re texture is really 800×600, using this overload (TextureLoader.FromFile) will ‘upscale’ the texture so that it is square and a power of two, in this case, 1024×1024.. Assuming 32bit color for each pixel, the default mip level would be:

1024x1024x4 = 4 megs of pixel data.

Now, this overload would also create a mipmap chain down to 1×1, so you would also have:

512x512x4 = 1 meg of pixel data

256x256x4 = 256k of pixel data

128x128x4 = 64k of pixel data

64x64x4 = 16k of pixel data

32x32x4 = 4k of pixel data

16x16x4 = 1k of pixel data

8x8x4 = 256 bytes of pixel data

4x4x4 = 64 bytes of pixel data

2x2x4 = 16 bytes of pixel data

1x1x4 = 4 bytes of pixel data

So, add it all up, and each texture you load would take (on average) 5.6 megs of pixel data.. So the numbers make sense to me..

Now that the Whidbey Beta is out, what about Managed DirectX?

With the release of the Whidbey (err, I mean Visual Studio.NET 2005) beta a few weeks ago, my thoughts have drifted towards how Managed DirectX may work in this new release. For those of you who haven’t seen VS2005, or the new CLR, the changes there are magnificent. Across the board, improvements have been made, and the thing is looking wonderful.

So my thoughts roll over to MDX, and I think to myself, “Self: wouldn’t it be awesome if we took advantage of some of those features?” Who wouldn’t want to declare the vertex data they’re about to use like:

VertexBuffer vb = null;
IndexBuffer ib = null;

Of course that just scratches the surface of the possibilities that would lie in a VS2005 specific version of Managed DirectX. Which begs the question, what do you think?