Pyrogon : Entertain Your Brain!

Today's Developer Diary installment is being guest written by Ryan Gordon, the person responsible for porting Candy Cruncher to Linux and other platforms. I've interspersed my own comments, mostly as amplifications of his own statements, but occasionally with clarifications or rebuttals.

Anyone that is interested in having their games ported to Linux or other platforms should definitely contact Ryan, he works quickly, finds bugs quick, and does amazingly high quality work. He can be contacted at icculus@icculus.org or http://www.icculus.org.

So now I hand the mike to Ryan.

To celebrate the completion of the Linux port of Candy Cruncher, I thought I'd put down some thoughts about porting games and writing portable software in general. This is a sort of tip-of-the-iceberg, random HOWTO on the subject.

Why port software? Especially in the game industry, is there much to gain by catering to minorities? MacOS is sitting on 5 percent of the world's desktops, and while that's a significant number of users, the other 95 percent is almost all Windows boxes, which means you can get most of the users from writing for one target, and think nothing of the Linux and BeOS users of the world. So what's the point?

Here are my reasons:

Choice: I like my Linux command lines, others like MacOS X's Aqua, others like WinXP. Making a portable game lets your users work and play in the environment they like best. The consumer wins.

Code quality: porting to a new platform exposes bugs you never knew existed in the first place. It forces you to remove assumptions from the program. It can make it easier to maintain. It can make it easier to license your code to others, or contract out work on that code. Having a game that runs cleanly across several platforms can actually lower the cost of maintenance if you do it right. It also helps maintain the programmers' sanity when problems like buffer overruns become obvious through different memory managers.

Consoles: You want a Playstation 2 port of a game? Your first stop should be Linux. The migration path is easier than going to the console directly, it's cheaper to develop for, and yes, Linux and the PSX2 use the same compiler.

I can't agree with this strongly enough. The porting process involves many different steps, including handling compiler, library and system dependencies. Trying to handle all of these simultaneously can be aggravating and difficult, so it's often easier to do the port in steps. For example, if you know you're going to port from Windows and Microsoft Visual C++ to the Mac and CodeWarrior, it might make sense to first port to CodeWarrior on the PC, THEN port to the Mac. Divide and conquer.

Financial reasons: If a program is done right from the start, the cost to move it to new platforms is small, but the potential for extra sales increases. Mac and Linux users do tell their friends about software they like, and they tend to be very vocal in their praise of the companies that are willing to support their platforms. Games for Linux tend to have an infinite shelf life, whereas their Win32 counterparts tend to land in the bargain bin in a month or so.

I'd like to add that the OS X version of Candy Cruncher is outselling the Windows version by over 2-to-1. If you look at market size alone, you'd think that the OS X version would get its butt kicked, but in fact it's the other way around. Quality of consumer can often make up for a huge difference in quantity of consumer -- this is why niche markets can survive.

However, I do feel obligated to mention a counterargument, and that is support costs. We're in a bit of a quandary right now with our BeOS version. It took almost no time to get Candy Cruncher working on BeOS, but we don't think we'll sell many (if any!) copies. The cost, for us, to launch and support the BeOS version is significantly higher than any income we'd derive from it. So we have three choices: release the BeOS version even though we lose money on it; don't release the BeOS version, which is unfair to BeOS users and goes against our techie souls; or release the BeOS version for free, possibly alienating our paying customers. It's a tough situation and we're not sure what we can do about it.

Warm fuzzies: I like supporting the underdog operating systems. It makes me feel good. Pyrogon's stated intention is to make quality games that don't cost millions of dollars to make. Having players enjoy their games is priority one, so it makes sense to get them into the hands of people that generally appreciate the games more than the twitchy overclockers that are giving Windows gaming a bad name. Note: not that we don't love you overclocking Windows users too!

I am speaking of games, but these statements are generally true for any kind of software.

So that we know why, let's explore how.

To start, I have to express the Tao of porting: no code is portable until it gets ported. Sure, we all write wonderful code, and choirs of angels sing while we type, but there will always be unexpected problems that won't be seen until the source is pushed through a strange compiler on a strange operating system running on a strange processor. The trick is to minimize the amount of non-portability right from the start. This takes diligence and a little bit of know-how on the part of the coder. Knowing what you're doing can literally reduce the porting time by weeks. This knowledge is best gathered through experience, but these guidelines can be a push down the road of that first experience. If any of this seems like common sense, then it just means you've been down that road before.

Rule #1: Think before you code.
Sounds simple, doesn't it? Unfortunately, even in the video game industry (ESPECIALLY in the video game industry!), it seems that many developers jump right in and start coding. This is wrong, wrong, wrong. It doesn't take long for a project to become an unwieldy, hardcoded mass of spaghetti, which leads to the usual set of problems; however, if even maintaining (or, heaven forbid, enhancing) the codebase is difficult, it will be twice as hard to make it portable. What you need is a blueprint. Sketch it out, write it down, babble incessantly about your plan to everyone around. Have an attitude that lets people tell you honestly if a plan is stupid, and prepare to revise details or whole subsystems. Better to do this now than find yourself reworking the program during crunch time.

Rule #2: Make abstractions.
If you're writing a Windows game, sooner or later, you are going to have to call a Windows-specific function. Things that can be done portably (like using stdio instead of the win32 API) should be done, but other things, like blitting to the screen, are system-specific. What you should do is take a few minutes and wrap things like DirectDraw in a simple class, and expose their functionality in general ways. Do NOT expose DirectDraw data types, to prevent the urge to bypass the abstraction. If something can't be done through the abstraction layer, then the abstraction layer should expand. Candy Cruncher does this very thing for audio, video, the "registry", input, etc. For example, Candy Cruncher has a "H2DDisplayDevice" class. This class is subclassed into a DirectDraw version, a GDI version, an SDL version (for Unix), and a Carbon version (for MacOS). There are some immediate benefits here, even in single-platform development. Note that two of those subclasses are for Windows; this gives Pyrogon the ability to choose the best balance of performance and stability at runtime for any given run of the game. Theoretically, they could add an OpenGL-based subclass of DisplayDevice2D that renders the sprites as textured quads to increase the framerate more, at their users' discretion. This would be added to the framework, without changing the actual game's code, and would benefit the next game they do, too. Good abstraction makes good design, and the benefits are quick ports to other architectures, more flexibility for the power users, and better support for various hardware on the primary platform. It's a huge win.

Rule #3: Be data driven.
Spend as little time in your program as possible. Branch gaming logic out into scripting languages as quickly as possible. This isn't an argument to write your whole game in Perl (unless you want to, I guess); instead, get the stuff that must run fast in C/C++ code (which is almost always the blitters in a 2D game, reference Rule #2) and get the game logic itself out to something you don't need to compile every time you tweak it. I say this not just because it's a good idea, but porting a script interpreter is frequently easier than looking for subtle problems in game logic in C. But what scripting language is best? It depends on what you need. If you need something basic, roll it yourself, but it's better to embed an existing scripting language; there are many that are portable, debugged, and supported to choose from. Perl, Python, and Scheme are just some options. The Pyrogon framework has Lua, which seems to be popular for scripting game logic.

A clarification here -- while we're big fans of Lua, Lua isn't actually used in Candy Cruncher. Our upcoming 3D game, ColdStar, however will be leveraging Lua significantly for its scripting and AI components.

Rule #4: Be sensitive to byte ordering and packing.
If I ever see another game that sends "sizeof (myStructure)" bytes over a network connection, I'm going to scream. I should scream right now, because I will no doubt see this again. Candy Cruncher is not a networked game, but it does run on both Intel (Windows and Linux) and PowerPC (MacOS X) systems, which means that it has to be careful about reading from and writing to files. Between processor types, operating systems, and even compilers, sizes of data types change. I'm talking about something more subtle than the classic C problem of an-int-is-not-always-the-same-size-everywhere, although that's important, too. Structures get packed differently (and not every compiler can understand #pragma pack), data has to be aligned differently, and data gets stored backwards on different processors. If you have to read or write structures to disk, a network socket, or anywhere that a different system may see it, you should send it, one scalar at a time, in an agreed upon format (bigendian or littleendian), and rebuild the structure on the other side of the connection. Do not send floating point numbers ever, if you can help it, since different CPUs have different precisions, and you can only correct for this so far (I can think of at least four ports I've worked on that got bitten by the floating point thing. Be wary.) If you do not do this now, it is nearly impossible to fix it later in a program of any size.

I'd like to add examples of the above that bit me while working on Candy Cruncher. The first place -- and this is extremely common -- was in my image file loading code. It's common practice -- and WRONG -- to try to format your structures to match the memory layout you expect. Then you see code like this:

TGAHeader *pHeader;

pHeader = ( char * ) someBufferLoadedFromDisk;

imageWidth = pHeader->width; //BAD!!!

The above code would work fine on a PC, since the TGA data is stored on disk in little endian format, but on a Mac it explodes horribly with wildly incorrect values. The correct way to handle this would be something like:

pHeader = ConvertLittleEndianToNativeEndian( pHeader->width );

The second bug I encountered was because the sizeof(bool) changed between CodeWarrior and ProjectBuilder/GCC on OS X. So when storing values to my preferences file I'd get mismatched sizes, which caused a lot of subtle (and difficult to find) problems.

Rule #5: Write what you have to, steal the rest.
I've just told you to write a scripting language and be very careful about how data gets manipulated. Right now you're probably wondering how any of this is supposed to make your job easier. Hey, I said this takes diligence! However, the secret is really in the open source community. Why should you write image decoders, and audio format decoders, and scripting languages, when they are freely available for the taking? Candy Cruncher takes advantage of several cross-platform libraries: Lua, zlib, and Ogg Vorbis, to name a few. The Linux and Mac Classic ports use SDL, SDL_ttf, and SDL_mixer, not to mention Loki Setup for the installer. This is literally years of development time that can just be dropped into place, and more importantly, all of these libraries are cross-platform to start with, so you don't have to wonder how you'll get that .OGG file to play on BeOS; it just will.

Rule #6: Don't use assembly language.
Just don't. If you must, you better write a C version and optimize based from that, so that there is a working fallback. But don't write assembly in the first place. 99.5% of the time you think you need it, you don't. Just say no; Candy Cruncher did.

I'd like to add a corollary here -- minimize system specific dependencies. For example, Candy Cruncher expects each system to implement audio mixing capabilities. This makes porting more arduous than necessary. In fact, I intend to write my own audio mixer just so that I can remove a lot of the gross system specific stuff I have in place right now.

A further corollary to this is to avoid depending on closed source, non-portable libraries for core aspects of your game. It might be great to license XYZ from another vendor, but if they have no plans to port to Linux, then guess what -- neither should you.

Rule #7: Listen to your beta testers.
Sooner or later, your port will be ready for external testing (you are going to do a beta test, right?) and you will be unleashing your baby into an unfamiliar world. If this isn't your primary development platform, chances are it isn't a platform you know all the ins and outs of. Even Linux users will find that different distributions do things very differently, and every Linux user has her own routines and traditions. Listen to what they ask you for, and give them what you can. One of the beta testers for Candy Cruncher noted that the game's response was a bit jerky on his box, and wondered if we could make it use the X11 cursor instead of drawing a sprite. We added that. Another wanted to have his Unix login name be the default when entering his high score. It was a good idea that never crossed my mind: added. People with keyboard layouts I've never heard of showed up: fixed. People with exotic display targets poked their heads up: tweaked. Odd sound problems on certain distros: debugged. These requests are to be expected, and are relatively trivial to implement, but they lead to happy customers and, again, a more flexible codebase. You do not want thousands of demo downloads from users that would have bought the game if only the mouse was a little more responsive. You could not have predicted it until someone came along and ran their X-server at an odd color depth. Anticipate possible differences in platforms, but be ready for anything.

Rule #8: Embrace Murphy.
Not every idea is a good one. If something isn't working, chuck it. If a subsystem isn't portable, make it so. If you are modular, and abstract, this makes the code easier to drop into a future product. Like I said, nothing is portable until it's ported, and all the planning in the world doesn't beat Murphy's Law. In such cases, don't be afraid to throw something out and replace it with something that works better. Struggling only makes it worse.

That's your moment of Zen. Now go forth and write portable software! (and be sure to buy a copy of Candy Cruncher for Linux, to help a starving hacker like me!)

Thanks to Ryan for a great Developer Diary! Okay, any developer reading this doesn't have a reason not to port to at least one other platform. And Win98 to WinXP doesn't count!