Boxer

Developer diary: plans and progress reports.

Positively agog Sunday 21st October 2012

So when I said back in July that I was taking a break because of real-life commitments, that was a teeny tiny fib. In fact I was beavering away undercover with the fine people at GOG.com to help them launch their DOS games catalogue on Mac.

On October 18th the fruits of our labours were finally revealed, and I can finally gush about what I’ve been working on all this time: Boxer standalone. This is a streamlined and state-of-the-art Boxer built for one purpose: releasing DOS games as individual apps. It's also a peek into Boxer’s future.

Why a separate version?

When we were figuring out how to deploy GOG.com's DOS games on the Mac, it became abundantly clear that Boxer in its current form wasn't gonna cut it. It sticks you with a folder full of sample games to help you figure out the emulator. Its gameboxes are document files that can’t be played without downloading a separate application. Its emulation is studded with peripheral features to help you install new games and configure them and add new drives to them and all that jazz. The entire app is designed around turning your box of old CDs into a games collection.

But GOG.com are selling games, not raw materials for someone else’s curated emulation experience. Their games already come ready to play, so they don't need any of the features to help you get there. Each game needs nothing but to look and feel like a native Mac game.

So I began macheteing off all the bits of Boxer that weren’t part of that experience. Sample games and game importing were the first to go; the preferences window, inspector panel and drag-drop drive addition all hit the cutting-room floor. The UI was tailored to clean away everything that smelled like emulator infrastructure instead of a native game.

Putting together a game app isn’t as easy as making a gamebox – there’s resources to bundle, app IDs to choose, help links to define, behaviour to tweak, branding to slap on – so I also developed a bundler utility that takes a gamebox and wraps it into an app according to your specifications. It’s not as slick as Boxer’s game importer and it’s not intended for end-users: you’ll need to build it and Boxer standalone from source yourself.


With Boxer’s dead weight liposuctioned away and gamebox-to-app conversion a reality, new challenges came to light. Ones that required exciting and shiny new features.

The launcher, rethought

Boxer’s program panel was designed for one task: choosing the right executable once and never touching it again.

But many of GOG.com’s games don’t have just one true executable: there may be a main game and expansion packs, there may be separate singleplayer and multiplayer options, there may be setup utilities and campaign editors. Boxer’s program panel was woefully inadequate for switching back and forth between different launch options.

Even worse, the program panel insisted on appearing alongside the wretched old DOS prompt, and it wouldn’t even appear at all in fullscreen mode. In a standalone game app, that makes for a terrible user experience.

So I threw it away and started again.

The first time you launch one of these games, you’ll be greeted with a tidy list of predefined launch options. Click an option to launch that program; quit back to DOS, and the app will return to the launch options. (You can also get back to the launch options at any time from the File menu.)

If you quit the app while a program is still running, the app will remember which launch option you’d chosen and start up with that next time. This way, it’s easy to switch back and forth between launch options, without needing to choose one every time you start up.

The launch options panel only appears in GOG games that actually have multiple options. For games with only a single launch option, the game will start up straight away and exiting to DOS will quit the app instead of returning to the launch options.

Shadow boxing

As you’d expect, inside each game app are all the files for that game. And as you also know, DOS games like to dump their savegames in the same place as all their other program files.

These two facts are on a head-on collision course. OS X apps aren’t expected to be self-modifying: and if you’re not running as an administrator, then apps in the Applications folder are not even allowed to write to themselves. This would prevent you from saving your game and in many cases would prevent the game from running at all.

The solution is one I’ve discussed in the past: write shadowing. Attempts to write to game files are instead written to a ‘shadow’ location inside the current user’s Application Support folder. The app reads changes from there first, before reading the original game files from inside the app. This way noone’s trying to modify the app itself and everyone’s happy.

This has several knock-on benefits:

Shaders a-go-go

Dropping OSX 10.5 support let me pull the trigger on a feature I’ve had in my sights for a long time: pixel shaders for rendering styles. Shaders are programmable effects that run on the GPU, and they allow for faster and considerably fancier scaling effects.


The game apps ship with three shader-powered rendering styles: the original untouched output, a 5xBR smoothing shader by Hyllian and crazy46guy, and a simply awesome curved CRT shader by cgwg, Themaister and DOLLS. I cherry-picked these from the thriving BSNES shader community; the coders deserve much love for creating such great-looking shaders.

For older Macs that can’t run pixel shaders at an acceptable speed, Boxer standalone falls back on the old software rendering styles you get in Boxer 1.3.

Naturally, the game apps also natively support those newfangled retina displays.

There’s always a but

The game apps are slick as hell, but that comes in exchange for flexibility and features. Hence some caveats:

Each game app has a gamebox inside, so if you prefer you can just run the gamebox in Boxer 1.3.2 instead. However, Boxer 1.3.2 does not utilise the new launch option structure, nor does it support write shadowing: your game state will not be consistent between Boxer and the game app, and any changes you make in Boxer will be saved permanently into the app.

So when do I get to see all this neat stuff in Boxer itself?

Soon! But there’s still a bunch of work to do before these features are ready to appear in Boxer 1.4:

For now, buy the hell out of GOG.com’s DOS game collection and get the goodies right away. If you don’t mind wet paint and you’re familiar with XCode, you could build Boxer 1.4pre from source and take advantage of all the new features; or even build Boxer standalone and its bundler utility and start churning out your own game apps (some assembly required!)

Closing remarks

It's been a huge pleasure to collaborate with the cool guys at GOG.com and big ups to them for choosing Boxer. By doing so, they’ve ensured that their Mac gamers get the best DOS gaming experience across any platform.

Commentary

  1. Awesome work, as always. I was almost at the point of lamenting the lack of save-game shadowing, but then you delivered!

  2. Great news, gratz on your work with GOG!

  3. I still don't understand why the inspector panel and access to the DOS promt had to be removed, I mean if people don't want to mess with them they just don't need to touch them. It makes installing for example the upgrade patches for the Ultima games more complicated than it needs. Oh well, as long as I can just take the gamebox out of the port, play it in Boxer with glorious MT32 sound and just throw the rest away I'm happy. Plus, now I don't need to install the game in Wine just to get the folder. One thing though, with GoG's DOS games a picture of the original cover was always included and Boxer would then pick that as the icon for the gamebox, that was a really neat feature, would be great if the gameboxes in the ports had those icons as well so they fit with the rest of my DOS library.

    on another note, I am really surprised how many users at GoG didn't know about Boxer and could play the DOS games. Mac support on GoG fantastic news and it's even better that they chose Boxer instead of just plain DOSBox.

  4. Great work, it'll be great to see Boxer exposed to a larger audience. I noticed a couple of files seem to be missing from the repository for the Boxer Bundler, the NSURL category in the BXToolkit file is listed in the project but isn't in the github repository for Bundler, or the main Boxer repository.

  5. Alun: I wondered, considering that emulation can now be limited to a single game, whether you have given any thought to removing the mouse-lock feature for games that don't require bounding the pointer at the emulated screen edges?

    There's probably not all that many, but for those that do, it may make the experience a little more transparent.

  6. @HIPHISH Surely it would be trivial to simply patch the .boxer bundle in the gog.com app directly (in Contents/Resources)?

  7. My concern since 1.0 is in making everything "too seamless" and removing the various tweaking options that some games require to work properly. It's all well and good to add features that improve usability Alun, but there still needs to be a way for DOSbox power users to be able to configure games as needed.

    Also, by potentially making certain games harder to install & configure than they were in previous versions, you might be putting off users who may only play a few DOS games here and there.

    Seamlessness is good, as along as it doesn't harm customization in the long run.

  8. Just installed Boxer after hearing about it on GOG. Works great! Thank you!

  9. Huge congrats and a fantastic job you did on this! Thank-you so much for ensuring we mac users aren't left out of this modern era of online game stores. Who knows, perhaps Steam and the App Store can now get on the ball and release dos games for the mac, or does Gog have exclusive rights to Boxer Standalone? As for the boxer poweruser/tinkerer, once boxer 1.4 is out, it will be small potatoes to grab the .boxer file in each game and use our own copies of boxer. Heck I usually am even tinkering with the dosbox build within boxer itself. XD

  10. Hiphish: I know this will be an unsatisfying answer, but the apps were not designed to be emulators but to be games. The less ‘surface area’ the emulator exposes, the less there is to document and explain, the less there is to screw up and the smaller the support burden is for GOG.com.

    As you've found, if you want to bring the full range of Boxer's tools to bear then you can just grab the gameboxes from inside the apps and play them in Boxer instead – or if you prefer, you make changes with Boxer and then go back to the game app.

  11. AndyH: There are several games in their catalogue that don't support mouse control, and for those it would theoretically make sense to remove the mouse-locking altogether. Early versions of Boxer did do this for games that didn't attempt to read mouse input, but it resulted in the mouse cursor floating around and getting in your way in fullscreen mode. This was irritating enough that it stopped being worthwhile to disable the mouse-locking.

  12. Shinohara: thanks for the heads-up, I'll try to get the repos fixed up sometime this week.

  13. Sivilius: It should be obvious by now that Boxer is not – never has been, never will be – designed with DOSBox power users in mind. Boxer is not a fancy window onto DOSBox’s feature set: it's a separate DOS emulator with very different design goals that happens to use DOSBox underneath. Protecting someone's investment in learning DOSBox is such a vanishingly irrelevant goal that it doesn’t even register on my radar.

    If a particular game doesn’t work without custom configuration, then my solution has always been to make Boxer aware of that game and automatically apply a tailored configuration for it: not to expose further knobs whose use requires an intimate familiarity with the inner workings of the emulation. Tinkering with CPU modes and soundcard compatibility settings is neither a necessary part of DOS gaming nor fun; it’s just work, work the emulator can do and should be doing for you.

    So a tremendous amount of effort goes into Boxer to catch problem games and make sure they'll work out of the box, for people who know nothing about DOS or Boxer or DOSBox. The users who only play a few DOS games are the point of all this effort, because these are precisely the people who are least willing and least able to invest time in figuring out an esoteric emulator just for one game.

    If there have been actual regressions in Boxer’s game support, such that a game which previously worked in Boxer or DOSBox can no longer install or run, these are not the inevitable cost of streamlining the user experience: these are bugs I need to know about.

  14. It's not an unsatisfying answer, it's exactly what I wanted to know. I understand that those standalone ports from GoG are not meant for me, they are meant for people who just want to play the games as they are right out of the box, and that's fine as long as I can still get the real Boxer experience by taking the gamebox out of the port.

  15. [...] [...]

  16. Well, this makes my GOG importer vanishingly usefull...

  17. That's one of the things I'm most relieved about actually - now I don't have to invest the time in coding up a complex rickety extraction process just for GOG.com's Innosetup archives!

  18. [...] with native mac support! OSX-ers rejoice! The developer of Boxer, who is working with them also posted a great article talking details of his work.Finally, SimCity 5 now has a release date!After an email, we get to the main topic, the great [...]

  19. [...] [...]

  20. I'm really happy for you Alun, although its gonna hurt my wallet now that I'm gonna buy so many games from GoG :-)

    How involved are you in the actual porting? Do you the boxer tool for them and they make the "port" or do you work with them all the way? Is built in support for controllers coming back? Other than that, I fully support you in that the experience should be as seemless as possible

    (Btw, pass them the info that some of the icons for the Mac games look, hmm, quite poor to put it nicely.)

  21. This is interesting. Will Boxer 1/3/x still be available after the release of 1.4 for people who need to do a little tweaking?

    Also is there some way I can supply GOG with some old DOS games/suggest titles to DOD that I would LOVE them to put into packages, as all my efforts at installing said DOS games via Boxer have failed?

  22. Cassandra: Perhaps there was some confusion from my post: Boxer 1.4 will still have all of the features of 1.3.x. Only the separate standalone Boxer version has had some features removed. The standard Boxer 1.4 release will behave just like 1.3.2, albeit with a redesigned program launcher and the addition of write-shadowing. Nothing is going away.

    That said, 1.3.x will still be offered as a download for users who can't upgrade to OSX 10.6 (i.e. PowerPC users).

    The GOG.com website has a wishlist feature for suggesting games and service improvements you'd like, and they appear to use that for triaging which games to focus on releasing next.

    But if there's games you haven't been able to install using Boxer, please let me know: I'll test them myself and try to improve Boxer's game importing to handle them.

  23. Jeaz: unfortunately I can't discuss the details of my involvement in the game porting process; I'll just say that I've worked closely with them on their initial releases.

    The standalone games have the same builtin controller support as Boxer 1.3.x, which is to say "plug it in and if it works then great." However, controller support is not a priority for GOG, and as far as I know they have not configured the games to enable joystick control or to use specific joystick modes. The UI for choosing a joystick mode is absent from the current standalone version so they'll use the standard 4-axis 4-button mode.

    As Boxer's own controller support improves (to allow axis and button rebinding etc.), those improvements will make their way into the standalone version and a UI for choosing the joystick mode will probably make a return then. It's up to GOG to update their existing game apps when new Boxer versions are available though.

Design by 40watt.