This year’s E3 will probably be remembered as the year that Microsoft and Sony launched their big counter-offensives against the Wii. Despite spending a great deal of money on developing and promoting both the 360 and PS3 as the hardcore gamer’s consoles of choice, both companies have now announced very big changes in direction for their consoles – intuitive new control devices that negated the need for traditional joypads, and plans to attract a ‘family audience’.

The above image shows the three rival motion control systems in action. (Click on it to see a larger version)

Sony will use the existing PlayStation Eye set-top camera/microphone and give users a little plastic wand with a glowing ball on one end, presumably with some buttons on the wand itself. The camera seems to identify your coloured ball and the relative direction of the wand and calculates the angle and position at which it is being held. Their tech demos involved using the wand to swing swords, torches, tennis rackets and other items around.

Microsoft’s exotically-named Project Natal is basically a pair of cameras and a microphone array. A conventional camera sees visible light, while the other uses infra-red light to create a kind of depth-map image. These two images can be used to calculate a kind of pseudo-3D skeletal tracking system, amongst other things. And the microphones are apparently quite good, although since most of the voice-recognition stuff will be software based (I think), the actual quality will probably vary from game to game.

Nintendo, of course, have the Wii remote. A ’sensor bar’ containing two infra-red lights sits next to your TV and your controller contains a sensor that can see their position and calculate its angle and position. Nintendo are currently building on the Wii remote’s capabilities by way of the Motion Plus add-on, which adds a gyroscope into the mix for added rotational accuracy.

I’ve read a lot of comments to the effect that Microsoft’s Project Natal and Sony’s motion control wand thing are just late-arriving knock-offs of the Wii remote. It’s a massive over-simplification that takes no account of their unique features, but I think there’s some truth in it.

Microsoft have said recently that it was always their plan to reach out to families and non-gamers, and that the previous brute force, high-def, hardcore profile of the 360 was always merely the first step in their long-term plan; I expect someone at Sony has made similar comments about how they’ve been developing camera technology for years now, as if they have always intended to release a motion control peripheral.

I’m sure both companies have been developing these technologies for many years now, but I’m also pretty sure that neither would have considered putting so much emphasis on them if the Wii proven to be such a phenomenal success. After all, the XBox Live Vision Camera and the PS Eye have both been around for years, but hardly any games have been made that really make use of them – have YOU played TotemBall? Clearly neither company has really wanted to push them as control devices until the last year or two.

In fact, the technology used in both new products seems to be quite old. A quick search on YouTube reveals videos of Richard Marks – creator of the EyeToy – demonstrating both the motion control wand and infra-red depth mapping back in 2004.

PS3 Motion Control Wand

Natal-like Infra-Red Depth Mapping

My understanding is that either of these systems could have been introduced years ago, if Sony or Microsoft had given them adequate support at the time – it took the runaway success of the Wii to make them sit up and take motion-sensitive interfaces seriously. But while they certainly appear to be reactions to the Wii remote, the actual interfaces are totally different – almost incomparable, in some regards. So if anyone tells you that they’re just Wii remote knock-offs, I would ask you to slap them in the face.

I assume that everyone reading this blog has used a Wii remote by now. You can wave it around, it has some buttons on it (although only the A and B buttons are in a comfortable position for regular use) and you can point it at your sensor bar to use a cursor. There is an expansion slot, although the only expansion worth talking about is the nunchuck, which just seems to be a concession for traditional gamers who still want to use an analogue stick. You can also turn it on its side and use it like a wireless NES pad, but very few people seem interested in this – the only (non-virtual console) game I ever play like this is Metal Slug Anthology.

It’s interesting that you can’t use all of these features at the same time. You can’t swing your remote around and keep it pointed at the screen, for instance. Even flagship first-party games like Super Mario Galaxy and Zelda: Twilight Princess don’t stray too far away from traditional control systems, and only really dip into the Wii remote’s unique features for short periods (guiding Mario through a web of blue stars, firing Link’s bow, etc). Notably, the Wii remote’s motion sensing tends to function as a kind of ‘virtual button’ – a button that tells Link to swing his sword, or Mario to do his funny little spin attack.

Natal and the PSEye take a different approach. While the Wii remote sits in your hand, detecting its movements and position and sending this information back to the console, these camera-based systems focus directly on the player. They have a lot in common – patents aside, I see no reason why Natal couldn’t handle a control wand of its own. The main difference is that the PSEye lacks Natal’s infra-red depth sensor, which means it can probably only track your body movements on a 2D plane – like in the old EyeToy Play games - while Natal (probably) can’t track handheld objects as naturally as the PSEye. And assuming the motion control wand has some buttons on it, PS3 players will also have a direct digital interface with their game, which would make a big difference.

What do they mean for games?

During their E3 media briefing, Microsoft unveiled this ‘product vision’. Don’t take it literally - it’s not footage of actual games, but a treatment of the kind of things you might expect from Natal.

So, what is their vision?

  • Beat-em-up where you have to do actual kung-fu

  • Racing game where you use an imaginary steering wheel

  • Some kind of kaiju rampage game

  • Penalty shoot-out game

  • Skateboard game where you do tricks on an imaginary deck

  • Virtual dress-up (with co-op over Live!)

  • Quiz games

  • Motion control dashboard interface

The beat-em-up is an obvious non-starter. You could probably get away with a Punch-Out style boxing sim, but any game that requires you to jump around and balance on one leg is going to end in tears – and lawsuits. The racing game would probably work – I gather there’s already a version of Burnout Paradise that can be played just using the camera. That said, you (apparently) control your speed using your feet, which makes me wonder how many people will have to move their coffee table across the room whenever they want to play. How many people have coffee tables, anyway? Valve regularly send out PC hardware surveys over Steam… perhaps Microsoft should start taking surveys of people’s living room furniture and lighting?

The kaiju game clip reveals one of the big problems I see for these camera-controlled systems. You can see the kid marching towards the TV, and the monster walking towards the screen in response, but the clip ends when the kid has just a few feet left in front of him. What happens next? It’s all very well being able to track skeletal movement, but players can only move around within a small area – usually, the space between their TV and their sofa. There are few in-game situations where this is adequate. As I will discuss later, designers will need to change their approach to motion if they want to keep presenting games in navigable 3D worlds.

Penalty shoot-outs? Fine. It might take some practice to learn where the virtual ball is, but perhaps you could put some kind of penalty spot on the floor. The skateboarding game suffers the earlier problem of jumping around, and I’m not sure how well players could do skateboard tricks with an imaginary skateboard – I’m not a skater, but I do have an A-level in physics, and I wonder how often players could ollie off a solid floor without breaking their ankles. The other features all seem feasible - if a little dull - but they hint towards the second big problem.

The one thing that all of these clips have in common is that every single controllable character is facing either directly into, or out of, the screen. This is a symptom of my second big problem with camera-controlled games - when placed beneath the TV, a camera will always be staring at the player’s front. Whatever gestures players might make, they will always be made in the direction of the screen, and I think many players would be a little put off if their characters weren’t facing the same direction, or mirrored back out of the screen.

Problem 1: Character Movement and Gestures

Character movement is something that I have yet to see any motion controlled game do well. In my opinion, Wii Sports remains one of the best showcases of the Wii’s motion control in action, but four of the five games require no movement whatsoever, and the tilt-to-sidestep boxing controls are only appropriate because you’re fighting in a confined space. Most Wii games get around this problem by either bringing in the nunchuck’s analogue stick or using some kind of point-and-click movement system, neither of which could be easily reproduced with just a camera.

Traditionally, players push on a d-pad, analogue stick, joystick or keyboard key to issue a kind of ‘movement impulse’ command. So long as you hold right on the d-pad, Mario will run towards the right of the screen; so long as you hold W, your Heavy Weapons Guy will keep plodding forward. Most other commands - kick, jump, use, shoot - are more like one-off instances. These are what you see a lot of in the product vision, because it’s relatively easy to equate a gesture (eg. you throwing a punch) with an in-game action (eg. Zangief throwing a punch). But following this kind of interface logic, maintaining an ‘impulse’ command - like pushing on an analogue stick to move - would equate to ‘posing’, rather than ‘gesturing’, and that doesn’t sound very immersive or intuitive at all.

I think the most common outcome will be that developers will create games that don’t require characters to move, in the conventional sense - games that are designed around a series of connected scenes, rather than contiguous 3D worlds. Think of games like Dragon’s Lair, Snatcher, Day of the Tentacle, or Mario Party, where you enter a scene, perform a few actions and maybe select an exit. This would tie in with the idea that motion-control systems are supposed to attract non-gaming audiences - after all, if people want to play traditional games, they can just pick up a control pad and play normally. There’s a great number of party game collections on the Wii that attest to this line of thinking.

I think a better solution is to develop games where movement is controlled through gestures. Thinking through the problem backwards, this means creating games that are controlled by instance-based commands rather than sustained impulses. The driving game is a good example - once you have a system to control your speed (Burnout’s foot control is an intuitive solution, since it vaguely imitates the use of pedals in a car), the player only really uses gestures to indicate a change of direction. The young skateboarder builds up speed by making a ‘pushing’ gesture with his right foot, and presumably stops by putting his foot down to the side of his imaginary deck. Another example would be some kind of 3D flying game where the player moves their arms to flap their virtual wings - each instance of the ‘flap’ gesture would generate a certain amount of thrust, similar to Joust.

Of course, some styles of traditional game would be very well-suited to this kind of gesture-based control scheme - imagine a game like Black & White played using the kind of dragging and stretching gestures associated with the iPhone. As a former Warhammer player, I quite like the idea of running a miniature kingdom, streaking across fields and villages with one sweep of my mighty hand! Since these kinds of management/strategy/God games don’t anchor the player to particular characters, they can drag, push and flick their way around a full 3D environment without questioning the laws of physics - in a sense, the omnipotent player is manipulating and moving the game world around their static position. Of course this doesn’t address the problem of character movement, it just ducks around it. But because these kinds of games are navigated by a series of unique viewpoint adjustments rather than a perpetual motion, they are very well-suited to a gesture-based interface.

The biggest Natal project revealed so far - Lionhead’s Milo & Kate - uses a disembodied camera for this reason. It evades the problem of character control by addressing the player directly, as they stand in their living room. The central character romps around the environment according to their own ‘free will’ (and a liberal dose of scripting, no doubt) and has a window into the real world that matches the player’s window into the virtual. Whatever you think about the game itself, this is clearly the kind of progressive, user-friendly interface that Microsoft hope will sell the 360 to confused non-gamers, but it comes with its own caveats. Firstly, the player is now anchored to an NPC who may or may not agree to explore the game world - Milo & Kate continues Peter Molyneux’s campaign to create emotionally engaging characters instead of intellectually engaging rules. Secondly, without some kind of on-screen representation (a floating hand, a crosshair, a character), it’s difficult for players to judge their relative position within the game - see my comments on that penalty shoot-out clip - which is pretty essential for any kind of dynamic interaction.

The PS3’s motion control wand sits somewhere between Natal and the Wii remote. The most natural use for the device - as shown in all the tech demos - is to manipulate virtual hand-held items from a first-person perspective. This could be great! I’d love to play a cross between Orcs & Elves and Mazan. But at the same time, it seems a bit more restrictive than the totally controller-free Natal. As the Wii has demonstrated, focusing your game interface onto a handheld device makes the rest of your body seem a bit useless, like you’re operating a one-armed robot. Even having an analogue stick on the device would be a bit pointless, if you’re expected to wave it around while holding down directions. But since the PSEye can effect some degree of bodily tracking too, the final outcome depends on how well developers can bring all of its capabilities to work together.

Problem 2: Camera Orientation

The other thing that bothers me about these camera-controlled systems is the way that they stare directly out from underneath the screen. In my opinion, space in games is best represented in the X and Y axis, using the width and height of the screen, so that players can accurately judge distance and motion. Unfortunately, these cameras are always positioned to look along the Z axis, so this becomes the logical axis on which action should take place - after all, players want to act in the direction they are facing. This is a troublesome conflict of priorities. As a result, it can be assumed that players will always want to face the camera. Whenever designers start thinking about how their players will move, they will have to keep this weird Vitruvian Man model in mind.

Of course people will always experiment. Here’s a handy video taken from Sega SuperStars for the EyeToy:

See what a mess this is? The guy’s either twisting his head round - definitely not good for your neck - or facing the screen and fighting sideways, which just looks ridiculous. I used to wonder whether it would be worth moving the camera to face the player’s side, so they they could at least face the screen while attacking across it, but I’m not sure that would be much better. A key element of immersion is to minimise the amount of time that players spend ‘thinking outside the game’ - the time they spend asking themselves questions like ‘Where is the grenade button?’ or ‘What is my Heal gesture?’, for example. Any control system that rotates your movement around 90º would force players to mentally translate game events and their own reactions between the two planes, which would be hugely distracting.

Going back to the Vitruvian Man comparison, I think it’s also worth asking whether players will want to stand up and jump around whenever they want to play a game. I know I don’t. And while variety games like Wii Sports or Rayman: Raving Rabbids continue to drag people off their sofa for a few minutes at a time, you’d be hard pushed to find a ’serious’ game that can’t be played while seated - I’m thinking of games like Mario, Zelda, and my beloved No More Heroes.  And I don’t think anyone would want to jump out of their seat and change a tyre whenever their friend decides to take a virtual pit stop.

At this stage, the precise capabilities of Natal are unclear, but if it really can track multiple players’ hand gestures then I think there’s some good potential for multiplayer games. Going back to the product vision video, the more realistic gaming situation can be seen in the quiz show game - the family are all sitting on the sofa, and the system is using facial recognition and motion tracking to keep track of which player is doing what. I think there’d be a great deal of interest in games that groups of players can enjoy from the comfort of their sofa - off the top of my head, maybe some kind of wizard duel game where you use hand gestures and keywords to cast spells. Come to think of it, that sounds like the ninjitsu techniques from Naruto… I wonder how long it will be before that game comes out?

Even then, there’s no way that a camera will ever be able to read gestures as quickly as a control pad can register button inputs. Gesture-controlled games must inevitably run at a slower pace than those with a digital interface - which may or may not be a good thing, depending on how much you enjoy studying frame data. The PS3’s motion control wand can bridge this gap a little with its mounted buttons, but Natal will always require a more relaxed sense of timing.

Wrapping Up

All in all, there’s no doubt that these new interface devices should lead to some interesting new game experiences. I don’t think they will totally reinvent gaming from the ground up, though. As I’ve said before, gamers love all the user-unfriendly junk that puts non-gamers off - the complex control schemes, the impossible AI opponents, the tedious stats. Milo & Kate looks interesting, and there’s no doubt I’m going to buy it, but if I had to choose between it or Street Fighter IV, I’d pick Zangief any day of the week. I think most self-confessed gamers would do the same.

That said, there’s plenty of scope for deisgners to build on existing game design principles, so long as they stay focused on the player’s experience. After all, there’s no point in building games around unique motion control features if people don’t enjoy playing them. Games should lessen their reliance on precise timing, and controls should be based on unique gestures rather than sustained commands. I wrote in my dissertation about how the development of 3D games led to player-characters becoming a focal object that often obstructs the player’s view of the game world… as camera-controlled games improve, I think it will often be a good idea to remove the on-screen player-character altogether.

Finally, following their continuing deviation away from the hardcore market, I think Sony and Microsoft will need to be careful not to alienate their core consumers with endless previews of touchy-feely ‘casual’ games. For their own sake, they need to maintain a balance between progressive, intuitive, gentle experiences, and punishing, hardcore challenges. In my experience, core gamers are responsible for the initial spread of new hardware, bringing it into homes and persuading their friends and relatives to give it a chance. It’s only after the sales figures have built up, people start coming out and talking about it to their friends at work, and a few key journalists have had a good go on their kids’ new Christmas presents, that we see the kind of mass market recognition and ballooning popularity that the Wii has enjoyed.