Keywords

1 Introduction

Nowadays, Virtual and Augmented Reality Head Mounted Displays (VR/AR HMDs) are more popular than ever due to their relative high quality and affordability. However, despite the wide availability of such technology, it is still in its infancy when it comes to gaming. For instance, most VR/AR games attempt to focus on a singular experience, such as the sole use of VR or AR, with limited interaction methods. While AR enables adding of digital contents to a real world’s visual view, VR enables immersive virtual environments and experiences that engage multiple senses. Taking advantage of both VR and AR within a gaming context has high potential to deliver a gaming experience that leverages the advances of both interaction mediums. Thus, we take the first steps to investigate a gaming experience that attempts to combine elements from both VR and AR.

We introduce WAR Bots, a mutiplayer battle game where users pilot a physical robot vehicle from within a virtual cockpit in first person. We discuss various design elements and explain how such experience is delivered to players. Finally, we present our future direction of this project.

2 Related Work

Previous work and projects have investigated the use of either AR or VR seperately to enhance the gaming experience. “AR Drone” [2] and “AR Quake” [3, 4], have introduced AR based gaming elements to piloting drones. Players can engage in a drone fight using the drones equipped cameras and sensors. The drone’s camera feed, viewed from a smartphone, is used to visualize missiles and opponents as well as other match related contents.

3 Approach

Improving on previous works, WAR Bots emphasizes the engagement of both VR and AR within a gaming experience for controlling robots. We extend each medium in the following method:

VR is utilized to immerse the player within the game’s vehicle cockpit (see Figs. 1 and 2). Here users are able to naturally view the vehicle cockpit, within which we are able to introduce interaction methods to enhance the player’s immersiveness and fun. Accordingly, this method enables flexible implementation of various interaction techniques and play mechanics that would suit various game types.

Fig. 1.
figure 1

Player’s view in cockpit, with live camera feed and AR contents in the middle.

Full size image
Fig. 2.
figure 2

Left and Right views of the cockpit as seen through the HMD.

Full size image

AR is used to extend the robot’s live camera feed of the real world by adding various 3D models, effects and other gaming elements. First, AR is used to enhance the opponents’ robot, by augmenting it’s physical appearance with a 3D model and energy shield (see Fig. 1). Elements such as player’s Heads-Up Display (HUD), and damage and weapon effects are also shown as part of the AR environment, enabling players to view weapon trajectories, executions and outcomes.

As a result, we believe that VR can be utilized to enhance immersiveness and introduce increasingly interesting interaction mechanisms to enhance the gaming experience. Likewise, AR enhances the physical camera view by enhancing real-world game elements whilst also introducing additional game related information.

4 Implementation

The game is implemented in a client-server design structure, each host maintaining it’s own connection to a single dedicated robot with all communications happening over wireless network (see Fig. 3). Game components such as hit points, shield points, ammunition, etc. are all held locally and are transmitted between corresponding clients, via the server, on such events as Shot Hit.

Fig. 3.
figure 3

WAR Bots system architecture.

Full size image

4.1 Hardware

The game will successfully run on any VR capable PC, such were those used in design and implementation. The HMD used during testing and implementation was the Oculus DK2 coupled an Xbox 360 controller which was mimiced in the cockpit design for added immersion (see Fig. 1). Two EZ-Robot Adventure Bots [1], strapped with HD webcams, used as the robot battle vehicles. Each of the robots was also fitted with an AR-marker shell (see Fig. 4 Left).

Fig. 4.
figure 4

Robot fitted with AR-marker shell and webcam (left) and overlayed AR model (right).

Full size image

4.2 Software

The entire project was developed within Unity3D game engine [7]. Vuforia [8] was the basis for all AR contents (see Fig. 4). Oculus DK2 HMD was utilized to deliver the VR experience (see Figs. 1 and 2). The robots were controlled wirelessly via a client-server architecture software script written between the robot and Unity3D.

5 Conclusion and Future Work

WAR Bots is a multiplayer game that combines elements of AR and VR. The game creates a sense of tele-presence for the user by having the player control a surrogate robot. The ability to tangibly interact with the robot in a cyber-physical environment, allows for a deeply immersive and highly enjoyable experience for all.

Future functionality includes upgraded interaction methods, such as motion and gesture control, customizable robots which are digitally represented in game including type benefits, augmenting environments with kinect sensors and projectors, and embedding nested markers.