Final Project Documentation

Project Overview


Project Summary

Individual immersion characterizes most VR experiences. The wearer puts the headset on and suddenly disconnected from his or her surroundings, dives into another space. Especially for exhibition purpose, other audiences without HMD cannot get involved in the VR experience. My thesis is trying to find a way to solve this particular problem by making shared awareness between HMD (Head-mounted display) wearer and other observers. I am aiming to examine the shared awareness in a designed roleplay experience with customized HMD, VR/AR apps and speculative futuristic scenario by combining cyberculture and criticizing human-animal relationship.

Research Questions

  • How to make shared awareness in a VR experience between HMD wearer and other observers to solve the problem of single immersion?
  • How to make active communications and interactions between HMD wearer and other observers?
  • How to incorporate cyberculture and cyberpunk in the design of the VR experience with shared awareness?
  • How to incorporate wearable technologies, 3D printing and laser cutting to customize the VR head-mounted display?
  • How to overcome the trust issues between HMD wearer and other observers?
  • How to set the goals, constraints for the HMD wearer and other observers in the shared VR experience?
  • How to use the VR experience with shared awareness between HMD wearer and other observers to discuss the relationship between human and animals by showing different perspectives?

My thesis is exploring possibilities to make the shared awareness between HMD wearer and other audiences based on the problem of single immersion that exists in most VR experience. I argue that a good VR experience should not only focus on the HMD wearer but also focus on other audiences/obersers. I am aiming to have several prototypes with user testing to make and improve the shared awareness by merging parallel technologies such as IoT, wearable computing and augmented reality. The outcome of my thesis will be a roleplay experience with customized HMD, VR and AR apps that take place in an exhibition for a group of people.

Why

The single player in VR dominant for a while, there is a need to explore other ways to engage audiences. Individual immersion in VR cuts off the social connections and awareness between the virtual world and physical world, between the HMD wearer and other viewers. As a consequence, people will get bored of VR very easily. Besides, the overall situation in the current market of HMD all has a dull appearance, which also shows the lack of attention for other audiences. I want to bridge the connections between the two types of people and also using my research results to speculate what future HMD will be.

How

The methodologies I intend to use for my thesis is the combination of Research Through Design, User-centered Design, Agile Design and Speculative Design. From the technical and physical making part, I will split different steps to make several prototypes in an iterated matter to help to improve the shared awareness in VR by researching and experimenting with wearable technologies, internet of things and augmented reality. From the user experience part, I will test out with physical and digital prototypes on a large range of testers to gain more feedbacks to contribute to my design process. I will also use speculative design to make a futuristic scenario as a background story as narrative cues for the roleplay experience.

So What

My thesis is hopefully contributing to the new wave of innovation of mixed reality that will bring more social aspects to this field. It’s becoming more and more important to share our own experiences with people. I want to encourage other artists, designers, researchers to pay more attention to the relationships between HMD wearer and other audiences when designing and developing a VR experience. Furthermore, I want to use the shared experience with different perspectives as an artistic medium to bring a critical discussion on the human-animal relationship.

Context Review


Artificial Nature: Inhabitat. – Haru Ji and Graham Wakefield

inhabitat-0 (1).jpg
Image source from: http://www.artificialnature.net/

Inhabitat was one of the research projects and an evolving series of art installations. Inhabitat offers a poetic, interactive experience of a mixed reality landscape with a virtual ecosystem that is populated by artificial organisms whose rules can be rewritten while participants interact within them. The artists have built in a feature to directly engage other audiences, by making their creatures vulnerable to extinction by the shadows that people cast when walking through the installation.

This project has two perspectives, one is from the HMD wearer who has the perspectives of the organism living in the artificial ecosystem, the other one is from the other observers who have the godlike perspectives. The artists designed the experience both for the HMD wearer and other audiences with interactions and different perspectives. I think this project successfully made the shared awareness between HMD wearer and other audiences and it overcame the problem of single immersion in VR.

Shooting Game. – EGO

IMG_4620
Image source from: http://www.sohu.com/a/204976575_555194

Shooting Game simultaneously demonstrates two parallel worlds by blurring the boundary between virtual reality and real life, which renders the artwork a profound sense of immersions. A participator was led onto a platform surrounded by projections on all four sides. The participator then started a light-hearted shooting game, where the system applauds every time when a paperman target gets to shoot. Meanwhile, in the outside world, the lifeless targets were transformed into running children. Audience standing on the outer circle were able to hear their screams and saw them dropping dead onto the ground. Yet the shooter remained oblivious to this, until at the end of the VR game when the screen finally showed a name list of the children he shot dead, actually the true victims shot dead in wars, and playback the whole shooting process.

This project provides two different perspectives with distinct visual contents. The artists designed both for the HMD wearer and other audiences. Although it didn’t make the shared awareness it overcame the problem of single immersion on HMD wearer and other audience who can also be part of the experience. I think this project brings a critical discussion on the virtual world and real world with humanities and ethics.

Flock – David Lobser

Flock is a shared, immersive, co-located, multi-user VR experience made for a group of people. The artists also describe it as a gamifies sandbox, an interactive music video and a ritualized LARP (Live Action Role Playing Game). Each participant wears an HMD, a headphone and a pair of wings attached to their arms. The participants are role playing birds and they will have their representative avatar in VR. In the experience, the participants are encouraged to call like birds and flapping their wings instead of speaking to each other. The more insects they eat, their birds’ avatars will grow and become more colorful.

Even this project belongs to another research field which is social VR and not directly having the problem of single immersions as the fact that every user has the access of an HMD, it still related to what I am aiming to do for my thesis. Firstly, the whole experience became a show, if there were extra audiences without an HMD they can still watch them with an awareness of what happened to these people because they were acting and behaving like birds even with a pair of physical wings. Secondly, it related to one of my other research questions on human-animal relationship and animal’s perspectives. Thirdly, it uses a role-playing game format which is also one of my intends to explore the narrative cues and interactions.

Prototype Process


Ideation

Scanned Documents_Page_1
Prototype suggestions from my cohorts

In the early ideation process of the designing the prototype, we had the group discussion and mind mapping. I wrote about my thesis title, short description and prototype ideas on a poster and my cohorts give me the suggestions as follows:

  • Role Play
  • Use case Diagram
  • Storyboard
  • Charades
  • Ask people if they would use your concept
  • Concept testing to get ideas of interactions people would want
  • Storyboarding for a multiplayer game people who is wearing VR and people who is not, need to work together to complete a goal. Observe the interactions between them.
  • Asymmetric design
  • Haptic feedback
  • Think of physical interactions between the person in + persons out
IMG_4673
My Prototype Mindmap

Then based on the suggestions I received, I sketched out a mindmap of my prototype with the key elements such as users, interactions, location, and objects. At this point, I have an initial plan of staying away from the technology first and focusing more on the actual experience. Thus I came out with the idea of making a physical game to explore possible interactions between HMD wearer and other audiences to answer my research question of making shared awareness between them as seen in the prototype sketch in figure 3.

IMG_4674
Physical Game Sketch

This physical game is splitting my classmates into a group of three people, person A to be a player, person B to be an interrupter, person C to be a commander. Each round lasts for 3 minutes. If person A reached the destination, commander win. If person A didn’t reach the destination, interrupter would win. Switch the positions for three times.

  • Person A – Player: You will be blindfolded and you need to reach out to the destination by the commands made by the commander, you can completely trust him/her and they will make you safe.
  • Person B – Interrupter: You are allowed to make sounds, move chairs around and touch person A. Your goal is to try your best to prevent Person A from reaching the destination.
  • Person C – Commander: You need to make clear commands to person A and help him/her to reach the destination. Your responsibility is to make person A safe and don’t make him/her stumbled and fell down.

In this prototype, I incorporated the theory of cybernetics and information communications with a sender, a receiver and noise. Person A is playing the role of the HMD wearer who has no control of the physical world and only has limited access to interact within the virtual world. Person B is playing the role of the audience who can interact with the HMD wearer in the virtual environment like moving objects around, making sound and haptic interactions. Person C is playing the role of the storytelling cues in the virtual reality for the player to finish the whole experience. I interviewed each participant to ask for the feedback to add to my user experience design.

Video records:

https://www.youtube.com/watch?v=Tpdg_qVE8BE

https://www.youtube.com/watch?v=uLtHilRp7ss&t=30s

https://www.youtube.com/watch?v=lNSQBwpe4Q4

https://www.youtube.com/watch?v=uJB2XRjSbfU

https://www.youtube.com/watch?v=lXkV1B_L24w

Design thinking

IMG_4768

Verbs:

HMD wearer: Stomp, Walk, Crunch, Jump

Other audience: Watch, Tap, Hold, Swipe

Value: Awareness + Cooperation

When I wrote down the verbs for my players, I considered both the HMD wearer and other audiences. As I am thinking to make the game characters to be animals who have social behaviors like rabbits, so I mainly use the verbs that rabbits would have, such as stomp, walk, crunch and jump. And for other audiences, I mainly use some verbs for the interactions on the smartphone, such as watch, tap, hold, swipe. There are surely more verbs I would use for both users, but I would like to start with these simple ones first to test out. The values I choose for my prototype are awareness and cooperation because the main purpose for my thesis is making shared awareness between the HMD wearer and other audiences. And after my physical prototype, I found the cooperation between two kinds of users are quite interesting.

Digital prototype

Based on my last prototype, I then went ahead to develop my digital prototype. From the more technical aspect of this user testing, I want to test out the wearability of my headset, the usability of the AR app, the functionality of the IoT communications. Beyond that, I also want to test the artistic concept behind relationship and different perspectives of human and animals. As the fact of the time scope, I wasn’t able to make the communications between AR and VR, but that part could be added to my next iteration.

This prototype including four parts:

  1. A DIY VR headset with LEDs, servos and felts marker for the HMD wearer. (Prototype from my independent study)
  2. An Android device running VR scene with the designed rabbit’s vision.
  3. An IOS device running AR app which can control the components on the headset.
  4. A background story

Research about rabbit vision:             

Screen Shot 2018-08-03 at 21.05.26
Image source from:https://www.youtube.com/watch?v=eFYKeLu9tG8&t=70s

I did some research on rabbit vision and want to figure out what color and angles that a rabbit can see. Evidently, they can discriminate between the wavelengths we call “green” and “blue.” Although rabbits may not perceive green and blue the way humans do, they can tell them apart. This means they have limited color vision, probably conferred by two different categories of cone cells (blue and green).  And I also found a video which simulates rabbit vision which basically only have gray and blue-greenish colors.

Technical specs:

  • MAC OS: High Sierra 10.13.5 (17F77)
  • iPhone OS: 11.3.1 (15E302)
  • Android: 7.0
  • Xcode: 9.4 (9F1027a)
  • Vuforia: 7.1.35
  • Unity 3D: 2018.1.4f1
  • Android JDK: 1.8.0

Based on the initial research, I then created the rabbit’s vision in Unity 3D with downloaded 3d assets of a forest. I used photoshop to change every texture to fit with the rabbit’s color sensing.

Screen Shot 2018-08-03 at 21.18.00.png
Working VR Scene in Unity
Screen Shot 2018-06-28 at 19.17.55
AR App Development in Unity (Prototype in my Independent Study)

I didn’t change the AR app very much which I have done in my independent study with some placeholders of fire camp and buttons. However, I did receive feedbacks that my users were wondering why there was a fire camp on the HMD wearer’s head and they also found it’s difficult to touch on the buttons in the AR app.

giphy
Interactions between the HMD wearer and other audiences

Revision Ideas:

I tested with this VR scene for the first time in a group of people with my cohorts and professors, one of the important feedbacks is that the testers would prefer more instructions. The other one is the improvement for the AR app. It’s also interesting to know most of my HMD testers feel the ears movement is very robotic, so I am thinking how about to just make her/him play the role of robotic rabbit (cyborg rabbit)?

So based on the feedbacks, firstly, I came up with the idea of speculating a background story which can both be instructions for the participants but also to immerse and position my participants in a cyberculture futurist setup to further introduce the idea of human-animal perspectives.

Secondly, I would like to improve both the AR and VR apps. For VR, I wanted to add a lab scene outside the forest to make sense of my background story and also improve the graphics quality. For AR, I wanted to replace the small buttons with a 3D rabbit model, so instead of touching on the buttons, my user can touch the specific parts of the rabbit model to interact with the corresponding parts on the headset.

Screen Shot 2018-08-03 at 11.19.13 (2).png
Final VR Scene with lab

Speculative background story: Today is 26th July 2038, My name is Yiyi Shao, I’m the chief scientist at AniBot Lab. Welcome welcome! It’s an exciting day to the public our new project RabBot. We implanted two robotics ears and eyes to our test subject. It’s still the first stage of our experiment, but we saw a remarkable success. Now may I invite our guest to wake up RabBot1.0.

critique.jpg

To conclude,  I added ta lab scene in the VR to aligned with my speculative background story. So my tester will have a sense of she/he is in an artificial forest that scientists create in the lab. I also improved the AR app by adding a rabbit 3d object placeholder, so when the AR tester touches the ears, the ears on the HMD will move and the LED will light up. When she/he touched the eyes, the LED will turn off.

Project Roadmaps


roeadmap.jpg

Above is the project roadmap for my thesis plan, I’ve put some important dates/deadline for each particular stage. The prototype and writing doc should be developed at the same time and contributes to each other. I am hoping to make two more prototypes to make it more polish also conduct more user testing to gain more feedbacks.

Flowcharts of user experience


Workflow.jpg

 

The diagram above is the initial user flow of the interactions between the HMD wearer and other observers which includes the whole loop between two of the users. The looping interactions are the fundamental condition to examine my research question on the shared awareness between the HMD wearer and other observers. As the ongoing development of the prototypes and user testing, more interactions will be added to the flowchart.

Test Report


Physical Prototype Report

Method: Group Testing, Interview, Open discussion.

Testers: My cohorts and instructors

Questions and interviews:

To person A

  • Have you tried VR before? Is it reminding you of any VR experience?
  • What do you think is the most challenging part for you to reach the destination?

To person B

  • What do you think is the most effective part you did to interrupt?
  • What do you wish you could have to stop person A to reach the destination?

To person C

  • What do you think is the most effective part you did to guide person A to reach the destination?
  • What other things do you wish you could have to help person A to reach the destination?

Feedbacks:

Person A

  • Most participants tried VR before and it’s reminding them of wearing headset but not knowing what happened outside of the VR.
  • Most participants can figure out a strategy quickly to move forward.
  • Vocal cues/commands from the commander are the most helpful part.
  • Other noises in the space are very destructed.
  • The lost of vision is difficult for most participants.
  • More constraints and rules to would be more difficult for the participants.

Person B

  • Some participants wish they can simulate the voice of the commander to confuse the player.
  • Most participants worried they were doing too much to interrupt and gave less interruptions towards the end of the game.

Person C

  • Secret code to communicate with the player
  • Clarifications of the directions with the player at the beginning of the game.
  • More clear and straightforward way to guide the player.
  • Body configurations

Summary:

  • Most participants tried VR before and it’s reminding them of wearing headset but not knowing what happened outside of the VR.
  • Most participants can figure out a strategy quickly to move forward.
  • Vocal cues/commands from the commander are the most helpful part.
  • Other noises in the space are very destructed.
  • The lost of vision is difficult for most participants.
  • More constraints and rules to would be more difficult for the participants.
  • Some participants wish they can simulate the voice of the commander to confuse the player.
  • Most participants worried they were doing too much to interrupt and gave fewer interruptions towards the end of the game.
  • Secret code to communicate with the player
  • Clarifications of the directions with the player at the beginning of the game.
  • More clear and straightforward way to guide the player.
  • Body configurations

Relations to the research question: My main research question is how to make shared awareness between HMD wearer and other observers. This prototype is to explore possible interactions which can improve the shared awareness of my target users through three directions: corporations, interruptions, instructions.

Digital Prototype Report

Method: One-to-one testing, Interview, Open discussion

Testers: Self-testing, My cohorts, Instructors, Professors, VR Experts, People I don’t know

In-session observations:

  • The HMD wearer moved their head in a large-range once they put on the headset to look around in the VR. The observer who is holding the ios device had to chase the movement of the headset to be able to trigger the AR control panel.
  • The HMD wearer didn’t feel any uncomfortable when wearing the headset and their body movement is pretty natural.
  • The HMD wearer tried to walk around and wondered the interactions in the VR.
  • The observer with the ios device had some difficult to touch the button in the AR app.
  • The sound in the VR scene is not loud enough.
  • Some people have some difficulties to adjust their eye focus in VR, I need to tell them to watch trees that far away, which helped a lot.

Questions and interviews:

To HMD wearer:

  1. How do you feel like as a rabbit in VR?
  2. How do you find the VR scene?
  3. How do you feel like when the ears are moving?
  4. Do you feel any uncomfortable when wearing the headset?
  5. Is the headset heavy to you?

To AR App observer:

  1. Do you find the app to be difficult to use?
  2. How do you feel when you can interact with the HMD wearer?

Feedbacks:

  • The headset is not heavy at all, It’s quite comfortable to wear.
  • I would prefer more instructions from you.
  • The menu in AR app is too small.
  • I wonder why there is a campfire on the headset.
  • The sound is too low, I can’t hear anything.
  • I wonder If there is any interaction I can do in VR.
  • If I were a rabbit, I would prefer to see my pink noise, furry feet and pows when I look down.
  • Wow, I feel like I am so small like a rabbit.
  • I prefer the small movements of the ears, I feel like the big movements are too robotics.
  • I can see some shadow of trees is pink.
  • I wonder if a rabbit can turn their neck at such a big angle.
  • I found the graphics in VR are a bit rough.
  • I feel it’s a bit blur in VR but when I look far it will become better.
  • Your headset can’t fit me, I can’t see it well.

This prototype provides three different perspectives on the participants. These perspectives including the HMD wearer, the AR observer and other observers (Who didn’t have access to any device). The shared awareness between the HMD wearer and other observer was examined through the following parts:

  • The speculative background story and roleplaying
  • The interactions of AR and components on the HMD
  • The design/appearance of the HMD
  • The design/appearance of augmented HMD wearer

Final Critique Report

We have a final critique section at the end of the prototyping course and I received very important feedback. One of the feedbacks is that both of the VR experts have the distortion problem after wearing the headset and they can’t see the VR scene very well. This happened only once during my user testing and it gets solved after I told the user to look something far away and focus on the radical pointer. However, the VR expert team still have the problem, and I didn’t have time to adjust the wheel for them. This brought my attention to the differences of each person’s face, although I’ve already chosen to hack an existing headset which already has the universal design with the consideration on this issue, it still won’t work in some cases.

Install sketches


install sketch.jpg

I am thinking to decorate the space to be aligned with the speculative background story like a futurist lab by either poster, small objects or projection mapping in a cyberpunk style. Maybe some LEDs, plants, previous prototype and a sign of the lab name.

Reflections


Physical Prototype:

What happened: Roleplay physical game to simulate the scenario of communications and interactions between HMD wearer and other audience. The blindfolded player can figure out a strategy and win very quickly.

What worked: Physical tasks worked very well, vocal commands are useful.

What didn’t work: Passive interruption.

What I learned: Lack of constraints/restrictions on players, interaction patterns for replayability (Comments from cohorts: A want for physical interaction between viewer & VR user). Demands on more physical interactions, succinct and encoded communications

What next: Moving on to a digital version with more constraints. (Comments from cohorts: Technology development or game development? Which elements of the physical experiment are you looking to preserve? How do people not in VR play in the same game?)

I think the whole ideation process with my cohorts and instructions is very useful. We helped each other with feedback and suggestions by sticky notes, group discussions, which helped me to gain a lot of insights to come up with the new design. I was able to run the user testing for this prototype 4 times on different people, through the observations and interviews, the results are providing me the fundamental supports for my future development. It also cleared out my assumptions of what my users would do to interact with each other, especially an experience for a group of people with different perspectives and roles. Based on the received feedback, more thoughtful rules need to be appropriately designed in the game experience. The changing of the VR scenes, responsive objects and sound will be designed with the interactions from the outside world. So for my next prototypes, I will incorporate these interactions together with social animal behaviors to make a shared awareness in VR and further discuss the relationship between human, animals and nature.

Digital Prototype:

What happened: Roleplay VR and AR experience with a speculative futuristic background story.

What worked: Interaction between AR and physical components on HMD. The speculative story. Roleplaying to provide shared awareness and different perspectives for an HMD wearer, observers who have access with a smartphone to run AR, and observers who don’t access on any device.

What didn’t work: Lack of instruction at user testing, Vision distortion in VR at final critique

What I learned: To be fully prepared to help the HMD wearer to adjust the vision distance because each person has different distances between eyes.

What next:  Communications between AR and VR app. Try with different headsets to figure out a better module to hack.

I have run the user testing for this prototype five times which covered a larger range of testers including myself, my cohorts, strangers, professors and VR experts. I found the user testing is so useful and contributed more to my design. For example, the speculative story came up after the first rounds of user testing on Tuesday because my testers didn’t have a clear idea of what they can do ( also happened in my physical prototype), so I think it’s important to make an instruction to my testers in a more interesting way. Based on the past experience on the physical prototype, I designed a role-play futurist scenario for everyone to position themselves to be part of this experience. Furthermore, It helped me to find a way to combine my theoretical framework of cybernetics, speculative design and set an artistic style (cyberpunk) for me to work on further for the next interactions. I do appreciate every feedback and questions I received so far, they made me more clear of what I am aiming to research on for my thesis. At this point, I want to make it clear that my research questions are all centered on the shared awareness between the HMD wearer and other audiences. Multiplayer VR is not within the field that having individual immersion issue. Redesigning a VR headset is not my research area and can’t fully make the shared awareness between the HMD wearer and other audiences. 

Future Plans and Next Steps

I want to keep working on a more consistent cyberpunk style, for example, the representation of the rabbit should be aligned with the rest of the other design. I want to find a way to solve the issue of the VR distortion that happened to different people, it’s interesting to know that it didn’t happen on the user testing on my cohorts, but it did happen on the VR experts. The speculative story seems to make more sense as part of the experience, I would like to work more on that to see if it will help to improve the shared awareness and human-animal relationship.

Furthermore, I definitely want to make the interactions and communications between VR and AR, for example, simulate what happened in my physical prototypes to make it digital like moving objects around in the VR scene. And I want to select a better VR headset case to hack on which will have more adjustments for each person.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s