Prototyping Your Immersive Experience (Blender)

Week 1- 360 3D Content using Blender

Figure 1: Underwater Life, Marsa Alam, Egypt. 360 video in 8K.

What is a 360° Video and How does it work?

A 360° video is a type of video that lets you see everything around you in all directions. Unlike regular videos that only show what’s in front of you, 360° videos let you look up, down, left, or right by moving your phone, dragging with your mouse, or turning your head in a VR headset.

These videos are made with special cameras that record every angle at once. The footage is then stitched together to create a complete, seamless view, so you can explore the entire scene as if you were there.

Things to Consider When Creating a 360° Video:

  1. Camera Placement: Position the camera where viewers would naturally stand, as everything around it will be visible.
  2. Storytelling: Plan for action in all directions, so viewers can explore the scene.
  3. Lighting: Ensure even lighting, as shadows or bright spots can distract across the wide view.
  4. Audio: Use spatial sound to guide the viewer’s attention.
  5. Stitching: Ensure smooth transitions between camera feeds to avoid visible seams.
  6. Viewer Comfort: Avoid quick movements to prevent motion sickness.

Workshop 1: Prototyping Your Immersive Experience with 360° 3D Content using Blender

Lab Exercise: Creating a 360° 3D Scene in Blender

Objective: In this lab, we will learn how to prototype immersive 360° environments using Blender. The goal is to create a simple scene that leverages the unique storytelling capabilities of a 360° canvas. We will focus on basic scene setup, lighting, and camera placement to explore how story spheres (narrative spaces) can guide user interaction and immersion in VR environments.

The process of creating a 360 animated scene:

Planning:

  • Starting by designing my story, mapping out what the audience should experience in all directions. Creating a storyboard to visualize key moments.
Figure 2: 360 Scene Storyboard created in Illustrator
  • Scene Design & User Perspective:
    Design your scene with movement not only in the user’s direct line of sight but across the entire 360° environment. Encourage the user to look around to fully experience the scene.
  • Camera Setup:
    Create a new VR camera and position it at the centre of your scene.
  • Animation Creation:
    Develop a 10-second animation at 30 frames per second, aiming for around 300 frames total. This ensures smooth motion while keeping the rendering time manageable, approximately 30 minutes.
  • Rendering Settings:
    Render using Blender, set at 1280×640 resolution with a 2:1 aspect ratio (ideal for YouTube 360° videos). For single image renders, use the JPEG file format. For animated sequences, render using PNG format.
  • Post-Production:
    After rendering, import your image sequence into Adobe Premiere to create a 10-second video. When exporting for 360° video, ensure that the frame layout is set to Side-by-Side, not Over/Under.
Figure 3: Creating 3D Environment in Blender

The concept for this scene is to depict a dynamic forest environment. Rocks will fall from the sky, smashing into larger rocks upon impact, breaking into smaller chunks that sink into the ground. Surrounding trees will interact with the movement—some trees will sink and rise out of the ground, while others will emerge from the earth.

To enhance the contrast and add a natural light source, I will import a sky texture to create a more realistic atmosphere.

 

Figure 4: Rendered 3D environment image

Figure 5: Testing my animation of 360 scene from blender.

Problem with 360 Scene Upload to YouTube

During the development of my 360 scene, I encountered an issue where the scene works flawlessly when tested on my media and TV app on my laptop, but fails to function properly when uploaded to YouTube. The 360 features and interactivity that should be available are not performing as expected on the YouTube platform.

To troubleshoot the problem, I attempted the following solutions:

  1. Switched to another web browser – I tried using different browsers to rule out any browser-specific issues, but the problem persisted.
  2. Updated the YouTube application – Ensuring the YouTube app was up to date, I tested the upload again, with no change in the results.
  3. Updated Graphics Card drivers – My graphics card is quite limited and cannot be upgraded, but I ensured the drivers were updated to the latest version available for my system.
  4. Cleared cache data – I cleared the cache on both YouTube and my web browsers to eliminate any potential cached data conflicts.
  5. Turned on Graphics acceleration – Enabling graphics acceleration where available in browser settings did not resolve the issue.
  6. Tried professional video repair software – As a last resort, I used Video Repair Software to see if there was any underlying corruption in the video, but it didn’t fix the problem.

Figure 6: Testing my animation of 360 scene with Films & TV.

Figure 7: 360 Scene created with Blender and uploaded in my YouTube Channel.

Conclusion

Despite these troubleshooting steps, the issue remains unresolved. I believe the limitations of my graphics card and potentially missing metadata during the YouTube upload process may be contributing factors.

Workshop 2: Blender: Geometry Nodes – Blender’s MASH equivalent

What is geometry nodes in Blender?

Geometry Nodes in Blender is a feature that allows users to create and manipulate 3D models using a node-based system. Instead of manually modeling objects, you can use nodes to define shapes, instances, and transformations, making it easier to create complex geometry and effects procedurally.

First Tutorial

In my first tutorial, I experimented with shattering objects using the “Cell Fracture” add-on combined with Geometry Nodes. This allowed me to break apart models in a controlled, procedural way, creating dynamic and visually striking effects.

Figure 8: Applying cell fracture to the monkey model

Testing shatter and destroy object with cell fracture in Blender.

Figure 9: Testing shatter and destroy object with cell fracture in Blender.

Second tutorial:

For my geometry nodes second tutorial, I wanted to experiment with an audio visualizer.

By adding audio, adjusting various values, and applying noise to the cubes, I’m creating a visual graphic that is manipulated by the soundtrack. This journey from simple cubes to an audio visualizer demonstrates how something impressive can be created from almost nothing.

Figure 10: Testing Audio Visualizer in Blender.

Figure 11: Audio Visualizer Blender Tutorial Animation.

Conclusion:

Geometry Nodes is an incredibly versatile tool, and I feel like I’ve only just begun to tap into its full potential. It’s fascinating to explore and has a lot of promise for future use. While my main focus is on a different project, I’m definitely considering how Geometry Nodes could contribute to creating more dynamic visual graphics for upcoming work. It’s a concept that I’ll be diving deeper into as I continue experimenting and pushing the limits of what it can do.

Week 2 – WebVR

Figure 12: Steps for Signing Up and Setting Up the Environment
Figure 13: Creating my avatar

When I first entered the FrameVR platform, I was asked to create both an account and an avatar. After completing the setup, I was transported into a world of my choosing. I started by experimenting with the environment, incorporating images, text, and objects to create a mini gallery. FrameVR offers great potential for creating fully immersive and interactive portfolio experiences. Unlike the previously mentioned 360VR videos, WebVR allows for full interactivity. This interactive nature deeply engages users, giving them options and the ability to physically explore different aspects of the virtual design.

FrameVR Experience link – Frame (framevr.io) 

The following screenshots showcase the development of my custom WebVR experience, which highlights the exploration of a new wine lounge.

Figure 14: Configuring and exploring features within FrameVR.
Figure 15: Incorporating 3D objects, such as the bar and winery, to enrich the visual experience of the VR narrative (via Sketchfab).
Figure 16: Integrating "Bar Sound" to enhance the dynamism of the experience (sourced from Pixabay).
Figure 17: Incorporating text to explain the VR scene and experimenting with various 3D text effects for added visual impact.
Figure 18: Exploring the VR scene from multiple angles for a comprehensive view.
Figure 19: Exploring the VR scene from multiple angles for a comprehensive view.
Figure 20: Integrating "Bar Sound" to enhance the dynamism of the experience (sourced from Pixabay).
Figure 21: Viewing the VR scene from a seated perspective for a more immersive experience.

Figure 22: Final screen recording of the FrameVR wine lounge experience, showcasing an in-depth exploration of the virtual space.

Conclusion:

The WebVR experience focuses on exploring a wine lounge, incorporating 3D assets to elevate the overall environment. These elements, such as wine cabinets, barrels, wall displays, the bar, and decor, contribute to a gallery-like ambiance while adding depth and dimension to the scene. The inclusion of sound effects enhances realism, immersing the user in the virtual setting. Incorporating interactivity would further enrich the experience, offering a fully immersive engagement. Future WebVR designs could benefit from these interactive elements, providing users with a more dynamic and engaging virtual exploration.

Reference List:

AirPano VR (2022) Underwater Life, Marsa Alam, Egypt. 360 video in 8K. [Video].  Underwater Life, Marsa Alam, Egypt. 360 video in 8K. (youtube.com) [Accessed 27 Sep 2024].

Amaksi (2024) Night Detective. Pixabay [Download]. Night Detective | Royalty-free Music – Pixabay [Accessed 1/ Oct 2024].

Maio, A. (2019) What is 360 Video? How to Create An Immersive Experience [Blog post]. StudioBinder. 5 December. What is 360 Video? How to Create An Immersive Experience (studiobinder.com) [Accessed 29 Sept 2024].

Pixabay (2022) General Chatter In Bar. Pixabay [Download]. Bar Sound | Royalty-free Music – Pixabay [Accessed 3/ Oct 2024].

Selin, E. (2022) Blender geometry nodes fundamentals guide [Blog post]. Artistic Render. 23 July. Blender geometry nodes fundamentals guide – Artisticrender.com [Accessed 1 Oct 2024].

Sketchfab Search (no date) Sketchfab. Available at: Sketchfab [Accessed: 03 October 2024]. 

Shukla, S. (2024) Fix: YouTube 360-degree video not working issue [Blog post]. RecoveryFix. 10 June. Fix: YouTube 360-degree video not working issue (recoveryfix.com) [Accessed 30 Sept 2024].