CJ Harris / University of Missouri
Oct. 25, 2022
Contact: Marcus Wilkins, wilkinsm@missouri.edu
A first-year nurse checks on a series of patients and goes about his rounds. With nothing serious to report, he heads down the hallway, sneakers squeaking.
Suddenly, a monitor alarm erupts and another caregiver shouts for help. A patient who was fine moments ago is now unresponsive — with dangerously low blood pressure and a stuttering heart rate. The nurse panics.
Should I call a code, or should I rush to the charge nurse? Where are my colleagues? Why are there so many people around suddenly? I can feel my chest tightening and my ears burning …
This is just one example of an escalating situation many nurses role-play en route to a degree — or as part of their professional training. But such exercises require a cast of simulation actors with memorized performances — not to mention dedicated time and space to act out the scenario.
Knoo Lee, a post-doctorate fellow in the Sinclair School of Nursing and the Preparing Future Faculty–Faculty Diversity program at the University of Missouri, is working on a virtual reality (VR) solution to traditional time-space-resource constraints that will supplement and reinforce real-life instruction.
Lee has teamed up with Fang Wang, associate teaching professor of engineering and information technology, to develop a VR training program that challenges nurses with unpredictable emergent circumstances to test their knowledge and nerve.
Welcome to Mizzou in the metaverse. Like Lee and Wang, thousands of Tigers across campus are learning practical skills by working in augmented, virtual and mixed realities.
“When nurses enter the workforce, we are seeing a significant portion leave within the first year,” Knoo said. “For some it’s because they are not prepared for these types of escalating situations, and they feel powerless. This virtual training program will serve to increase confidence and better prepare prospective nurses.”
To support the technical side, Wang’s team in the College of Engineering’s Collaborative Research Environments for Extended Reality Lab (CREXR) specializes in the creation of metaverse experiences that immerse humans into digital surroundings. The metaverse — an ever-closer future of cyberspace in which users are connected in virtual-, augmented- and mixed-reality environments — is the frontier of these technologies.
Wang’s lab recently developed a VR program for training students to operate equipment in an engineering laboratory. The team also collaborated with the MU Museum of Art and Archeology to create a virtual museum for the Missouri 2021 Bicentennial celebration. The program allows users to view paintings by artists from the Show-Me State and was recently released on the Oculus Store (like an app store but for VR).
“The nursing VR project has been uniquely interdisciplinary because, when we are building the environment and the characters, there are details about the emergency room and the equipment that we, as engineers, might not fully understand. Our partnership with nursing is what makes it accurate and immersive,” Wang said. “What makes VR, and eventually the metaverse, so exciting is that it makes the physical world accessible to almost everyone — regardless of location. It is especially powerful for these types of trainings.”
Read on about a few of the myriad Mizzou projects that are pushing technical boundaries, expanding classroom concepts and changing the way humans work together across space and time.
College of Arts and Science
Students use AR, VR and a 20-foot long immersive 3D screen for a variety of projects in the Immersive Visualization Lab housed in the Department of Architectural Studies and led by assistant professor Jong Bum Kim. In the AT&T Connected Climate Initiative Project, students explore 5G network-powered “digital twins” of MU campus buildings (Stanley and Gwynn halls) to reduce energy use. The research employs building information modeling, a process central to VR.
The Department of Textile and Apparel Management uses 3D apparel product development software allowing students to design, engineer and test new products using avatars in virtual settings. Students are also working with The Mizzou Store on the MizzouMirror project, an augmented reality tool that enhances the apparel-shopping experience by allowing customers to interact virtually with characters — such as Truman the Tiger — on a full-length screen.
Trulaske College of Business
The MU Innovation Space, housed in Cornell Hall, features AT&T 5G wireless technology and hardware that harnesses its capabilities. VR headsets are available to students, staff and faculty for gaming research and experimentation. They can also experience Healium, an immersive, therapeutic VR software that employs guided meditation in lifelike “outdoor” environments. In addition, the space provides 360-degree cameras for checkout.
College of Education and Human Development
The research of Xinhao Xu, an assistant professor in the School of Information Science and Learning Technologies and the director of the MU Technology to Enhance Learning Lab, focuses on how to help people use technology to learn. In the VR world, Xu has collaborated with Wang to refine the virtual engineering lab. He works with Sue Yun Fowler and Hillary Claunch, instructors in the Sinclair School of Nursing, as well as Jason Snyder, associate director of simulation in the Sheldon Clinical Simulation Center, on virtual patient-intake trainings for nursing students. He also developed a virtual commencement ceremony for use during the early part of the pandemic in 2020, a VR-based new student orientation system.
College of Engineering
In addition to the CREXR lab, engineering boasts cutting-edge research on artificial intelligence (AI) and a state-of-the-art transportation simulation lab. Derek Anderson, an associate professor of electrical engineering and computer science, and team are using simulated environments to train AI — essentially teaching machines to think more like humans — at fantastic rates. Carlos Sun, professor of civil and environmental engineering, and his team created ZouSim, a lab that collaborates with the Missouri Department of Transportation to track human behavior within simulated situations to make highways, sidewalks, airports and other transportation systems safer and more efficient.
School of Health Professions
Rachel Proffitt, an associate professor of occupational therapy and director of the VR-Augmented Rehab Lab, works with Marjorie Skubic — Curators’ Distinguished Professor, Robert H. Buescher Faculty Fellow and professor of electrical engineering and computer science — to improve the health and well-being of adults through the use of smart, home-based technology. Specific topics include in-home sensors to monitor sleep quality, vital signs, and to detect falls for older adults. Proffitt also researches the effectiveness of video-game based therapy for stroke patients.
School of Journalism
Students use 360-degree cameras to capture immersive content, learning how to extract themselves from the content while telling 3D stories. Taught by Jim Flink, the class uses Insta360 One X2 cameras. This semester, Flink is collaborating with Wang’s lab to convert the video to fully immersive content that can be viewed using a mobile device. Past projects have included interactive maps of Mizzou’s Botanic Gardens and Shelter Gardens in Columbia. Students have also captured skateboarding footage, guided museum tours and other virtual experiences.
