From the smartphone to the Cloud and back again
Mike Panciera had already helped a blind man navigate the perilous fantasy worlds of video games. It made sense that the next step would be to design a mobile app to help the blind find their way through the interiors of real buildings. It’s a challenge working in the real world, but at least there won’t be any monsters.
Panciera ’14 was a computer science major and one of the students in Robert Dickerson’s inaugural Mobile Cloud Computing I class. The goal of the class is to create apps—the compact programs that add functionality to smartphones and tablets —that take advantage of the enormous computing power and data-storage potential of the Cloud.
Dickerson is a visiting assistant professor in William & Mary’s Department of Computer Science. He says that there are a number of reasons to yoke a smartphone app to the Cloud.
“Apps are often designed with power savings in mind. They stay on as long as people are interacting with it, but if no key presses have been seen in a while, the app will turn itself off and save the program data into the phone,” he said. An app connected with the Cloud can store more data and use the ever-increasing processing power of the devices themselves to retrieve the data, he noted.
Many of the most popular apps are games and it’s appropriate that Mobile Cloud Computing’s class project is the Werewolf Game. Dickerson came up with the Werewolf Game concept and the class was tasked with making it work and refining the rules. Here’s how a finished version of the game might work:
Let’s say you’ve been one of the class members who have been selected, randomly and anonymously, as one of the werewolves. You’re at dinner one evening when your phone buzzes. The game has determined that the virtual moon is full. It doesn’t matter that you haven’t sprouted the diagnostic facial hair—you just got your werewolf on.
You resume your dinner and your phone buzzes again. The alert lets you know that there is a townsperson—your legitimate prey—within “scent radius.” Across the dining room you see the townsperson, a classmate, sitting there eating pizza with her friends. You get up, and amble over. You need to get within “attack radius” of your victim. You pull out your phone, as if you’ve just received a text. You punch a few keys—and your classmate puts down her slice, pulls out her phone and learns that the population of the town has gone down by one—her. But Dickerson said that it’s not in your self-interest to howl or otherwise celebrate your kill; exposure can be deadly for the werewolf.
“In the morning all the townspeople get a report that this townsperson was killed at such-and-such a location. During the daytime all the townspeople vote on who they think the werewolf is,” Dickerson said. “And the townspeople assemble to get rid of that person because they think they’re a werewolf. So the game continues until the number of werewolves exceeds the number of townspeople or all the werewolves have been eliminated.”
The Werewolf Game is a fun, light-hearted vehicle for introducing students to the possibilities of tapping the Cloud and the demands of working in the Android and iOS platforms, but the final projects were more practical.
“People can work by themselves or with another partner on this project and I give them a lot of freedom,” Dickerson explained. “I’ve really encouraged them to build both the web service side and also the app side. I’m looking for a strong motivation that this app needs to exist; I want them to figure out what similar apps exist and how their app would improve the state of the art or fill a niche.”
Panciera’s sightless navigation app fills a niche, but even this very practical app has a basis in video games. Panciera had gone to school in Blacksburg with a guy named Ameen Ahmed, but they lost touch when Ahmed was held back. Panciera said he didn’t realize until much later that Ahmed didn’t have academic troubles—he had been losing his sight. The two reconnected about two or three years ago, when Ahmed had become legally blind.
“He needed help playing video games. He likes to play video games. It’s like Pinball Wizard,” Panciera said. “That’s how our friendship got started up again.”
Panciera found that Ahmed was playing video games without sight, specializing in the Final Fantasy series. Ahmed needed a sighted assistant’s help to explain how to navigate a few of the more heavily visual aspects of the game. Panciera stressed that his friend didn’t need much help.
“These games are in no way intended for a blind person to use it. He’s able to situate himself by the sound of footsteps and listening to the audio clues built into the game,” Panciera explained.
He conceived of an app that would harness the same skills Ahmed uses to navigate the virtual worlds of the Final Fantasy franchise and apply them to help the blind find their way through real brick-and-mortar buildings. The app uses audio clues, such as footsteps and sound indicators for nearby doors.
Panciera’s goal is to produce a finished app that is both a mobility-training aid as well as a navigation device. The app shows its video game DNA in its training aspect—users collect keys and jewels as they learn to navigate the main services building of the Virginia Rehabilitation Center for the Blind and Visually Impaired. Panciera refined his app in Dickerson’s Mobile Cloud Computing II class, a seminar-style extension of the fall semester class.
“Mike has built relationships with some of the people in the Rehabilitation Center,” Dickerson said. “He’s looking to make this more of a kind of research project, involving several blind people in his study.”
Many projects from Mobile Cloud Computing I were influenced by Dickerson’s own background in ubiquitous computing and the development of sensor networks with health care applications. Oren Ely’s Aego App, for instance, would organize, via mobile device, a network of caregivers for a homebound person. Another app, being developed by Sami Mirimiri, Ryan Novak and Alan Tennent, proposes to make a smartphone into a portable electroencephalography (EEG) device.
“With people who suffer from epilepsy, neurologists want to know what kind of brain signals are happening in the head before their seizure,” Dickerson said. “So, we were wondering—can you wear an EEG all the time?”
The students’ EEG Mobil App proposes to collect brainwave data and perform signal processing algorithms via a headset attached to a smartphone. The app stores the data in the Cloud, where it can be accessed by a neurologist, who can examine the brain activity immediately preceding a seizure.
Cloud-using apps are a challenge and Dickerson said he knew that some of the apps would require development beyond the time allotted to for the courses. He said he was looking for killer apps, not necessarily finished apps.
“I encouraged them to go ahead and take risks, aim for the stars, and come up with a prototype,” he said. “It might not be polished and perfect, but go ahead and figure out the core technical contribution you want to try to make and what new novel application area you want to target. Then I’m impressed.”