MERCED - A year from now, if you're watching a person in a motion-capture exoskeleton interacting with a giant high-resolution screen, you could be watching the special features on the latest CGI-effects action movie - or you could be watching a research demonstration in the Cognitive Sensorium and Visualization Facility at the University of California, Merced.
Professors Marcelo Kallmann and Shawn Newsam of UC Merced's School of Engineering have teamed up with Professor Teenie Matlock of the School of Social Sciences, Humanities and Arts and won a $250,000 grant from the National Science Foundation's Major Research Instrumentation program to build the facility.
Inside, the Cognitive Sensorium and Visualization Facility will house eye tracking equipment, a full-body motion capture system and data gloves for real-time body and hand motion capture, and 3-D projection capability with active stereo eye wear, along with a 7'x15' "Power Wall" screen. Everything will be integrated - the immersive environment projected on the screen, the eye tracking, and the motion capture - so that the researchers can use the data they gather to see how it all fits together.
That integration and the resulting interdisciplinary collaboration will set this facility apart from other research universities, Matlock said.
"We are starting with the idea of using interactive full-body motion capture and eye tracking combined with immersive virtual reality for doing cognitive science research" she explained.
She is planning collaborative studies in the new facility that can track eye movements in very large simulated scenarios - something not possible on the small screens she currently uses in her research.
In addition to the cognitive science research that Matlock does - learning where people point and look when they describe an environment, for example - the facility will be used to gain data that could later be used in robotics or virtual characters for computer games.
Kallmann specializes in computer algorithms for generating complex motions when a person does more than one thing at a time - like approaching a bookshelf while reaching for an item.
"We'll use sensors to capture a person's motion in real-time, even down to shoulder and knee joint rotations," he said. "Then we can analyze and understand that data, and finally apply it, developing better algorithms that can control robots or virtual characters."
Kallmann already has an undergraduate student working on integrating a data glove he will use in the new facility with a computer model of a skeletal human hand.
"When we get data about how humans view images, that can better inform the way computers process images," said Newsam. "For example, I'm planning to study how we view satellite images like the ones we see on Google Earth. In the future that may improve our automatic analysis of those images."
Other applications may be possible as well, the professors said. For example, biology researchers could use it for huge, high-definition visualization of unfolding proteins. Or it could become part of UC Merced's distance medicine efforts. The professors are working with the campus administration to find the best space to take advantage of all the equipment's potential.
"I think NSF has recognized the opportunity to make a difference early on with UC Merced," Newsam said. "We're hoping this facility will become a showpiece for the campus."
The material collected in the facility will be useful in courses taught on campus, Kallmann added, like his computer graphics and motion planning courses.
"Students can complete their class projects in the lab, or even participate in the studies", he said.
"Once we have this equipment, it will open up many possibilities for additional funding," Matlock added. "The preliminary experiments we complete will help us justify the need for more interdisciplinary work in cognitive science and computer science and engineering."
Plans for the new facility should progress rapidly once a space on campus is identified. Matlock, Newsam and Kallmann anticipate that they will start receiving the equipment in a few months and start to set it up around the beginning of next academic year.
###
