By Samantha Phillips
Youngstown State University student Jenna Wise and professor Bonita Sharif traveled to Italy in September to demonstrate and present a paper on iTrace — a software tool developed in the software engineering and empirical studies lab in the department of computer science and information systems.
iTrace tracks and collects eye movements of developers while they’re reading code. This helps researchers and developers figure out what process someone took to fix a bug in the system.
“You can imagine a scenario where, solely based on your eye movements, you would want to recommend someone something when they’re stuck,” Sharif said. “Our eye movements change a lot when we have a highly cognitive load. We can recommend things for them. That’s what we’re looking into, how we can recommend things on the fly.”
Sharif and her students have been working on iTrace for about two years. Three of the students who worked on it have graduated. Wise was recruited about six months ago to continue development and add new features.
“It took a while, but now we have it in a functional prototype form and people can actually use it,” Sharif said. “It is open source, so [people] can download it and use it.”
Wise helped improve this device over the last six months. She added a display that helps researchers calibrate their eyes with the program and set up an organization system for the collected data. She said she is passionate about this project.
“We can see the specific method they’re looking at and for how long, and where are they jumping to with their eyes,” Wise said. “So we can really track their patterns and where they’re looking, and from there we can really determine what they were thinking at that specific time. You can really do detailed data analysis to determine that, with the data we’re outputting.”
Sharif said the tool sits underneath the monitor and shines infrared lights on a developer’s eyes. The reflection tells the eye tracker where the researcher is looking on the screen and can even pinpoint the XY coordinate. iTrace then collects data on where the eye gaze is, maps it to the source code element and stores the information to a specific folder.
“The cool thing about it is that it’s not limited to one screen. They can be scrolling stuff around, they can be opening many different files, and all this time it’s collecting data,” Sharif said. “[You don’t need to use your hands for it], and it’s not on them at all. It collects all this data without any intrusive equipment.”
One of the papers they presented in Italy was about a research study that was conducted at the ABB Corporate Research Center in North Carolina. Sharif and Braden Walters, another student, brought the equipment to the company and had 10 professional developers come in and use it. They had them look through lines of codes to find and eliminate the bugs in them while recording their eye movements.
“We would see how the software engineering developers would fix the bug, and compare it to how it was actually fixed,” Sharif said. “You might fix something in file A, but that’s just a fix, we don’t know how you got to it. The eye tracking tells us you looked at B, C and D and then you fixed A.”
Wise said that is something developers do on a daily basis, and that iTrace showed them how it can make finding a solution easier.
Together with a team of researchers and developers in Switzerland, Sharif and Wise wrote a paper on the study and submitted it to the Foundations of Software Engineering conference.
“We got accepted, which is pretty amazing. We had really good reviews,” Sharif said. “We also submitted a tool paper on iTrace, and that’s what Jenna was involved in, she actually helped to make some additions to iTrace to make this in publishable form and submitted that. So we had two papers.”
Wise has been to many conferences in the U.S., but had never traveled abroad. That made the trip special for her.
“I was super excited we got accepted, and was very excited to travel abroad for once in my life. Dr. Sharif told me that about only 20 percent of papers that are submitted got accepted, so it was a big deal to be accepted,” Wise said.
The National Science Foundation provided funds for Wise to travel overseas and present the paper. They took a smaller version of the tool because customs wouldn’t let them take the bigger one. High-profile researchers in the software engineering community watched the presentations given by Sharif and Wise.
“People who we referenced were in the room. I was nervous and excited at the same time. We had really good feedback and everyone really had good criticisms on how we could improve this tool,” Sharif said. “We made more collaborations; we were working on other things with people, so that’s always a good thing.”
Wise said Sharif introduced her to the professors she knew, from all across the world. Some of them invited her to do graduate student research at their schools. She said she would have never made the connections she made if she hadn’t attended the conference.
“Jenna did a good job presenting the system there, and everybody was interested in what we were doing and they gave us some nice suggestions to actually improve the program as well,” Sharif said. “So, we are planning a future grant submission on it; hopefully we can get more money and support more students, to improve this even faster.”
Wise said they plan on adding a new feature that will allow iTrace to not only map a developer’s gaze to elements or codes, but also to bug reports and stack overflow documents, which usually contains the solution to a technical problem.
“We are working on getting more money for iTrace to do more studies. We are working with a team in Canada. So, the new feature, once we implement it, they are going to use it in a study and we are going to help them along with that study. And that will be a paper or a couple papers submitted,” she said.
Sharif sees a lot of potential in her students and wants to get more of them involved in the project.
“We have some pretty amazing students,” she said. “They just need the motivation and the incentive to build something, and they can do it.”