Research Studies in Computer Engineering Rewarded: “Smart” Lives with Sound and Visual

Boğaziçi University students Heysem Kaya and Tuğçe Özkaptan, working under the guidance of Asst. Prof. Albert Ali Salah and Prof. Sadık Fikret Gürgen, won an important overseas award in paralinguistics at “Interspeech 2014”.  

Under the guidance of Asst. Prof. Salah, second year Gül Varol, a 2nd year master’s student in Computer Engineering, was also recognized for her work entitled “Extreme Learning Machine for Large-Scale Action Recognition” at the “Thumos Challenge 2014”. We talked about the international achievements of Boğaziçi University students with Asst. Prof. Salah, who was the advisor of the students and their projects. 

“Interspeech 2014”, where Heysem Kaya received an award, was held in Singapore in September. The Interspeech conference brings together researchers who work on speech recognition and includes workshops in various fields, and scientific and industrial exhibitions; over a thousand participants attend it every year. The field of “paralinguistics” aims to process the human voice and automatically reveal the human condition; for example, to understand if a person is stressed, afraid, depressive or happy, or to ascertain certain characteristics about a person, such as personality traits. Heysem Kaya, who works in this field, told us about the competition process:

“The competition we took part in involved understanding whether a person was fatigued from varied variations in vocal cues and predicting cognitive preoccupation. Competition officials collected the speech data and experts marked them. After the competition began, they sent the data to participating groups and the marks by the experts were kept a secret. Once we set up our system, we sent in the system’s predictions on the basis of the data. All groups were allowed to send in a total of five different results based on all the data, so we didn’t have the chance to repeatedly try different system parameters. Because we were making predictions on data we couldn’t see, it was important for the system to make generalizations, and we succeeded at doing that.”

Potential Use in Distance Learning and in the Education of Children with Autism

Asst. Prof. Salah talked about how the project could be used in daily life, noting that vocal cues provided a lot of information about a person’s state. “The immediate computer analysis of human behavior brings with it many potential fields of application. For example, people with autism experience difficulties in perceiving emotions; we can fill in these gaps with various technologies or help educate them. We can monitor children with autism continually as we teach them to speak and provide automatic feedback. We can transfer some of these tasks, especially those that require constant repetition, to computers and lighten the workload of educators and care providers. Our efforts in this area are ongoing.” Salah also noted that this project could be applied in the field of distance learning.

“Distance learning is presently a very important topic. Assessments show that students’ focused attention span is extremely short. Consequently, a more interactive dimension needs to be added to education. Computers are currently unable to do this, but with this project, we can get computers to perceive how fatigued students are or whether or not they have lost their focus, and act accordingly.”

Diagnosing Affect Based on Vocal Cues

Salah said this project could make it possible to detect depression and affect (anger, sadness, surprise, joy), determine fatigue level, and, for instance, enable instant analysis of the users or employees of companies that provided services by telephone. These technologies are expected to play a significant role in many areas, such as predicting driver fatigue in intelligent cars, adapting to the user in intelligent environments, modeling customer behavior, and monitoring the elderly in various modes. One of the premier conferences on this topic, the ACM International Conference on Multimedia Interaction, was held at Boğaziçi University in November.

Gül Varol Places Third at the Thumos Challenge 2014

Boğaziçi University student Gül Varol won an international award and placed third at the Thumos Challenge 2014, marking a significant achievement. The project that Gül Varol, 2nd year master’s student in Computer Engineering, accomplished with Asst. Prof. Salah as her advisor is called “Extreme Learning Machine for Large-Scale Action Recognition.” A total of 11 universities took part in Thumos this year, which is a competition held at the European Conference on Computer Vision, one of the most important conferences in the field of computer vision.

An Image System That Recognizes Movement

Gül Varol explained the details of the project that aims to recognize motion from video images and said they began the study, called Action Recognition, by identifying certain attributes in a 254-hour video data set. “After working on it for weeks, we wrote a computer program that can automatically classify 101 different actions from the data by using machine-learning techniques. We might say this process makes many different actions recognizable by the program. The program might, for instance, make it possible to direct the camera toward a fight that broke out at a metro station, and detect movements leading to vandalism. This can make it possible to take security measures faster.”

Boğaziçi University master’s student Gül Varol said she would be writing her thesis on the same topic; Varol aims to continue working on perceiving human behavior.

Asst. Prof. Salah spoke about how the project could be used in daily life, and said the ultimate objective was to automatically recognize each specific action. However, Salah noted that work was ongoing because much more data had to be collected and processed to reach this objective, adding that the program could also be very beneficial in Internet searches, making it possible to search not on the basis of writing but images.