Staff

[ Project Leader, Professor* ]
Satoshi Shioiri

Research Activities

While online classes and conferences have become popular and the limitations compared to face-to-face meetings have been identified, online communication is attracting many people in terms of new types of information usages. For example, facial expressions, which could be automatically recognized from video images of participants, have begun to be used for estimating mental states of participants. According to rapid increase of online classes, it is an urgent issue to research mental state estimations for the appropriate and effective use of video teaching materials, tablet teaching materials, etc. Knowing student’s mental states is useful for removing obstacles during learning and developing teaching methods of good creative activities, we expect that such technology improves learning environment dramatically. In this research, we aim to develop the method to estimate student’s attention state and use it for vicarious experiment for better and efficient education in the highly information-oriented society.
Assuming the use of video teaching materials and tablet teaching materials as a learning environment, the target of the present study is the position of attention, depth of attention, frequency of attention shifts and concentration of attention. Using the electroencephalogram measurement, the change in the attention state during learning is measured, which is used as the reference (ground torus). The purpose is to estimate the attention state from facial expressions and voices and to model the relationship with attention states. For the purpose, we measure facial images and voices as well as EEG during learning. To analyze these time varying measures online, it is necessary to use high speed and performance system based on the latest AI technology and FPGA (Field Programmable Gate Array). Since improving the learning environment is more important for people with disabilities, we will consider the system for people with visual/hearing impairments in this study. Vicarious experience of cognitive experience of persons with disabilities can be expected to make a great contribution not only to learning but also to support for and to communicate with persons with disabilities.