Kaist

New Insights into How the Human Brain Solves Compl..

A new study on meta reinforcement learning algorithms helps us understand how the human brain learns to adapt to complexity and uncertainty when learning and making decisions. A research team, led by Professor Sang Wan Lee at KAIST jointly with John O’Doherty at Caltech, succeeded in discovering both a computational and neural mechanism for human meta reinforcement learning, opening up the possibility of porting key elements of human intelligence into artificial intelligence algorithms. This study provides a glimpse into how it might ultimately use computational models to reverse engineer human reinforcement learning.  This work was published on Dec 16, 2019 in the journal Nature Communications. The title of the paper is “Task complexity interacts with state-space uncertainty in the arbitration between model-based and model-free learning.” Human reinforcement learning is an inherently complex and dynamic process, involving goal setting, strategy choice, action selection, strategy modification, cognitive resource allocation etc. This a very challenging problem for humans to solve owing to the rapidly changing and multifaced environment in which humans have to operate. To make matters worse, humans often need to often rapidly make important decisions even before getting the opportunity to collect a lot of information, unlike the case when using deep learning methods to model learning and decision-making in artificial intelligence applications. In order to solve this problem, the research team used a technique called 'reinforcement learning theory-based experiment design' to optimize the three variables of the two-stage Markov decision task - goal, task complexity, and task uncertainty. This experimental design technique allowed the team not only to control confounding factors, but also to create a situation similar to that which occurs in actual human problem solving. Secondly, the team used a technique called ‘model-based neuroimaging analysis.’ Based on the acquired behavior and fMRI data, more than 100 different types of meta reinforcement learning algorithms were pitted against each other to find a computational model that can explain both behavioral and neural data. Thirdly, for the sake of a more rigorous verification, the team applied an analytical method called ‘parameter recovery analysis,’ which involves high-precision behavioral profiling of both human subjects and computational models. In this way, the team was able to accurately identify a computational model of meta reinforcement learning, ensuring not only that the model’s apparent behavior is similar to that of humans, but also that the model solves the problem in the same way as humans do. The team found that people tended to increase planning-based reinforcement learning (called model-based control), in response to increasing task complexity. However, they resorted to a simpler, more resource efficient strategy called model-free control, when both uncertainty and task complexity were high. This suggests that both the task uncertainty and the task complexity interact during the meta control of reinforcement learning. Computational fMRI analyses revealed that task complexity interacts with neural representations of the reliability of the learning strategies in the inferior prefrontal cortex. These findings significantly advance understanding of the nature of the computations being implemented in the inferior prefrontal cortex during meta reinforcement learning as well as providing insight into the more general question of how the brain resolves uncertainty and complexity in a dynamically changing environment. Identifying the key computational variables that drive prefrontal meta reinforcement learning, can also inform understanding of how this process might be vulnerable to break down in certain psychiatric disorders such as depression and OCD. Furthermore, gaining a computational understanding of how this process can sometimes lead to increased model-free control, can provide insights into how under some situations task performance might break down under conditions of high cognitive load. Professor Lee said, “This study will be of enormous interest to researchers in both the artificial intelligence and human/computer interaction fields since this holds significant potential for applying core insights gleaned into how human intelligence works with AI algorithms.” This work was funded by the National Institute on Drug Abuse, the National Research Foundation of Korea, the Ministry of Science and ICT, Samsung Research Funding Center of Samsung Electronics.    Figure 1 (modified from the figures of the original paper doi:10.1038/s41467-019-13632-1). Computations implemented in the inferior prefrontal cortex during meta reinforcement learning. (A) Computational model of human prefrontal meta reinforcement learning (left) and the brain areas whose neural activity patterns are explained by the latent variables of the model. (B) Examples of behavioral profiles. Shown on the left is choice bias for different goal types and on the right is choice optimality for task complexity and uncertainty. (C) Parameter recoverability analysis. Compared are the effect of task uncertainty (left) and task complexity (right) on choice optimality.   -Profile Professor Sang Wan Lee sangwan＠kaist.ac.kr Department of Bio and Brain Engineering Director, KAIST Center for Neuroscience-inspired AI KAIST Institute for Artificial Intelligence (http://aibrain.kaist.ac.kr) KAIST Institute for Health, Science, and Technology KAIST (https://www.kaist.ac.kr)

Professor Junil Choi Receives Stephen O. Rice Priz..

< Professor Junil Choi (second from the left) >  Professor Junil Choi from the School of Electrical Engineering received the Stephen O. Rice Prize at the Global Communications Conference (GLOBECOM) hosted by the Institute of Electrical and Electronics Engineers (IEEE) in Hawaii on December 10, 2019.   The Stephen O. Rice Prize is awarded to only one paper of exceptional merit every year. The IEEE Communications Society evaluates all papers published in the IEEE Transactions on Communications journal within the last three years, and marks each paper by aggregating its scores on originality, the number of citations, impact, and peer evaluation.   Professor Choi won the prize for his research on one-bit analog-to-digital converters (ADCs) for multiuser massive multiple-input and multiple-output (MIMO) antenna systems published in 2016. In his paper, Professor Choi proposed a technology that can drastically reduce the power consumption of the multiuser massive MIMO antenna systems, which are the core technology for 5G and future wireless communication. Professor Choi’s paper has been cited more than 230 times in various academic journals and conference papers since its publication, and multiple follow-up studies are actively ongoing.   In 2015, Professor Choi received the IEEE Signal Processing Society Best Paper Award, an award equals to the Stephen O. Rice Prize. He was also selected as the winner of the 15th Haedong Young Engineering Researcher Award presented by the Korean Institute of Communications and Information Sciences (KICS) on December 6, 2019 for his outstanding academic achievements, including 34 international journal publications and 26 US patent registrations.

Professor Shin-Hyun Kim Receives the Young Scienti..

    Professor Shin-Hyun Kim from the Department of Chemical and Biomolecular Engineering received the Young Scientist Award from the Korean Academy of Science and Technology.   The Young Scientist Award is presented to a promising young Korean scientist under the age of 40 who shows significant potential, passion, and remarkable achievement.   Professor Kim was lauded for his research of intelligent soft materials. By applying his research, he developed a capsule sensor material that can not only be used for sensors, but also for displays, color aesthetics, anti-counterfeit technology, residual drug detection, and more.   The award ceremony took place on December 14 at the Gwacheon National Science Museum. The Korean minister of Science and ICT delivered words of encouragement, reminding everyone that “the driving force behind creative performance of scientists is the provision of continuous support.” He added, “Researchers of Korea deserve greater public attention and support.”

Team Geumo Wins Consecutive Victories in K-Cyber S..

   < Professor Sang Kil Cha >       < Masters Candidate Kangsu Kim and Researcher Corentin Soulet >   Team Geumo, led by Professor Sang Kil Cha from the Graduate School of Information Security, won the K-Cyber Security Challenge in the AI-based automatic vulnerability detection division for two consecutive years in 2018 and 2019.   The K-Cyber Security Challenge is an inter-machine hacking competition. Participants develop and operate AI-based systems that are capable of independently identifying software vulnerabilities and gaining operating rights through hacking. The K-Cyber Security Challenge, inspired by the US Cyber Grand Challenge launched by the Defense Advanced Research Projects Agency (DARPA), is hosted by the Ministry of Science and ICT and organized by the Korea Internet and Security Agency.   Researcher Corentin Soulet of the School of Computing and master’s student Kangsu Kim of the Graduate School of Information Security teamed up for the competition. Professor Cha, who has led the research on software and systems security since his days at Carnegie Mellon University, succeeded in establishing a world-class system using domestic technology.   In a recent collaboration with the Cyber Security Research Center, Professor Cha achieved a ten-fold increase in the speed of binary analysis engines, a key component of AI-based hacking systems. For this accomplishment, he received the Best Paper Award at the 2019 Network and Distributed System Security Workshop on Binary Analysis Research (NDSS BAR).   Kangsu Kim said, "It is a great honor to win the competition two years in a row. I will continue to work hard and apply my knowledge to serve society.”

Two Professors Receive Awards from the Korea Robot..

< Professor Jee-Hwan Ryu and Professor Ayoung Kim >   The Korea Robotics Society (KROS), in recognition of their achievements and contributions to the development of robotic technology and industry in 2019, awarded two KAIST professors from the Department of Civil and Environmental Engineering.   Professor Jee-Hwan Ryu has been actively engaged in researching the field of teleoperation, and this led him to win the KROS Robotics Innovation (KRI) Award. The KRI Award was newly established in 2019 by the KROS, in order to encourage researchers who have made innovative achievements in robotics. Professor Ryu shared the honor of being the first winners of this award with Professor Jaeheung Park of Seoul National University.   Professor Ayoung Kim, from the same department, received the Young Investigator Award presented to emerging robitics researchers under 40 years of age.

New Liquid Metal Wearable Pressure Sensor Created ..

Soft pressure sensors have received significant research attention in a variety of fields, including soft robotics, electronic skin, and wearable electronics. Wearable soft pressure sensors have great potential for the real-time health monitoring and for the early diagnosis of diseases. A KAIST research team led by Professor Inkyu Park from the Department of Mechanical Engineering developed a highly sensitive wearable pressure sensor for health monitoring applications. This work was reported in Advanced Healthcare Materials on November 21 as a front cover article. This technology is capable of sensitive, precise, and continuous measurement of physiological and physical signals and shows great potential for health monitoring applications and the early diagnosis of diseases. A soft pressure sensor is required to have high compliance, high sensitivity, low cost, long-term performance stability, and environmental stability in order to be employed for continuous health monitoring. Conventional solid-state soft pressure sensors using functional materials including carbon nanotubes and graphene have showed great sensing performance. However, these sensors suffer from limited stretchability, signal drifting, and long-term instability due to the distance between the stretchable substrate and the functional materials. To overcome these issues, liquid-state electronics using liquid metal have been introduced for various wearable applications. Of these materials, Galinstan, a eutectic metal alloy of gallium, indium, and tin, has great mechanical and electrical properties that can be employed in wearable applications. But today’s liquid metal-based pressure sensors have low-pressure sensitivity, limiting their applicability for health monitoring devices. The research team developed a 3D-printed rigid microbump array-integrated, liquid metal-based soft pressure sensor. With the help of 3D printing, the integration of a rigid microbump array and the master mold for a liquid metal microchannel could be achieved simultaneously, reducing the complexity of the manufacturing process. Through the integration of the rigid microbump and the microchannel, the new pressure sensor has an extremely low detection limit and enhanced pressure sensitivity compared to previously reported liquid metal-based pressure sensors. The proposed sensor also has a negligible signal drift over 10,000 cycles of pressure, bending, and stretching and exhibited excellent stability when subjected to various environmental conditions. These performance outcomes make it an excellent sensor for various health monitoring devices. First, the research team demonstrated a wearable wristband device that can continuously monitor one’s pulse during exercise and be employed in a noninvasive cuffless BP monitoring system based on PTT calculations. Then, they introduced a wireless wearable heel pressure monitoring system that integrates three 3D-BLiPS with a wireless communication module. Professor Park said, “It was possible to measure health indicators including pulse and blood pressure continuously as well as pressure of body parts using our proposed soft pressure sensor. We expect it to be used in health care applications, such as the prevention and the monitoring of the pressure-driven diseases such as pressure ulcers in the near future. There will be more opportunities for future research including a whole-body pressure monitoring system related to other physical parameters.” This work was supported by a National Research Foundation of Korea (NRF) grant funded by the Ministry of Science and ICT.     < Figure 1. The front cover image of Advanced Healthcare Materials, Volume 8, Issue 22. > < Figure 2. Highly sensitive liquid metal-based soft pressure sensor integrated with 3D-printed microbump array. > < Figure 3. High pressure sensitivity and reliable sensing performances of the proposed sensor and wireless heel pressure monitoring application. >   Profile: Prof. Inkyu Park inkyu＠kaist.ac.kr Micro/Nano Transducers Laboratory  http://mintlab1.kaist.ac.kr/  Department of Mechanical Engineering KAIST

New IEEE Fellow, Professor Jong Chul Ye

(Professor Jong Chul Ye at the Department of Bio and Brain Engineering)   Professor Jong Chul Ye from the Department of Bio and Brain Engineering was named a new fellow of the Institute of Electrical and Electronics Engineers (IEEE). IEEE announced this on December 1 in recognition of Professor Ye’s contributions to the development of signal processing and artificial intelligence (AI) technology in the field of biomedical imaging.   As the world’s largest society in the electrical and electronics field, IEEE names the top 0.1％ of their members as fellows based on their research achievements.Professor Ye has published more than 100 research papers in world-leading journals in the biomedical imaging field, including those affiliated with IEEE.   He also gave a keynote talk at the yearly conference of the International Society for Magnetic Resonance Imaging (ISMRM) on medical AI technology. In addition, Professor Ye has been appointed to serve as the next chair of the Computational Imaging Technical Committee of the IEEE Signal Processing Society, and the chair of the IEEE Symposium on Biomedical Imaging (ISBI) 2020 to be held in April in Iowa, USA.   Professor Ye said, “The importance of AI technology is developing in the biomedical imaging field. I feel proud that my contributions have been internationally recognized and allowed me to be named an IEEE fellow.”

Professor Il-Doo Kim Named Scientist of the Year b..

             (Professor Il-Doo Kim from the Department of Materials Science and Engineering) Professor Il-Doo Kim from the Department of Materials Science and Engineering was named the 2019 Scientist of the Year by Korean science journalists. The award was conferred at the 2019 Science Press Night ceremony of the Korea Science Journalists Association (KSJA) on November 29.   Professor Kim focuses on developing nanofiber gas sensors for diagnosing diseases in advance by analyzing exhaled biomarkers with electrospinning technology. His outstanding research was praised and selected as one of the top 10 nanotechnology of 2019 by the Korea Nano Technology Research Society (KoNTRS), the Ministry of Science and ICT (MSIT), and the Ministry of Trade, Industry and Energy (MOTIE).   Professor Kim was honored with the QIAN Baojun Fiber Award, which is awarded every two years by Donghua University in Shanghai, China to recognize outstanding contributions in fiber science and technology. Professor Kim was also elected as an academician of the Asia Pacific Academy of Materials (APAM) on November 21 in Guangzhou, China.   In May, Professor Kim was appointed as an associate editor of ACS Nano, a leading international research journal in the field of nanoscience. In his editorial published in the May issue of ACS Nano, Professor Kim introduced and shared the history of KAIST and its vision for the future with other members of the journal. He hopes this will help with promoting a closer relationship between the members of the journal and KAIST moving forward.   “Above all,” he said in his acceptance speech, “the greatest news for me as an educator is that the first PhD graduate from our lab, Dr. Seonjin Choi, was appointed as the youngest professor in the Division of Materials Science and Engineering at Hanyang University on September 1.”

A System Controlling Road Active Noise to Hit the ..

The research team led by Professor Youngjin Park of the Department of Mechanical Engineering has developed a road noise active noise control (RANC) system to be commercialized in partnership with Hyundai Motor Group.  On December 11, Hyundai Motor Group announced the successful development of the RANC system, which significantly reduces the road noise flowing into cars. The carmaker has completed the domestic and American patent applications for the location of sensors and the signal selection method, the core technology of RANC. RANC is a technology for reducing road noise during driving. This system consists of an acceleration sensor, digital signal processor (the control computer to analyze sound signals), microphone, amplifier, and audio system. To make the system as simple as possible, the audio system utilizes the original audio system embedded in the car instead of a separate system. The acceleration sensor first calculates the vibration from the road into the car. The location of the sensor is important for accurately identifying the vibration path. The research team was able to find the optimal sensor location through a number of tests. The System Dynamics and Applied Control Laboratory of Professor Park researched ways to significantly reduce road noise with Hyundai Motor Group for four years from 1993 as a G7 national project and published the results in international journals. In 2002, the researchers published an article titled “Noise Quietens Driving” in Nature, where they announced the first success in reducing road noise in actual cars. The achievement did not lead to commercialization, however, due to the lack of auxiliary technologies at the time, digital amplifiers and DSP for cars for example, and pricing issues. Since 2013, Professor Park’s research team has participated in one technology transfer and eight university-industry projects. Based on these efforts, the team was able to successfully develop the RANC system with domestic technology in partnership with Hyundai’s NVH Research Lab (Research Fellow, Dr. Gangdeok Lee; Ph.D. in aviation engineering, 1996), Optomech (Founder, Professor Gyeongsu Kim; Ph.D. in mechanical engineering, 1999), ARE (CEO Hyeonseok Kim; Ph.D. in mechanical engineering, 1998), WeAcom, and BurnYoung. Professor Park’s team led the project by performing theory-based research during the commercialization stage in collaboration with Hyundai Motor Group. For the commercialization of the RANC system, Hyundai Motor Group is planning to collaborate with the global car audio company Harman to increase the degree of completion and apply the RANC system to the GV 80, the first SUV model of the Genesis brand. “I am very delighted as an engineer to see the research I worked on from my early days at KAIST be commercialized after 20 years,” noted Professor Park. “I am thrilled to make a contribution to such commercialization with my students in my lab.”   

New Members of KAST 2020

   < Professor Zong-Tae Bae (Left) and Professor Sang Ouk Kim (Right) >   Professor Zong-Tae Bae from the School of Management Engineering and Professor Sang Ouk Kim from the Department of Materials Science and Engineering became new fellows of the Korean Academy of Science and Technology (KAST) along with 22 other scientists in Korea.   On November 22, KAST announced 24 new members for the year 2020. This includes seven scientists from the field of natural sciences, six from engineering, four from medical sciences, another four from policy research, and three from agriculture and fishery.   The new fellows will begin their term from January next year, and their fellowships wll be conferred during the KAST’s New Year Reception to be held on January 14  in Seoul.

KAIST and Google Jointly Develop AI Curricula

  KAIST selected the two professors who will develop AI curriculum under the auspices of the KAIST-Google Partnership for AI Education and Research. The Graduate School of AI announced the two authors among the 20 applicants who will develop the curriculum next year. They will be provided 7,500 USD per subject. Professor Changho Suh from the School of Electrical Engineering and Professor Yong-Jin Yoon from the Department of Mechanical Engineering will use Google technology such as TensorFlow, Google Cloud, and Android to create the curriculum.   Professor Suh’s “TensorFlow for Information Theory and Convex Optimization “will be used for curriculum in the graduate courses and Professor Yoon’s “AI Convergence Project Based Learning (PBL)” will be used for online courses. Professor Yoon’s course will explore and define problems by utilizing AI and experiencing the process of developing products that use AI through design thinking, which involves product design, production, and verification. Professor Suh’s course will discus“information theory and convergence,” which uses basic sciences and engineering as well as AI, machine learning, and deep learning.

‘Carrier-Resolved Photo-Hall’ to Push Semiconducto..

     (Professor Shin and Dr. Gunawan (left))   An IBM-KAIST research team described a breakthrough in a 140-year-old mystery in physics. The research reported in Nature last month unlocks the physical characteristics of semiconductors in much greater detail and aids in the development of new and improved semiconductor materials. Research team under Professor Byungha Shin at the Department of Material Sciences and Engineering and Dr. Oki Gunawan at IBM discovered a new formula and technique that enables the simultaneous extraction of both majority and minority carrier information such as their density and mobility, as well as gain additional insights about carrier lifetimes, diffusion lengths, and the recombination process. This new discovery and technology will help push semiconductor advances in both existing and emerging technologies. Semiconductors are the basic building blocks of today’s digital electronics age, providing us with a multitude of devices that benefit our modern life. To truly appreciate the physics of semiconductors, it is very important to understand the fundamental properties of the charge carriers inside the materials, whether those particles are positive or negative, their speed under an applied electric field, and how densely they are packed into the material.  Physicist Edwin Hall found a way to determine those properties in 1879, when he discovered that a magnetic field will deflect the movement of electronic charges inside a conductor and that the amount of deflection can be measured as a voltage perpendicular to the flow of the charge. Decades after Hall’s discovery, researchers also recognized that they can measure the Hall effect with light via “photo-Hall experiments”. During such experiments, the light generates multiple carriers or electron–hole pairs in the semiconductors. Unfortunately, the basic Hall effect only provided insights into the dominant charge carrier (or majority carrier). Researchers were unable to extract the properties of both carriers (the majority and minority carriers) simultaneously. The property information of both carriers is crucial for many applications that involve light such as solar cells and other optoelectronic devices. In the photo-Hall experiment by the KAIST-IBM team, both carriers contribute to changes in conductivity and the Hall coefficient. The key insight comes from measuring the conductivity and Hall coefficient as a function of light intensity. Hidden in the trajectory of the conductivity, the Hall coefficient curve reveals crucial new information: the difference in the mobility of both carriers. As discussed in the paper, this relationship can be expressed elegantly as: Δµ = d (σ²H)/dσ The research team solved for both majority and minority carrier mobility and density as a function of light intensity, naming the new technique Carrier-Resolved Photo Hall (CRPH) measurement. With known light illumination intensity, the carrier lifetime can be established in a similar way. Beyond advances in theoretical understanding, advances in experimental techniques were also critical for enabling this breakthrough. The technique requires a clean Hall signal measurement, which can be challenging for materials where the Hall signal is weak due to low mobility or when extra unwanted signals are present, such as under strong light illumination.  The newly developed photo-Hall technique allows the extraction of an astonishing amount of information from semiconductors. In contrast to only three parameters obtained in the classic Hall measurements, this new technique yields up to seven parameters at every tested level of light intensity. These include the mobility of both the electron and hole; their carrier density under light; the recombination lifetime; and the diffusion lengths for electrons, holes, and ambipolar types. All of these can be repeated N times (i.e. the number of light intensity settings used in the experiment). Professor Shin said, “This novel technology sheds new light on understanding the physical characteristics of semiconductor materials in great detail.” Dr. Gunawan added, “This will will help accelerate the development of next-generation semiconductor technology such as better solar cells, better optoelectronics devices, and new materials and devices for artificial intelligence technology.” Profile: Professor Byungha Shin Department of Materials Science and Engineering KAIST byungha＠kaist.ac.kr http://energymatlab.kaist.ac.kr/