Sangmin Bae

Research Scientist, OSI Lab

Graduate School of AI, KAIST
85 Hoegi-ro, Dongdaemun-gu, Seoul, Korea

Email: bsmn0223xkxkxk@kaist.ac.kr / bsmn0223xkxkxk@gmail.com
Google Scholar, CV, Github, Linkedin, X

Welcome to my page! I am a Research Scientist with a strong desire to become a versatile expert in AI. I have explored NLP, CV, Audio, Tabular, and Multimodal, to broaden my knowledge and expertise.

My research interests lie in Efficient AI, which entails exploring inference-, training- or data-efficient approaches. I'm lately interested in LLM Inference Acceleration via adaptive computation algorithms or various techniques. Moreover, I have experienced in developing Foundation Models---pretraining LLMs optimized for accelerated inference, and self-supervised learning for visual models.

News

Sep. 2024: 🎊 A paper on 'Global-to-Local Language Modeling for Fast Inference' accepted at NeurIPS 2024.

Aug. 2024: 🎊 A paper on 'Robust Audio-Visual Speech Recognition' accepted at SLT 2024.

Jun. 2024: 🎊 A paper on 'Contrastive Decoding for Multi-modal Models' accepted at ICMLW 2024.

May. 2024: 💻 Starting an internship at Google DeepMind.

May. 2024: 🎊 A paper on 'Hard Prompt Optimization with RL' accepted at ACL 2024.

Education

Ph.D. student in Graduate School of AI, KAIST. Advised by Prof. Se-Young Yun. Mar. 2021 - Present

M.S. in Industrial and Systems Engineering, KAIST. Advised by Prof. Se-Young Yun. Mar. 2019 - Feb. 2021

B.S. in Industrial and Systems Engineering. Mar. 2014 - Feb. 2019

Publications Google Scholar *: 1st co-authors, ^†: corresponding authors, C: conferences, J: journals, W: workshops, P: preprints

2024

	[P6] As a co-author. Continual Learning for Large Language Models. Under Review 2024.
	[P5] As a co-author. Robust Audio-Visual Speech Recognition. Under Review 2024.
	[P4] Sangmin Bae, Adam Fisch, Hrayr Harutyunyan, Ziwei Ji, Seungyeon Kim, Tal Schuster^†. Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA. Preprint (Under Review) 2024. [pdf]
	[P3] Yongjin Yang, Sihyeon Kim, Hojung Jung, Sangmin Bae, SangMook Kim, Se-Young Yun^†, Kimin Lee^†. Automated Filtering of Human Feedback Data for Aligning Text-to-Image Diffusion Models. Preprint (Under Review) 2024. [pdf]
	[P2] Felix den Greejen, Sangmin Bae, Stephen Cha, Se-Young Yun^†. Fine-tuned In-Context Learning Transformers are Excellent Tabular Data Classifiers. Preprint (Under Review)* 2024. [pdf] [code]
	[C12] Namgyu Ho, Sangmin Bae, Taehyeon Kim, Hyunjik Jo, Yireun Kim, Tal Schuster, Adam Fisch, James Thorne^†, Se-Young Yun^†. Block Transformer: Global-to-Local Language Modeling for Fast Inference*. Conference on Neural Information Processing Systems* (NeurIPS) 2024. [pdf] [code]
	[W7] Sihyeon Kim, Boryeong Cho, Sangmin Bae, Sumyeong Ahn^†, Se-Young Yun^†. VACoDe: Visual Augmented Contrastive Decoding. International Conference on Machine Learning Workshop on Trustworthy Multi-modal Foundation Models and AI Agents (ICMLW) Under Review 2024. [pdf]
	[C11] Sungnyun Kim, Kangwook Jang, Sangmin Bae, Hoirin Kim^†, Se-Young Yun^†. Learning Video Temporal Dynamics with Asymmetric Cross-Modal Attention for Robust Audio-Visual Speech Recognition. IEEE Spoken Language Technology Workshop (SLT) 2024. [pdf]
	[C10] Yunseon Choi, Sangmin Bae, Seonghyun Ban, Minchan Jeong, Chuheng Zhang, Lei Song, Li Zhao, Jiang Bian, Kee-Eung Kim^†. Hard Prompts Made Interpretable: Sparse Entropy Regularization for Prompt Tuning with RL. The Association for Computational Linguistics (ACL) 2024. Oral Presentation. [pdf] [code]
	[C9] June-Woo Kim, Miika Toikkanen, Sangmin Bae, Minseok Kim^†, Ho-Young Jung^†. RepAugment: Input-Agnostic Representation-Level Augmentation for Respiratory Sound Classification. International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2024. [pdf]
	[C8] Yujin Kim, Jaehong Yoon, Seonghyeon Ye, Sangmin Bae, Namgyu Ho, Sung Ju Hwang^†, Se-Young Yun^†. Carpe diem: On the Evaluation of World Knowledge in Lifelong Language Models. Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) Long Paper 2024. [pdf] [code]
	[C7] June-Woo Kim, Sangmin Bae, Won-Yang Cho, Byungjo Lee, Ho-Young Jung^†. Stethoscope-guided Supervised Contrastive Learning for Cross-domain Adaptation on Respiratory Sound Classification. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2024. [pdf] [code]

2023

	[W6] June-Woo Kim, Chihyeon Yoon, Miika Toikkanen, Sangmin Bae, Ho-Young Jung^†. Adversarial Fine-tuning using Generated Respiratory Sound to Address Class Imbalance. Neural Information Processing Systems Workshop on Deep Generative Models for Health (NeurIPSW) 2023. [pdf] [code]
	[W5] Felix den Breejen, Sangmin Bae, Stephen Cha, Tae-Young Kim, Seoung-Hyun Koh, Se-Young Yun^†. Exploring the Retrieval Mechanism for Tabular Deep Learning. Neural Information Processing Systems Workshop on Table Representation Learning (NeurIPSW) 2023. [pdf]
	[C6] Sangmin Bae, Jongwoo Ko, Hwanjun Song^†, Se-Young Yun^†. Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding. Conference on Empirical Methods in Natural Language Processing (EMNLP) Long Paper 2023. [pdf] [code]
	[C5] Sangmin Bae, June-Woo Kim, Won-Yang Cho, Hyerim Baek, Soyoun Son, Byungjo Lee, Changwan Ha, Kyongpil Tae, Sungnyun Kim^†, Se-Young Yun^†. Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification. Conference of the International Speech Communication Association (INTERSPEECH) 2023. [pdf] [code]
	[C4] Sungnyun Kim, Sangmin Bae, Se-Young Yun^†. Coreset Sampling from Open-Set for Fine-Grained Self-Supervised Learning*. International Conference on Computer Vision and Pattern Recognition* (CVPR) 2023. [pdf] [code]
	[C3] Sangmook Kim, Sangmin Bae, Hwanjun Song^†, Se-Young Yun^†. Re-thinking Federated Active Learning based on Inter-class Diversity*. International Conference on Computer Vision and Pattern Recognition* (CVPR) 2023. [pdf] [code]
	[C2] Sangmin Bae, Sungnyun Kim, Jongwoo Ko, Gihun Lee, Seungjong Noh, Se-Young Yun^†. Self-Contrastive Learning: Single-viewed Supervised Contrastive Framework using Sub-network. The Association for the Advancement of Artificial Intelligence (AAAI) 2023. Oral Presentation. [pdf] [code]

2022

	[C1] Gihun Lee, Minchan Jeong, Yongjin Shin, Sangmin Bae, Se-Young Yun^†. Preservation of Global Knowledge by Not-True Distillation in Federated Learning. Neural Information Processing Systems (NeurIPS) 2022. [pdf] [code]
	[W4] Sungnyun Kim, Sangmin Bae, Se-Young Yun^†. Coreset Sampling from Open-Set for Fine-Grained Self-Supervised Learning*. Neural Information Processing Systems Workshop on Self-Supervised Learning: Theory and Practice* (NeurIPSW) 2022. [pdf]
	[W3] Sangmook Kim, Sangmin Bae, Hwanjun Song^†, Se-Young Yun^†. LG-FAL: Federated Active Learning Strategy using Local and Global Models*. International Conference on Machine Learning Workshop on Adaptive Experimental Design and Active Learning in the Real World* (ICMLW) 2022. [pdf]

2020

	[W2] Sungnyun Kim, Gihun Lee, Sangmin Bae, Se-Young Yun^†. MixCo: Mix-up Contrastive Learning for Visual Representation. Neural Information Processing Systems Workshop on Self-Supervised Learning: Theory and Practice* (NeurIPSW) 2020. [pdf] [code]
	[P1] Taehyeon Kim, Sangmin Bae, Jin-woo Lee, Se-Young Yun^†. Accurate and Fast Federated Learning via Combinatorial Multi-Armed Bandits*. Preprint* 2020. [pdf]
	[W1] Gihun Lee, Sangmin Bae, Jaehoon Oh, Se-Young Yun^†. SIPA: A Simple Framework for Efficient Networks*. IEEE International Conference on Data Mining Workshop on Big Data Analysis for Smart Engergy* (ICDMW) 2020. [pdf] [code]

Patents

Se-Young Yun, Seongyoon Kim, Woojin Chung, Sangmin Bae. Toward Enhanced Representation for Federated Re-Identification by Not-True Self
Knowledge Distillation. Korea Patent Application. Aug. 2022

Jaehoon Oh, Sangmook Kim, Se-Young Yun, Sangmin Bae, Jaewoo Shin, Seongyoon Kim, Woojin Chung. Federated Learning System for Performing Individual Data Customized Federated Learning, Method for Federated Learning, and Client Aratus for Performing Same. Korea and US Patent Application.
Jun. 2022, Oct. 2022

Gihun Lee, Minchan Jeong, Se-Young Yun, Sangmin Bae, Jaeyeon Ahn, Seongyoon Kim, Woojin Chung. System, Method, Computer-Readable Storage Medium and Computer Program for Federated Learning of Local Model based on Learning Direction of Global Model. Korea and US Patent Application.
Jun. 2022, Oct. 2022

Awards and Honors

$30,000 in Google Cloud Credits from Google Cloud Platform (GCP). Aug. 2024

Silver Award in Signal Processing from Samsung Humantech Paper Awards. Jan. 2024

Two Best Presentation Awards from Korea Computing Congress (KCC). Aug. 2022

Best Paper Award (5th Place) from Korean AI Association and LG AI Research (JKAIA). Nov. 2021

MicroNet Challenge 4th Place at NeurIPS Workshop. Oct. 2019

Alumni Scholarship from KAIST. Mar. 2017 - Feb. 2019

Dean's List (Top 3%) at Faculty of Engineering Department in KAIST. Spring 2017

Research Experience

Research Collaboration with Google, advised by Tal Schuster, Adam Fisch, Seungyeon Kim, Ziwei Ji, and Hrayr Harutyunyan. Aug. 2024 - Present

Research Internship at Google DeepMind, advised by Tal Schuster and Adam Fisch. May. 2024 - Aug. 2024

Research Collaboration with MODULABS. Sep. 2022 - Jan. 2024

Research Collaboration with NAVER AI, advised by Hwanjun Song. Jan. 2022 - Jan. 2023

Research Internship at Kakao Recommendation Team. Sep. 2018 - Feb. 2019

Research Internship at Optimization and Statistical Inference Lab, KAIST. Jul. 2018 - Aug. 2018

Research Internship at Human Factors and Ergonomics Lab, KAIST. Dec. 2017 - Jun. 2018

Exchange Student at Linköping University, Sweden. Jul. 2017 - Aug. 2017

Research Projects

[KT] Efficient large language model inference algorithm. Sep. 2024 - Oct. 2024

[NIER] Short-term Prediction of Particulate Matter via Artificial Intelligence. Project Manager. Mar. 2023 - May. 2024

[KT] Neural Architecture Search for Detecting Communication Network Failure. Project Manager. Apr. 2022 - Feb. 2023

[ETRI] Lightweight Edge Device Technology via Federated Learning. Project Manager. Mar. 2021 - Sep. 2022

[SK Hynix] Semantic Segmentation to Detect Errors in Wafer Process. Feb. 2021 - Sep. 2021

[ETRI] Data-efficient Unsupervised Representation Learning. Mar. 2020 - Dec. 2020

[ETRI] Model Compression for Big Data Ddge Analysis. Jun. 2019 - Oct. 2019

[Hankook Tire and Technology] Compound Prediction with Artificial Intelligence and Auto-ML. Mar. 2019 - Feb. 2020

Services

Server Manager at KAIST AI. Mar. 2021 - Feb. 2023

Student Leader at OSI Lab, KAIST. Mar. 2021 - Mar. 2022

Teaching Assistant.

− KAIST AI505 Optimization for AI. Fall 2021, Fall 2022, Fall 2023
− KAIST AI603 Machine Learning Theory . Spring 2021, Spring 2023
− KAIST AI611 Deep Reinforcement Learning. Spring 2022

Instructor on DL and ML courses.

− MetaCode: Machine Learning Course. [video] (views 120K) Jun. 2021 - Dec. 2022
− ForumM: Recommendation Seminar. Nov. 2022
− LG-KAIST AI: Computer Vision Course. Oct. 2020, Oct. 2021
− Korea Blockchain Institute: Machine Learning Course. Dec. 2020
− Samsung DS: Deep Learning Course. Jul. 2020

	[W6] June-Woo Kim, Chihyeon Yoon, Miika Toikkanen, Sangmin Bae, Ho-Young Jung^†. Adversarial Fine-tuning using Generated Respiratory Sound to Address Class Imbalance. Neural Information Processing Systems Workshop on Deep Generative Models for Health (NeurIPSW) 2023. [pdf] [code]
	[W5] Felix den Breejen, Sangmin Bae, Stephen Cha, Tae-Young Kim, Seoung-Hyun Koh, Se-Young Yun^†. Exploring the Retrieval Mechanism for Tabular Deep Learning. Neural Information Processing Systems Workshop on Table Representation Learning (NeurIPSW) 2023. [pdf]
	[C6] Sangmin Bae, Jongwoo Ko, Hwanjun Song^†, Se-Young Yun^†. Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding. Conference on Empirical Methods in Natural Language Processing (EMNLP) Long Paper 2023. [pdf] [code]
	[C5] Sangmin Bae, June-Woo Kim, Won-Yang Cho, Hyerim Baek, Soyoun Son, Byungjo Lee, Changwan Ha, Kyongpil Tae, Sungnyun Kim^†, Se-Young Yun^†. Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification. Conference of the International Speech Communication Association (INTERSPEECH) 2023. [pdf] [code]
	[C4] Sungnyun Kim, Sangmin Bae, Se-Young Yun^†. Coreset Sampling from Open-Set for Fine-Grained Self-Supervised Learning*. International Conference on Computer Vision and Pattern Recognition* (CVPR) 2023. [pdf] [code]
	[C3] Sangmook Kim, Sangmin Bae, Hwanjun Song^†, Se-Young Yun^†. Re-thinking Federated Active Learning based on Inter-class Diversity*. International Conference on Computer Vision and Pattern Recognition* (CVPR) 2023. [pdf] [code]
	[C2] Sangmin Bae, Sungnyun Kim, Jongwoo Ko, Gihun Lee, Seungjong Noh, Se-Young Yun^†. Self-Contrastive Learning: Single-viewed Supervised Contrastive Framework using Sub-network. The Association for the Advancement of Artificial Intelligence (AAAI) 2023. Oral Presentation. [pdf] [code]