Changkyu Choi

Changkyu Choi

Associate professor, Department for Informatics, University of Oslo

MishMash role: Member · WP7

Multimodality Computer Vision Artificial Intelligence Class Imbalance Problem for Machine Learning Unsupervised Machine Learning Algorithms

Other projects

Latest results

Type

Journal article

  1. Journal article, 2026

    Suppressing Non-Semantic Noise in Masked Image Modeling Representations

    Martine Hjelkrem Tan ; Marius Aasan ; Rwiddhi Chakraborty ; Gabriel Yanci Arteaga ; Changkyu Choi ; Adín Ramírez Rivera

    Masked Image Modeling (MIM) has become a ubiquitous self-supervised vision paradigm. In this work, we show that MIM objectives cause the learned representations to retain non-semantic information, which ultimately hurts performance during inference. We introduce a model-agnostic score for semantic invariance using Principal Component Analysis (PCA) on real and synthetic non-semantic images. Based on this score, we propose a simple method, Semantically Orthogonal Artifact Projection (SOAP), to directly suppress non-semantic information in patch representations, leading to consistent improvements in zero-shot performance across various MIM-based models. SOAP is a post-hoc suppression method, requires zero training, and can be attached to any model as a single linear head.

  3. Journal article, 2026

    Remember Your Trace: Memory-Guided Long-Horizon Agentic Framework for Consistent and Hierarchical Repository-Level Code Documentation

    Bae, Suyoung ; Lee, Jaehoon ; Changkyu Choi ; Choi, YunSeok ; Lee, Jee-Hyong

    Automated code documentation is essential for modern software development, providing the contextual grounding that both human developers and coding agents rely on to navigate large codebases. Existing repository-level approaches process components independently, causing redundant retrieval and conflicting descriptions across documents while producing outputs that lack hierarchical structure. Therefore, we propose MemDocAgent, a long-horizon agentic framework that generates documentation within a single, integrated context spanning the entire repository. It combines two components: (i) Dependency-Aware Traversal Guiding that predetermines a traversal order respecting dependency and granularity hierarchies; (ii) Memory-Guided Agentic Interaction, in which the agent interacts with RepoMemory, a shared memory accumulating prior work traces through read, write, and verify operations. Through an in-depth multi-criteria evaluation, MemDocAgent achieves the best performance over both open and closed-source baselines and demonstrates practical applicability in real software development workflows.

  4. Journal article, 2026

    PubTables-QA: A Benchmark Toward Cross-Page Table Reasoning in Table Visual Question Answering

    Jiin Han ; Suyoung Bae ; Yerim Choi ; Sangyun Lee ; Kyutae Kim ; Ryunho Kim ; Jee-Hyong Lee ; Changkyu Choi ; Yunseok Choi

    Tables are central to scientific communication, yet their interpretation depends critically on layout, continuity, and surrounding context. While existing multi-page and multi-table benchmarks formally require document-level reasoning, models can often succeed through stochastic shortcut search—locating local answer-bearing regions without navigating table structure. To address this gap, we introduce PubTables-QA, a multi-page TableVQA benchmark designed to audit responsible cross-page visual reasoning in academic documents. Built on PubTables-v2’s structural annotations, PubTables-QA provides 2,106 QA pairs that are unanswerable from any single page, compelling models to reconstruct table continuity across pages or integrate evidence across multiple tables. We further introduce a three-level QA taxonomy spanning Document, Table, and Cell/column levels. Experiments on recent MLLMs, including GPT-4o, Gemini-2.5-Pro, and Gemma-4, show that even the strongest model reaches only 32.6% accuracy. Oracle-setting analysis further shows that performance depends strongly on access to the correct page, table region, and cell-level evidence, highlighting persistent failures in faithful cross-page evidence integration.

  5. Journal article, 2026

    Selective Disclosure: Controlling Information Leakage in DocVQA Explanations

    Kangsoo Jung ; Mohamed Ali Souibgui ; Changkyu Choi ; Catuscia Palamidessi

    Explainable document visual question answering systems improve transparency by visualizing document regions relevant to a query, but they can unintentionally expose sensitive information through explanation outputs, even when textual answers are restricted. We analyze privacy risks arising from coarse explanations and adversarial jailbreaking queries. To mitigate this, we propose a policy-aware visual explanation sanitization framework based on a Role-centric Attribute-Based Access Control (RABAC) model, combining document structure analysis and line-level localization to mask restricted regions. Our method is model-agnostic, supports dynamic policies, and significantly reduces sensitive information leakage while preserving interpretability.

Conference

  2. Conference poster, 2026

    Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models

    Changkyu Choi

    Dual-encoder vision-language models systematically fail at negation, scoring "a cat without a dog"' nearly identically to "a cat with a dog.'' We show that this failure is inherent to the single-embedding framework. Under contrastive training, single-embedding methods can at best ignore the negated concept (dilution) but never penalize its presence (rejection). We propose Know-Thy-Not (KTN), a training-free method that introduces an image-side rejection dimension to penalize the negated concept directly in the visual representation. We also introduce COCO-Confusion, the first benchmark requiring genuine rejection to succeed. Across 9 backbones, KTN improves COCO-Confusion mAP by up to +25.6 over vanilla VLMs and +9.6 over the strongest existing method.

  3. Conference abstract, 2025

    Proceedings of NORA’s annual conference 2025

    Geir Halnes ; Anam Javaid ; Michael Solvang ; Stefano Nichele ; Michael Riegler ; Bjørn-Jostein Singstad ; Baltasar Beferull-Lozano ; Emilio Ruiz Moreno ; Luis M. Lopez-Ramos ; Mehrzad Abdi Khalife ; Ola Huse Ramstad ; Hamze Issa ; Arina Surko ; Axel Sandvig ; Hasan Ogul ; Daniele Fantin ; Ioanna Sandvig ; Christopher Vibe ; Kushtrim Visoka ; Mehdi HoushmandSarkhoosh ; Aaron de Leyos ; Sinan Ugur Umu ; Klaus Johannsen ; Kjetil Indrehus ; Malcom McMillan ; Julia Kropiunig ; Ryan Anthony Marinelli ; Fadi Al Machot ; Xue-Cheng Tai ; Andrea Alessandro Gasparini ; David Parkes ; Semra Oztemel Sari ; Gro Fonnes ; Cise Midoglu ; Anton Tkachenko ; Maria Bashir ; Kari-Anne Kallerud Lyng ; Florenc Demrozi ; Kate Briggs ; Junyong You ; Signe Riemer-Sørensen ; Benjamin Daniel Adolphi ; Martin Thomas Horsch ; Arangan Subramaniam ; Ibrahim Riza Hallac ; Lina Plataniti ; Hao Liu ; Mikkel Elle Lepperød ; Changkyu Choi ; Preben Castberg ; Abdelaziz Qassi ; Raymond H. Chan ; Anja Stein ; Heinz Adolf Preisig ; Alexander Johannes Stasik ; Saeed Shafiee Sabet ; Nils Olav Handegard ; Robert Jenssen ; Solve Sæbø ; Synnøve Rubach ; Waldir Leoncio Netto ; Pankaj Pandey ; Jan Wuite ; Arezo Shakeri ; Shailendra Singh ; Ali Ramezani-Kebrya ; Pål Halvorsen ; David S. Leslie ; Matteo Iervasi ; Mathis Korseberg Stokke ; Tomas Kupka ; Lingfeng Li ; Helge Fredriksen ; Shakiba Sadat Mirbagheri ; Ali Ramezanikebrya ; Jacob Alexander Hay ; Aslak Djupskås ; Mina FNorwayarmanbar ; Claudio Sartori ; Felix Simon Reimers ; Thomas Nagler ; Amber Leeson

More results in NVA…