Mengwei Ren

I am a Research Scientist at Adobe. I obtained my Ph.D. in Computer Science from NYU (2023), supervised by Prof. Guido Gerig under Visualization and Data Analytics Research (VIDA) Center . I have been fortunate to intern at Adobe, Google Research, and Siemens Healthineers.

My research broadly lies at the intersection of computer vision , deep learning, and biomedical image analysis. Particularly, I am interested in generative models, representation learning and spatiotemporal analysis.

I am looking for a research intern for the summer of 2025 (current PhD students in US), working on image composition and generative models. Please feel free to email your resume and background to my email .

Email  /  CV  /  Google Scholar  /  LinkedIn  /  Github

profile photo
Activities
09-2024 I will present at Adobe MAX Sneak 2024. Stay tuned!
04-2024 I received the Pearl Brownstein Doctoral Research Award from NYU CSE for “doctoral research which shows the greatest promise”.
02-2024 Our work on lighting-aware background replacement has been accepted to CVPR2024.
12-2023 Passed my Ph.D thesis defense :)
09-2023 Our work on keypoint augmented self-supervised learning has been accepted to NeurIPS2023.
07-2023 Our work on structure guided diffusion model for deblurring has been accepted to ICCV2023.
07-2023 Our work on data synthesis for microscopy segmentation has been accepted to MICCAI DALI.
05-2023 Starting my internship at Adobe.
10-2022 I received a Scholar Award from NeurIPS2022.
09-2022 Our work on spatiotemporal representation learning has been accepted to NeurIPS2022 (oral).
08-2022 I gave a talk on my PhD research on image-to-image translation at Luma seminar, Google Research.
07-2022 I gave an invited presentation on longitudinal neuroimage analysis at Stanford Research Institute & Computational Neuroimage Science Laboratory. Milestone: my first in-person talk :p
06-2022 Starting my internship at Computational Imaging (LUMA) Team, Google Research.
04-2022 Guest lecture on "Deep Learning for Computer Vision" for NYU Tandon CS-GY 6643 Computer Vision.
07-2021 Our work on spatiotemporal brain atlas synthesis has been accepted to ICCV2021.
06-2021 Our work on diffusion-weighted brain image synthesis has been accepted to MICCAI2021 (oral).
05-2021 Starting a Machine Learning research internship @Siemens Healthineer.
04-2021 Guest lecture on "Deep generative models (w/ a focus on VAE/GANs)" for NYU Tandon CS-GY 6643 Computer Vision.
02-2021 My first journal paper was accepted by IEEE Transactions on Medical Imaging!
Research

Relightful Harmonization: Lighting-aware Portrait Background Replacement
Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang, HyunJoon Jung, Guido Gerig, He Zhang
CVPR, 2024
project page, arXiv, bibtex

We introduce Relightful Harmonization, a lighting-aware diffusion model designed to seamlessly harmonize sophisticated lighting effect for the foreground portrait using any background image.

Keypoint-Augmented Self-Supervised Learning for Medical Image Segmentation with Limited Annotation
Zhangsihao Yang*, Mengwei Ren*, Kaize Ding, Guido Gerig, Yalin Wang.
NeurIPS, 2023
arXiv, bibtex

CNN-extracted global and local features are limited in capturing long-range spatial dependencies that are essential in biological anatomy. We present a keypoint-augmented fusion layer that extracts representations preserving both short- and long-range self-attention in a self-supervised manner.

Multiscale Structure Guided Diffusion for Image Deblurring
Mengwei Ren, Mauricio Delbracio, Hossein Talebi, Guido Gerig, Peyman Milanfar.
ICCV, 2023
arXiv, bibtex

Image-conditioned Diffusion Probablistic Models (icDPMs) for restoration work well on benchmarks but not real images. We introduce a simple yet effective structure guidance that leads to significantly better visual quality on unseen images.

Local Spatiotemporal Representation Learning for Longitudinally-consistent Neuroimage Analysis
Mengwei Ren, Neel Dey, Martin A. Styner, Kelly N. Botteron, Guido Gerig.
NeurIPS (oral), 2022
arXiv, github, project page, bibtex

We propose a local and multi-scale spatiotemporal representation learning method for image-to-image architectures trained on longitudinal (non i.i.d) images.

Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization
Mengwei Ren, Neel Dey, James Fishbaugh, Guido Gerig
Transaction on Medical Imaging, 2021
arXiv, github, bibtex

Based on an underlying assumption that morphological shape is consistent across imaging sites, we propose a segmentation-renormalized image translation framework to reduce inter-scanner heterogeneity while preserving anatomical layout.

Q-space Conditioned Translation Networks for Directional Synthesis of Diffusion Weighted Images from Multi-modal Structural MRI
Mengwei Ren, Heejong Kim, Neel Dey, Guido Gerig
MICCAI (oral), 2021
arXiv, github, bibtex

We propose a generative adversarial translation framework for high-quality DW image estimation with arbitrary Q-space sampling given commonly acquired structural images.

Generative Adversarial Registration for Improved Conditional Deformable Templates
Neel Dey, Mengwei Ren, Adrian Dalca, Guido Gerig
ICCV, 2021
arXiv, github, bibtex

We reformulate deformable registration and conditional template estimation as an adversarial game wherein we encourage realism in the moved templates with a generative adversarial registration framework conditioned on flexible image covariates.

MDA-Net: Memorable Domain Adaptation Network for Monocular Depth Estimation
Jing Zhu, Yunxiao Shi, Mengwei Ren, Yi Fang
BMVC, 2020
Link, bibtex

3D-A-Nets: 3D Deep Dense Descriptor for Volumetric Shapes with Adversarial Networks
Mengwei Ren, Liang Niu, Yi Fang
Thesis for B.S. degree, 2017
arXiv, bibtex

Photography
Another side of me is a photographer, and my cat Sudo is a very responsible 24/7 model. Check out my photo page here.

cr: Jon Baron