Preprint

We propose motion-textual inversion, a general method to transfer the semantic motion of a given reference motion video to given target images. We thereby optimize a motion representation composed of a set of text/image embedding tokens using a frozen, pre-trained image-to-video diffusion model. Our method generalizes across various domains and supports multiple types of motions, including full-body, face, camera, and even hand-crafted motions.

Manuel Kansy (DisneyResearch|Studios / ETH Zurich), Jacek Naruniec (DisneyResearch|Studios), Christopher Schroers (DisneyResearch|Studios), Markus Gross (DisneyResearch|Studios / ETH Zurich), Romann M. Weber (DisneyResearch|Studios)

Project Page   —   Paper

Preprint

We show that applying classifier-free guidance (CFG) does not require any specific training procedure (e.g., inserting a null condition during training), and CFG can be extended to a more general method that is applicable to any diffusion model, including unconditional ones.

Seyedmorteza Sadat (DisneyResearch|Studios / ETH Zurich), Manuel Kansy (DisneyResearch|Studios / ETH Zurich), Otmar Hilliges (ETH Zurich), Romann M. Weber (DisneyResearch|Studios)

Paper

IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023

We tackle the challenging task of inverting the latent space of pre-trained face recognition models without full model access (i.e. black-box setting). Our method, the identity denoising diffusion probabilistic model (ID3PM), leverages the stochastic nature of the denoising diffusion process to produce high-quality, identity-preserving face images with various backgrounds, lighting, poses, and expressions.

Manuel Kansy (DisneyResearch|Studios / ETH Zurich), Anton Raël (ETH Zurich), Graziana Mignone (DisneyResearch|Studios), Jacek Naruniec (DisneyResearch|Studios), Christopher Schroers (DisneyResearch|Studios), Markus Gross (DisneyResearch|Studios / ETH Zurich), Romann M. Weber (DisneyResearch|Studios)

Project Page   —   Paper   —   Supplementary Material

IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, 2023

The terms high-resolution and high-quality are not equivalent, and high-resolution does not always imply high-quality. In this paper, we motivate and precisely define the concept of effective resolution and propose a novel self-supervised learning scheme to train a neural network for effective resolution estimation. We demonstrate that our method outperforms state-of-the-art image quality assessment methods in estimating the sharpness of real and generated human faces, despite using only unlabeled data during training.

Manuel Kansy (DisneyResearch|Studios / ETH Zurich), Julian Balletshofer (DisneyResearch|Studios), Jacek Naruniec (DisneyResearch|Studios), Christopher Schroers (DisneyResearch|Studios), Graziana Mignone (DisneyResearch|Studios), Markus Gross (DisneyResearch|Studios / ETH Zurich), Romann M. Weber (DisneyResearch|Studios)

Project Page   —   Video   —   Paper   —   Supplementary Material

Master’s Thesis (2020 - 2021)

• Adapted Faster R-CNN to produce spatio-temporal object proposals (STPs)
• Adapted Message Passing Network (MPN) tracking framework to jointly perform detection and data association given the STPs
• Evaluated different design choices and data sets to improve detection performance

Used technologies: Python, PyTorch, Ubuntu

Course: Master Practical (2019 - 2020)

• Created dataset with drone videos of different interestingness levels
• Implemented novel neural network architectures to predict drone video interestingness
• Developed pipeline to output edited composite video given several raw videos as input

Used technologies: Python, PyTorch, Ubuntu

Course: Reinforcement Learning for Robotics (2019 - 2020)

• Created simple environments: Inverted pendulum, cart-pole
• Implemented classic function approximators: Tabular, variable resolution (constant, linear combination of basis functions), Gaussian mixture model
• Implemented and compared reinforcement learning and optimal control (MPC) methods

Used technologies: MATLAB, Simulink, MacOS

Course: Advanced Deep Learning for Computer Vision (2019)

• Created synthetic dataset of unstabilized videos by applying transformations to videos
• Reimplemented and extended image inpainting architecture to inpaint video edge regions (Python, PyTorch, Ubuntu)
• Compared results qualitatively and quantitatively to state-of-the-art methods

Used technologies: Python, PyTorch, Ubuntu

(2015 - 2018)

• Semester 6 (Bachelor Thesis): Development and Validation of a Maneuver Strategy to Avoid Collision with Oncoming Traffic
• Semester 5+6 (Student Research Project): Development of a Concept to Optically Localize a Measuring Device
• Semester 5 (Semester Project): Development of a Testing Concept to Visualize Accidents with Oncoming Traffic and Validation of the Sensors Necessary for the Corresponding Safety Function
• Semester 4 (Semester Project, Thailand): Integrating and Evaluating the Reduced Order Model of the Magnetic Circuit of the Flow Control Valve of a High Pressure Pump in a Simulation Software
• Semester 3 (Semester Project): Enhancement of a Software to Calculate the Fuel Benefit of a Vehicle Equipped with the Coasting Function
• Semester 2 (Semester Project): Development of a VBA Tool to Automatize the Creation of Software Component Release Notes

Used technologies: Simulink, MATLAB, C++, C#, Unity, Windows, MacOS, iOS, Embedded Linux, Vehicle Deployment

Master’s Thesis (Fall 2024)

Master’s Thesis (Fall 2023)

Master’s Thesis (Fall 2023)

Semester Thesis (Spring 2023)

Bachelor’s Thesis (Spring 2023)

Master’s Thesis (Spring 2023)

Master’s Thesis (Fall 2022)

Bachelor’s Thesis (Fall 2022)

Bachelor’s Thesis (Spring 2022)

Bachelor’s Thesis (Spring 2022)

Master’s Thesis (Spring 2022)

Backoffice TA (Fall 2024)

Head TA (Fall 2023)

Head TA (Fall 2023)

Head TA (Fall 2022)

Head TA (Fall 2022)

Regular TA (Spring 2022)

Regular TA (Fall 2021)

Student organization (2016 - 2018)

• Managed and represented student council as Vice President
• Led and re-organized commission that decides over 500 000€ university budget
• Helped obtain funding and machinery for the Formula Student team
• Kept close contact with university administration
• Held several other positions in the engineering faculty, local senate, student parliament, and student union
• Was awarded the KOMMUNITY award (given to one student per faculty every year for exceptional extracurricular commitment; with cash prize)