Jimei Yang

 Researcher
Runway
Mountain View, CA

jimeiyang at gmail.com
[Google Scholar] [LinkedIn] [Twitter]

Research

My research mission is learning fundamental structures from large-scale visual data that connect real and virtual worlds and solving problems in human sensing, character animation, image and video synthesis.

Highlights

  • I joined Runway to work on video generative models and controllability. Check out our latest model Gen 3 Alpha.
  • Our pose-based image search work, Project OnPoint was featured at Adobe MAX Sneaks 2021.
  • Our webcam-based real-time body animation technology was shipped in Adobe Character Animator.
  • Our dance video editing technology, Project On-The-Beat was featured at Adobe MAX Sneaks 2020, and covered by Protocol.
  • Our video-based motion capture technology was published at ECCV 2020 as Spotlight, and covered by Synced
  • Our body tracking technology in After Effects, Project GoFigure was featured at Adobe MAX Sneaks 2019 , and covered by The Verge.
    •

    Selected Publications

    Audio-driven Neural Gesture Reenactment with Video Motion Graphs
    Yang Zhou, Jimei Yang, Dingzeyu Li, Jun Saito, Deepali Aneja, Evangelos Kalogerakis
    to appear in CVPR, 2022.
    [Paper] [Webpage] [GitHub] [Video]
    Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
    Badour AlBahar, Jingwan Lu, Jimei Yang, Zhixin Shu, Eli Shechtman, Jia-Bin Huang
    in SIGGRAPH Asia, 2021.
    [Paper] [GitHub]
    HuMoR: 3D Human Motion Model for Robust Pose Estimation
    Davis Rempe, Tolga Birdal, Aaron Hertzmann, Jimei Yang, Srinath Sridhar, Leonidas J. Guibas
    in ICCV (oral), 2021.
    [Paper] [Webpage] [GitHub] [Video]
    Stochastic Scene-Aware Motion Prediction
    Mohamed Hassan, Duygu Ceylan, Ruben Villegas, Jun Saito, Jimei Yang, Yi Zhou, Michael Black
    in ICCV, 2021.
    [Paper] [Webpage] [GitHub]
    Contact-Aware Retargetting of Skinned Motion
    Ruben Villegas, Duygu Ceylan, Aaron Hertzmann, Jimei Yang, Jun Saito
    in ICCV, 2021.
    [Paper] [Webpage]
    Interactive Liquid Splash Modeling by User Sketches
    Guowei Yan, Zhili Chen, Jimei Yang, Huamin Wang
    in SIGGRAPH Asia, 2020.
    [Paper] [Video]
    Contact and Human Dynamics from Monocular Video
    Davis Rempe, Leonidas J. Guibas, Aaron Hertzmann, Bryan Russell, Ruben Villegas, Jimei Yang
    in ECCV (spotlight), 2020.
    [Paper] [Webpage] [GitHub] [Video]
    High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling
    Yu Zeng, Zhe Lin, Jimei Yang, Jianming Zhang, Eli Shechtman, Huchuan Lu
    in ECCV, 2020.
    [Paper] [Webpage]
    Statistics-based Motion Synthesis for Social Conversations
    Yanzhe Yang, Jimei Yang, Jessica K. Hodgins
    in SCA, 2020.
    [Paper] [Webpage] [Video]
    Attribute-Conditioned Layout GAN for Automatic Graphic Design
    Jianan Li, Jimei Yang, Jianming Zhang, Chang Liu, Christina Wang, Tingfa Xu
    in IEEE Transactions on Visualization and Computer Graphics, 2020.
    [Paper]
    3D Ken Burns Effect from a Single Image
    Simon Niklaus, Long Mai, Jimei Yang, Feng Liu
    in SIGGRAPH Asia, 2019.
    [Paper] [Webpage]
    Free-Form Image Inpainting with Gated Convolution
    Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, Thomas Huang
    in ICCV (oral), 2019.
    [Paper] [GitHub]
    FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images
    Christian Zimmermann, Duygu Ceylan, Jimei Yang, Bryan Russell, Max Argus, Thomas Brox
    in ICCV, 2019.
    [Paper] [Webpage] [GitHub]
    Multimodal Style Transfer via Graph Cuts
    Yulun Zhang, Chen Fang, Yilin Wang, Zhaowen Wang, Zhe Lin, Yun Fu Jimei Yang
    in ICCV, 2019.
    [Paper]
    Foreground-aware Image Inpainting
    Wei Xiong, Zhe Lin, Jimei Yang, Xin Lu, Connelly Barnes, Jiebo Luo
    in CVPR, 2019.
    [Paper] [Webpage]
    On the Continuity of Rotation Representations in Neural Networks
    Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, Hao Li
    in CVPR, 2019.
    [Paper] [Webpage]
    LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators
    Jianan Li, Jimei Yang, Aaron Hertzmann, Jianming Zhang, Tingfa Xu
    in ICLR, 2019.
    [Paper] [GitHub] [Journal version]
    BodyNet: Volumetric Inference of 3D Human Body Shapes
    Gül Varol, Duygu Ceylan, Bryan Russell, Jimei Yang, Ersin Yumer, Ivan Laptev, Cordelia Schmid
    in ECCV, 2018.
    [Paper] [Github]
    Flow-Grounded Spatial-Temporal Video Prediction from Still Images
    Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, Ming-Hsuan Yang
    in ECCV, 2018.
    [Paper] [Github]
    Brush Stroke Synthesis with a Generative Adversarial Network Driven by Physically Based Simulation
    Rundong Wu, Zhili Chen, Zhaowen Wang, Jimei Yang, Steve Marschner
    in Expressive, 2018.
    [Paper]
    Generative Image Inpainting with Contextual Attention
    Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, Thomas Huang
    in CVPR, 2018.
    [Paper] [GitHub]
    Neural Kinematic Networks for Unsupervised Motion Retargeting
    Ruben Villegas, Jimei Yang, Duygu Ceylan, Honglak Lee
    in CVPR (oral), 2018.
    [Paper] [Webpage] [Github]
    PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image
    Chen Liu, Jimei Yang, Duygu Ceylan, Ersin Yumer, Yasutaka Furukawa
    in CVPR (spotlight), 2018.
    [Paper] [Webpage] [Github]
    MatNet: Modular Attention Network for Referring Expression Comprehension
    Licheng Yu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, Mohit Bansal>, Tamara Berg
    in CVPR, 2018.
    [Paper] [Webpage] [Github]
    Universal Style Transfer via Feature Transforms
    Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, Ming-Hsuan Yang
    in NeurIPS, 2017.
    [Paper] [Github]
    PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image
    Guilin Liu, Duygu Ceylan, Ersin Yumer, Jimei Yang, Jyh-Ming Lien
    in ICCV (spotlight), 2017.
    [Paper]
    Recurrent Multimodal Interaction for Referring Image Segmentation
    Chenxi Liu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, Alan Yuille
    in ICCV, 2017.
    [Paper]
    3D-PRNN: Generating Shape Primitives With Recurrent Neural Networks
    Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, Derek Hoiem
    in ICCV, 2017.
    [Paper]
    Learning to Generate Long-term Future via Hierarchical Prediction
    Ruben Villegas, Jimei Yang, Yuliang Zou, Sungryull Sohn, Xunyu Lin, Honglak Lee
    in ICML, 2017.
    [Paper] [Webpage] [Github]
    Transformation-Grounded Image Generation Network for Novel 3D View Synthesis
    Eunbyung Park, Jimei Yang, Ersin Yumer, Duygu Ceylan, Alex Berg
    in CVPR, 2017.
    [Paper] [Webpage] [Github]
    Forecasting Human Dynamics from Static Images
    Yu-Wei Chao, Jimei Yang, Brian Price, Scott Cohen, Jia Deng
    in CVPR, 2017.
    [Paper] [Webpage] [Github]
    Generative Face Completion
    Yijun Li, Sifei Liu, Jimei Yang, Ming-Hsuan Yang
    in CVPR, 2017.
    [Paper] [Webpage] [Github]
    Diversified Texture Synthesis with Feed-forward Networks
    Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, Ming-Hsuan Yang
    in CVPR (spotlight), 2017.
    [Paper] [Webpage] [Github]
    Decomposing Motion and Content for Natural Video Sequence Prediction
    Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin, Honglak Lee
    in ICLR, 2017.
    [Paper] [Webpage] [Github]
    Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
    Xinchen Yan, Jimei Yang, Ersin Yumer, Yijie Guo, Honglak Lee
    in NeurIPS, 2016.
    [Paper] [Github]
    Attribute2Image: Conditional Image Generation from Visual Attributes
    Xinchen Yan, Jimei Yang, Kihyuk Sohn, Honglak Lee
    in ECCV, 2016.
    [Paper] [Webpage] [Github] [Video]
    Object Contour Detection with a Fully Convolutional Encoder-Decoder Network
    Jimei Yang, Brian Price, Scott Cohen, Honglak Lee, Ming-Hsuan Yang
    in CVPR (spotlight), 2016.
    [Paper]
    Deep Interactive Object Selection
    Ning Xu, Brian Price, Scott Cohen, Jimei Yang, Thomas Huang
    in CVPR, 2016.
    [Paper] [Webpage]
    Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis
    Jimei Yang, Scott Reed, Ming-Hsuan Yang, Honglak Lee
    in NeuIPS, 2015.
    [Paper] [Video] [GitHub] [Talk]