Ruben Villegas

Research Scientist, Adobe Research
villegas [at] adobe.com
Github / Google scholar / Twitter

I completed my PhD at the Computer Science & Engineering department at the University of Michigan, Ann Arbor. My PhD adviser was Professor Honglak Lee. During my PhD, I interned at Google Brain, Adobe Research and NVIDIA Research. My general research interest lies on modeling motion with deep neural networks. Particularly, my interest spans across the areas of computer vision, graphics and model-based reinforcement learning. Most of my work has been in video prediction/generation, motion retargeting and human motion synthesis.

Fun Facts: I played for my national basketball team (I am originally from Ecuador). I was also second best scorer in the nation in a national championship I played back in the day. I was part of a team that beat the media's projected champion during a championship in Quito (the guy that was best scorer in the national championship played for the other team :P). Let's have a Curry-range 3-point shootout. Ok, I'll stop now ...

Internship

If you are a PhD student interested in working on research projects at Adobe, please send me your resume and a one page research statement.

News

  • 09/2019: Two papers accepted to NeurIPS 2019!
  • 04/2019: I will be joining Adobe Research as Research Scientist!
  • 04/2019: Our paper "Learning Latent Dynamics for Planning from Pixels" has been accepted to ICML 2019 taking place in Long Beach, California, USA.
  • 06/2018: Our paper "MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics" has been accepted to ECCV 2018 taking place in Munich, Germany.
  • 05/2018: Our paper on Hierarchical Long-term Video Prediction without Supervision has been accepted to ICML 2018 taking place in Stockholm, Sweden.
  • 03/2018: Our paper on Neural Kinematic Networks for Unsupervised Motion Retargetting has been accepted as Oral Presentation to CVPR 2018 taking place in Salt Lake City, Utah, USA.

 

Publications

2019

High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks New!
Ruben Villegas, Arkanath Pathak, Harini Kannan, Dumitru Erhan, Quoc V. Le, Honglak Lee
In Advances in Neural Information Processing Systems (NeurIPS), 2019
Paper [Coming soon] ArXiv Project page

Unsupervised Learning of Object Structure and Dynamics from Videos New!
Matthias Minderer, Chen Sun, Ruben Villegas, Forrester Cole, Kevin Murphy, Honglak Lee
In Advances in Neural Information Processing Systems (NeurIPS), 2019
Paper ArXiv Project page

Learning Latent Dynamics for Planning from Pixels New!
Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee,
James Davidson
In Proceedings of the 36th International Conference on Machine Learning (ICML), 2019
Paper ArXiv Project page

2018

MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics
Xinchen Yan, Akash Rastogi, Ruben Villegas, Kalyan Sunkavalli, Eli Shechtman,
Sunil Hadap, Ersin Yumer, Honglak Lee
In European Conference on Computer Vision (ECCV), 2018
Paper ArXiv Project page

Hierarchical Long-term Video Prediction without Supervision
Nevan Wichers, Ruben Villegas, Dumitru Erhan, Honglak Lee
In Proceedings of the 35th International Conference on Machine Learning (ICML), 2018
Paper ArXiv Code Project page

Neural Kinematic Networks for Unsupervised Motion Retargetting Oral Presentation
Ruben Villegas, Jimei Yang, Duygu Ceylan, Honglak Lee
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018
(Oral acceptance rate: 2.1%)
Paper Supplementary ArXiv Code Demo Project page Slides Poster

2017

Learning to Generate Long-term Future via Hierarchical Prediction
Ruben Villegas, Jimei Yang, Yuliang Zou, Sungryull Sohn, Xunyu Lin, Honglak Lee
In Proceedings of the 34th International Conference on Machine Learning (ICML), 2017
Paper Supplementary ArXiv Code Project page Slides Poster

Decomposing Motion and Content for Natural Video Sequence Prediction
Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin, Honglak Lee
In International Conference on Learning Representations (ICLR), 2017
Paper ArXiv Code Project page

2015

Improving Object Detection with Deep Convolutional Networks
via Bayesian Optimization and Structured Prediction
Oral Presentation
Yuting Zhang, Kihyuk Sohn, Ruben Villegas, Gang Pan, Honglak Lee
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015
(Oral acceptance rate: 3.3%)
Paper ArXiv Code CV Community Top Paper Award

2014

Who Do I Look Like? Determining Parent-Offspring Resemblance via Genetic Features
Afshin Dehghan, Enrique G. Ortiz, Ruben Villegas, Mubarak Shah
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014
Paper

Teaching

  • Graduate Student Instructor for EECS 598 @University of Michigan: Deep Learning (Winter 2019)
  • Guest Lecture for EECS 492 @University of Michigan: Introduction to Artificial Intelligence (Winter 2019)

Professional Activities

  • Program Committee Member
    • First Workshop on Learning From Unlabeled Videos @ CVPR 2019
  • Conference Reviewer
    • CVPR 2019, 2020
    • CoRL 2018
    • Eurographics 2020
    • ICCV 2019
    • ICLR 2018, 2019, 2020
    • ICML 2019, 2020
    • ICRA 2020
    • NeurIPS 2016, 2018, 2019
    • Pacific Graphics 2019
    • SIGGRAPH 2019
    • UAI 2018, 2019
  • Journal Reviewer
    • IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
    • International Journal on Computer Vision (IJCV)
    • Transactions on Graphics (ToG)

Invited Talks

  • Tech Talk @Waymo, Mountain View, US March 2019
  • Tech Talk @Uber ATG, Toronto, CA Feb 2019
  • Tech Talk @Adobe Research, San Jose, US Feb 2019

Awards

  • Rackham Conference Travel Grant ICML 2017, ICLR 2017, CVPR 2018
  • Conference Travel Award ICML 2017, ICLR 2017
  • University of Michigan Rackham Merit Fellowship Sep 2014 - Present
  • Google Scholar / Hispanic Scholarship Fund Scholarship Sep 2013 - May 2014
  • STATESS Scholarship, University of Central Florida Sep 2011- May 2014