Ruben Villegas

Research Scientist, Adobe Research
villegas [at] adobe.com
Github / Google scholar / Twitter / CV

I completed my PhD from the Computer Science & Engineering department at the University of Michigan, Ann Arbor. My adviser is Professor Honglak Lee. My general research interest lies on modeling structured motion with deep neural networks. Particularly, my interest spans across the areas of computer vision, graphics and model-based reinforcement learning. Most of my work has been in video prediction/generation, motion retargeting and human motion synthesis.

Fun Facts: I played for my national basketball team (I am originally from Ecuador). I was also second best scorer in the nation in a national championship I played back in the day. I was part of a team that beat the media's projected champion during a championship in Quito (the guy that was best scorer in the national championship played for the other team :P). Let's have a Curry-range 3-point shootout. Ok, I'll stop now ...

News

  • 04/2019: I will be joining Adobe Research as Research Scientist this Summer!
  • 04/2019: Our paper "Learning Latent Dynamics for Planning from Pixels" has been accepted to ICML 2019 taking place in Long Beach, California, USA.
  • 06/2018: Our paper "MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics" has been accepted to ECCV 2018 taking place in Munich, Germany.
  • 05/2018: Our paper on Hierarchical Long-term Video Prediction without Supervision has been accepted to ICML 2018 taking place in Stockholm, Sweden.
  • 03/2018: Our paper on Neural Kinematic Networks for Unsupervised Motion Retargetting has been accepted as Oral Presentation to CVPR 2018 taking place in Salt Lake City, Utah, USA.

 

Publications

Preprints

Unsupervised Learning of Object Structure and Dynamics from Videos New!
Matthias Minderer, Chen Sun, Ruben Villegas, Forrester Cole, Kevin Murphy, Honglak Lee
Paper ArXiv Project page

2019

Learning Latent Dynamics for Planning from Pixels New!
Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, James Davidson
In Proceedings of the 36th International Conference on Machine Learning (ICML), 2019
Paper ArXiv Project page

2018

MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics
Xinchen Yan, Akash Rastogi, Ruben Villegas, Kalyan Sunkavalli, Eli Shechtman,
Sunil Hadap, Ersin Yumer, Honglak Lee.
In European Conference on Computer Vision (ECCV), 2018
Paper ArXiv Project page

Hierarchical Long-term Video Prediction without Supervision
Nevan Wichers, Ruben Villegas, Dumitru Erhan, Honglak Lee
In Proceedings of the 35th International Conference on Machine Learning (ICML), 2018
Paper ArXiv Code Project page

Neural Kinematic Networks for Unsupervised Motion Retargetting Oral Presentation
Ruben Villegas, Jimei Yang, Duygu Ceylan, Honglak Lee
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018
(Oral acceptance rate: 2.1%)
Paper Supplementary ArXiv Code Demo Project page Slides Poster

2017

Learning to Generate Long-term Future via Hierarchical Prediction
Ruben Villegas, Jimei Yang, Yuliang Zou, Sungryull Sohn, Xunyu Lin, Honglak Lee
In Proceedings of the 34th International Conference on Machine Learning (ICML), 2017
Paper Supplementary ArXiv Code Project page Slides Poster

Decomposing Motion and Content for Natural Video Sequence Prediction
Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin, Honglak Lee
In International Conference on Learning Representations (ICLR), 2017
Paper ArXiv Code Project page

2015

Improving Object Detection with Deep Convolutional Networks
via Bayesian Optimization and Structured Prediction
Oral Presentation
Yuting Zhang, Kihyuk Sohn, Ruben Villegas, Gang Pan, Honglak Lee
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015
(Oral acceptance rate: 3.3%)
Paper ArXiv Code CV Community Top Paper Award

2014

Who Do I Look Like? Determining Parent-Offspring Resemblance via Genetic Features
Afshin Dehghan, Enrique G. Ortiz, Ruben Villegas, Mubarak Shah
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014
Paper

Teaching

  • Graduate Student Instructor for EECS 598 @University of Michigan: Deep Learning (Winter 2019)
  • Guest Lecture for EECS 492 @University of Michigan: Introduction to Artificial Intelligence (Winter 2019)

Professional Activities

  • Program Committee Member
    • First Workshop on Learning From Unlabeled Videos @ CVPR 2019
  • Conference Reviewer
    • Pacific Graphics 2019
    • ICCV 2019
    • SIGGRAPH 2019
    • ICML 2019
    • CVPR 2019
    • ICLR 2018, 2019, 2020
    • NeurIPS 2016, 2018, 2019
    • CoRL 2018
    • UAI 2018, 2019
  • Journal Reviewer
    • IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
    • International Journal on Computer Vision (IJCV)

Invited Talks

  • Tech Talk @Waymo, Mountain View, US March 2019
  • Tech Talk @Uber ATG, Toronto, CA Feb 2019
  • Tech Talk @Adobe Research, San Jose, US Feb 2019

Awards

  • Rackham Conference Travel Grant ICML 2017, ICLR 2017, CVPR 2018
  • Conference Travel Award ICML 2017, ICLR 2017
  • University of Michigan Rackham Merit Fellowship Sep 2014 - Present
  • Google Scholar / Hispanic Scholarship Fund Scholarship Sep 2013 - May 2014
  • STATESS Scholarship, University of Central Florida Sep 2011- May 2014