Danny Driess

I am a PhD student in Machine Learning and Robotics since 2019, affiliated with the TU Berlin, SCIoI and the IMPRS-IS.
My advisor is Marc Toussaint.
In 2020, I visited Russ Tedrake's Lab at MIT.
Currently, I am a Student Researcher at Google Brain, working with Pete Florence.
I organized three RSS workshops in 2020, 2021 discussing the synergies between learning and planning, and 2022 on Implicit Representations for Robotics.

Email: danny dot driess at gmail dot com

Google Scholar Twitter

Publications

For a full list of publications see Google Scholar

PaLM-E: An Embodied Multimodal Language Model

Danny Driess, Fei Xia, Mehdi Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, Pete Florence
ICML, 2023

PDF Videos arXiv

@inproceedings{driess2023palme,
    title={PaLM-E: An Embodied Multimodal Language Model},
    author={Driess, Danny and Xia, Fei and Sajjadi, Mehdi S. M. and Lynch, Corey and Chowdhery, Aakanksha and Ichter, Brian and Wahid, Ayzaan and Tompson, Jonathan and Vuong, Quan and Yu, Tianhe and Huang, Wenlong and Chebotar, Yevgen and Sermanet, Pierre and Duckworth, Daniel and Levine, Sergey and Vanhoucke, Vincent and Hausman, Karol and Toussaint, Marc and Greff, Klaus and Zeng, Andy and Mordatch, Igor and Florence, Pete},
    booktitle={Proceedings of the International Conference on Machine Learning},
    year={2023}
}
Reinforcement Learning with Neural Radiance Fields

Danny Driess, Ingmar Schubert, Pete Florence, Yunzhu Li, Marc Toussaint
NeurIPS, 2022

PDF Video arXiv

@InProceedings{22-driess-NeRF-RL,
  title={Reinforcement Learning with Neural Radiance Fields},
  author={Driess, Danny and Schubert, Ingmar and Florence, Pete and Li, Yunzhu and Toussaint, Marc},
  booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
  year={2022}
}
Learning Multi-Object Dynamics with Compositional Neural Radiance Fields

Danny Driess, Zhiao Huang, Yunzhu Li, Russ Tedrake, Marc Toussaint
CoRL, 2022

PDF Video arXiv

@InProceedings{2022-driess-compNerf,
  title = {Learning Multi-Object Dynamics with Compositional Neural Radiance Fields},
  author = {Driess, Danny and Huang, Zhiao and Li, Yunzhu and Tedrake, Russ and Toussaint, Marc},
  booktitle = {Proc{.} of the Annual Conf{.} on Robot Learning (CoRL)},
  pages = {1755--1768},
  year = {2023},
  volume = {205},
  series = {Proceedings of Machine Learning Research},
  month = {14--18 Dec},
  publisher = {PMLR},
}
Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning

Danny Driess, Jung-Su Ha, Marc Toussaint, Russ Tedrake
CoRL 2021

PDF Video

@InProceedings{21-driess-CORL,
  title = {Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning},
  author = {Driess, Danny and Ha, Jung-Su and Toussaint, Marc and Tedrake, Russ},
  booktitle = {Proc{.} of the Annual Conf{.} on Robot Learning (CoRL)},
  year = {2021},
}
Learning to Solve Sequential Physical Reasoning Problems from a Scene Image

Danny Driess, Jung-Su Ha, Marc Toussaint
IJRR 2021

PDF Video DOI

@article{21-driess-IJRR,
  title = {Learning to Solve Sequential Physical Reasoning Problems from a Scene Image},
  author = {Driess, Danny and Ha, Jung-Su and Toussaint, Marc},
  journal = {The International Journal of Robotics Research (IJRR)},
  year = {2021},
  doi = {10.1177/02783649211056967},
}
Learning Geometric Reasoning and Control for Long-Horizon Tasks from Visual Input

Danny Driess, Jung-Su Ha, Russ Tedrake, Marc Toussaint
ICRA 2021

PDF Video

@InProceedings{21-driess-ICRA,
  title = {Learning Geometric Reasoning and Control for Long-Horizon Tasks from Visual Input},
  author = {Danny Driess and Jung-Su Ha and Russ Tedrake and Marc Toussaint},
  booktitle = {Proc{.} of the IEEE Int{.} Conf{.} on Robotics and Automation (ICRA)},
  year = {2021}
}
Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image

Danny Driess, Jung-Su Ha, Marc Toussaint
R:SS 2020

PDF Video

@InProceedings{20-driess-RSS,
  title = {Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image},
  author = {Danny Driess and Jung-Su Ha and Marc Toussaint},
  booktitle = {Proc{.} of Robotics: Science and Systems (R:SS)},
  year = {2020}
}
Deep Visual Heuristics: Learning Feasibility of Mixed-Integer Programs for Manipulation Planning

Danny Driess, Ozgur Oguz, Jung-Su Ha, Marc Toussaint
ICRA 2020

Nominated for Best Paper Award in Automation

PDF Video

@InProceedings{20-driess-ICRA,
  title = {Deep Visual Heuristics: Learning Feasibility of Mixed-Integer Programs for Manipulation Planning},
  author = {Danny Driess and Ozgur Oguz and Jung-Su Ha and Marc Toussaint},
  booktitle = {Proc{.} of the IEEE Int{.} Conf{.} on Robotics and Automation (ICRA)},
  year = {2020}
}
Probabilistic framework for constrained manipulations and task and motion planning under uncertainty

Jung-Su Ha, Danny Driess, Marc Toussaint
ICRA 2020

PDF Video

@InProceedings{20-ha-ICRA,
  title={Probabilistic framework for constrained manipulations and task and motion planning under uncertainty},
  author={Ha, Jung-Su and Driess, Danny and Toussaint, Marc},
  booktitle = {Proc{.} of the IEEE Int{.} Conf{.} on Robotics and Automation (ICRA)},
  year = {2020}
}
Optimality Principles in Human Point-to-Manifold Reaching Accounting for Muscle Dynamics

Isabell Wochner, Danny Driess, Heiko Zimmermann, Daniel F. B. Haeufle, Marc Toussaint, Syn Schmitt
Frontiers in Computational Neuroscience 2020

Paper

@ARTICLE{20-wochner-frontiers,
  AUTHOR={Wochner, Isabell and Driess, Danny and Zimmermann, Heiko and Haeufle, Daniel F. B. and Toussaint, Marc and Schmitt, Syn},
  TITLE={Optimality Principles in Human Point-to-Manifold Reaching Accounting for Muscle Dynamics},
  JOURNAL={Frontiers in Computational Neuroscience},
  VOLUME={14},
  PAGES={38},
  YEAR={2020},
}
Hierarchical Task and Motion Planning using Logic-Geometric Programming (HLGP)

Danny Driess, Ozgur Oguz, Marc Toussaint
R:SS 2020 Workshop on Robust Task and Motion Planning

PDF Video

@Misc{19-driess-RSSws,
  title = {Hierarchical Task and Motion Planning using Logic-Geometric Programming (HLGP)},
  author = {Danny Driess and Ozgur Oguz and Marc Toussaint},
  year = {2019},
  howpublished = {RSS Workshop on Robust Task and Motion Planning}
}
Active Inverse Model Learning with Error and Reachable Set Estimates

Danny Driess, Syn Schmitt, Marc Toussaint
IROS 2019

PDF Video

@InProceedings{19-driess-IROS,
  title = {Active Inverse Model Learning with Error and Reachable Set Estimates},
  author = {Danny Driess and Syn Schmitt and Marc Toussaint},
  booktitle = {Proc{.} of the IEEE Int{.} Conf{.} on Intelligent Robots and Systems (IROS)},
  year = {2019}
}
Active Multi-Contact Continuous Tactile Exploration with Gaussian Process Differential Entropy

Danny Driess, Daniel Hennes, Marc Toussaint
ICRA 2019

PDF Video

@InProceedings{19-driess-ICRA,
  title = {Active Multi-Contact Continuous Tactile Exploration with Gaussian Process Differential Entropy},
  author  = {Danny Driess and Daniel Hennes and Marc Toussaint},
  booktitle  = {Proc{.} of the IEEE Int{.} Conf{.} on Robotics and Automation (ICRA)},
  year = {2019},
}
Learning to Control Redundant Musculoskeletal Systems with Neural Networks and SQP: Exploiting Muscle Properties

Danny Driess, Heiko Zimmermann, Simon Wolfen, Dan Suissa, Daniel Haeufle, Daniel Hennes, Marc Toussaint, Syn Schmitt
ICRA 2018

PDF Video

@InProceedings{18-driess-ICRA,
  title = {Learning to Control Redundant Musculoskeletal Systems with Neural Networks and {SQP}: Exploiting Muscle Properties},
  author = {Danny Driess and Heiko Zimmermann and Simon Wolfen and Dan Suissa and Daniel Haeufle and Daniel Hennes and Marc Toussaint and Syn Schmitt},
  booktitle = {Proc{.} of the IEEE Int{.} Conf{.} on Robotics and Automation (ICRA)},
  year = {2018}
}
Active Learning with Query Paths for Tactile Object Shape Exploration

Danny Driess, Peter Englert, Marc Toussaint
IROS 2017

PDF Video

@InProceedings{17-driess-IROS,
  author = {Danny Driess and Peter Englert and Marc Toussaint},
  title = {Active Learning with Query Paths for Tactile Object Shape Exploration},
  booktitle = {Proc. of the IEEE Int. Conf on Intelligent Robots and Systems (IROS)},
  year = {2017}
}
Constrained Bayesian Optimization of Combined Interaction Force/Task Space Controllers for Manipulations

Danny Driess, Peter Englert, Marc Toussaint
ICRA 2017

PDF Video

@InProceedings{17-driess-ICRA,
  author  = {Danny Driess and Peter Englert and Marc Toussaint},
  title = {Constrained Bayesian Optimization of Combined Interaction Force/Task Space Controllers for Manipulations},
  booktitle  = {Proc. of the IEEE Int. Conf. on Robotics and Automation (ICRA)},
  year = {2017}
}