The Center for Autonomy Computing (CAC) is dedicated to creating innovative algorithms and software for energy-efficient computing hardware on autonomous platforms. We perform research in perception and control of autonomous systems and in the processing and analytics of data generated by such systems. Our goal is to enable all these computations on the autonomous platforms in an energetically efficient way. Moreover, we discover new algorithms for autonomy functions that exploit unconventional computing hardware.
Autonomy functions focus on perception and control of autonomous systems, signal processing, and robotic swarm behavior.
The information & Systems Sciences Lab has extensive experience in intelligence, surveillance, and reconnaissance technologies, including bio-inspired attention, neuromorphic object recognition, 3D recognition using LIDAR, tracking, and sensor fusion. For applications like autonomous driving and flight, we are developing more accurate object and situation recognition systems that are robust to errors in individual sensors.
HRL’s Cognitive Signal Processing technology combines innovations from machine learning, dynamical system theory, and real-time optimization to perform rapid short-term prediction, anomaly detection, and de-noising of wideband signals in less than 10 nanoseconds. These algorithms are highly scalable (both in bandwidth and complexity) and have been successfully demonstrated to reduce noise by 20-40dB on a wide variety of challenging synthetic and experimental RF data sets.
This material is based upon work supported by the Office of Naval Research, under contract number N00014-12-C-0027. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Office of Naval Research.
Accurate prediction of swarm behavior is a crucial step in counter-swarm tactical adjustment. To this end, HRL’s new innovations in swarm technologies has been extended to make predictions of multi-agent adversarial movements. We have introduced different techniques and evaluate them with real-world datasets from team sport events, predicting, e.g., adversary basketball player positions with less than 87cm mean square error.
Unconventional computing currently focuses on synchronization-based computing, neuromorphic and probabilistic computing, and self-organized criticality.
HRL is exploring new ways to do energy efficient computing using combinations of novel emerging devices along with computing architectures that allow high accuracy results with low-precision devices. Using arrays of coupled oscillators, it is possible to perform a variety of basic operations commonly found in deep neural networks. These arrays can be constructed from new devices developed from nano structures that can operate with extremely low power. The architectures developed for these novel devices incorporate unique sparsity techniques that can also be applied to conventional CMOS circuits, allowing neural networks for deep learning to be constructed with low-power low-precision elements that rival the performance of their high-precision counterparts. This architecture has led to the development of custom CMOS circuits that are able to achieve over 1.4 Tera-ops per watt. Potential applications for these devices include:
This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR0011-13-C-0052. The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government.
Neuromorphic computing, with synaptic plasticity, is capable of performing highly scalable Bayesian computation using few resources, accelerating traditional computational problems. Synaptic plasticity can also be used to learn a task through intermittent rewards.
We have found that computer models of a phenomenon in the brain called self-organized criticality (SOC) can be used to calculate optimal conditions within complex networks. Surprisingly, the search patterns produced by an SOC process have unique properties that lead to more efficient solutions than conventional optimization methods.
Authors | Title | Publication | Year |
---|---|---|---|
S.R. Park, S. Kolouri, S. Kundu, G.K. Rohde | The cumulative distribution transform and linear pattern classification | Applied and Computational Harmonic Analysis, 45(3), pp.616-641 | 2019 |
M. Rostami, S. Kolouri, E. Eaton, Kim, K. | Deep Transfer Learning for Few-Shot SAR Image Classification | Remote Sensing, 11(11), p.1374 | 2019 |
Heiko Hoffmann | Sparse Associative Memory | Neural Computation, Vol. 31(5), pp. 998-1014 | 2019 |
H. Hoffmann, D. W. Payton | Optimization by Self-Organized Criticality | Scientific Reports | 2018 |
V. De Sapio, D. Earl, R. Green, K. Saul | Advanced Analytical Dynamics: Theory and Applications | Cambridge University Press | 2017 |
Y Cao, Y Chen, D Khosla | Spiking deep convolutional neural networks for energy-efficient object recognition | International Journal of Computer Vision, Volume 113, Issue 1, pp 54–66 | 2015 |
N. Srinivasa | Design considerations for a computational architecture of human cognition | Emerging Nanoelectronic Devices, pp. 456-466, John Wiley and Sons | 2015 |
N. D. Stepp, D. Plenz, N. Srinivasa | Synaptic Plasticity Enables Adaptive Self-Tuning Critical Networks | PLOS Computational Biology 11(1) | 2015 |
V. De Sapio, N. Srinivasa | A methodology for controlling motion and constraint forces in holonomically constrained systems | Multibody System Dynamics, 33(2):179– 204 | 2015 |
Dmitriy Korchev, Hyukseong Kwon, Yuri Owechko | Detecting small, low-contrast moving targets in infrared video produced by inconsistent sensor with bad pixels | Optical Engineering, 54(11) | 2015 |
D. Khosla, Y. Chen, and K. Kim | A neuromorphic system for video object recognition | Frontiers of Computational Neuroscience | 2014 |
J. Cruz-Albrecht, T. Derosier, N. Srinivasa | Scalable neural chip with synaptic electronics using CMOS integrated memristors | Nanotechnology, Special Issue on Synaptic Electronics, vol. 24, 384011 (11pp), doi:10.1088/0957-4484/24/38/384011 | 2013 |
N. Srinivasa, Q. Jiang | Stable learning of functional maps in self-organizing spiking neural networks with continuous synaptic plasticity | Front. Comput. Neurosci., 7:10. doi: 10.3389/fncom.2013.00010 | 2013 |
K. Minkovich, N. Srinivasa, J. M. Cruz-Albrecht, Y. K. Cho, A. Nogin | Programming Time-Multiplexed Reconfigurable Hardware Using a Scalable Neuromorphic Compiler | IEEE Trans. on Neural Networks and Learning Systems, vol. 23, no. 6, pp. 889-901 | 2012 |
Authors | Title | Conference | Year |
---|---|---|---|
AM Rahimi, K Lee, A Agarwal, H Kwon, R Bhattacharyya | Toward Improving the Visual Characterization of Sport Activities With Abstracted Scene Graphs | Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition | 2021 |
Johnson, S., Warmsley, D., Martin, C. | Mitigating Limited Observability via Proxy Utilities in Game Theoretic Control | In Proceedings of the 2021 Conference on Complex Systems (CSS) | 2021 |
Sean Soleyman, Fan Hung, Deepak Khosla, Yang Chen, Joshua Fadaie, Navid Naderi | Multi-Agent Autonomous Battle Management Using Deep Neuroevolution | SPIE 11758, Unmanned Systems Technology XXIII, 117580C | 2021 |
H Kwon, A Rahimi, KG Lee, A Agarwal, R Bhattacharyya | CogSense: A Cognitively Inspired Framework for Perception Adaptation | arXiv preprint arXiv:2107.10456 | 2021 |
Joseph F. Comer, Reed W. Andrews, Navid Naderializadeh, Soheil Kolouri, Heiko Hoffman | SAR automatic target recognition with less labels | SPIE 11758, Unmanned Systems Technology XXIII, 117580C SPIE 11394, Automatic Target Recognition XXX, 113940Q; doi: 10.1117/12.2564875 | 2020 |
Sean Soleyman, Deepak Khosla | Multi-Agent Mission Planning with Reinforcement Learning | AAAI Spring Symposium Series Proceedings | 2020 |
M. Rostami, S. Kolouri, E. Eaton, K. Kim | SAR Image Classification Using Few-Shot Cross-Domain Transfer Learning | Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops | 2019 |
Kolouri, Soheil, Phillip E. Pope, Charles E. Martin, and Gustavo K. Rohde | Sliced-Wasserstein Autoencoders | International Conference on Learning Representations (ICLR) | 2019 |
Phillip Pope, Soheil Kolouri, Mohammad Rostami, Charles E Martin, Heiko Hoffmann | Explainability Methods for Graph Convolutional Neural Networks | Conference on Computer Vision and Pattern Recognition (CVPR) | 2019 |
Xi Hang Cao, Kyungnam Kim, Zoran Obradovic | A Simple yet Effective Model for Zero-Shot Learning | IEEE Winter Conf. on Applications of Computer Vision | 2018 |
Soheil Kolouri, Mohammad Rostami, Yuri Owechko, Kyungnam Kim | Joint Dictionaries for Zero-Shot Learning | Thirty-Second AAAI Conference on Artificial Intelligence (AAAI) | 2018 |
Mohammad Rostami, Soheil Kolouri, Kyungnam Kim, Eric Eaton | Multi-Agent Distributed Lifelong Learning for Collective Knowledge Acquisition | AAMAS (International Conference on Autonomous Agents and Multiagent Systems) | 2018 |
Stepp, N., Jammalamadaka, A | A Dynamical Systems Approach to Neuromorphic Computation of Conditional Probabilities | Proceedings of the International Conference on Neuromorphic Systems (p. 7). ACM | 2018 |
Shay Deutsch, Soheil Kolouri, Kyungnam Kim, Yuri Owechko, Stefano Soatto | Zero Shot Learning via Multi-Scale Manifold Regularization | AAAI Conference on Artificial Intelligence | 2018 |
Soheil Kolouri, Gustavo K. Rohde, Heiko Hoffmann | Sliced Wasserstein Distance for Learning Gaussian Mixture Models | Conference on Computer Vision and Pattern Recognition | 2018 |
Zak Murez, Soheil Kolouri, David Kriegman, Ravi Ramamoorthi, Kyungnam Kim | Image to Image Translation for Domain Adaptation | IEEE Conference on Computer Vision and Pattern Recognition (CVPR) | 2018 |
Hyukseong Kwon, Kyungnam Kim, Jean Dolne | Improving 3D registration by upsampling of sparse point cloud through fusion with high-resolution 2D image | SPIE Proceedings Volume 10410, Unconventional and Indirect Imaging, Image Reconstruction, and Wavefront Sensing | 2017 |
Ryan Uhlenbrock, Kyungnam Kim, Heiko Hoffmann, Jean Dolne | Rapid 3D registration using local subtree caching in iterative closest point (ICP) algorithm | SPIE Proceedings Volume 10410, Unconventional and Indirect Imaging, Image Reconstruction, and Wavefront Sensing | 2017 |
Deepak Khosla, Yang Chen, Jiejun Xu, Kyungnam Kim | Online Location recognition for drift-free state estimation and efficient autonomous exploration | SPIE Defense and Security | 2017 |
Qin Jiang, Yuri Owechko, Brendan Blanton | Visibility enhancement of multi-waveband IR images from degraded visual environment | SPIE Proceeding | 2017 |
David Huber, Kyungnam Kim, Deepak Khosla | Motion-Seeded Object-Based Attention for Dynamic Visual Imagery | SPIE Defense and Security | 2017 |
A. M. Rahimi, S. Kolouri, R. Bhattacharyya | Automatic Tactical Adjustment in Real-Time: Modeling Adversary Formations With Radon-Cumulative Distribution Transform and Canonical Correlation Analysis | IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 83-90 | 2017 |
S. M. Salas, R. P. Patrick, S. Roach, M. E. Phillips, N. D. Stepp, J. Cruz-Albrecht, V. De Sapio, T-C. Lu, V. Sritapan. | Neuromorphic and early warning behavior- based authentication in common theft scenarios | Proceedings of the 2017 IEEE International Symposium on Technologies for Homeland Security (HST) | 2017 |
V. De Sapio, M. Howard, D. Korchev, R. Green, R. Gardner, L. Bruchal | Demographic specific musculoskeletal models of factory worker performance, fatigue, and injury | Proceedings of the 2016 IEEE International Aerospace Conference | 2016 |
Phillips, M. E., Stepp, N. D., Cruz-Albrecht, J., De Sapio, V., Lu, T. C., Sritapan, V | Neuromorphic and early warning behavior-based authentication for mobile devices | Technologies for Homeland Security (HST), 2016 IEEE Symposium on (pp. 1-5). IEEE | 2016 |
Qin Jiang, Yuri Owechko, Brendan Blanton | Wavelet based visibility enhancement of IR images | SPIE Proceedings | 2016 |
M. Mansouri, V. De Sapio, J. Reinbolt | Prioritized task-based control of movement with supporting contacts using OpenSim and Matlab | Abstracts for the XV International Symposium on Computer Simulation in Biomechanics | 2015 |
Kyungnam Kim, David J. Huber, Jiejun Xu, Deepak Khosla | Efficient Algorithms for Indoor MAV Flight using Vision and Sonar Sensors | 11th International Symposium on Visual Computing (ISVC), Las Vegas, Nevada, USA | 2015 |
Jiejun Xu, Kyungnam Kim, Lei Zhang, Deepak Khosla | 3D Perception for Autonomous Robot Exploration | 11th International Symposium on Visual Computing (ISVC), Las Vegas, Nevada, USA | 2015 |
V. De Sapio, D. Earl, R. Green, K. Saul | Human factors simulation using demographically tuned biomechanical models | Proceedings of the 2014 International Annual Meeting of the Human Factors and Ergonomics Society, volume 58, pages 944– 948 | 2014 |
S. Goldfarb, D. Earl, V. De Sapio, M. Mansouri, J. Reinbolt | An approach and implementation for coupling neurocognitive and neuromechanical models | Proceedings of the 2014 IEEE International Conference on Systems, Man, and Cybernetics, pages 406–413 | 2014 |
V. De Sapio | An approach for goal-oriented neuromuscular control of digital humans | International Journal of Human Factors Modelling and Simulation, 4(2):121– 144 | 2014 |
Authors | Title | Patent # | Date |
---|---|---|---|
Qin Jiang, Yuri Owechko, Ken Kyungnam | Apparatus, system and method for determining an object’s location in image video data | US10549853 | 02/04/2020 |
Qin Jiang, Yuri Owechko | Apparatus, system and method for enhancing image video data | US10176557 | 01/08/2020 |
Heiko Hoffmann, Ken Kim | Robust recognition on degraded imagery by exploiting known image transformation under motion | US10332265 | 06/25/2019 |
Yongqiang Gao, Qin Jiang, Yang Chen, Deepak Khosla | Method for object detection in digital image and video using spiking neural networks | US10198689 | 02/05/2019 |
Qin Jiang, Yuri Owechko | Apparatus, system and method for enhancing image video data | US10176557 | 01/08/2019 |
Heiko Hoffmann, Jaehoon Choe, Corey Thibeault | Device and method to automatically tune the nerve stimulation pattern of a sensory feedback capable prosthesis | US10166394 | 01/01/2019 |
V. De Sapio, S. E. Chelian, R. Bhattacharyya, M. E. Phillips, M. D. Ziegler, D. W. Payton | In-home patient-focused rehabilitation system | US10130311 | 11/20/2018 |
Heiko Hoffmann | Neural network device with engineered delays for pattern storage and matching | US10095976 | 10/09/2018 |
Vincent DeSapio, Michael D. Howard, Matthew E. Phillips, Kevin R. Martin, Heiko Hoffmann, David W. Payton | System and method for assistive gait intervention and fall prevention | US10052062 | 08/21/2018 |
Heiko Hoffmann, David W. Payton | Adaptive control system capable of recovering from unexpected situations | US10001760 | 06/19/2018 |
Shuoqin Wang, Qin Jiang | A method for on-line state of power (SOP) estimation of a Li-ion battery with high accuracy | US9989595 | 06/05/2018 |
Hyukseong Kwon, Kyungnam Kim | System and method for upsampling of sparse point cloud for 3D registration | US9972067 | 05/15/2018 |
Heiko Hoffmann, Michael J. Daily | System and method for robot supervisory control with an augmented reality user interface | US9880553 | 01/30/2018 |
Swarup Medasani, Jason Meltzer, Jiejun Xu, Zhichao Chen, Rashmi Sundareswara, David Payton, Ryan Uhlenbrock, Leandro Barajas, Kyungnam Kim | Method for object localization and pose estimation for an object of interest | US9875427 | 01/23/2018 |
Darren J. Earl, Ryan M. Uhlenbrock, Heiko Hoffmann | Device and method for merging 3D point clouds from sparsely distributed viewpoints | US9858640 | 01/02/2018 |
V. De Sapio, D. J. Earl | Method and system for tuning a musculoskeletal model | US9858391 | 01/02/2018 |
David Payton, Kyungnam Kim, Zhichao Chen, Ryan Uhlenbrock, Li Yang Ku | Robotic device including machine vision | US9844881 | 12/19/2017 |
Heiko Hoffmann, David W Payton, Vincent DeSapio | Method for tele-robotic operations over time-delayed communication links | US9776325 | 10/03/2017 |
Hyukseong Kwon, Kyungnam Kim, Yuri Owechko | Method and apparatus for tracking targets | US9727785 | 08/08/2017 |
Hyukseong Kwon, Kyungnam Kim, Yuri Owechko | Method and apparatus for detecting targets | US9715639 | 07/25/2017 |
Heiko Hoffmann, Behnam Salemi | Robotic control device and method for manipulating a hand-held tool | US9613180 | 04/04/2017 |
V. De Sapio, M. D. Howard, R. F. Green | Quantifying muscle and tendon fatigue during physical exertion | US9610036 | 04/04/2017 |
V. De Sapio, N. Srinivasa | System for controlling brain machine interfaces and neural prosthetic systems | US9566174 | 02/14/2017 |
Heiko Hoffmann, David W Payton | Self-stabilizing system for multiple interacting controllers | US9557722 | 01/31/2017 |
Terrell N. Mundhenk, Arturo Flores, Heiko Hoffmann | Method for classification and segmentation and forming 3D models from images | US9530218 | 12/27/2016 |
Qin Jiang, Yuri Owechko | Apparatus, System, and Method for Enhancing Image Data | US9508134 | 11/29/2016 |
Dmitriy Korchev, Yuri Owechko, Hyukseong Kwon | Methods and systems for detecting moving objects in a sequence of image frames produced by sensors with inconsistent gain, offset, and dead pixels | US9501839 | 11/22/2016 |
Darren Earl, Derek Mitchell, Heiko Hoffmann | System and method for quick scripting of tasks for autonomous robotic manipulation | US9486918 | 11/08/2016 |
Darren Earl, Derek Mitchell, Heiko Hoffmann | System and method for quick scripting of tasks for autonomous robotic manipulation | US9486918 | 11/08/2016 |
Hyukseong Kwon, Yuri Owechko, Kyungnam Kim | Occlusion-robust visual object fingerprinting using fusion of multiple sub-region signatures | US9483839 | 11/01/2016 |
Karim El Defrawy, Joshua D. Lampkins | Method for secure and resilient distributed generation of elliptic curve digital signature algorithm (ECDSA) based digital signatures with proactive security | US9467451 | 10/11/2016 |
David Payton, Michael Daily | Systems, methods, and apparatus for neuro-robotic goal selection | US9445739 | 09/20/2016 |
David Payton, Ryan Uhlenbrock, Li Yang Ku | Rapid robotic imitation learning of force-torque tasks | US9403273 | 08/02/2016 |
Derek Mitchell, Heiko Hoffmann | Dynamic obstacle avoidance in a robotic system | US9403275 | 08/02/2016 |
Leandro Barajas, David Payton, Li Yang Ku, Ryan Uhlenbrock, Darren Earl | Visual debugging of robotic tasks | US9387589 | 07/12/2016 |
Heiko Hoffmann, David W. Payton, Derek Mitchell | Dynamical system-based robot velocity control | US9381643 | 07/05/2016 |
Vincent DeSapio, Heiko Hoffmann | A system for controlling motion and constraint forces in a robotic system without the need for force sensing | US9364951 | 06/14/2016 |
Zhichao Chen, Heiko Hoffmann | Device and method to localize and control a tool tip with a robot arm | US9259840 | 02/16/2016 |
Heiko Hoffmann, Hooman Kazemi, Michael J Daily | System and Method for Fast Template Matching in 3D | US9171247 | 10/27/2015 |
Qin Jiang, Michael J. Daily, Richard Michael Kremer | Acoustic Ranging System Using Atmospheric Dispersion | US9146295 | 09/29/2015 |
Michael Daily, Michael Howard, Yang Chen, David Payton, Rashmi Sundareswara | Recall system using spiking neuron networks | US9020870 | 04/28/2015 |
Qin Jiang, Yang Chen | System for automatic data clustering utilizing bio-inspired computing models | US9009156 | 04/14/2015 |
Suhas E Chelian, Rashmi N Sundareswara, Heiko Hoffmann | Robotic visual perception system | US9002098 | 04/07/2015 |
William Noble, Serdar Gokcen, Michael Howard | Track prediction and identification via particle motion with intent | US9002642 | 04/07/2015 |
Leandro Barajas, Eric Martinson, David Payton, Ryan Uhlenbrock | Method and system for training a robot using human-assisted task demonstration | US8843236 | 09/23/2014 |
Qin Jiang | Radar pulse detection using a digital radar receiver | US8803730 | 08/12/2014 |
David Payton, Michael Daily | Systems, methods, and apparatus for neuro-robotic tracking point selection | US8788030 | 07/22/2014 |
Michael Daily, Michael Howard, Yang Chen, Rashmi Sundareswara, David Payton | System for representing, storing, and reconstructing an input signal | US8756183 | 06/17/2014 |
David Payton, Michael Daily | Systems, methods, and apparatus for neuro-robotic tracking point selection | US8483816 | 07/09/2013 |
Qin Jiang | Active sonar system and active sonar method using fuzzy logic | US8320216 | 11/27/2012 |
Qin Jiang, Shubha Kadambe | Active sonar system and active sonar method using noise reduction techniques and advanced signal processing techniques | US8116169 | 02/14/2012 |
Qin Jiang, Shubha Kadambe | System and method for enhancing weak target signals for a sensor array | US8068385 | 11/29/2011 |
David Payton, Michael Daily, Mike Howard | Distributed display composed of active fiducials | US8035078 | 10/11/2011 |
Michael Howard, David Payton | System and method for distributed engagement | US7912631 | 03/22/2011 |
Michael Howard, David Payton, Wendell Bradshaw, Timothy Smith | System and method for automated search by distributed elements | US7908040 | 03/15/2011 |
David Payton | Method and system for independently observing and modifying the activity of an actor processor | US7877347 | 01/25/2011 |
David Payton | Event localization within a distributed sensor array | US7786885 | 08/31/2010 |
David Payton, Scott Smith | Iterative particle reduction methods and systems for localization and pattern recognition | US7613673 | 11/03/2009 |
David Payton, Mike Daily, Mike Howard, Craig Lee | Distributed display composed of active fiducials | US7612324 | 11/03/2009 |
Peter Tinker, David Payton | System and method for computing reachable areas | US7599814 | 10/06/2009 |
David Payton, Eric Martinson | Arranging mobile sensors into a predetermined pattern | US7379840 | 05/27/2008 |
David Payton | Method and apparatus for providing directed communications through a networked array of nodes | US7158511 | 01/02/2007 |
David Payton, Bruce Hoff, Mike Howard, Craig Lee | Method and apparatus for signaling among a plurality of agents | US7113746 | 09/26/2006 |
David Payton, Regina Estkowski | Motion prediction within an amorphous sensor array | US6885303 | 04/26/2005 |
David Payton, Craig Lee, Bruce Hoff, Mike Howard, Mike Daily | Method and apparatus for terrain reasoning with distributed embedded processing elements | US6580979 | 06/17/2003 |
David Payton, Mike Howard, Mike Daily, Craig Lee, Bruce Hoff | Method and apparatus for controlling the movement of a plurality of agents | US6507771 | 01/14/2003 |
David Payton, David Keirsey | System and method for rapid determination of visibility-based terrain properties over broad regions | US6173067 | 01/09/2001 |
David Payton | System and method for processing commands from a plurality of control sources | US5285380 | 02/08/1994 |
Media coverage coming soon.
June 21, 2019
|
HRL Laboratories - Malibu, CA
Computer Vision Frontier Workshop
Email: cemartin[at]hrl.com
Dr. Charles E. Martin
Research Scientist
Leader, Center for Autonomy Computing
Information and Systems Sciences Lab
HRL Laboratories, LLC
3011 Malibu Canyon Road
Malibu, CA 90265
USA