Autonomy Functions Jump to section
Unconventional Computing Jump to section

Center for Autonomy Computing (CAC)

The Center for Autonomy Computing (CAC) is dedicated to creating innovative algorithms and software for energy-efficient computing hardware on autonomous platforms. We perform research in perception and control of autonomous systems and in the processing and analytics of data generated by such systems. Our goal is to enable all these computations on the autonomous platforms in an energetically efficient way. Moreover, we discover new algorithms for autonomy functions that exploit unconventional computing hardware.


Autonomy Functions

Autonomy functions focus on perception and control of autonomous systems, signal processing, and robotic swarm behavior.


The information & Systems Sciences Lab has extensive experience in intelligence, surveillance, and reconnaissance technologies, including bio-inspired attention, neuromorphic object recognition, 3D recognition using LIDAR, tracking, and sensor fusion. For applications like autonomous driving and flight, we are developing more accurate object and situation recognition systems that are robust to errors in individual sensors.

Selected Publications:

  • D. Khosla, Y. Chen, and K. Kim. A neuromorphic system for video object recognition. Frontiers of Computational Neuroscience, November 28, 2014

Signal Processing

HRL’s Cognitive Signal Processing technology combines innovations from machine learning, dynamical system theory, and real-time optimization to perform rapid short-term prediction, anomaly detection, and de-noising of wideband signals in less than 10 nanoseconds. These algorithms are highly scalable (both in bandwidth and complexity) and have been successfully demonstrated to reduce noise by 20-40dB on a wide variety of challenging synthetic and experimental RF data sets.

This material is based upon work supported by the Office of Naval Research, under contract number N00014-12-C-0027. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Office of Naval Research.

Swarm Behavior

Accurate prediction of swarm behavior is a crucial step in counter-swarm tactical adjustment. To this end, HRL’s new innovations in swarm technologies has been extended to make predictions of multi-agent adversarial movements. We have introduced different techniques and evaluate them with real-world datasets from team sport events, predicting, e.g., adversary basketball player positions with less than 87cm mean square error.

Unconventional Computing

Unconventional computing currently focuses on synchronization-based computing, neuromorphic and probabilistic computing, and self-organized criticality.

Synchronization-based Computing

HRL is exploring new ways to do energy efficient computing using combinations of novel emerging devices along with computing architectures that allow high accuracy results with low-precision devices. Using arrays of coupled oscillators, it is possible to perform a variety of basic operations commonly found in deep neural networks. These arrays can be constructed from new devices developed from nano structures that can operate with extremely low power. The architectures developed for these novel devices incorporate unique sparsity techniques that can also be applied to conventional CMOS circuits, allowing neural networks for deep learning to be constructed with low-power low-precision elements that rival the performance of their high-precision counterparts. This architecture has led to the development of custom CMOS circuits that are able to achieve over 1.4 Tera-ops per watt. Potential applications for these devices include:

  1. Autonomous driving
  2. Persistent surveillance
  3. Internet of Things
  4. Airport security

This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under Contract No. HR0011-13-C-0052. The views expressed are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government.

Neuromorphic and Probabilistic Computing

Neuromorphic computing, with synaptic plasticity, is capable of performing highly scalable Bayesian computation using few resources, accelerating traditional computational problems. Synaptic plasticity can also be used to learn a task through intermittent rewards.

Self-organized Criticality

We have found that computer models of a phenomenon in the brain called self-organized criticality (SOC) can be used to calculate optimal conditions within complex networks. Surprisingly, the search patterns produced by an SOC process have unique properties that lead to more efficient solutions than conventional optimization methods.

Selected Publications:

  • H. Hoffmann and D. W. Payton. Optimization by Self-Organized Criticality. Scientific Reports, Feb. 5, 2018


James Comer

Joshua Fadaie

Charles Martin

Wyatt McAllister

Neale Ratzlaff

Ling Zhang


Journals (14)

Authors Title Publication Year
S.R. Park, S. Kolouri, S. Kundu, G.K. Rohde The cumulative distribution transform and linear pattern classification Applied and Computational Harmonic Analysis, 45(3), pp.616-641 2019
M. Rostami, S. Kolouri, E. Eaton, Kim, K. Deep Transfer Learning for Few-Shot SAR Image Classification Remote Sensing, 11(11), p.1374 2019
Heiko Hoffmann Sparse Associative Memory Neural Computation, Vol. 31(5), pp. 998-1014 2019
H. Hoffmann, D. W. Payton Optimization by Self-Organized Criticality Scientific Reports 2018
V. De Sapio, D. Earl, R. Green, K. Saul Advanced Analytical Dynamics: Theory and Applications Cambridge University Press 2017
Y Cao, Y Chen, D Khosla Spiking deep convolutional neural networks for energy-efficient object recognition International Journal of Computer Vision, Volume 113, Issue 1, pp 54–66 2015
N. Srinivasa Design considerations for a computational architecture of human cognition Emerging Nanoelectronic Devices, pp. 456-466, John Wiley and Sons 2015
N. D. Stepp, D. Plenz, N. Srinivasa Synaptic Plasticity Enables Adaptive Self-Tuning Critical Networks PLOS Computational Biology 11(1) 2015
V. De Sapio, N. Srinivasa A methodology for controlling motion and constraint forces in holonomically constrained systems Multibody System Dynamics, 33(2):179– 204 2015
Dmitriy Korchev, Hyukseong Kwon, Yuri Owechko Detecting small, low-contrast moving targets in infrared video produced by inconsistent sensor with bad pixels Optical Engineering, 54(11) 2015
D. Khosla, Y. Chen, and K. Kim A neuromorphic system for video object recognition Frontiers of Computational Neuroscience 2014
J. Cruz-Albrecht, T. Derosier, N. Srinivasa Scalable neural chip with synaptic electronics using CMOS integrated memristors Nanotechnology, Special Issue on Synaptic Electronics, vol. 24, 384011 (11pp), doi:10.1088/0957-4484/24/38/384011 2013
N. Srinivasa, Q. Jiang Stable learning of functional maps in self-organizing spiking neural networks with continuous synaptic plasticity Front. Comput. Neurosci., 7:10. doi: 10.3389/fncom.2013.00010 2013
K. Minkovich, N. Srinivasa, J. M. Cruz-Albrecht, Y. K. Cho, A. Nogin Programming Time-Multiplexed Reconfigurable Hardware Using a Scalable Neuromorphic Compiler IEEE Trans. on Neural Networks and Learning Systems, vol. 23, no. 6, pp. 889-901 2012

Conferences (32)

Authors Title Conference Year
AM Rahimi, K Lee, A Agarwal, H Kwon, R Bhattacharyya Toward Improving the Visual Characterization of Sport Activities With Abstracted Scene Graphs Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2021
Johnson, S., Warmsley, D., Martin, C. Mitigating Limited Observability via Proxy Utilities in Game Theoretic Control In Proceedings of the 2021 Conference on Complex Systems (CSS) 2021
Sean Soleyman, Fan Hung, Deepak Khosla, Yang Chen, Joshua Fadaie, Navid Naderi Multi-Agent Autonomous Battle Management Using Deep Neuroevolution SPIE 11758, Unmanned Systems Technology XXIII, 117580C 2021
H Kwon, A Rahimi, KG Lee, A Agarwal, R Bhattacharyya CogSense: A Cognitively Inspired Framework for Perception Adaptation arXiv preprint arXiv:2107.10456 2021
Joseph F. Comer, Reed W. Andrews, Navid Naderializadeh, Soheil Kolouri, Heiko Hoffman SAR automatic target recognition with less labels SPIE 11758, Unmanned Systems Technology XXIII, 117580C SPIE 11394, Automatic Target Recognition XXX, 113940Q; doi: 10.1117/12.2564875 2020
Sean Soleyman, Deepak Khosla Multi-Agent Mission Planning with Reinforcement Learning AAAI Spring Symposium Series Proceedings 2020
M. Rostami, S. Kolouri, E. Eaton, K. Kim SAR Image Classification Using Few-Shot Cross-Domain Transfer Learning Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops 2019
Kolouri, Soheil, Phillip E. Pope, Charles E. Martin, and Gustavo K. Rohde Sliced-Wasserstein Autoencoders International Conference on Learning Representations (ICLR) 2019
Phillip Pope, Soheil Kolouri, Mohammad Rostami, Charles E Martin, Heiko Hoffmann Explainability Methods for Graph Convolutional Neural Networks Conference on Computer Vision and Pattern Recognition (CVPR) 2019
Xi Hang Cao, Kyungnam Kim, Zoran Obradovic A Simple yet Effective Model for Zero-Shot Learning IEEE Winter Conf. on Applications of Computer Vision 2018
Soheil Kolouri, Mohammad Rostami, Yuri Owechko, Kyungnam Kim Joint Dictionaries for Zero-Shot Learning Thirty-Second AAAI Conference on Artificial Intelligence (AAAI) 2018
Mohammad Rostami, Soheil Kolouri, Kyungnam Kim, Eric Eaton Multi-Agent Distributed Lifelong Learning for Collective Knowledge Acquisition AAMAS (International Conference on Autonomous Agents and Multiagent Systems) 2018
Stepp, N., Jammalamadaka, A A Dynamical Systems Approach to Neuromorphic Computation of Conditional Probabilities Proceedings of the International Conference on Neuromorphic Systems (p. 7). ACM 2018
Shay Deutsch, Soheil Kolouri, Kyungnam Kim, Yuri Owechko, Stefano Soatto Zero Shot Learning via Multi-Scale Manifold Regularization AAAI Conference on Artificial Intelligence 2018
Soheil Kolouri, Gustavo K. Rohde, Heiko Hoffmann Sliced Wasserstein Distance for Learning Gaussian Mixture Models Conference on Computer Vision and Pattern Recognition 2018
Zak Murez, Soheil Kolouri, David Kriegman, Ravi Ramamoorthi, Kyungnam Kim Image to Image Translation for Domain Adaptation IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018
Hyukseong Kwon, Kyungnam Kim, Jean Dolne Improving 3D registration by upsampling of sparse point cloud through fusion with high-resolution 2D image SPIE Proceedings Volume 10410, Unconventional and Indirect Imaging, Image Reconstruction, and Wavefront Sensing 2017
Ryan Uhlenbrock, Kyungnam Kim, Heiko Hoffmann, Jean Dolne Rapid 3D registration using local subtree caching in iterative closest point (ICP) algorithm SPIE Proceedings Volume 10410, Unconventional and Indirect Imaging, Image Reconstruction, and Wavefront Sensing 2017
Deepak Khosla, Yang Chen, Jiejun Xu, Kyungnam Kim Online Location recognition for drift-free state estimation and efficient autonomous exploration SPIE Defense and Security 2017
Qin Jiang, Yuri Owechko, Brendan Blanton Visibility enhancement of multi-waveband IR images from degraded visual environment SPIE Proceeding 2017
David Huber, Kyungnam Kim, Deepak Khosla Motion-Seeded Object-Based Attention for Dynamic Visual Imagery SPIE Defense and Security 2017
A. M. Rahimi, S. Kolouri, R. Bhattacharyya Automatic Tactical Adjustment in Real-Time: Modeling Adversary Formations With Radon-Cumulative Distribution Transform and Canonical Correlation Analysis IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 83-90 2017
S. M. Salas, R. P. Patrick, S. Roach, M. E. Phillips, N. D. Stepp, J. Cruz-Albrecht, V. De Sapio, T-C. Lu, V. Sritapan. Neuromorphic and early warning behavior- based authentication in common theft scenarios Proceedings of the 2017 IEEE International Symposium on Technologies for Homeland Security (HST) 2017
V. De Sapio, M. Howard, D. Korchev, R. Green, R. Gardner, L. Bruchal Demographic specific musculoskeletal models of factory worker performance, fatigue, and injury Proceedings of the 2016 IEEE International Aerospace Conference 2016
Phillips, M. E., Stepp, N. D., Cruz-Albrecht, J., De Sapio, V., Lu, T. C., Sritapan, V Neuromorphic and early warning behavior-based authentication for mobile devices Technologies for Homeland Security (HST), 2016 IEEE Symposium on (pp. 1-5). IEEE 2016
Qin Jiang, Yuri Owechko, Brendan Blanton Wavelet based visibility enhancement of IR images SPIE Proceedings 2016
M. Mansouri, V. De Sapio, J. Reinbolt Prioritized task-based control of movement with supporting contacts using OpenSim and Matlab Abstracts for the XV International Symposium on Computer Simulation in Biomechanics 2015
Kyungnam Kim, David J. Huber, Jiejun Xu, Deepak Khosla Efficient Algorithms for Indoor MAV Flight using Vision and Sonar Sensors 11th International Symposium on Visual Computing (ISVC), Las Vegas, Nevada, USA 2015
Jiejun Xu, Kyungnam Kim, Lei Zhang, Deepak Khosla 3D Perception for Autonomous Robot Exploration 11th International Symposium on Visual Computing (ISVC), Las Vegas, Nevada, USA 2015
V. De Sapio, D. Earl, R. Green, K. Saul Human factors simulation using demographically tuned biomechanical models Proceedings of the 2014 International Annual Meeting of the Human Factors and Ergonomics Society, volume 58, pages 944– 948 2014
S. Goldfarb, D. Earl, V. De Sapio, M. Mansouri, J. Reinbolt An approach and implementation for coupling neurocognitive and neuromechanical models Proceedings of the 2014 IEEE International Conference on Systems, Man, and Cybernetics, pages 406–413 2014
V. De Sapio An approach for goal-oriented neuromuscular control of digital humans International Journal of Human Factors Modelling and Simulation, 4(2):121– 144 2014

Patents (68)

Authors Title Patent # Date
Qin Jiang, Yuri Owechko, Ken Kyungnam Apparatus, system and method for determining an object’s location in image video data US10549853 02/04/2020
Qin Jiang, Yuri Owechko Apparatus, system and method for enhancing image video data US10176557 01/08/2020
Heiko Hoffmann, Ken Kim Robust recognition on degraded imagery by exploiting known image transformation under motion US10332265 06/25/2019
Yongqiang Gao, Qin Jiang, Yang Chen, Deepak Khosla Method for object detection in digital image and video using spiking neural networks US10198689 02/05/2019
Qin Jiang, Yuri Owechko Apparatus, system and method for enhancing image video data US10176557 01/08/2019
Heiko Hoffmann, Jaehoon Choe, Corey Thibeault Device and method to automatically tune the nerve stimulation pattern of a sensory feedback capable prosthesis US10166394 01/01/2019
V. De Sapio, S. E. Chelian, R. Bhattacharyya, M. E. Phillips, M. D. Ziegler, D. W. Payton In-home patient-focused rehabilitation system US10130311 11/20/2018
Heiko Hoffmann Neural network device with engineered delays for pattern storage and matching US10095976 10/09/2018
Vincent DeSapio, Michael D. Howard, Matthew E. Phillips, Kevin R. Martin, Heiko Hoffmann, David W. Payton System and method for assistive gait intervention and fall prevention US10052062 08/21/2018
Heiko Hoffmann, David W. Payton Adaptive control system capable of recovering from unexpected situations US10001760 06/19/2018
Shuoqin Wang, Qin Jiang A method for on-line state of power (SOP) estimation of a Li-ion battery with high accuracy US9989595 06/05/2018
Hyukseong Kwon, Kyungnam Kim System and method for upsampling of sparse point cloud for 3D registration US9972067 05/15/2018
Heiko Hoffmann, Michael J. Daily System and method for robot supervisory control with an augmented reality user interface US9880553 01/30/2018
Swarup Medasani, Jason Meltzer, Jiejun Xu, Zhichao Chen, Rashmi Sundareswara, David Payton, Ryan Uhlenbrock, Leandro Barajas, Kyungnam Kim Method for object localization and pose estimation for an object of interest US9875427 01/23/2018
Darren J. Earl, Ryan M. Uhlenbrock, Heiko Hoffmann Device and method for merging 3D point clouds from sparsely distributed viewpoints US9858640 01/02/2018
V. De Sapio, D. J. Earl Method and system for tuning a musculoskeletal model US9858391 01/02/2018
David Payton, Kyungnam Kim, Zhichao Chen, Ryan Uhlenbrock, Li Yang Ku Robotic device including machine vision US9844881 12/19/2017
Heiko Hoffmann, David W Payton, Vincent DeSapio Method for tele-robotic operations over time-delayed communication links US9776325 10/03/2017
Hyukseong Kwon, Kyungnam Kim, Yuri Owechko Method and apparatus for tracking targets US9727785 08/08/2017
Hyukseong Kwon, Kyungnam Kim, Yuri Owechko Method and apparatus for detecting targets US9715639 07/25/2017
Heiko Hoffmann, Behnam Salemi Robotic control device and method for manipulating a hand-held tool US9613180 04/04/2017
V. De Sapio, M. D. Howard, R. F. Green Quantifying muscle and tendon fatigue during physical exertion US9610036 04/04/2017
V. De Sapio, N. Srinivasa System for controlling brain machine interfaces and neural prosthetic systems US9566174 02/14/2017
Heiko Hoffmann, David W Payton Self-stabilizing system for multiple interacting controllers US9557722 01/31/2017
Terrell N. Mundhenk, Arturo Flores, Heiko Hoffmann Method for classification and segmentation and forming 3D models from images US9530218 12/27/2016
Qin Jiang, Yuri Owechko Apparatus, System, and Method for Enhancing Image Data US9508134 11/29/2016
Dmitriy Korchev, Yuri Owechko, Hyukseong Kwon Methods and systems for detecting moving objects in a sequence of image frames produced by sensors with inconsistent gain, offset, and dead pixels US9501839 11/22/2016
Darren Earl, Derek Mitchell, Heiko Hoffmann System and method for quick scripting of tasks for autonomous robotic manipulation US9486918 11/08/2016
Darren Earl, Derek Mitchell, Heiko Hoffmann System and method for quick scripting of tasks for autonomous robotic manipulation US9486918 11/08/2016
Hyukseong Kwon, Yuri Owechko, Kyungnam Kim Occlusion-robust visual object fingerprinting using fusion of multiple sub-region signatures US9483839 11/01/2016
Karim El Defrawy, Joshua D. Lampkins Method for secure and resilient distributed generation of elliptic curve digital signature algorithm (ECDSA) based digital signatures with proactive security US9467451 10/11/2016
David Payton, Michael Daily Systems, methods, and apparatus for neuro-robotic goal selection US9445739 09/20/2016
David Payton, Ryan Uhlenbrock, Li Yang Ku Rapid robotic imitation learning of force-torque tasks US9403273 08/02/2016
Derek Mitchell, Heiko Hoffmann Dynamic obstacle avoidance in a robotic system US9403275 08/02/2016
Leandro Barajas, David Payton, Li Yang Ku, Ryan Uhlenbrock, Darren Earl Visual debugging of robotic tasks US9387589 07/12/2016
Heiko Hoffmann, David W. Payton, Derek Mitchell Dynamical system-based robot velocity control US9381643 07/05/2016
Vincent DeSapio, Heiko Hoffmann A system for controlling motion and constraint forces in a robotic system without the need for force sensing US9364951 06/14/2016
Zhichao Chen, Heiko Hoffmann Device and method to localize and control a tool tip with a robot arm US9259840 02/16/2016
Heiko Hoffmann, Hooman Kazemi, Michael J Daily System and Method for Fast Template Matching in 3D US9171247 10/27/2015
Qin Jiang, Michael J. Daily, Richard Michael Kremer Acoustic Ranging System Using Atmospheric Dispersion US9146295 09/29/2015
Michael Daily, Michael Howard, Yang Chen, David Payton, Rashmi Sundareswara Recall system using spiking neuron networks US9020870 04/28/2015
Qin Jiang, Yang Chen System for automatic data clustering utilizing bio-inspired computing models US9009156 04/14/2015
Suhas E Chelian, Rashmi N Sundareswara, Heiko Hoffmann Robotic visual perception system US9002098 04/07/2015
William Noble, Serdar Gokcen, Michael Howard Track prediction and identification via particle motion with intent US9002642 04/07/2015
Leandro Barajas, Eric Martinson, David Payton, Ryan Uhlenbrock Method and system for training a robot using human-assisted task demonstration US8843236 09/23/2014
Qin Jiang Radar pulse detection using a digital radar receiver US8803730 08/12/2014
David Payton, Michael Daily Systems, methods, and apparatus for neuro-robotic tracking point selection US8788030 07/22/2014
Michael Daily, Michael Howard, Yang Chen, Rashmi Sundareswara, David Payton System for representing, storing, and reconstructing an input signal US8756183 06/17/2014
David Payton, Michael Daily Systems, methods, and apparatus for neuro-robotic tracking point selection US8483816 07/09/2013
Qin Jiang Active sonar system and active sonar method using fuzzy logic US8320216 11/27/2012
Qin Jiang, Shubha Kadambe Active sonar system and active sonar method using noise reduction techniques and advanced signal processing techniques US8116169 02/14/2012
Qin Jiang, Shubha Kadambe System and method for enhancing weak target signals for a sensor array US8068385 11/29/2011
David Payton, Michael Daily, Mike Howard Distributed display composed of active fiducials US8035078 10/11/2011
Michael Howard, David Payton System and method for distributed engagement US7912631 03/22/2011
Michael Howard, David Payton, Wendell Bradshaw, Timothy Smith System and method for automated search by distributed elements US7908040 03/15/2011
David Payton Method and system for independently observing and modifying the activity of an actor processor US7877347 01/25/2011
David Payton Event localization within a distributed sensor array US7786885 08/31/2010
David Payton, Scott Smith Iterative particle reduction methods and systems for localization and pattern recognition US7613673 11/03/2009
David Payton, Mike Daily, Mike Howard, Craig Lee Distributed display composed of active fiducials US7612324 11/03/2009
Peter Tinker, David Payton System and method for computing reachable areas US7599814 10/06/2009
David Payton, Eric Martinson Arranging mobile sensors into a predetermined pattern US7379840 05/27/2008
David Payton Method and apparatus for providing directed communications through a networked array of nodes US7158511 01/02/2007
David Payton, Bruce Hoff, Mike Howard, Craig Lee Method and apparatus for signaling among a plurality of agents US7113746 09/26/2006
David Payton, Regina Estkowski Motion prediction within an amorphous sensor array US6885303 04/26/2005
David Payton, Craig Lee, Bruce Hoff, Mike Howard, Mike Daily Method and apparatus for terrain reasoning with distributed embedded processing elements US6580979 06/17/2003
David Payton, Mike Howard, Mike Daily, Craig Lee, Bruce Hoff Method and apparatus for controlling the movement of a plurality of agents US6507771 01/14/2003
David Payton, David Keirsey System and method for rapid determination of visibility-based terrain properties over broad regions US6173067 01/09/2001
David Payton System and method for processing commands from a plurality of control sources US5285380 02/08/1994



June 21, 2019


HRL Laboratories - Malibu, CA

Computer Vision Frontier Workshop

Current Openings for CAC

No current openings at this time.


Email: cemartin[at]

Dr. Charles E. Martin
Research Scientist
Leader, Center for Autonomy Computing

Information and Systems Sciences Lab
HRL Laboratories, LLC
3011 Malibu Canyon Road
Malibu, CA 90265