It is important to miniaturize robot systems while maintaining advantages such as high responsiveness and functionality for human-machine interactions and for achieving integration with other robotic systems such as drones. In this research, we focused on the miniaturization of a high-speed visual feedback system, and developed a “portable saccade mirror,” which is a system that can realize active target tracking using 1000 Hz image capturing, processing, and feedback actuation with only 3 ms latency in a handheld device. By using a three-dimensionally-stacked vision chip, the proposed system achieved high speed, low latency, low power consumption and compact size, and therefore, can be considered as a good example of a miniaturized high-speed visual feedback system. In this study, we evaluated the performance of the proposed system in comparison with the conventional optical gaze controller, and demonstrated some applications, such as tracking field scope and panorama target scanning.

1. [1] Y. Yamakawa, K. Kuno, and M. Ishikawa, “Human-Robot Cooperative Task Realization Using High-speed Robot Hand System,” Proc. of IEEE Int. Conf. on Advanced Intelligent Mechatronics (AIM), pp. 281-286, 2015.
2. [2] W. Tooyama, S. Huang, K. Murakami, Y. Yamakawa, and M. Ishikawa, “Development of an Assistive System for Position Control of a Human Hand with High Speed and High Accuracy,” Proc. of IEEE Int. Conf. on Humanoid Robots, pp. 230-235, 2016.
3. [3] K. Murakami, S. Huang, H. Sumi, M. Ishikawa, and Y. Yamakawa, “Towel-Like Object Alignment with Human-Robot Cooperation and High-Speed Robotic Manipulation,” Proc. of IEEE Int. Conf. on Robotics and Biomimetics, pp. 772-777, 2018.
4. [4] M. Ikura, L. Miyashita, and M. Ishikawa, “Real-time Landing Gear Control System Based on Adaptive 3D Sensing for Safe Landing of UAV,” Proc. of Int. Symposium on System Integration (SII), pp. 759-764, 2020.
5. [5] L. Miyashita, T. Yamazaki, K. Uehara, Y. Watanabe, and M. Ishikawa, “Portable Lumipen: Dynamic SAR in Your Hand,” Proc. Int. Conf. on Multimedia and Expo (ICME), pp. 1-6, 2018.
6. [6] L. Miyashita and M. Ishikawa, “Wearable DPM System with Intelligent Imager and GPU,” Proc. of Int. Conf. on Artificial Intelligence Circuits and Systems (AICAS), Live Demos, pp. 129-130, 2020.
7. [7] J. Park, D. H. Kim, Y. S. Shin, and S. Lee, “A comparison of convolutional object detectors for real-time drone tracking using a PTZ camera,” Proc. 17th Int. Conf. on Control, Automation and Systems (ICCAS), pp. 696-699, 2017.
8. [8] D. He, H.-M. Chuang, J. Chen, J. Li, and A. Namiki, “Real-Time Visual Feedback Control of Multi-Camera UAV,” J. Robot. Mechatron., Vol.33, No.2, pp. 263-273, 2021.
9. [9] M. Ishikawa, K. Ogawa, T. Komuro, and I. Ishii, “A CMOS Vision Chip with SIMD Processing Element Array for 1ms Image Processing,” Proc. of IEEE Int. Solid-State Circuits Conf. (ISSCC), pp. 206-207, 1999.
10. [10] T. Komuro, I. Ishii, M. Ishikawa, and A. Yoshida, “A Digital Vision Chip Specialized for High-speed Target Tracking,” IEEE Trans. on Electron Devices, Vol.50, No.1, pp. 191-199, 2003.
11. [11] T. Komuro, S. Kagami, and M. Ishikawa, “A Dynamically Reconfigurable SIMD Processor for a Vision Chip,” IEEE J. of Solid-State Circuits, Vol.39, No.1, pp. 265-268, 2004.
12. [12] T. Komuro, A. Iwashita, and M. Ishikawa, “A QVGA-size Pixel-parallel Image Processor for 1,000-fps Vision,” IEEE Micro, Vol.29, No.6, pp. 58-67, 2009.
13. [13] K. Okumura, H. Oku, and M. Ishikawa, “High-speed gaze controller for millisecond-order pan/tilt camera,” Proc. of IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 6186-6191, 2011.
14. [14] K. Okumura, K. Yokoyama, H. Oku, and M. Ishikawa, “1 ms Auto Pan-Tilt -video shooting technology for objects in motion based on Saccade mirror with background subtraction,” Advanced Robotics, Vol.29, Issue 7, pp. 457-468, 2015.
15. [15] K. Iida and H. Oku, “Saccade Mirror 3: high-speed gaze controller with ultra wide gaze control range using triple rotational mirrors,” Proc. of IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 624-629, 2016.
16. [16] D. Wood and M. Bishop, “A Novel Approach to 3D Laser Scanning,” Proc. of Australasian Conf. on Robotics and Automation (ACRA), 2012.
17. [17] H. Kawakami, S. Igarashi, Y. Sasada, and J. Akita, “Column-Parallel Architecture for Line-of-Sight Detection Image Sensor Based on Centroid Calculation,” Proc. of Int. Image Sensor Workshop, pp. 149-152, 2013.
18. [18] C. Shi, J. Yang, Y. Han, Z. Cao, Q. Qin, L. Liu, N.-J. Wu, and Z. Wang, “A 1000 fps Vision Chip Based on a Dynamically Reconfigurable Hybrid Architecture Comprising a PE Array Processor and Self-Organizing Map Neural Network,” IEEE J. of Solid-State Circuits, Vol.49, No.9, pp. 2067-2082, 2014.
19. [19] T. Yamazaki, H. Katayama, S. Uehara, A. Nose, M. Kobayashi, S. Shida, M. Odahara, K. Takamiya, Y. Hisamatsu, S. Matsumoto, L. Miyashita, Y. Watanabe, T. Izawa, Y. Muramatsu, and M. Ishikawa, “A 1ms High-Speed Vision Chip with 3D-Stacked 140GOPS Column-Parallel PEs for Spatio-Temporal Image Processing,” Proc. IEEE Int. Solid-State Circuits Conf. (ISSCC), pp. 82-83, 2017.
20. [20] A. Nose, T. Yamazaki, H. Katayama, S. Uehara, M. Kobayashi, S. Shida, M. Odahara, K. Takamiya, S. Matsumoto, L. Miyashita, Y. Watanabe, T. Izawa, Y. Muramatsu, Y. Nitta, and M. Ishikawa, “Design and Performance of a 1 ms High-Speed Vision Chip with 3D-Stacked 140 GOPS Column-Parallel PEs,” Sensors, Vol.18, No.5, Article No.1313, 2018.
21. [21] I. Ishii and M. Ishikawa, “Self Windowing for High-Speed Vision,” Proc. of IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 51-58, 1999.