Cellular neural network

In computer science and machine learning, cellular neural networks (CNN) or cellular nonlinear networks (CNN) are a parallel computing paradigm similar to neural networks, with the difference that communication is allowed between neighbouring units only. Typical applications include image processing, analyzing 3D surfaces, solving partial differential equations, reducing non-visual problems to geometric maps, modelling biological vision and other sensory-motor organs.

CNN is not to be confused with convolutional neural networks (also colloquially called CNN).

CNN architecture

Due to their number and variety of architectures, it is difficult to give a precise definition for a CNN processor. From an architecture standpoint, CNN processors are a system of finite, fixed-number, fixed-location, fixed-topology, locally interconnected, multiple-input, single-output, nonlinear processing units. The nonlinear processing units are often referred to as neurons or cells. Mathematically, each cell can be modeled as a dissipative, nonlinear dynamical system where information is encoded via its initial state, inputs and variables used to define its behavior. Dynamics are usually continuous, as in the case of Continuous-Time CNN (CT-CNN) processors, but can be discrete, as in the case of Discrete-Time CNN (DT-CNN) processors.

Each cell has one output, by which it communicates its state with both other cells and external devices. Output is typically real-valued, but can be complex or even quaternion, i.e. a Multi-Valued CNN (MV-CNN). Most CNN processors, processing units are identical, but there are applications that require non-identical units, which are called Non-Uniform Processor CNN (NUP-CNN) processors, and consist of different types of cells.

Chua-Yang CNN

In the original Chua-Yang CNN (CY-CNN) processor, the state of the cell was a weighted sum of the inputs and the output was a piecewise linear function. However, like the original perceptron-based neural networks, the functions it could perform were limited: specifically, it was incapable of modeling non-linear functions, such as XOR. More complex functions are realizable via Non-Linear CNN (NL-CNN) processors.

Cells are defined in a normed gridded space like two-dimensional Euclidean geometry. However, the cells are not limited to two-dimensional spaces; they can be defined in an arbitrary number of dimensions and can be square, triangle, hexagonal, or any other spatially invariant arrangement. Topologically, cells can be arranged on an infinite plane or on a toroidal space. Cell interconnect is local, meaning that all connections between cells are within a specified radius (with distance measured topologically). Connections can also be time-delayed to allow for processing in the temporal domain.

Most CNN architectures have cells with the same relative interconnects, but there are applications that require a spatially variant topology, i.e. Multiple-Neighborhood-Size CNN (MNS-CNN) processors. Also, Multiple-Layer CNN (ML-CNN) processors, where all cells on the same layer are identical, can be used to extend the capability of CNN processors.

The definition of a system is a collection of independent, interacting entities forming an integrated whole, whose behavior is distinct and qualitatively greater than its entities. Although connections are local, information exchange can happen globally through diffusion. In this sense, CNN processors are systems because their dynamics are derived from the interaction between the processing units and not within processing units. As a result, they exhibit emergent and collective behavior. Mathematically, the relationship between a cell and its neighbors, located within an area of influence, can be defined by a coupling law, and this is what primarily determines the behavior of the processor. When the coupling laws are modeled by fuzzy logic, it is a fuzzy CNN. When these laws are modeled by computational verb logic, it becomes a computational verb CNN. Both fuzzy and verb CNNs are useful for modelling social networks when the local couplings are achieved by linguistic terms.

History

The idea of CNN processors was introduced by Leon Chua and Lin Yang in 1988. In these articles, Chua and Yang outline the underlying mathematics behind CNN processors. They use this mathematical model to demonstrate, for a specific CNN implementation, that if the inputs are static, the processing units will converge, and can be used to perform useful calculations. They then suggest one of the first applications of CNN processors: image processing and pattern recognition (which is still the largest application to date). Leon Chua is still active in CNN research and publishes many of his articles in the International Journal of Bifurcation and Chaos, of which he is an editor. Both IEEE Transactions on Circuits and Systems and the International Journal of Bifurcation also contain a variety of useful articles on CNN processors authored by other knowledgeable researchers. The former tends to focus on new CNN architectures and the latter more on the dynamical aspects of CNN processors.

In 1993, Tamas Roska and Leon Chua introduced the first algorithmically programmable analog CNN processor in the world. The multi-national effort was funded by the Office of Naval Research, the National Science Foundation, and the Hungarian Academy of Sciences, and researched by the Hungarian Academy of Sciences and the University of California. This article proved that CNN processors were producible and provided researchers a physical platform to test their CNN theories. After this article, companies started to invest into larger, more capable processors, based on the same basic architecture as the CNN Universal Processor. Tamas Roska is another key contributor to CNNs. His name is often associated with biologically inspired information processing platforms and algorithms, and he has published numerous key articles and has been involved with companies and research institutions developing CNN technology.

Literature

Two references are considered invaluable since they manage to organize the vast amount of CNN literature into a coherent framework:

• An overview by Valerio Cimagalli and Marco Balsi. The paper provides a concise intro to definitions, CNN types, dynamics, implementations, and applications.
• "Cellular Neural Networks and Visual Computing Foundations and Applications", written by Leon Chua and Tamas Roska, which provides examples and exercises. The book covers many different aspects of CNN processors and can serve as a textbook for a Masters or Ph.D. course.

Other resources include

• The proceedings of "The International Workshop on Cellular Neural Networks and Their Applications" provide much CNN literature.
• The proceedings are available online, via IEEE Xplore, for conferences held in 1990, 1992, 1994, 1996, 1998, 2000, 2002, 2005 and 2006.
• There was also a workshop held in Santiago de Composetela, Spain. Topics included theory, design, applications, algorithms, physical implementations and programming and training methods.
• For an understanding of the analog semiconductor based CNN technology, AnaLogic Computers has their product line, in addition to the published articles available on their homepage and their publication list. They also have information on other CNN technologies such as optical computing. Many of the commonly used functions have already been implemented using CNN processors. A good reference point for some of these can be found in image processing libraries for CNN based visual computers such as Analogic’s CNN-based systems.

Related processing architectures

CNN processors could be thought of as a hybrid between ANN and Continuous Automata (CA).

Artificial Neural Networks

The processing units of CNN and NN are similar. In both cases, the processor units are multi-input, dynamical systems, and the behavior of the overall systems is driven primarily through the weights of the processing unit’s linear interconnect.

However, in CNN processors, connections are made locally, whereas in ANN, connections are global.

For example, neurons in one layer are fully connected to another layer in a feed-forward NN and all the neurons are fully interconnected in Hopfield networks. In ANNs, the weights of interconnections contain information on the processing system’s previous state or feedback.

But in CNN processors, the weights are used to determine the dynamics of the system.

Furthermore, due to the high inter-connectivity of ANNs, they tend not exploit locality in either the data set or the processing and as a result, they usually are highly redundant systems that allow for robust, fault-tolerant behavior without catastrophic errors.

A cross between an ANN and a CNN processor is a Ratio Memory CNN (RMCNN). In RMCNN processors, the cell interconnect is local and topologically invariant, but the weights are used to store previous states and not to control dynamics. The weights of the cells are modified during some learning state creating long-term memory.

Continuous Automata

The topology and dynamics of CNN processors closely resembles that of CA. Like most CNN processors, CA consists of a fixed-number of identical processors that are spatially discrete and topologically uniform. The difference is that most CNN processors are continuous-valued whereas CA have discrete-values. Furthermore, the CNN processor's cell behavior is defined via some non-linear function whereas CA processor cells are defined by some state machine.

However, there are some exceptions. Continuous Valued Cellular Automata are CA with continuous resolution. Depending on how a given Continuous Automata is specified, it can also be a CNN.

There are also Continuous Spatial Automata, which consist of an infinite number of spatially continuous, continuous-valued automata. There is considerable work being performed in this field since continuous spaces are easier to mathematically model than discrete spaces, thus allowing a more quantitative approach as opposed to an empirical approach taken by some researchers of cellular automata. Continuous Spatial Automata processors can be physically realized though an unconventional information processing platform such as a chemical computer. Furthermore, it is conceivable that large CNN processors (in terms of the resolution of the input and output) can be modeled as a Continuous Spatial Automata.

Model of computation

The dynamical behavior of CNN processors can be expressed using differential equations, where each equation represents the state of an individual processing unit. The behavior of the entire CNN processor is defined by its initial conditions, inputs, cell interconnections (topology and weights), and the cells themselves. One possible use of CNN processors is to generate and respond to signals of specific dynamical properties. For example, CNN processors have been used to generate multiscroll chaos (like the Chen attractor), synchronize with chaotic systems, and exhibit multi-level hysteresis. CNN processors are designed specifically to solve local, low-level, processor intensive problems expressed as a function of space and time. For example, CNN processors can be used to implement high-pass and low-pass filters and morphological operators. They can also be used to approximate a wide range of Partial differential equations (PDE) such as heat dissipation and wave propagation.

Reaction-Diffusion

CNN processors can be used as Reaction-Diffusion (RD) processors. RD processors are spatially invariant, topologically invariant, analog, parallel processors characterized by reactions, where two agents can combine to create a third agent, and diffusion, the spreading of agents. RD processors are typically implemented through chemicals in a Petri dish (processor), light (input), and a camera (output) however RD processors can also be implemented through a multi-layer CNN processor. D processors can be used to create Voronoi diagrams and perform skeletonisation. The main difference between the chemical implementation and the CNN implementation is that CNN implementations are considerably faster than their chemical counterparts and chemical processors are spatially continuous whereas the CNN processors are spatially discrete. The most researched RD processor, Belousov-Zhabotinsky (BZ) processors, has already been simulated using a four-layer CNN processors and has been implemented in a semiconductor.

Boolean functions

Like CA, computations can be performed through the generation and propagation of signals that either grow or change over time. Computations can occur within a signal or can occur through the interaction between signals. One type of processing, which uses signals and is gaining momentum is wave processing, which involves the generation, expanding, and eventual collision of waves. Wave processing can be used to measure distances and find optimal paths. Computations can also occur through particles, gliders, solutions, and filterons localized structures that maintain their shape and velocity. Given how these structures interact/collide with each other and with static signals, they can be used to store information as states and implement different Boolean functions. Computations can also occur between complex, potentially growing or evolving localized behavior through worms, ladders, and pixel-snakes. In addition to storing states and performing Boolean functions, these structures can interact, create, and destroy static structures.

The applications of CNNs to Boolean functions is discussed in the paper by Fangyue Chen, Guolong He, Xiubin Xu, and Guanrong Chen, "Implementation of Arbitrary Boolean Functions via CNN".

Automata and Turing machines

Although CNN processors are primarily intended for analog calculations, certain types of CNN processors can implement any Boolean function, allowing simulating CA. Since some CA are Universal Turing machines (UTM), capable of simulating any algorithm can be performed on processors based on the von Neumann architecture, that makes this type of CNN processors, universal CNN, a UTM. One CNN architecture consists of an additional layer. CNN processors have resulted in the simplest realization of Conway’s Game of Life and Wolfram’s Rule 110, the simplest known universal Turing Machine. This unique, dynamical representation of an old systems, allows researchers to apply techniques and hardware developed for CNN to better understand important CA. Furthermore, the continuous state space of CNN processors, with slight modifications that have no equivalent in Cellular Automata, creates emergent behavior never seen before.

Any information processing platform that allows the construction of arbitrary Boolean functions is called universal, and as result, this class CNN processors are commonly referred to as universal CNN processors. The original CNN processors can only perform linearly separable Boolean functions. By translating functions from digital logic or look-up table domains into the CNN domain, some functions can be considerably simplified. For example, the nine-bit, odd parity generation logic, which is typically implemented by eight nested exclusive-or gates, can also be represented by a sum function and four nested absolute value functions. Not only is there a reduction in the function complexity, but the CNN implementation parameters can be represented in the continuous, real-number domain.

There are two methods by which to select a CNN processor along with a template or weights. The first is by synthesis, which involves determine the coefficients offline. This can be done by leveraging previous work, i.e. libraries, papers, and articles, or by mathematically deriving co that best suits the problem. The other is through training the processor. Researchers have used back-propagation and genetic algorithms to learn and perform functions. Back-propagation algorithms tend to be faster, but genetic algorithms are useful because they provide a mechanism to find a solution in a discontinuous, noisy search space.

Physical implementations

There are toy models simulating CNN processors using billiard balls, but these are used for theoretical studies. In practice, CNN are physically implemented on hardware and current technologies such as semiconductors. There are plans to migrate CNN processors to emerging technologies in the future.

Semiconductors

Semiconductor-based CNN processors can be segmented into analog CNN processors, digital CNN processors, and CNN processors emulated using digital processors. Analog CNN processors were the first to be developed. Analog computers were fairly common during the 1950 and 1960s, but they gradually were replaced by digital computers the 1970s. Analog processors were considerably faster in certain applications such as optimizing differential equations and modeling nonlinearities, but the reason why analog computing lost favor was the lack of precision and the difficulty to configure an analog computer to solve a complex equation.

Analog CNN processors share some of the same advantages as their predecessors, specifically speed. The first analog CNN processors were able to perform real-time ultra-high frame-rate (>10,000 frame/s) processing unachievable by digital processors. The analog implementation of CNN processors requires less area and consumes less power than their digital counterparts. Although the accuracy of analog CNN processors does not compare to their digital counterparts, for many applications, noise and process variances are small enough not to perceptually affect the image quality.

The first algorithmically programmable, analog CNN processor was created in 1993. It was named the CNN Universal Processor because its internal controller allowed multiple templates to be performed on the same data set, thus simulating multiple layers and allowing for universal computation. Included in the design was a single layer 8x8 CCN, interfaces, analog memory, switching logic, and software. The processor was developed in order to determine CNN processor producibility and utility. The CNN concept proved promising and by 2000, there were at least six organizations designing algorithmically programmable, analog CNN processors.

AnaFocus, AnaLogic

In the 2000s, AnaFocus, a mixed-signal semiconductor company from the University of Seville, introduced their ACE prototype CNN processor product line. Their first ACE processor contained 20x20 B/W processor units; and subsequent processors provided 48x48 and 128x128 grayscale processor units, improving the speed and processing elements. AnaFocus also had a multilayer CASE prototype CNN processors line. Their processors allowed real-time interaction between the sensing and processing. In 2014, AnaFocus had been sold to e2v technologies.

Another company, AnaLogic Computers was founded in 2000 by many of the same researchers behind the first algorithmically programmable CNN Universal Processor. In 2003, AnaLogic Computers developed a PCI-X visual processor board that included the ACE 4K processor, with a Texas Instrument DIP module and a high-speed frame-grabber. This allowed CNN processing to be easily included in a desktop computer. In 2006, AnaLogic Computers developed their Bi-I Ultra High Speed Smart Camera product line, which includes the ACE 4K processor in their high-end models.

In 2006, Roska et al. published a paper on designing a Bionic Eyeglass for AnaLogic. The Bionic Eyeglass is a dual-camera, wearable platform, based on the Bi-I Ultra High Speed Smart Camera, designed to provide assistance to blind people. Some of its functions include route number recognition and color processing.

Analog CNN processors

Some researchers developed their own custom analog CNN processors. For example:

• A research team from University degli Studi di Catania made one in order to generate gaits for a hexapod robot.
• Chung-Yu Wu and Chiu-Hung Cheng from National Chiao Tung University designed a RM-CNN processor to learn more about pattern learning and recognition.
• Researchers from the National Lien-Ho Institute of Technology (W. Yen, R. Chen and J. Lai) developed a Min-Max CNN (MMCNN) processor to learn more about CNN dynamics.

Despite their speed and low power consumption, there are some significant drawbacks to analog CNN processors. First, analog CNN processors can potentially create erroneous results due to environment and process variation. In most applications, these errors are not noticeable, but there are situations where minor deviations can result in catastrophic system failures. For example, in chaotic communication, process variation will change the trajectory of a given system in phase space, resulting in a loss of synchronicity/stability. Due to the severity of the problem, there is considerable research being performed to ameliorate the problem. Some researchers are optimizing templates to accommodate greater variation. Other researchers are improving the semiconductor process to more closely match theoretical CNN performance. Other researchers are investigating different, potentially more robust CNN architectures. Lastly, researchers are developing methods to tune templates to target a specific chip and operating conditions. In other words, the templates are being optimized to match the information processing platform. Not only does process variation limit what can be done with current analog CNN processors, it is also a barrier for creating more complex processing units. Unless this process variation is resolved, ideas such as nested processing units, non-linear inputs, etc. cannot be implemented in a real-time analog CNN processor. Also, the semiconductor "real estate" for processing units limits the size of CNN processors.

Currently the largest AnaVision CNN-based vision processor consists of a 4K detector, which is significantly less than the megapixel detectors found in affordable, consumer cameras. Unfortunately, feature size reductions, as predicted by Moore’s Law, will only result in minor improvements. For this reason, alternate technologies such as Resonant Tunneling Diodes and Neuron-Bipolar Junction Transistors are being explored. Also, CNN processor architecture is being re-evaluated. For example, Star-CNN processors, where one analog multiplier is time-shared between multiple processor units, have been proposed and are expected to result in processor unit reduction size of 80%.

Digital CNN processors, FPGA

Although not nearly as fast and energy efficient, digital CNN processors do not share the problems of process variation and feature size of their analog counterparts. This allows digital CNN processors to include nested processor units, non-linearities, etc. In addition, digital CNN are more flexible, cost less and are easier to integrate. The most common implementation of digital CNN processors uses an FPGA. Eutecus, founded in 2002 and operating in Berkeley, provides intellectual property that can be synthesized into an Altera FPGA. Their digital 320x280, FPGA-based CNN processors run at 30 frame/s and there are plans to make a fast digital ASIC. Eustecus is a strategic partner of AnaLogic computers, and their FPGA designs can be found in several of AnaLogic’s products. Eutecus is also developing software libraries to perform tasks including but not limited to video analytics for the video security market, feature classification, multi-target tracking, signal and image processing and flow processing. Many of these routines are derived using CNN-like processing. For those wanting to perform CNN simulations for prototyping, low-speed applications, or research, there are several options. First, there are precise CNN emulation software packages like SCNN 2000. If the speed is prohibitive, there are mathematical techniques, such as Jacobi’s Iterative Method or Forward-Backward Recursions that can be used to derive the steady state solution of a CNN processor. Lastly, digital CNN processors can be emulated on highly parallel, application-specific processors, such as graphics processors. Implementing neural networks using graphics processors is an area of further research.

Holography, nanotechnology

Researchers are also perusing alternate technologies for CNN processors. Although current CNN processors circumvent some of the problems associated with their digital counterparts, they do share some of the same long-term problems common to all semiconductor-based processors. These include, but are not limited to, speed, reliability, power-consumption, etc. AnaLogic Computers, is developing optical CNN processors, which combine optics, lasers, and biological and holographic memories. What initially was technology exploration resulted in a 500x500 CNN processor able to perform 300 giga-operations per second.

Another promising technology for CNN processors is nanotechnology. One nanotechnology concept being investigated is using single electron tunneling junctions, which can be made into single-electron or high-current transistors, to create McCulloch-Pitts CNN processing units. In summary, CNN processors have been implemented and provide value to their users. They have been able to effectively leverage the advantages and address some of the disadvantages associated with their underling technology, i.e. semiconductors. Researchers are also transitioning CNN processors into emerging technologies. Therefore, if the CNN architecture is suited for a specific information processing system, there are processors available for purchase (as there will be for the foreseeable future).

Applications

CNN researchers have diverse interests, ranging from physical, engineering, theoretical, mathematical, computational, and philosophical applications.

Image processing

CNN processors were designed to perform image processing; specifically, real-time ultra-high frame-rate (>10,000 frame/s) processing for applications like particle detection in jet engine fluids and spark-plug detection. Currently, CNN processors can achieve up to 50,000 frames per second, and for certain applications such as missile tracking, flash detection, and spark-plug diagnostics these microprocessors have outperformed a conventional supercomputer. CNN processors lend themselves to local, low-level, processor intensive operations and have been used in feature extraction, level and gain adjustments, color constancy detection, contrast enhancement, deconvolution, image compression, motion estimation, image encoding, image decoding, image segmentation, orientation preference maps, pattern learning/recognition, multi-target tracking, image stabilization, resolution enhancement, image deformations and mapping, image inpainting, optical flow, contouring, moving object detection, axis of symmetry detection, and image fusion.

Due to their processing capabilities and flexibility, CNN processors have been used and prototyped for novel field applications such as flame analysis for monitoring combustion at a waste incinerator, mine-detection using infrared imagery, calorimeter cluster peak for high energy physics, anomaly detection in potential field maps for geophysics, laser dot detection, metal inspection for detecting manufacturing defects, and seismic horizon picking. They have also been used to perform biometric functions such as fingerprint recognition, vein feature extraction, face tracking, and generating visual stimuli via emergent patterns to gauge perceptual resonances.

Biology and medicine

CNN processors have been used for medical and biological research in performing automated nucleated cell counting for detecting hyperplasia, segment images into anatomically and pathologically meaningful regions, measure and quantify cardiac function, measure the timing of neurons, and detect brain abnormalities that would lead to seizures.

One potential future application of CNN microprocessors is to combine them with DNA microarrays to allow for a near-real time DNA analysis of hundreds of thousands of different DNA sequences. Currently, the major bottleneck of DNA microarray analysis is the amount of time needed to process data in the form of images, and using a CNN microprocessor, researchers have reduced the amount of time needed to perform this calculation to 7ms.

Texture recognition

CNN processors have also been used to generate and analyze patterns and textures. One motivation was to use CNN processors to understand pattern generation in natural systems. They were used to generate Turing patterns in order to understand the situations in which they form, the different types of patterns which can emerge, and the presence of defects or asymmetries. Also, CNN processors were used to approximate pattern generation systems that create stationary fronts, spatio-temporal patterns oscillating in time, hysteresis, memory, and heterogeneity. Furthermore, pattern generation was used to aid high-performance image generation and compression via real-time generation of stochastic and coarse-grained biological patterns, texture boundary detection, and pattern and texture recognition and classification.

Control and Actuator Systems

There is an ongoing effort to incorporate CNN processors into sensory-computing-actuating machines as part of the emerging field of Cellular Machines. The basic premise is to create an integrated system that uses CNN processors for the sensory signal-processing and potentially the decision-making and control. The reason is that CNN processors can provide a low power, small size, and eventually low-cost computing and actuating system suited for Cellular Machines. These Cellular Machines will eventually create a Sensor-Actuator Network (SAN), a type of Mobile Ad Hoc Networks (MANET) which can be used for military intelligence gathering, surveillance of inhospitable environments, maintenance of large areas, planetary exploration, etc.

CNN processors have been proven versatile enough for some control functions. They have been used to optimize function via a genetic algorithm, to measure distances, to perform optimal path-finding in a complex, dynamic environment, and theoretically can be used to learn and associate complex stimuli. They have also been used to create antonymous gaits and low-level motors for robotic nematodes, spiders, and lamprey gaits using a Central Pattern Generator (CPG). They were able to function using only feedback from the environment, allowing for a robust, flexible, biologically inspired robot motor system. CNN-based systems were able to operate in different environments and still function if some of the processing units are disabled.

Communication systems

The variety of dynamical behavior seen in CNN processors make them intriguing for communication systems. Chaotic communications using CNN processors is being researched due to their potential low power consumption, robustness and spread spectrum features. The premise behind chaotic communication is to use a chaotic signal for the carrier wave and to use chaotic phase synchronization to reconstruct the original message. CNN processors can be used on both the transmitter and receiver end to encode and decode a given message. They can also be used for data encryption and decryption, source authentication through watermarking, detecting of complex patterns in spectrogram images (sound processing), and transient spectral signals detection.

CNN processors are neuromorphic processors, meaning that they emulate certain aspects of biological neural networks. The original CNN processors were based on mammalian retinas, which consist of a layer of photodetectors connected to several layers of locally coupled neurons. This makes CNN processors part of an interdisciplinary research area whose goal is to design systems that leverage knowledge and ideas from neuroscience and contribute back via real-world validation of theories. CNN processors have implemented a real-time system that replicates mammalian retinas, validating that the original CNN architecture chosen modeled the correct aspects of the biological neural networks used to perform the task in mammalian life. However, CNN processors are not limited to verifying biological neural networks associated with vision processing; they have been used to simulate dynamic activity seen in mammalian neural networks found in the olfactory bulb and locust antennal lobe, responsible for pre-processing sensory information to detect differences in repeating patterns.

CNN processors are being used to understand systems that can be modeled using simple, coupled units, such as living cells, biological networks, physiological systems, and ecosystems. The CNN architecture captures some of the dynamics often seen in nature and is simple enough to analyze and conduct experiments. They are also being used for stochastic simulation techniques, which allow scientists to explore spin problems, population dynamics, lattice-based gas models, percolation, and other phenomena. Other simulation applications include heat transfer, mechanical vibrating systems, protein production, Josephson Junction problems, seismic wave propagation, and geothermal structures. Instances of 3d CNN have been used to prove certain emergent phenomena in complex systems, establishing a link between art, dynamical systems and VLSI technology.

CNN processors have been used to research a variety of mathematical concepts, such as non-equilibrium systems, constructing non-linear systems of arbitrary complexity, emergent chaotic dynamics, and discovering new dynamic behavior. They are often used in researching systemics, a trans-disciplinary, scientific field that studies natural systems. The goal of systemics researchers is to develop a conceptual and mathematical framework necessary to analyze, model, and understand systems, including, but not limited to, atomic, mechanical, molecular, chemical, biological, ecological, social and economic systems. Topics explored are emergence, collective behavior, local activity and its impact on global behavior, and quantifying the complexity of an approximately spatial and topologically invariant system. With another definition of complexity (MIT professor Seth Lloyd has identified 32 different definitions of complexity), it can potentially be mathematically advantageous when analyzing systems such as economic and social systems.

References

• The Chua Lectures: A 12-Part Series with Hewlett Packard Labs [1]
• D. Balya, G, Tímar, G. Cserey, and T. Roska, "A New Computational Model for CNN-UMs and its Computational Complexity", Int’l Workshop on Cellular Neural Networks and Their Applications, 2004.
• L. Chua and L. Yang, "Cellular Neural Networks: Theory," IEEE Trans. on Circuits and Systems, 35(10):1257-1272, 1988. [2]
• L. Chua and L. Yang, "Cellular Neural Networks: Applications" IEEE Trans. on Circuits and Systems, 35(10):1273:1290, 1988.
• L. Chua, T. Roska, Cellular Neural Networks and Visual Computing: Foundations and Applications, 2005.
• V. Cimagalli, M. Balsi, "Cellular Neural Networks: A Review", Neural Nets WIRN Vietri, 1993.
• S. Majorana and L. Chua, "A Unified Framework for Multilayer High Order CNN", Int’l Journal of Circuit Theory and Applications, 26:567-592, 1998.
• T. Roska, L. Chua, "The CNN Universal Machine: An Analogic Array Computer", IEEE Trans. on Circuits and Systems-II, 40(3): 163-172, 1993.
• T. Roska and L. Chua, "Cellular Neural Networks with Non-Linear and Delay-Type Template Elements and Non-Uniform Grids", Int’l Journal of Circuit Theory and Applications, 20:469-481, 1992.
• K. Yokosawa, Y. Tanji and M. Tanaka, "CNN with Multi-Level Hysteresis Quantization Output" Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• T. Nakaguchi, K. Omiya and M. Tanaka, "Hysteresis Cellular Neural Networks for Solving Combinatorial Optimization Problems", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• K. Crounse, C. Wee and L. Chua, "Linear Spatial Filter Design for Implementation on the CNN Universal Machine", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
• J. Poikonen1 and A. Paasio, "Mismatch-Tolerant Asynchronous Grayscale Morphological Reconstruction", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
• M. Gilli, T. Roska, L. Chua, and P. Civalleri, "CNN Dynamics Represents a Broader Range Class than PDEs", Int’l Journal of Bifurcations and Chaos, 12(10):2051-2068, 2002.
• Fangyue Chen, Guolong He, Xiubin Xu, and Guanrong Chen, "Implementation of Arbitrary Boolean Functions via CNN", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• R. Doguru and L. Chua, "CNN Genes for One-Dimensional Cellular Automata: A Multi-Nested Piecewise-Linear Approach", Int’l Journal of Bifurcation and Chaos, 8(10):1987-2001, 1998.
• R. Dogaru and L. Chua, "Universal CNN Cells", Int’l Journal of Bifurcations and Chaos, 9(1):1-48, 1999.
• R. Dogaru and L. O. Chua, "Emergence of Unicellular Organisms from a Simple Generalized Cellular Automata", Int’l Journal of Bifurcations and Chaos, 9(6):1219-1236, 1999.
• T. Yang, L. Chua, "Implementing Back-Propagation-Through-Time Learning Algorithm Using Cellular Neural Networks", Int’l Journal of Bifurcations and Chaos, 9(6):1041-1074, 1999.
• J. Nossek, G. Seiler, T. Roska, and L. Chua, "Cellular Neural Networks: Theory and Circuit Design," Int’l Journal of Circuit Theory and Applications, 20: 533-553, 1998.
• A. Zarandry, S. Espejo, P. Foldesy, L. Kek, G. Linan, C. Rekeczky, A. Rodriguez-Vazquez, T. Roska, I. Szatmari, T. Sziranyi and P. Szolgay, "CNN Technology in Action ", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
• L. Chua, L. Yang, and K. R. Krieg, "Signal Processing Using Cellular Neural Networks", Journal of VLSI Signal Processing, 3:25-51, 1991.
• T. Roska, L. Chua, "The CNN Universal Machine: An Analogic Array Computer", IEEE Trans. on Circuits and Systems-II, 40(3): 163-172, 1993.
• T. Roska and A. Rodriguez-Vazquez, "Review of CMOS Implementations of the CNN Universal Machine-Type Visual Microprocessors", International Symposium on Circuits and Systems, 2000
• T. Roska, "Cellular Wave Computers and CNN Technology – a SoC architecture with xK Processors and Sensor Arrays", Int’l Conference on Computer Aided Design Accepted Paper, 2005.
• K. Karahaliloglu, P. Gans, N. Schemm, and S. Balkir, "Optical sensor integrated CNN for Real-time Computational Applications", IEEE Int’l Symposium on Circuits and Systems, pp. 21–24, 2006.
• C. Wu and C. Cheng, "The Design of Cellular Neural Network with Ratio Memory for Pattern Learning and Recognition", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
• W. Yen, R. Chen and J. Lai, "Design of Min/Max Cellular Neural Networks in CMOS Technology", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
• Z. Gallias and M. Ogorzalek, "Influence in System Nonuniformity on Dynamic Phenomenon in Arrays of Coupled Nonlinear Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002
• A. Paasio and J. Poikonen, "Programmable Digital Nested CNN", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• M. Hanggi, R. Dogaru, and L. Chua, "Physical Modeling of RTD-Based CNN Cells", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
• Z. Voroshazi, Z. Nagy, A. Kiss and P. Szolgay, "An Embedded CNN-UM Global Analogic Programming Unit Implementation on FPGA", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• A. Loncar, R. Kunz and R. Tetzaff, "SCNN 2000 - Part I: Basic Structures and Features of the Simulation System for Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
• V. Tavsanoglu, "Jacobi’s Iterative Method for Solving Linear Equations and the Simulation of Linear CNN", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• B. Shi, "Estimating the Steady State using Forward and Backward Recursions", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• S. Tokes, L. Orzo, and A. Ayoub, "Programmable OASLM as a Novel Sensing Cellular Computer", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• J. Flak, M. Laiho1, and K Halonen, "Programmable CNN Cell Based on SET Transistors", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• A. Zarandry, S. Espejo, P. Foldesy, L. Kek, G. Linan, C. Rekeczky, A. Rodriguez-Vazquez, T. Roska, I. Szatmari, T. Sziranyi and P. Szolgay, "CNN Technology in Action ", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
• L. Chua, S. Yoon and R. Dogaru, "A Nonlinear Dynamics Perspective of Wolfram’s New Kind of Science. Part I: Threshold of Complexity," Int’l Journal of Bifurcation and Chaos, 12(12):2655-2766, 2002.
• P. Ecimovic and J. Wu, "Delay Driven Contrast Enhancement using a Cellular Neural Network with State Dependent Delay", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
• H. Aomori, T. Otaket, N. Takahashi, and M. Tanaka, "A Spatial Domain Sigma Delta Modulator Using Discrete Time Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• H. Kim, H. Son. J. Lee, I. Kim and I. Kim, "An Analog Viterbi Decoder for PRML using Analog Parallel Processing Circuits of the CNN", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• S. Chen, M. Kuo and J. Wang, "Image Segmentation Based on Consensus Voting", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
• Y. Cheng, J. Chung, C. Lin and S. Hsu, "Local Motion Estimation Based On Cellular Neural Network Technology for Image Stabilization Processing", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
• P. Korbel and K. Slot, "Modeling of Elastic Inter-node Bounds in Cellular Neural Network-based Implementation of the Deformable Grid Paradigm", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• C. Amenta, P. Arena, S. Baglio, L. Fortuna, D. Richiura, M.Xibilia and L. Vu, "SC-CNNs for Sensors Data Fusion and Control in Space Distributed Structures", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
• T. Su, Y. Du, Y. Cheng, and Y. Su, "A Fingerprint Recognition System Using Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• M. Bucolo, L. Fortuna, M. Frasca, M. La Rosa, D. Shannahoff-Khalsa, "A CNN Based System to Blind Sources Separation of MEG Signals", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
• F. Dohler, A. Chernihovskyi, F. Mormann, C. Elger, and K. Lehnertz, "Detecting Structural Alterations in the Brain using a Cellular Neural Network based Classification of Magnetic Resonance Images", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• L. Fortuna, P. Arena, D. Balya, and A. Zarandy, "Cellular Neural Networks: A Paradigm for Nonlinear Spatio-Temporal Processing", IEEE Circuits and Systems Magazine, 1(4): 6-21, 2001.
• L. Goras, L. Chua, and D. Leenearts, "Turing Patterns in CNNs – Part I: Once Over Lightly", IEEE Trans. on Circuits and Systems – I, 42(10):602-611, 1995.
• L. Goras, L. Chua, and D. Leenearts, "Turing Patterns in CNNs – Part II: Equations and Behavior", IEEE Trans. on Circuits and Systems – I, 42(10):612-626, 1995.
• L. Goras, L. Chua, and D. Leenearts, "Turing Patterns in CNNs – Part III: Computer Simulation Results", IEEE Trans. on Circuits and Systems – I, 42(10):627-637, 1995.
• L. Kornatowski, K. Slot, P. Dȩbiec, and H. Kim, "Generation of Patterns with Predefined Statistical Properties using Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• C. Lin and S. Chen, "Biological Visual Processing for Hybrid-Order Texture Boundary Detection with CNN-UM", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• G. Costantini, D. Casali, and M. Carota, "A Pattern Classification Method Based on a Space-Variant CNN Template", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• T. Roska and L. O. Chua, "The CNN Universal Machine: 10 Years Later, Journal of Circuits, Systems, and Computers", Int’l Journal of Bifurcation and Chaos, 12(4):377-388, 2003.
• R. Bise, N. Takahashi and T. Nishi, "On the Design Method of Cellular Neural Networks for Associate Memories Based on Generalized Eigenvalue Problem", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
• I. Szatmhri, "The Implementation of a Nonlinear Wave Metric for Image Analysis and Classification on the 64x64 I/O CNN-UM Chip", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
• P. Arena, L. Fortuna, M. Frasca, and L. Patane, "CNN Based Central Pattern Generators with Sensory Feedback", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
• R. Caponetto, L. Fortuna, L. Occhipiniti, and M. G. Xibilia, "SC-CNN Chaotic Signals Generation", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
• R. Chen and J. Lai, "Data Encryption Using Non-uniform 2-D Von Neumann Cellular Automata", Int’l Workshop on Cellular Neural Networks and Their Applications, 2005.
• A. Chernihovskyi, C. Elger, and K. Lehnertz, "Effect of in Inhibitory Diffusive Coupling on Frequency-Selectivity of Excitable Media Simulated With Cellular Neural Networks", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• P. Arena, M. Bedia, L. Fortuna, D. Lombardo, L. Patane, and M. Velardet, "Spatio-temporal Patterns in CNNs for Classification: the Winnerless Competition Principle", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• V. Perez-Munuzuri, A. P. Munuzuri, M. Gomez-Gesteria, V. Perez-Villar, L. Pivka, and L. Chua, "Nonlinear Waves, Patters, and Spatio-Temporal Chaos in Cellular Neural Networks," Phil. Trans. R. Soc. Lond. A, (353): 101-113, 1995.
• M. Ercsey-Ravasz, T. Roska and Z. Neda, "Random Number Generator and Monte Carlo type Simulations on the CMM-UM", Int’l Workshop on Cellular Neural Networks and Their Applications, 2006.
• P. Szolgay, T. Hidvegi, Z. Szolgay and P. Kozma, "A Comparison of the Different CNN Implementations in Solving the Problem of Spatiotemporal Dynamics in Mechanical Systems ", Int’l Workshop on Cellular Neural Networks and Their Applications, 2000.
• R. Brown and L. Chua, "Chaos or Turbulence", Int’l Journal of Bifurcation and Chaos, 2(4):1005-1009, 1992.
• M. Gilli, F. Corinto, and P. Checco, "Periodic Oscillations and Bifurcations in Cellular Nonlinear Networks", IEEE Trans. on Circuits and Systems – I, 51(5):948-962, 2004.
• K. A. Richardson, "Systems Theory and Complexity: Part 1", Emergence: Complexity and Organization, 6(3):75-79.
• K. A. Richardson, "Systems Theory and Complexity: Part 2", Emergence: Complexity and Organization, 6(4):77-82.
• K. A. Richardson, "Systems Theory and Complexity: Part 3", Emergence: Complexity and Organization, 7(2):104-114.
• P. Anderson, "Emergence", Proceedings of the Second Int’l Conference on Complex Systems, 2004.
• K. Mainzer, "CNN and the Evolution of Complex Information Systems in Nature and Technology", Int’l Workshop on Cellular Neural Networks and Their Applications, 2002.
• L. Chua, "Local Activity is the Origin of Complexity", Int’l Journal of Bifurcation and Chaos, 15(11):3435-2456, 2005.
• P. Arena, M. Bucolo, S. Fazzino, L. Fortuna, M. Frasca, ‘The CNN Paradigm: Shapes and Complexity’, International Journal of Bifurcations and Chaos (2005), Vol. 15, No. 7, pp. 2063–2090 – World Scientific.

External links

• Eutecus Homepage