BELBIC

In recent years, the use of biologically inspired methods such as the evolutionary algorithm have been increasingly employed to solve and analyze complex computational problems. BELBIC (Brain Emotional Learning Based Intelligent Controller) is one such controller which is proposed by Caro Lucas, Danial Shahmirzadi and Nima Sheikholeslami and adopts the network model developed by Moren and Balkenius to mimic those parts of the brain which are known to produce emotion (namely, the amygdala, orbitofrontal cortex, thalamus and sensory input cortex).[1]

Emotions and learning

Traditionally, the study of learning in biological systems was conducted at the expense of overlooking its lesser known counterparts: motivation and emotion. However these phenomena can not be separated.[2] Motivation is the drive that causes any system to do anything – without it, there is no reason to act. Emotions indicate how successful a course of actions have been and whether another set of actions should have been taken instead – they are a constant feedback to the learning system. Learning on the other hand, guarantees that motivation and emotional subsystems are able to adapt to constantly changing conditions.[3]

Thus, in the study of biological organisms, emotions have arisen to prominence as an integral part of any biologically inspired system. But how does any living organism benefit from its emotions? It is crucial to answer this question as we attempt to increasingly employ biologically inspired methods in solving computational problems.

Every creature has innate abilities that accommodate its survival in the world. It can identify food, shelter, partners, and danger. But these "simple mappings between stimuli and reactions will not be enough to keep the organisms from encountering problems."[4] For example, if a given animal knows that its predator has qualities A, B and C, it will escape all creatures that have those qualities. And thus waste much of its energy and resources on non-existent danger.

We can not expect evolution to provide more advanced algorithms for assessing danger, because the predator is also evolving at the same speed. Thus, biological systems need to be equipped with the ability to learn. This learning and re-learning mechanism allows them to adapt to highly complex and advanced situations.[3]

A Learning Organism

To learn effectively, every learning organism needs an evaluation of the current situation and also feedback on how beneficial the results of learning were.[4] On the most part, these evaluation mechanisms are built-in. And so we encounter a new problem: whereas creatures take appropriate measures in real time based on their evaluations, these built-in evaluation procedures are developed in evolutionary time. But all creatures need to learn of new evaluation techniques in their lifetime just as they learn the proper reactions.

This is where the ability to condition emotional reactions comes into play. Biological organisms associate innate emotional stimuli with other stimuli they encounter in the world and thus give them an emotional significance when needed.[4] These evaluations can be monitored to operate at very specific times, specific places or when accompanied by other specific stimuli.

There is another reason why these observations are so significant and that is the creation of artificial systems. These systems do not evolve over time but are designed with certain abilities from the start. Thus, their adaptability must be built-in.

Computational model
The Brain Limbic System
The Amygdala in the Human Brain

A model is a simplified description of a phenomenon. It brings to life some aspects of this phenomenon while overlooking others. What aspects are kept in the model and what are overlooked greatly depends on the topic of study. Thus, the nature of a model depends on the purpose the investigator plans to carry out. A computational model is one which can be mathematically analyzed, tested and simulated using computer systems.[4]

To construct a computational model of emotional learning in the brain requires a thorough analysis of the amygdala and the orbitofrontal cortex and the interaction between them:[5]

In mammals, emotional responses are processed in a part of the brain called the limbic system which lies in the cerebral cortex. The main components of the limbic system are the amygdala, orbitofrontal cortex, thalamus and the sensory cortex.

The amygdala is an almond shaped area that is placed such that it can communicate with all other cortices within the limbic system. The primary affective conditioning of the system occurs within the amygdala. That is, the association between a stimulus and its emotional consequence takes place in this region.[4]

It has been suggested that learning takes place in two fundamental steps.[6] First, a particular stimulus is correlated with an emotional response. This stimulus can be an endless number of phenomena from observing a face, to detecting a scent, hearing a noise, etc. Second, this emotional consequence shapes an association between the stimulus and the response.[6] This analysis is quite influential in part because it was one of the first to suggest that emotions play a key part in learning.[5] In more recent studies, it has been shown that the association between a stimulus and its emotional consequence take place in the amygdala.[3][7] "In this region, highly analyzed stimulus representations in the cortex are associated with an emotional value. Therefore, emotions are properties of stimuli".[5]

The task of the amygdala is thus to assign a primary emotional value to each stimulus that has been paired with a primary reinforcer[7] – the reinforcer is the reward and punishment that the mammal receives. This task is aided by the orbitofrontal complex. "In terms of learning theory, the amygdala appears to handle the presentation of primary reinforcement, while the orbitofrontal cortex is involved in the detection of omission of reinforcement."[5]

The first thing we notice in the computational model developed by Moren and Balkenius is that quite a number of interacting learning systems exist in the brain that deal with emotional learning. The computational model is presented below where:

  • Th : Thalamus
  • CX : Sensory Cortex
  • A : Input structures in the amygdala
  • E : Output structures in the amygdala
  • O : Orbitofrontal Cortex
  • Rew/Pun : External signals identifying the presentation of reward and punishment
  • CR/UR : conditioned response/unconditioned response
  • V : Associative strength from cortical representation to the amygdala that is changed by learning
  • W : Inhibitory connection from orbitofrontal cortex to the amygdala that is changed during learning

This image shows that the sensory input enters through the thalamus TH. In biological systems, the thalamus takes on the task of initiating the process of a response to stimuli. It does so by passing the signal to the amygdala and the sensory cortex.[8]

This signal is then analyzed in the cortical area – CX. In biological systems, the sensory cortex operates by distributing the incoming signals appropriately between the amygdala and the orbitofrontal cortex.[9] This sensory representation in CX is then sent to the amygdala A, through the pathway V.

This is the main pathway for learning in this model. Reward and punishment enter the amygdala to strengthen the connection between the amygdala and the pathway. At a later stage if a similar representation is activated in the cortex, E becomes activated and produces an emotional response.

O, the orbitofrontal cortex, operates based on the difference between the perceived (i.e. expected) reward/punishment and the actual received reward/punishment. This perceived reward/punishment is the one that has been developed in the brain over time using learning mechanisms and it reaches the orbitofrontal cortex via the sensory cortex and the amygdala. The received reward/punishment on the other hand, comes courtesy of the outside world and is the actual reward/punishment that the specie has just obtained. If these two are identical, the output is the same as always through E. If not, the orbitofronal cortex inhibits and restrains emotional response to make way for further learning. So the path W is only activated in such conditions.

Controller

In most industrial processes that contain complex nonlinearities, control algorithms are used to create linearized models.[10] One reason is that these linear models are developed using straightforward methods from process test data.

However, if the process is highly complex and nonlinear, subject to frequent disturbances, a nonlinear model will be required.[10] Biologically motivated intelligent controllers have been increasingly employed in these situations. Amongst them, fuzzy logic, neural networks and genetic algorithms are some of the most widely employed tools in control applications with highly complex, nonlinear settings.[11][12]

BELBIC is one such nonlinear controller – a neuromorphic controller based on the computational learning model shown above to produce the control action. This model is employed much like an algorithm in these control engineering applications. In these new approaches, intelligence is not given to the system from the outside but is actually acquired by the system itself.[1][10]

This simple model has been employed as a feedback controller to be applied to control design problems. One logic behind this use in control engineering is a belief held by many experts in the field that there has been too much focus on fully rational deliberative approaches, whereas in many real-world circumstances, we are only provided with a bounded rationality. Factors like computational complexity, multiplicity of objectives and prevalence of uncertainty lead to a desire to obtain more ad-hoc, rule-of-thumb approaches. Emotional decision making is highly capable of addressing these issues because it is neither fully cognitive nor fully behavioral.[13]

BELBIC, which is a model free controller, suffers from the same drawback of all intelligent model free controllers: it cannot be applied on unstable systems or systems with unstable equilibrium point. This is a natural result of the trial and error manner of the learning procedure, i.e. exploration for finding the appropriate control signals can lead to instability.[14][15] By integrating imitative learning and fuzzy inference systems, BELBIC is generalized in order to be capable of controlling unstable systems.[14][15]

Applications

To date, BELBIC and its modified versions have been tested on the following applications:[16]

  • HVAC Systems (heating, ventilating and air conditioning): these are some of the most challenging plants in control systems which consume 50% of the total world energy consumption.[17]
  • Unstable Systems (or stable systems with unstable equilibrium point) [14][15][18]
  • Inverted pendulum systems [15]
  • Nonlinear systems[11][12]
  • Cell-to-Cell Mapping Algorithm[19]
  • Electrically heated micro heat exchanger: this device has been developed to accelerate fluid and heat exchange in reduced systems.
  • The Motion Control of three-wheeled robots: three-wheeled robots are commonly used in RoboCup soccer because they are omnidirectional with minimum wheels.[20]
  • RoboCup Rescue Simulation: a large, multi-agent system is one of the most challenging environments to control and coordinate because there needs to be a precise coordination between agents.[21]
  • Control of intelligent washing machines: intelligent control of home appliances has gained considerable attention by scientists and the industry in recent years. In the case of washing machines, intelligent control could mean both easier use and energy and water conservation.[22]
  • Autolanding system[23]
  • Speed Regulation of DC motors[24]
  • Active queue management[25]
  • Aerospace launch vehicle control[26]
  • Impossibles AIBO 4-legged Robocup competition
  • Predicting geomagnetic activity index;[27] In this application, the various extended models are proposed by researchers. Babaei et al. presented multi agent model of brain emotional learning and Lotfi and Akbarzadeh proposed supervised learning version of brain emotional learning to forecast Geomagnetic Activity Indices.,[28][29]
  • Gene expression microarray classification.,[29][30]
  • Speed control of switched reluctance motor
  • Intelligent control of Micro Heat Exchanger[31]
  • Model Free Control of Overhead Travelling Crane[32][33]
  • Autopilot Control Design for a 2-DOF Helicopter Model[34]
  • Path Tracking for a Car[35]
  • Attitude Control of a quadrotor [36]
  • Digital Servo System [37]
  • Multi-Agent Systems [38][39]
  • Secondary Control of Microgrids [40]
  • Position control of a real laboratorial EHS actuator: Electrohydraulic servo valves are known to be nonlinear and non-smooth due to many factors such as leakage, friction, hysteresis, null shift, saturation, dead zone, and especially fluid flow expression through the servomechanism.[41]

See also

References
  1. Lucas, C.; Shahmirzadi, D.; Sheikholeslami, N. (2004), "Introducing BELBIC: Brain Emotional Learning Based Intelligent Controller", Intelligent Automation and Soft Computing, 10: 11–22, doi:10.1080/10798587.2004.10642862, S2CID 12854189
  2. LeDoux, J. E. (1995), "In Search of an Emotional System in the Brain: Leaping from Fear to Emotion and Consciousness", in Gazzaniga, M. S (ed.), The Cognitive Neurosciences, Hillsdale, NJ: Lawrence Erlbaum, pp. 1049–1061
  3. LeDoux, J.E. (1996), The Emotional Brain, Simon and Schuster, New York
  4. Moren, Jan (2002). "Thesis: Emotion and Learning". Lund University. Cite journal requires |journal= (help)
  5. Moren, J.; Balkenius, C. (2000), A Computational Model of Emotional Learning in the Amygdala, MIT Press
  6. Mower, O. H. (1973) [1960], Learning Theory and Behavior, New York: Wiley
  7. Rolls, E. T. (1995), "A theory of Emotion and Consciousness, and its Application to Understanding the Neural Basis of Emotion", in Gazzaniga, M. S. (ed.), The Cognitive Neurosciences, Hillsdale, NJ: Lawrence Erlbaum, pp. 127–155
  8. Kelly, J. P. (1991), The Neural Basis of Perception and Movement, Principles of Neural Science, London: Prentice Hall
  9. Shahmirzadi, D. (2005). "Thesis: Computational Modeling of the Brain Limbic System and its Application in Control Engineering". Texas A & M University. Cite journal requires |journal= (help)
  10. Rouhani, H.; Jalili, M.; Araabi, B. N.; Eppler, W.; Lucas, C. (2006), "Brain Emotional Learning Based Intelligent Controller Applied to Neurofuzzy Model of Micro-Heat Exchanger", Expert Systems with Applications, 32 (3): 911–918, doi:10.1016/j.eswa.2006.01.047
  11. Mehrabian, A. R.; Lucas, C. (2007), "Intelligent Adaptive Control of Non-Linear Systems Based on Emotional Learning Approach", International Journal on Artificial Intelligence Tools, 16 (1): 69–85, doi:10.1142/S0218213007003205
  12. Mehrabian, A.R.; Lucas, C. (2006), "Emotional Learning Based Intelligent Robust Adaptive Controller for Stable Uncertain Nonlinear Systems", International Journal of Computational Intelligence, 2 (4): 246–252
  13. Mehrabian, A. R.; Lucas, C.; Roshanian, J. (2006), "Aerospace Launch Vehicle Control: An Intelligent Adaptive Approach", Aerospace Science and Technology, 10 (2): 149–155, doi:10.1016/j.ast.2005.11.002
  14. Javan Roshtkhari, M.; Arami, A.; Lucas, C. (2010), "Imitative Learning Based Emotional Controller for Unknown Systems with Unstable Equilibrium" (PDF), International Journal of Intelligent Computing and Cybernetics, 3 (2): 334–359, doi:10.1108/17563781011049232, archived from the original (PDF) on 2015-06-10, retrieved 2012-09-26
  15. Javan Roshtkhari, M.; Arami, A.; Lucas, C. (2009), Emotional Control of Inverted Pendulum System, A soft switching from Imitative to emotional learning, pp. 651–656
  16. Lucas, C. (2011), "BELBIC and Its Industrial Applications: Towards Embedded Neuroemotional Control Codesign", Integrated Systems, Design and Technology: 203–214, doi:10.1007/978-3-642-17384-4_17, ISBN 978-3-642-17383-7
  17. Sheikholeslami, N.; Shahmirzadi, D.; Semsar, E.; Lucas, C.; Yazdanpanah, M. J. (2005), "Applying Brain Emotional Learning Algorithm for Multivariable Control of HVAC Systems", Intelligent and Fuzzy Systems, 16: 1–12
  18. Valipour, Mohammad Hadi; Niki Maleki, Khashayar; Shiry Ghidary, Saeed (2015). "Optimization of Emotional Learning Approach to Control Systems with Unstable Equilibrium". Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing. Studies in Computational Intelligence. 569: 45–56. doi:10.1007/978-3-319-10389-1_4. ISBN 978-3-319-10388-4.
  19. Shahmirzadi, D.; Langari, R. (2005), "Stability of Amygdala Learning system using Cell-to-Cell Mapping Algorithm", International Journal of Computational Intelligence
  20. Sharbafi, M. A.; Lucas, C.; Yaghobi, M. (2006), "Designing a Football Team of Robots from Beginning to End", International Journal of Information Technology, 3: 101–108
  21. Sharbafi, M. A.; Lucas, C.; Toroghi Haghighat, A.; Amirghiasvand, O.; Aghazade, O. (2006), "Using Emotional Learning in Rescue Simulation Environment", Transactions of Engineering, Computing and Technology, 13: 333–337
  22. Milasi, R. M.; Lucas, C.; Araabi, B. N. (2005), Intelligent Modeling and Control of Washing Machines Using LLNF Modeling and Modified BELBIC, pp. 812–817
  23. Lucas, C.; Moghimi, S., Appying BELBIC (Brain Emotional Learning Based Intelligent Controller) to an Auto Landing System
  24. Rashidi, F.; Rashidi, M.; Hashemi, A., Applying Intelligent Controllers for Speed Regulation of DC Motors
  25. Jalili, M., Application of Brain Emotional Learning Based Intelligent Controller (BELBIC) to Active Queue Management, 3037/2004, pp. 662–665
  26. Mehrabian, A.R.; Lucas, C.; Roshanian, J. (2008), "Design of an Aerospace Launch Vehicle Autopilot Based on Optimized Emotional Learning Algorithm", Cybernetics and Systems, 39 (3): 1–18, doi:10.1080/01969720801944364, S2CID 28928524
  27. Gholipour, A.; Lucas, C.; Shahmirzadi, D. (2003), Predicting Geomagnetic Activity Index by Brain Emotional Learning Algorithm
  28. Lotfi, E.; Akbarzadeh-T., M.R. (2012), "Supervised Brain Emotional Learning", The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–6, doi:10.1109/IJCNN.2012.6252391, ISBN 978-1-4673-1490-9, S2CID 6159346
  29. {http://www.bitools.ir}%5B%5D
  30. Lotfi, E.; Keshavarz., A. (2014), "Gene expression microarray classification using PCA–BEL", Computers in Biology and Medicine, 54: 180–187, doi:10.1016/j.compbiomed.2014.09.008, PMID 25282708
  31. Jalili, M., Intelligent control of Micro Heat Exchanger with Locally Linear Identifier and Emotional Based Controller, 3578/2005, pp. 179–186
  32. Jamali, M.R.; Arami, A., "Real Time Emotional Control for Anti-Swing and Positioning Control of SIMO Overhead Travelling Crane", International Journal of Innovative Computing, Information and Control, 4 (5/2008): 2333–2344
  33. Arami, A.; Javan Roshtkhari, M.; Lucas, C. (2008), A Fast Model Free Intelligent Controller Based on Fused Emotions: A Practical Case Implementation, pp. 596–602
  34. Jafarzadeh, S.; Mirheidari, R.; Jahed-Motlagh, M. R.; Barkhordari, M., "Intelligent Autopilot Control Design for a 2-DOF Helicopter Model", International Journal of Computers, Communications & Control, 3 (Proceedings of ICCCC 2008): 337–342
  35. Jafarzadeh, S.; Mirheidari, R.; Jahed-Motlagh, M. R.; Barkhordari, M., "Designing PID and BELBIC Controllers in Path Tracking Problem", International Journal of Computers, Communications & Control, 3 (Proceedings of ICCCC 2008): 343–348
  36. Jafari, M.; Shahri, A. M.; Shouraki, S. B. (2013-08-01). "Attitude control of a Quadrotor using Brain Emotional Learning Based Intelligent Controller". 2013 13th Iranian Conference on Fuzzy Systems (IFSC): 1–5. doi:10.1109/IFSC.2013.6675672. ISBN 978-1-4799-1228-5. S2CID 2173266.
  37. Jafari, M.; shahri, A. Mohammad; Shuraki, S. B. (2013-02-01). "Speed control of a Digital Servo System using Brain Emotional Learning Based Intelligent Controller". Power Electronics, Drive Systems and Technologies Conference (PEDSTC), 2013 4th: 311–314. doi:10.1109/PEDSTC.2013.6506724. ISBN 978-1-4673-4484-5. S2CID 36370603.
  38. Jafari, M.; Xu, H.; Carrillo, L. R. G. (May 2017). "Brain Emotional Learning-Based Intelligent Controller for flocking of Multi-Agent Systems". 2017 American Control Conference (ACC): 1996–2001. doi:10.23919/ACC.2017.7963245.
  39. Jafari, Mohammad; Xu, Hao (2019-03-01). "A biologically-inspired distributed fault tolerant flocking control for multi-agent system in presence of uncertain dynamics and unknown disturbance". Engineering Applications of Artificial Intelligence. 79: 1–12. doi:10.1016/j.engappai.2018.12.001. ISSN 0952-1976.
  40. Jafari, Mohammad; Ghasemkhani, Amir; Sarfi, Vahid; Livani, Hanif; Yang, Lei; Xu, Hao (2019-06-24). "Biologically inspired adaptive intelligent secondary control for MGs under cyber imperfections". IET Cyber-Physical Systems: Theory & Applications. 4 (4): 341–352. doi:10.1049/iet-cps.2018.5003. ISSN 2398-3396.
  41. Sadeghieh, A.; Sazgar, H.; Goodarzi, K.; Lucas, C. (2012), "Identification and real-time position control of a servo-hydraulic rotary actuator by means of a neurobiologically motivated algorithm", ISA Transactions, 51: 208–219, doi:10.1016/j.isatra.2011.09.006, ISSN 0019-0578
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.