Dopamine and temporal difference learning: A fruitful relationship between neuroscience and AI

Learning and motivation are driven by internal and external rewards. Many of our day-to-day behaviours are guided by predicting, or anticipating, whether a given action will result in a positive (that is, rewarding) outcome. The study of how organisms learn from experience to correctly anticipate rewards has been a productive research field for well over a century, since Ivan Pavlov’s seminal psychological work. In his most famous experiment, dogs were trained to expect food some time after a buzzer sounded. These dogs began salivating as soon as they heard the sound, before the food had arrived, indicating they’d learned to predict the reward. In the original experiment, Pavlov estimated the dogs’ anticipation by measuring the volume of saliva they produced. But in recent decades, scientists have begun to decipher the inner workings of how the brain learns these expectations. Meanwhile, in close contact with this study of reward learning in animals, computer scientists have developed algorithms for reinforcement learning in artificial systems. These algorithms enable AI systems to learn complex strategies without external instruction, guided instead by reward predictions.

The contribution of our new work, published in Nature (PDF), is finding that a recent development in computer science – which yields significant improvements in performance on reinforcement learning problems – may provide a deep, parsimonious explanation for several previously unexplained features of reward learning in the brain, and opens up new avenues of research into the brain’s dopamine system, with potential implications for learning and motivation disorders.

Feedback without Criticism

I have yet to meet a person who likes criticism. Instead, what most of us do is contract inside when we hear a criticism. Sometimes we respond defensively, sometimes we add the criticism to our pile of self-judgment, and sometimes we deflect and ignore what’s being said. In the process, we rarely manage to make use of the vital information and opportunities that useful feedback can provide: learning, better teamwork, or simply insight and understanding.

What Is Open Space Technology?

Open Space Technology is one way to enable all kinds of people, in any kind of organization, to create inspired meetings and events. Over the last 30+ years, it has also become clear that opening space, as an intentional leadership practice, can create inspired organizations, where ordinary people work together to create extraordinary results with regularity.

Strengthening Collaboration Through Encouraging Dissent

The first time I heard that groups thrive on dissent, I didn’t like the idea. It came up in conversations with Tom Atlee of the Co-Intelligence Institute, back in the mid-1990s. Tom was clear, based on his experience in activist movements and especially on a cross-country peace march, that dissent is essential for groups to function intelligently. So much so that if a group had too little dissent, he advocated for actively cultivating it to keep the group fresh and creative.

Warm Data Labs

Warm Data Labs are group processes, which illustrate interdependency and generate understandings of systemic patterns for people with no previous exposure to systems theory. Warm Data Labs enable new societal responses to complex challenges.

Linear Programming | Applications Of Linear Programming

Optimization is the way of life. We all have finite resources and time and we want to make the most of them. From using your time productively to solving supply chain problems for your company – everything uses optimization. It’s an especially interesting and relevant topic in data science.

It is also a very interesting topic – it starts with simple problems, but it can get very complex. For example, sharing a bar of chocolate between siblings is a simple optimization problem. We don’t think in mathematical terms while solving it. On the other hand, devising inventory and warehousing strategy for an e-tailer can be very complex. Millions of SKUs with different popularity in different regions to be delivered in defined time and resources – you see what I mean!

Linear programming (LP) is one of the simplest ways to perform optimization. It helps you solve some very complex optimization problems by making a few simplifying assumptions. As an analyst, you are bound to come across applications and problems to be solved by Linear Programming.

For some reason, LP doesn’t get as much attention as it deserves while learning data science. So, I thought let me do justice to this awesome technique. I decided to write an article that explains Linear programming in simple English. I have kept the content as simple as possible. The idea is to get you started and excited about Linear Programming.

What is TRIZ

Imagine the biggest study of human creativity ever conducted. Picture the systematic study of over two million of the world’s most successful patents, and the construction of a problem solving method which then combines those solutions into a whole that strips away all boundaries between different industries. Now imagine that it exists. What you’re seeing is TRIZ. The reason you may not have heard of it before, is that it was initially devised and developed in the former Soviet Union, and practically no-one outside the Eastern Bloc had heard of it before the fall of the Berlin Wall. In this paper, we examine what that Soviet research achieved and how that platform has now been transformed into a comprehensive Systematic Innovation methodology, suitable for all types of innovation and innovation management issues. In the paper we show how today’s version of the method is helping users to systematically and reliably create breakthrough solutions to problems of all descriptions.

Neuroception: A Subconscious System for Detecting Threats and Safety

The author describes recent findings on the neurobiological mechanisms involved in perceptions of risk and safety. The term “Neuroception” describes how neural circuits distinguish whether situations or people are safe, dangerous, or life threatening. Neuroception explains why a baby coos at a caregiver but cries at a stranger, or why a toddler enjoys a parent’s embrace but views a hug from a stranger as an assault. The author explains the Polyvagal Theory, which posits that mammals–especially primates–have evolved brain structures that regulate both social and defensive behaviors. The Polyvagal Theory describes three developmental stages of a mammal’s autonomic nervous system: immobilization, mobilization, and social communication or social engagement. A neuroception of safety is necessary before social engagement behaviors can occur. Infants, young children, and adults need appropriate social engagement strategies in order to form positive attachments and social bonds. Faulty neuroception might lie at the root of several psychiatric disorders, including autism, schizophrenia, anxiety disorders, depression, and Reactive Attachment Disorder.

Tesla is going to ‘kill’ the auto industry with Elon Musk’s way of thinking about manufacturing, says SpaceX CTO

Tesla was for a long time a “product company” – meaning that it focused on creating great products first. It has been quite successful at it with vehicles, like the Model S, winning almost all car awards out there. But CEO Elon Musk has recently shifted the automaker’s focus to manufacturing – saying that the factory itself is probably more important than the product it is making.

Fear in the Brain

Unraveling the mystery of how the mind experiences fear is one of the most interesting quests in recent neuroscience.