Dopamine and temporal difference learning: A fruitful relationship between neuroscience and AI

Learning and motivation are driven by internal and external rewards. Many of our day-to-day behaviours are guided by predicting, or anticipating, whether a given action will result in a positive (that is, rewarding) outcome. The study of how organisms learn from experience to correctly anticipate rewards has been a productive research field for well over a century, since Ivan Pavlov’s seminal psychological work. In his most famous experiment, dogs were trained to expect food some time after a buzzer sounded. These dogs began salivating as soon as they heard the sound, before the food had arrived, indicating they’d learned to predict the reward. In the original experiment, Pavlov estimated the dogs’ anticipation by measuring the volume of saliva they produced. But in recent decades, scientists have begun to decipher the inner workings of how the brain learns these expectations. Meanwhile, in close contact with this study of reward learning in animals, computer scientists have developed algorithms for reinforcement learning in artificial systems. These algorithms enable AI systems to learn complex strategies without external instruction, guided instead by reward predictions.

The contribution of our new work, published in Nature (PDF), is finding that a recent development in computer science – which yields significant improvements in performance on reinforcement learning problems – may provide a deep, parsimonious explanation for several previously unexplained features of reward learning in the brain, and opens up new avenues of research into the brain’s dopamine system, with potential implications for learning and motivation disorders.

LEARNING 10,000 PICTURES

Four experiments are reported which examined memory capacity and retrieval speed for pictures and for words. Single-trial learning tasks were employed throughout, with memory performance assessed by forced-choice recognition, recall measures or choice reaction-time tasks. The main experimental findings were: (I) memory capacity, as a function of the amount of material presented, follows a general power law with a characteristic exponent for each task; (2) pictorial material obeys this power law and shows an overall superiority to verbal material. The capacity of recognition memory for pictures is almost limitless, when measured under appropriate conditions; (3) when the recognition task is made harder by using more alternatives, memory capacity stays constant and the superiority of pictures is maintained; (4) picture memory also exceeds verbal memory in terms of verbal recall; comparable recognition/recall ratios are obtained for pictures, words and nonsense syllables; (5) verbal memory shows a higher retrieval speed than picture memory, as inferred from reaction-time measures. Both types of material obey a power law, when reaction-time is measured for various sizes of learning set, and both show very rapid rates of memory search.

From a consideration of the experimental results and other data it is concluded that the superiority of the pictorial mode in recognition and free recall learning tasks is well established and cannot be attributed to methodological artifact.

Cartographers of the Brain: Mapping the Connectome

Scientists are attempting to map the wiring of the nearly 100 billion neurons in the human brain. Are we close to uncovering the mysteries of the mind or are we only at the beginning of a new frontier?

PARTICIPANTS: Deanna Barch, Jeff Lichtman, Nim Tottenham, David Van Essen
MODERATOR: John Hockenberry
Original program date: JUNE 4, 2017

WATCH THE TRAILER: https://youtu.be/lX5S_1bXUhw
WATCH THE LIVE Q&A W/ JEFF LICHTMAN: https://youtu.be/h14hcBrqGSg

Imagine navigating the globe with a map that only sketched out the continents. That’s pretty much how neuroscientists have been operating for decades. But one of the most ambitious programs in all of neuroscience, the Human Connectome Project, has just yielded a “network map” that is shedding light on the intricate connectivity in the brain. Join leading neuroscientists and psychologists as they explore how the connectome promises to revolutionize treatments for psychiatric and neurological disorders, answer profound questions regarding the electrochemical roots of memory and behavior, and clarify the link between our upbringing and brain development.

Illusions, delusions and the brain. A Ramachandran lecture on body image and mind body interactions.

This lecture by Prof. V.S. Ramachandran (University of California, San Diego) will focus on body image and mind body interactions.
https://www.glasgow.ac.uk%2Findex.html%3Frefer%3Dguyoutube&event=video_description&v=kcR8-Sq8dZk

This year’s prestigious University of Glasgow Gifford Lecture Series will feature three talks from V.S. Ramachandran, the Director of the Centre for Brain and Cognition and Distinguished Professor with the Psychology Department and Neurosciences Program at the University of California.

Founded in 1887 by the bequest of Lord Gifford, the annual Gifford Lecture Series was established to promote, advance and diffuse the study of Natural Theology in the widest sense of that term. The focus of this series will be ‘Body and Mind; Insights from Neuroscience.’

Ramachandran said: “Monday’s lecture will focus on body image and mind body interactions, while Wednesday’s lecture will deal with understanding higher brain functions through studies of synesthesia and other types of intersensory interactions, including discoveries of mirror neurons. I will also touch on the important deeper philosophical implications that surround these subject areas.”

Dr Susan Stuart, Senior Lecturer in Philosophy at the University of Glasgow said: “It is our great pleasure that Vilayanur S. Ramachandran has agreed to present the 2012 Gifford lectures at the University of Glasgow. Ramachandran is one of the world’s leading neuroscientists; he has been responsible for groundbreaking work in the fields of behavioural neurology and psychophysics.

“His work on autism, visual processing, and synaesthesia is truly pioneering and during his career he has carried out marvellous work on understanding and treating phantom limb disorders as well as linking between temporal lobe epilepsy and hyper-religiosity, a field now known as ‘neurotheology’ or ‘spiritual neuroscience’.”

Biography:
V.S. Ramachandran is Director of the Center for Brain and Cognition and Distinguished Professor with the Psychology Department and Neurosciences Program at the University of California, San Diego, and Adjunct Professor of Biology at the Salk Institute. Ramachandran initially trained as a doctor and subsequently obtained a Ph.D. from Trinity College at the University of Cambridge. Ramachandran’s early work was on visual perception but he is best known for his experiments in behavioral neurology which, despite their apparent simplicity, have had a profound impact on the way we think about the brain. He has been called “The Marco Polo of neuroscience” by Richard Dawkins and “The modern Paul Broca” by Eric Kandel.
In 2005 he was awarded the Henry Dale Medal and elected to an honorary life membership by the Royal Instituion of Great Britain, where he also gave a Friday evening discourse (joining the ranks of Michael Faraday, Thomas Huxley, Humphry Davy, and dozens of Nobel Laureates). His other honours and awards include fellowships from All Souls College, Oxford, and from Stanford University (Hilgard Visiting Professor); the Presidential Lecture Award from the American Academy of Neurology, two honorary doctorates, the annual Ramon Y Cajal award from the International Neuropsychiatry Society, and the Ariens-Kappers medal from the Royal Netherlands Academy of Sciences. In 2003 he gave the annual BBC Reith lectures and was the first physician/psychologist to give the lectures since they were begun by Bertrand Russel in 1949. In 1995 he gave the Decade of the Brain lecture at the 25th annual (Silver Jubilee) meeting of the Society for Neuroscience. In 2010 he delivered the annual Jawaharlal Nehru memorial lecture in New Delhi, India. Most recently the President of India conferred on him the second highest civilian award and honorific title in India, the Padma Bhushan. And TIME magazine named him on their list of the 100 most influential people in the world.

Ramachandran has published over 180 papers in scientific journals (including five invited review articles in the Scientific American). He is author of the acclaimed book “Phantoms in the Brain” that has been translated into nine languages and formed the basis for a two part series on Channel Four TV (UK) and a 1 hour PBS special in USA. NEWSWEEK magazine has named him a member of “The Century Club” — one of the “hundred most prominent people to watch in the next century.” He has been profiled in the New Yorker Magazine and appeared on the Charlie Rose Show. His new book, “The Tell Tale Brain” was on the New York Times best-seller list.

In addition, Ramachandran has an interest in history and archaeology (see his article on the Indus Valley Code).

Closed-loop brain training: The science of neurofeedback

Neurofeedback is a psychophysiological procedure in which online feedback of neural activation is provided to the participant for the purpose of self-regulation. Learning control over specific neural substrates has been shown to change specific behaviours. As a progenitor of brain–machine interfaces, neurofeedback has provided a novel way to investigate brain function and neuroplasticity. In this Review, we examine the mechanisms underlying neurofeedback, which have started to be uncovered. We also discuss how neurofeedback is being used in novel experimental and clinical paradigms from a multidisciplinary perspective, encompassing neuroscientific, neuroengineering and learning-science viewpoints.

Thanks for the Feedback: The Science and Art of Receiving Feedback Well

We get feedback every day of our lives, from friends and family, colleagues, customers, and bosses, teachers, doctors, and strangers.  We’re assessed, coached, and criticized about our performance, personalities, and appearance.

We know that feedback is essential for professional development and healthy relationships—but we dread it and often dismiss it.  That’s because receiving feedback sits at the junction of two conflicting human desires. We want to learn and grow, but we also want to be accepted and respected just as we are now.  Thanks for the Feedback is the first book to address this tension head on.  It explains why getting feedback is so crucial yet so challenging and offers a simple framework and powerful tools to help us take on life’s blizzard of offhand comments, annual evaluations, and unsolicited advice with curiosity and grace.

The business world spends billions of dollars and millions of hours each year teaching people how to give feedback more effectively.  Stone and Heen argue that we’ve got it backwards and show us why the smart money is on educating receivers – in the workplace as well as in personal relationships. It’s the receivers, after all, who interpret what they’re hearing and decide whether and how to change.

Coauthors of the international bestseller Difficult Conversations, Stone and Heen have discovered that while receiving feedback can be fraught, doing it well can be taught.  With humor and clarity, the book blends the latest insights from neuroscience and psychology with practical, hard-headed advice. It is destined to become a classic in the world of leadership, organizational behavior, and education.

Neuroception: A Subconscious System for Detecting Threats and Safety

The author describes recent findings on the neurobiological mechanisms involved in perceptions of risk and safety. The term “Neuroception” describes how neural circuits distinguish whether situations or people are safe, dangerous, or life threatening. Neuroception explains why a baby coos at a caregiver but cries at a stranger, or why a toddler enjoys a parent’s embrace but views a hug from a stranger as an assault. The author explains the Polyvagal Theory, which posits that mammals–especially primates–have evolved brain structures that regulate both social and defensive behaviors. The Polyvagal Theory describes three developmental stages of a mammal’s autonomic nervous system: immobilization, mobilization, and social communication or social engagement. A neuroception of safety is necessary before social engagement behaviors can occur. Infants, young children, and adults need appropriate social engagement strategies in order to form positive attachments and social bonds. Faulty neuroception might lie at the root of several psychiatric disorders, including autism, schizophrenia, anxiety disorders, depression, and Reactive Attachment Disorder.

Cultivating Communities of Practice: A Guide to Managing Knowledge

Today’s economy is fueled by knowledge. Every leader knows this to be true, yet few have systematic methods for converting organizational knowledge into economic value. This book argues that communities of practice–groups of individuals formed around common interests and expertise–provide the ideal vehicle for driving knowledge-management strategies and building lasting competitive advantage. Written by leading experts in the field, Cultivating Communities of Practice is the first book to outline models and methods for systematically developing these essential groups. Through compelling research and company examples, including DaimlerChrysler, McKinsey & Company, Shell, and the World Bank, authors Etienne Wenger, Richard McDermott, and William M. Snyder show how world-class organizations have leveraged communities of practice to drive strategy, generate new business opportunities, solve problems, transfer best practices, develop employees’ professional skills, and recruit and retain top talent. Underscoring the new central role communities of practice are playing in today’s knowledge economy, Cultivating Communities of Practice is the definitive guide to fostering, designing, and developing these powerful groups within and across organizations.

Michel Thomas – The Language Master

Michel Thomas spends 5 days with students who have previously failed to learn languages and, in the words of their teacher, they learn as much from Michel in those 5 days as they would normally learn in 5 years.

All without homework, writing or revision

Learning How to Learn: Powerful mental tools to help you master tough subjects

This course gives you easy access to the invaluable learning techniques used by experts in art, music, literature, math, science, sports, and many other disciplines. We’ll learn about the how the brain uses two very different learning modes and how it encapsulates (“chunks”) information. We’ll also cover illusions of learning, memory techniques, dealing with procrastination, and best practices shown by research to be most effective in helping you master tough subjects.

Using these approaches, no matter what your skill levels in topics you would like to master, you can change your thinking and change your life. If you’re already an expert, this peep under the mental hood will give you ideas for: turbocharging successful learning, including counter-intuitive test-taking tips and insights that will help you make the best use of your time on homework and problem sets. If you’re struggling, you’ll see a structured treasure trove of practical techniques that walk you through what you need to do to get on track. If you’ve ever wanted to become better at anything, this course will help serve as your guide.

This course can be taken independent of, concurrent with, or prior to, its companion course, Mindshift. (Learning How to Learn is more learning focused, and Mindshift is more career focused.)