1
|
Bowers JS, Malhotra G, Dujmović M, Montero ML, Tsvetkov C, Biscione V, Puebla G, Adolfi F, Hummel JE, Heaton RF, Evans BD, Mitchell J, Blything R. Clarifying status of DNNs as models of human vision. Behav Brain Sci 2023; 46:e415. [PMID: 38054298 DOI: 10.1017/s0140525x23002777] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
On several key issues we agree with the commentators. Perhaps most importantly, everyone seems to agree that psychology has an important role to play in building better models of human vision, and (most) everyone agrees (including us) that deep neural networks (DNNs) will play an important role in modelling human vision going forward. But there are also disagreements about what models are for, how DNN-human correspondences should be evaluated, the value of alternative modelling approaches, and impact of marketing hype in the literature. In our view, these latter issues are contributing to many unjustified claims regarding DNN-human correspondences in vision and other domains of cognition. We explore all these issues in this response.
Collapse
Affiliation(s)
- Jeffrey S Bowers
- School of Psychological Science, University of Bristol, Bristol, UK ; https://jeffbowers.blogs.bristol.ac.uk/
| | - Gaurav Malhotra
- School of Psychological Science, University of Bristol, Bristol, UK ; https://jeffbowers.blogs.bristol.ac.uk/
| | - Marin Dujmović
- School of Psychological Science, University of Bristol, Bristol, UK ; https://jeffbowers.blogs.bristol.ac.uk/
| | - Milton L Montero
- School of Psychological Science, University of Bristol, Bristol, UK ; https://jeffbowers.blogs.bristol.ac.uk/
| | - Christian Tsvetkov
- School of Psychological Science, University of Bristol, Bristol, UK ; https://jeffbowers.blogs.bristol.ac.uk/
| | - Valerio Biscione
- School of Psychological Science, University of Bristol, Bristol, UK ; https://jeffbowers.blogs.bristol.ac.uk/
| | | | - Federico Adolfi
- Ernst Strüngmann Institute (ESI) for Neuroscience in Cooperation with Max Planck Society, Frankfurt am Main, Germany
| | - John E Hummel
- Psychology Department, University of Illinois Urbana-Champaign, Champaign, IL, USA
| | - Rachel F Heaton
- Psychology Department, University of Illinois Urbana-Champaign, Champaign, IL, USA
| | - Benjamin D Evans
- Department of Informatics, School of Engineering and Informatics, University of Sussex, Brighton, UK
| | - Jeffrey Mitchell
- Department of Informatics, School of Engineering and Informatics, University of Sussex, Brighton, UK
| | - Ryan Blything
- School of Psychology, Aston University, Birmingham, UK
| |
Collapse
|
2
|
Vannuscorps G, Caramazza A. Effector-specific motor simulation supplements core action recognition processes in adverse conditions. Soc Cogn Affect Neurosci 2023; 18:nsad046. [PMID: 37688518 PMCID: PMC10576201 DOI: 10.1093/scan/nsad046] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 08/10/2023] [Accepted: 09/05/2023] [Indexed: 09/11/2023] Open
Abstract
Observing other people acting activates imitative motor plans in the observer. Whether, and if so when and how, such 'effector-specific motor simulation' contributes to action recognition remains unclear. We report that individuals born without upper limbs (IDs)-who cannot covertly imitate upper-limb movements-are significantly less accurate at recognizing degraded (but not intact) upper-limb than lower-limb actions (i.e. point-light animations). This finding emphasizes the need to reframe the current controversy regarding the role of effector-specific motor simulation in action recognition: instead of focusing on the dichotomy between motor and non-motor theories, the field would benefit from new hypotheses specifying when and how effector-specific motor simulation may supplement core action recognition processes to accommodate the full variety of action stimuli that humans can recognize.
Collapse
Affiliation(s)
- Gilles Vannuscorps
- Psychological Sciences Research Institute, Université catholique de Louvain, Place Cardinal Mercier 10, 1348, Louvain-la-Neuve, Belgium
- Institute of Neuroscience, Université catholique de Louvain, Avenue E. Mounier 53, Brussels 1200, Belgium
- Department of Psychology, Harvard University, Kirkland Street 33, Cambridge, MA 02138, USA
| | - Alfonso Caramazza
- Department of Psychology, Harvard University, Kirkland Street 33, Cambridge, MA 02138, USA
- CIMEC (Center for Mind-Brain Sciences), University of Trento, Via delle Regole 101, Mattarello TN 38123, Italy
| |
Collapse
|
3
|
Vannuscorps G, Galaburda A, Caramazza A. From intermediate shape-centered representations to the perception of oriented shapes: response to commentaries. Cogn Neuropsychol 2023; 40:71-94. [PMID: 37642330 DOI: 10.1080/02643294.2023.2250511] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Revised: 07/14/2023] [Accepted: 07/31/2023] [Indexed: 08/31/2023]
Abstract
In this response paper, we start by addressing the main points made by the commentators on the target article's main theoretical conclusions: the existence and characteristics of the intermediate shape-centered representations (ISCRs) in the visual system, their emergence from edge detection mechanisms operating on different types of visual properties, and how they are eventually reunited in higher order frames of reference underlying conscious visual perception. We also address the much-commented issue of the possible neural mechanisms of the ISCRs. In the final section, we address more specific and general comments, questions, and suggestions which, albeit very interesting, were less directly focused on the main conclusions of the target paper.
Collapse
Affiliation(s)
- Gilles Vannuscorps
- Department of Psychology, Harvard University, Cambridge, MA, USA
- Institute of Psychological Sciences, Université catholique de Louvain, Louvain-la-Neuve, Belgium
- Institute of Neuroscience, Université catholique de Louvain, Louvain-la-Neuve, Belgium
- Louvain Bionics, Université catholique de Louvain, Louvain-la-Neuve, Belgium
| | - Albert Galaburda
- Department of Neurology, Harvard Medical School and Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Alfonso Caramazza
- Department of Psychology, Harvard University, Cambridge, MA, USA
- Center for Mind/Brain Sciences (CIMeC), Università degli Studi di Trento, Rovereto, Italy
| |
Collapse
|
4
|
The merits and future of data deep-dives for cognitive neuroscience. Neuropsychologia 2022; 176:108394. [DOI: 10.1016/j.neuropsychologia.2022.108394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
5
|
Popovkina DV, Pasupathy A. Davida's deficits: weak encoding of impoverished stimuli or faulty egocentric representation? Cogn Neuropsychol 2022; 39:99-102. [PMID: 35676872 DOI: 10.1080/02643294.2022.2083947] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Abstract
Vannuscorps and colleagues present the fascinating case of Davida, a young person who makes systematic errors in judgments related to orientations of sharp or high-contrast visual stimuli. In this commentary, we discuss the findings in the context of observations from mid-level ventral visual stream physiology. We propose two additional interpretations for the specificity of the behavioural deficits: the observed impairments in orientation judgments may be consistent with a system that is not able to unambiguously represent certain impoverished stimuli, or with a system that is not able to translate visual input into head- or body-centered coordinates. Davida's case offers a unique glimpse into the complex cascade of transformations that enable accurate orientation judgments, and sparks curiosity about which mechanistic disruptions can produce such specific unstable percepts.
Collapse
Affiliation(s)
- Dina V Popovkina
- Department of Psychology, University of Washington, Seattle, WA, USA
| | - Anitha Pasupathy
- Department of Biological Structure, University of Washington, Seattle, WA, USA.,Washington National Primate Research Center, University of Washington, Seattle, WA, USA
| |
Collapse
|
6
|
Medina J. Using single cases to understand visual processing: The magnocellular pathway. Cogn Neuropsychol 2022; 39:106-108. [PMID: 35677970 DOI: 10.1080/02643294.2022.2083949] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Affiliation(s)
- Jared Medina
- Department of Psychological and Brain Sciences, University of Delaware, Newark, DE, USA
| |
Collapse
|
7
|
Almeida J. Precedence of parvocellular- over magnocellular-biased information for 2D object-related shape processing. Cogn Neuropsychol 2022; 39:95-98. [PMID: 35603605 DOI: 10.1080/02643294.2022.2076584] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Affiliation(s)
- Jorge Almeida
- Proaction Lab, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal.,CINEICC, Faculdade de Psicologia e de Ciências da Educação, Universidade de Coimbra, Coimbra, Portugal
| |
Collapse
|
8
|
Vannuscorps G, Galaburda A, Caramazza A. Shape-centered representations of bounded regions of space mediate the perception of objects. Cogn Neuropsychol 2021; 39:1-50. [PMID: 34427539 DOI: 10.1080/02643294.2021.1960495] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/18/2023]
Abstract
We report the study of a woman who perceives 2D bounded regions of space ("shapes") defined by sharp edges of medium to high contrast as if they were rotated by 90, 180 degrees around their centre, mirrored across their own axes, or both. In contrast, her perception of 3D, strongly blurred or very low contrast shapes, and of stimuli emerging from a collection of shapes, is intact. This suggests that a stage in the process of constructing the conscious visual representation of a scene consists of representing mutually exclusive bounded regions extracted from the initial retinotopic space in "shape-centered" frames of reference. The selectivity of the disorder to shapes originally biased toward the parvocellular subcortical pathway, and the absence of any other type of error, additionally invite new hypotheses about the operations involved in computing these "intermediate shape-centered representations" and in mapping them onto higher frames for perception and action.
Collapse
Affiliation(s)
- Gilles Vannuscorps
- Department of Psychology, Harvard University, Cambridge, MA, USA.,Institute of Psychological Sciences, Université catholique de Louvain, Louvain-la-Neuve, Belgium.,Institute of Neuroscience, Université catholique de Louvain, Louvain-la-Neuve, Belgium.,Louvain Bionics, Université catholique de Louvain, Louvain-la-Neuve, Belgium
| | - Albert Galaburda
- Department of Neurology, Harvard Medical School and Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Alfonso Caramazza
- Department of Psychology, Harvard University, Cambridge, MA, USA.,Center for Mind/Brain Sciences (CIMeC), Università degli Studi di Trento, Rovereto, Italy
| |
Collapse
|