Extensional Superposition and Its Relation to Compositionality in Language and Thought

On the ability of standard and brain-constrained deep neural networks to support cognitive superposition: a position paper

The ability to coactivate (or "superpose") multiple conceptual representations is a fundamental function that we constantly rely upon; this is crucial in complex cognitive tasks requiring multi-item working memory, such as mental arithmetic, abstract reasoning, and language comprehension. As such, an artificial system aspiring to implement any of these aspects of general intelligence should be able to support this operation. I argue here that standard, feed-forward deep neural networks (DNNs) are unable to implement this function, whereas an alternative, fully brain-constrained class of neural architectures spontaneously exhibits it. On the basis of novel simulations, this proof-of-concept article shows that deep, brain-like networks trained with biologically realistic Hebbian learning mechanisms display the spontaneous emergence of internal circuits (cell assemblies) having features that make them natural candidates for supporting superposition. Building on previous computational modelling results, I also argue that, and offer an explanation as to why, in contrast, modern DNNs trained with gradient descent are generally unable to co-activate their internal representations. While deep brain-constrained neural architectures spontaneously develop the ability to support superposition as a result of (1) neurophysiologically accurate learning and (2) cortically realistic between-area connections, backpropagation-trained DNNs appear to be unsuited to implement this basic cognitive operation, arguably necessary for abstract thinking and general intelligence. The implications of this observation are briefly discussed in the larger context of existing and future artificial intelligence systems and neuro-realistic computational models.

The Puzzling Chasm Between Cognitive Representations and Formal Structures of Linguistic Meanings

Natural language meaning has properties of both (embodied) cognitive representations and formal/mathematical structures. But it is not clear how they actually relate to one another. This article argues that how properties of cognitive representations and formal/mathematical structures of natural language meaning can be united remains one of the puzzles in cognitive science. That is primarily because formal/mathematical structures of natural language meaning are abstract, logical, and truth-conditional properties, whereas cognitive/conceptual representations are embodied and grounded in sensory-motor systems. After reviewing the current progress, this work offers, in outline, the general formulations that show how these two different kinds of representations for semantic structures can (potentially) be unified and also proposes three desiderata for testing, in brain dynamics, the mathematical equivalence between formal symbolic representations (and their transitions), and neuronal population codes (and their transitions).