TRENDS in Cognitive Sciences
Vol.7 No.6 June 2003
What is a visual object?
Department of Psychology, Center for Cognitive Science, Rutgers University, New Brunswick, NJ 08903, USA
The concept of an ‘object’ plays a central role in cognitive science, particularly in vision, reasoning and conceptual development – but it has rarely been given a concrete formal deﬁnition. Here I argue that visual objects cannot be deﬁned according to simple physical properties but can instead be understood in terms of the hierarchical organization of visual scene interpretations. Within the tree describing such a hierarchical description, certain nodes make natural candidates as the ‘joints’ between objects, representing division points between parts of the image that cohere internally but do not perceptually group with one another. Thus each subtree hanging from such a node corresponds to a single perceived ‘object’. This formal deﬁnition accords with several intuitions about the way objects behave. Objects are everywhere in cognitive science. Objects are thought to be the building blocks of children’s conception of the physical world [1,2]; to delineate the boundaries respected by visual attention [3 – 5]; and to inﬂuence neural processing even at the earliest stages of visual cortex [6,7]. But what exactly is an object? In a phrase due to the American jurist Potter Stewart, ‘we know one when we see one’ – but what does the word actually mean? How do we know where one object ends and the next begins? The premise of this article is that this is not (as it were) an ‘objective’ question, but rather one that relates to how we mentally divide the world up into coherent units. That is, it is less about physics and more about the mental assumptions lurking behind the word ‘coherent’. The division of the world into objects seems so intuitive and effortless, at least under everyday conditions, that we speak about this division as if the world provided it overtly, without any contribution from our brains. But if cognitive science has shown anything, it has shown that what seems subjectively obvious is often the result of complex and subtle computations. The division of the world into objects is a case in point. The fallacy (philosophers would call it ‘Naive Realism’) is epitomized by Woody Allen’s tale of the Great Roe, a mythical beast with ‘the head of a lion, and the body of a lion, although not the same lion’ . If (and only if) it looks like an object, and quacks like an object, it’s an object. Perception dictates. In the same way that a ﬁst is something that a set of ﬁve ﬁngers turns into only when they are organized a particular way, objects are subsets of the world to which has been attached – by the perceiver – a particular kind of Corresponding author: Jacob Feldman (email@example.com).
subjective organization. And like a ﬁst, objects take on special signiﬁcance and deﬁnite properties only by virtue of this organization. Therefore, in seeking a deﬁnition of objects, I would argue, we need to focus not on how the world is structured, but rather on how our subjective perceptual interpretations are organized, and then ask how this kind of organization most naturally decomposes into object-like components. But what kind of organization turns inchoate visual ‘stuff ’ into a coherent object? Unfortunately, no simple answer to this question is to be found in the literature on perceptual grouping. First, no single grouping cue deﬁnes objects. ‘Real’ objects and object boundaries tend to obey a variety of nice properties – including closure [9,10], connectedness , convexity [12,13], good continuation in their contours [14 –18], regularity of shape [19,20], and so forth. But although each of these properties contributes to the perception of objects, none is, in and of itself, essential. Counterexamples can be found for each principle – perfectly good objects that are concave, have irregular...