Abstraction as Resistance to Racialized Surveillance: The Intersections of Art, Technology, and Identity in the Age of Computer Vision

Abstraction as a representational and cognitive apparatus has received a great deal of criticism in much critical theory and in a number of academic disciplines within the social sciences and humanities. Feminist and anti-racist scholars in particular have been at the forefront of the charges against abstraction for its tendency to evacuate lived specificities and the multiplicity of reality in favor of simplified and universalized models which are ostensibly objective and rational, but in practice have a tendency to privilege a white, male, western subject position.1 Theorists such as Philip Brian Harper are increasingly giving attention to the necessary and potentially recuperative aspects of abstraction. His book Abstractionist Aesthetics begins by detailing the ambivalent history of abstraction towards “black persons and populations,” often used as a method of marginalizing black people culturally as well as socially and politically through genericizing and commodifying them according to an abstract, reductive, and ultimately arbitrary calculus which presents itself as objective, logical, and infallible. This entails a translation of the embodied subject into informatics such as tallies and plots on grids for summation, a form of effacement that produces them as an object of exchange. The artistic reaction Harper identifies is one towards a figurative, referential, and realist aesthetic affect to invoke positive depiction of “blackness” per se within “African American expressive culture” which he defers to conventional notions to define as “works and practices that both originate among and in some way represent the experiences of African American people.”2 Harper by contrast advocates for a deliberate strategy of “abstractionist aesthetics” to both co-opt abstraction’s historically adverse relationship to black identity, as well as counter the realist empirical tendency which he views as stunting the polemic compelled by African American culture insofar as such cultural production adheres to the previously given definition and thereby “forces the perennially contestable question of how best to make a racial-political stand.”3 This agonistic rationale is solicited by the demands of such culture to elucidate the “racial-political” context in which African American experience occurs, a discursive horizon structurally unavailable to Realism as it merely brackets off the historic and ongoing hegemonic strategies of abstraction applied to the black subject rather than meaningfully engaging them. It is these considerations which cause Harper to cite Fred Wilson’s 2011 proposed work E Pluribus Unum as a paradigmatic example of abstractionist aesthetics’ distancing qualities through its “metarepresentational quality,” that is, its indexical representation of the signifying apparatus.4 Perhaps counterintuitively, the figural character of the work is key to its productively abstractionist quality for Harper, as it elicits critical questions about the contingency of paradigms of representation.

This emphasis on engaging the aesthetic sensibilities of an audience however is myopic if one’s aim is to trigger a reassessment of how identity formation is hegemonically shaped, and misses entirely the role surveillance plays in such formations, as well as the role abstraction plays in surveillance. As Simone Browne outlines in her book Dark Matters, the surveillance of blackness is overwritten with strategies of analytic fragmentation and deterritorialization, all with the aim of reifying the black body as a highly “readable” object. Citing Fanon, Browne highlights the advent of “digital epidermalization,” the imposition of race on a body; a process now made high-tech in the digital age but which she traces back to the practices of slave surveillance in the United States, including branding that made bodies literally legible. For her, such practices find their modern correlate in digital biometric information technologies which take aspects and parts of the human form and “reduce flesh to pure information.”5 An artist prescient to the contingent and abstractionist logic of digital biometrics and information processing systems in rendering images of humans readable, storable, and manipulable, is David Rokeby. Said to be an artist of systems rather than pictures,6 Rokeby’s works, especially those from the early 2000s, place the human subject within a system of analytical fragmentation using the apparatuses of mass surveillance. These include installations such as Seen (2002), Taken (2002), and Sorting Daemon (2003), all of which digitally capture and fragment the human form according to a reductive algorithmic logic. While these works are illustrative of the “metarepresentational” qualities of Harper’s abstractionist aesthetics, and the “critical biometric consciousness” of Browne’s interrogation of surveillance, only Sorting Daemon hints at a formalist framework which meaningfully both illustrates and serves as a model of disruption called for by Browne and Harper. Both authors privilege narrative and text to this end.

Rokeby’s Sorting Daemon and its function as a tool of biometric classification will serve as a common thread in this article to elucidate theories of surveillance and abstractionism in Browne and Harper’s texts, as all the tendencies described by these authors have become more diffuse, saturated, and intensified through the deployment of computer vision and technologies of surveillance. While Browne addresses the impact of computer vision directly, Harper’s account omits it entirely. Nonetheless, I see Rokeby’s work as bearing upon Harper’s schema, as computer vision presents a newfound permeability between image and text as subjects and objects alike are rendered as “data objects” to be collected, sorted, and circulated within a web of endlessly deferred meaning. Preoccupation with legibility of the body is the etiology diagnosed by Browne and Harper in their longue durée historical analyses of the management and depictions of blackness. While both authors reference visual artworks to develop their arguments, Harper’s advancement and problematization of narrative as the most promising arena for abstractionist aesthetics is not only in accordance with Browne’s own recognition of narrative as a fraught field for contesting or asserting personhood, it also recognizes the productive power of defamiliarizing strategies of syntactical fragmentation and non-linearity, strategies contained within the “heap.” To better connect Rokeby’s work with such strategies, I will be deploying Sianne Ngai’s theory of the “heap” as an aesthetic device to examine Rokeby’s work in relation to the aesthetic strategies espoused by Browne and Harper. Sorting Daemon will be used to elaborate the qualities of computer vision more broadly as it relates to the ongoing history of subject formation in which the archaic coalesces into the modern in a way disproportionally catastrophic to the black community. As a heuristic device, computer vision, or “operational images,” will, in their intended function, be analyzed as a device of linear logical ordering against which Sorting Daemon’s “heapness” stages a confrontation and serves as foil.

Harper’s enthusiasm for abstractionist aesthetics as potentially disruptive is informed by the theories of Berthold Brecht’s Verfremdungseffekt (alienation effect); however, he registers a “conundrum” inherent to African Americanist abstractionism that is indexical to the very troubled nature of abstraction’s historical relationship to the black population in the US. Artwork that engages the terrain of African American abstractionism must strike a dialectical balance between situating itself sufficiently within the lived reality of the racial order to register as “black” while also maintaining an acceptable degree of distance from that reality to be “productively abstractionist.”7 This is the source of tension that leads Harper to declare the contemporary visual arts as an “unreliable” avenue for a sustained abstractionist program.8

However, Harper further elucidates his understanding of African American expressive culture by later noting that he would rather emphasize it more so as a “practice” rather than an “institution.”9 Whereas the institutional framework assumes such cultural expression to be “distinct,” “coherent,” and “evidently unified,” a focus on practice implies mutability and process. This more clearly situates his choice of Brecht as a positive example, whose own practice is locateable within the orbit of the Frankfurt School of critical theory, which generally emphasizes art’s historical contingency and dialectical elements rather than any mystifying irreducible ontology inherent to art or identity. Brecht’s “alienation effect” is attractive to Harper precisely because it is not dependent on empathy, and understands social facts themselves to be historically contingent, unlike the more superficial register of “race-proud” realist art.10 As Harper puts it, Empathy, in the paradigm of Brechtian theatre, is a device which elicits “inertial passivity” in its audience by connoting a kind of immutability in the condition of its characters through how they are represented rather than calling for “active social engagement” by pulling the audience out of conventions of reception to understand the artificial and contingent character of oppressive social relations.11

The abstraction of bodies and construction of subjectivities to which both Browne and Harper provide a longue durée historical analysis is not only ongoing, but far more intensified in the present day through obscured technological means. The abstractive logic and racialized surveillance applied to blackness provide the practical and theoretic framework for the quantification required to turn people into commodities.12 As with reification broadly, this allows for the trafficking of humans as sign values and the personification of objects. After moving on from examples provided in so called “black Americana,” Harper describes a totalizing abstract rubric applied to both the black and white populations in the US to determine their place in political and social bodies whereby individual difference is evacuated in favor of a conception of classes of subjects who are differentiated from each other, but amongst themselves are homogenous and interchangeable. What relegated the black population to chattel slavery elevated the white population to political and economic enfranchisement.13 This process is echoed in the present day in terms of hiring practices, rental agreements, and the inequitable distribution of loans and criminal sentences, processes which are becoming increasingly automated using algorithms and artificial intelligence. Quoting surveillance scholars, Browne outlines the concept of the “surveillant assemblage” which “sees the observed human body ‘broken down by being abstracted from its territorial setting’ and then reassembled elsewhere (a credit reporting database, for example) to serve as virtual data doubles.”14 Her intervention, among others, is to add the concept of “racializing surveillance” to the discursive fields of surveillance and black representation in order to describe not just any “race inflected social situation” that sees the production of racial norms, but more precisely “moments when enactments of surveillance reify boundaries, borders, and bodies along racial lines.”15 Following the literal definition Harper provides of abstraction as “withdrawal from some originary point,”16 abstraction, which necessarily serves as the precondition for “abstractionism,”17 functions by a process of simultaneously genericizing an individual to a group, while at the same time establishing irreducible differences between groups—in the case of the US setting up one as foil to the other—all derived according to rational and as such, moral principles. For Harper, this has a visual corollary in the grid, which is instructive as to abstraction’s interlinking of cognitive models and formal tools which shape the world instead of being shaped by it. This marks real-world phenomena as both metaphor within the model and matter to be shaped by that model, bearing upon Browne’s theorization of blackness as both metaphor and lived material reality.18

Visualizing but invisible, algorithmic computer systems draw from and participate in that ongoing history of political subject formation through abstraction at a register which nullifies any sensible distinction between metaphor and lived material reality, or an abstractionist or realist aesthetic, that is, a register of abstract visual language that is withdrawn from the world, but also constitutive of it, that of concrete abstraction.19 Concrete abstraction is the principle that capitalist societies are overridden with abstract concepts that permeate quotidian reality as immaterial concept but manifest in direct material and consequential ways which makes them concrete. It is a dialogue between the two that produces social reality. Harper draws from the assumption that all artwork, even that functioning through a realist affect, is sine qua non operating at a distance from social reality via the intrinsic nature of all representation as representation.20 However, this position becomes problematized when one considers how that social totality itself is founded in abstraction.

In the introduction and first chapter of his book, “Black Personhood in the Maw of Abstraction,” Harper details how representations of black figures have been the site of polemics and fraught contestation over black personhood itself, hosting harmful caricatures as well as artistic strategies for denaturalizing and highlighting the contrived nature of those caricatures. Largely focusing on the silhouette wall art of Kara Walker which negotiates exactly that balance between the referential and fantastical, Harper makes the case that her recognizable forms are arguably more abstract than avowedly abstract modernist works which occupy a discursive space of material reality and conditions of display which Harper calls “objective facticity.”21 It is only from the perspective of “representational referentiality” that non-figural works become totally abstract. Yet in having clear ambitions beyond the realm of modernist self-referentiality, Walker implicates the contrived character of stereotypical caricature inherent to popular culture portrayals of African Americans, taking away from their givenness and subjecting them to the kind of productive interrogation called for in a Brechtian intervention. If nothing else, Walker’s tableaux demonstrate human capacity to “engineer both the forms of our aesthetic experience and the conditions of our social existence.”22 Yet this same abstractive logic which constructs caricature in the first place struggles to contend with its historical legacy of earnest and collective caricaturization of African Americans, often realized in degrading trinkets and marketing materials collectively labelled “black Americana.”

Harper locates “black Americana” within the same totalizing discourse of commodity relations which set in place the social-political abstraction faced by America’s black population as they were relegated to commodity slave status. This necessitated a kind of social technology capable of eliminating individual difference to better accommodate the interchangeability required of commodity status as imposed on an entire demographic in order to render them a “massified source of… labor.”23 Yet the massified source of enfranchised citizens in America was formulated in much the same mass application of an abstracting rubric. This totalizing abstract rubric was applied to both the black and white populations in the US to determine their place in political and social bodies; individual difference is evacuated in favor of a conception of classes of subjects who are rendered internally homogenous but externally heterogeneous according to reified boundaries. Such abstract reckoning was understood “as the very epitome of rational thought” by the class of “white, landowning male[s]” of the early American republic.24 Since this condition is situated within a larger analysis of the visual arts’ negotiation of abstraction broadly and abstractionism’s application to the human form specifically, a suitably rationalizing, organizing, and totalizing visual apparatus is located by Harper in the grid.

Brought up as a pervasive motif and criterion of the abstracting impulse, the grid is first discussed by Harper in relation to Benjamin Franklin’s procedural attempt at moral perfection as determined through selected virtues charted on a grid.25 In this case, it is the inverse of the profiling and categorization of risk into categories to anticipate potential dangerous behavior, intensified post-9/11 as Browne details.26 The grid serves as an abstracting device and visualizing perceptual apparatus that accompanies and accommodates this mode of thinking, that is of defining worth through tallying behavioral data points. Prescriptive rather than mimetic, the grid refers to nothing other than a principle of abstraction of which it is the criterion.27 It is used in the real world as an abstract, archetypal ideal against which human subject and landscape are judged. This recourse to a mode of geometric abstraction for articulating a subject and making value judgments about them, their capacities, and their belonging, has undergone a computerized intensification and saturation which broadly applies Harper’s abstractive process and formally applies Browne’s theory of racializing surveillance.

First articulated as such by the video artist Harun Farocki in his piece War at a Distance (2003), operational images are computer-generated images which function by reducing the world and the objects within it to component lines and curves or geometric shapes which are then compared by an algorithm to models stored in memory. Uniquely programmed to function in specific environments, they are produced solely by machines, for machines and as such have the capacity to “figure” the exclusion of all or specific human subjects. A certain compatibility can be drawn with Harper’s location of the grid as a formal device with intrinsic social consequences. In establishing a “noble genericism” linked with white masculinity as per Franklin, the grid thereby excluded non-white males from the polity insofar as they were unwilling or unable to accommodate the grid through the self-effacement of their own particulars.28 Indeed, the voice-over narrator in Farocki’s video announces that “war in the electronic age presents itself as being an event free of people. It takes no account of people, though they may still be involved.” Operational images exist in all sites of enclosure, from quality control and manufacturing to self-driving cars, cruise missiles, and facial recognition and surveillance technology, yet are perceptually invisible to humans. Their widespread and rapid proliferation has made them the largest source of image production today.29 They are a kind of digitized equivalent of the platonic ideal of abstract personhood Harper cites, which has its own “graphic emblem” in the grid.30 Harper notes this formation of a geometrically and as such rationally determined universal ideal subject can be read as an “update” of a classical European tendency which simultaneously and paradoxically establishes the ideal human form while genericizing it.31 Such abstract personhood has a number of normative effects which Harper illustrates in reference to the Vitruvian Man, which is “effectively an avatar of the white-masculine form itself, whose own supposed generic-representative character has made it the archetypal image of everything from the citizen-soldier to the prospective consumer.”32 Operational images and the algorithms that employ them similarly determine ideal subjects from a functionalist if not pseudo-Platonic point of view.33 These can range from prospective consumers to suspicious persons and even terrorists.

The point of Farocki’s video investigating operational imagery is to demonstrate a common aesthetic framework between sites of production and destruction, but in a sense it also empirically demonstrates Harper’s argument that economic relations serve as the locus for the abstracting impulse made on the human form. This project of a universalizing subsumption of particulars is the other element of operational images as an aesthetic strategy for recognizing subjects through physical resemblance and even patterns of movement. Although the Vitruvian Man was the search for an ideal, however rarefied, new archetypal figures are the outcome of a pragmatic search for the statistical median, however rarefied. While the former model was proactive, the latter is reactive, allowing the modellers to better plot members of existing populations into the structure of significations. The legitimacy of both is underwritten through the conceit of rational derivation, both being determined through geometric proofs. Harper claims that the Vitruvian Man gives objectifying form to the concept of the statistical majority and sets a standard for personhood against which the rest of the members of society are evaluated. This process is far more actualized in Woody Bledsoe’s “Standard Head,” which was rendered by the AI researcher as part of his larger CIA supported project beginning in the early 1960s to develop the first computerized facial recognition technology.34 Due at least in part to technical limitations of the time, Bledsoe made a composite virtual head with a white male appearance composed of the median average features of a sample size of just seven people to create a universal model that would stand in for all of humanity as far as rule based AI programs were concerned.35 Rather than being relegated to past naiveté, as could be inferred from Harper’s account, this optimization of personhood has only undergone an intensification both in terms of its application and in terms of its reliance on rational principles and the conceit of high technology.

An increasing number of artists have worked through the logic of these systems and the formal principles of operational imagery in recent decades. David Rokeby is one who has sustained an early engagement with the discursive framework presented by Harper and recognition technology. Particularly of interest is Sorting Daemon, a kind of surveillance system as an installation piece originally assembled in 2003. Utilizing a surveillance camera that tracks passersby on the streets outside the gallery window, the artist establishes a system which locates the form of what appears to be a person from the background for further scrutiny. It then separates the identified forms from the frame and breaks them down into two or three component color blocks determined by any exposed skin and articles of clothing. These are quickly arranged on a monitor into a collage of matching color blocks from other human forms which have likewise been broken apart and ordered by size and hue, coming to form an amalgam. Rokeby is a Canadian computer and cybernetic artist who began in the 1980s to take an interest “in ways that the computer and computer technology were challenging the body.”36 Although his work takes the form of room-sized installations, his practice can’t really be described as solely or even primarily installation art; the focus is more located in the procedural and recursive logic of algorithmic systems. When he develops a new art project, Rokeby describes how he is taking on a lot more than just that piece, but a whole state of mind of an artificial intelligence researcher: a state of mind which is one of formal, unambiguous problem solving. He seeks to find a means to subsume the particulars of the real world into an abstract universal. This mode of thinking resonates with Harper’s grid concept, a normative apparatus aiding in the formation of what is understood “as the very epitome of rational thought.”37 It’s also a mode of thinking still embedded in the surveillance apparatus used by the state.

By 2012, the NYPD had access to IBM software which automatically tags video frames containing human figures and stores them within a database that is searchable according to bodily attributes such as clothing type and color, but also skin color. The NYPD claims they never used the skin color search function and discontinued their partnership with IBM in 2016.38 Computer vision is, fundamentally, a technology which compels the production of identity without the participation of the subject. Operational images are constructed on the basis of rote detection and the comparison of pixel light values which determine the plotting and trajectory of their lines. The system does not a priori understand from what element a pixel is or the nature of its object. Such technology is irreducibly racializing and fragmenting. An immense effort would be required to teach an algorithm to not make distinctions based on protected aspects of a person’s identity, or not to perform what Fanon, as quoted by Browne from Black Skin, White Masks, experiences “where the white gaze fixes him as an object among objects and, he says, ‘the white gaze, the only valid one, is already dissecting me,’”39 a strikingly literal allusion from the 1950s to the operative process of biometric surveillance. Sorting Daemon vividly visualizes this process. Through décollage and collage, Rokeby’s work performs the analytical process of dissection, a kind of negating violence which assumes an object’s linear reduction will provide insight into its inner character. This analytical consumption resonates with Browne’s concept of “digital epidermalization,” when bodily identification is made biometrically through digitized code as a “means through which the body, or more specifically parts, pieces, and, increasingly, performances of the body are mathematically coded as data, making for unique templates for computers to then sort by relying on a searchable database.”40

In a presentation of his work at UCLA, Rokeby notes the critique that interactive art is impossible without surveillance. There is, however, more than that going on in the systems Rokeby designs, which evoke the increasingly ubiquitous conditions of surveillance and algorithmic organization. Rokeby takes issue with the narrowly inscribed parameters and assumptions of how interactivity is staged in conventional fine arts settings which, as he puts it, stage a kind of formulaic user-as-god who receives a direct and immediate response from an artwork as if from a mirror.41

            Browne cites John Fiske’s examination of video surveillance as he describes how its “penetration is differential” in regards to demographics in modern society. He notes that behavior by black individuals is far more likely to be coded as suspicious than behavior engaged in by whites, concluding “today’s seeing eye is white.”42 Considering the normative techniques in developing facial recognition technology, it is perhaps not surprising that such biases have become literally encoded and empirically verifiable. 28 NFL players were misidentified as felons by Amazon’s “Rekognition” algorithm in a 2019 test.  The same year, the National Institute of Standards and Technology tested over 50 facial recognition systems in development and found they were less likely to falsely match white males with mugshots.43 The conceit of such technology is that it is presented as disembodied, detached, and therefore unbiased, recalling for Browne Donna Haraway’s articulation of a “‘conquering gaze from nowhere,’ a gaze that is always unmarked, and therefore already markedly white and male, and one that claims a power to ‘represent while escaping representation.’”44

For Browne, the desire to see without being seen is the distinguishing feature of the panopticon. Browne acknowledges the “overreliance” on the panopticon “as an explanatory metaphor” and offers her own critical reinterpretation through a survey of existing scholarship involving and reevaluating its legacy, including its myopic misapplication.45  She acknowledges Foucault as having the greatest influence on panopticon discourse, but points out “that hierarchical observation and normalizing judgment combine in the examination.” The examination perhaps represents a greater fidelity to the abstracting rubrics and the techniques of operational images and digital biometrics than the panopticon. Both the panopticon and the examination have an outcome of producing homogenizing effects of power, but with very different aims: the panopticon purports to correct behavior, to instill “a real subjection . . . born mechanically from a fictitious relation,”46 while the examination merely maintains categories of official difference with no real aim to alter behavior. As Foucault puts it:

The examination combines the techniques of an observing hierarchy and those of a normalizing judgement. It is a normalizing gaze, a surveillance that makes it possible to qualify, to classify and to punish. It establishes over individuals a visibility through which one differentiates them and judges them. That is why, in all the mechanisms of discipline, the examination is highly ritualized. In it are combined the ceremony of power and the form of the experiment, the deployment of force and the establishment of truth. At the heart of the procedures of discipline, it manifests the subjection of those who are perceived as objects and the objectification of those who are subjected. The superimposition of the power relations and knowledge relations assumes in the examination all its visible brilliance.47

The examination remains a prominent epistemic model throughout the history of surveillance, beginning with Foucault’s identification of a modern “disciplinary society” which polices the body through the control of spaces. Yet it becomes more essential to the digital information flows and computerized surveillance characteristic of postmodern “control society,”48 which instead polices the body through the control of information it generates, including through examinations. Browne further contextualizes that “(t)he examination places the individual in a ‘network of writing,’” a more fundamental Foucaldian device under which the exam is subsumed.49 A network of writing emphasizes the structural relations of meaning and power, and situates writing’s use as a tool of surveillance and control, as texts become historically and institutionally embedded, and meaning policed by a web of disciplines and practices. It also emphasizes writing’s diffuse character, able to circulate in a way spaces and bodies cannot, and it is in both these senses that Sorting Daemon specifically, and operational images generally, can be located as belonging to such a system. A tool of communication and control, writing as it is linked to power relations constructs subjects always already embedded within a discourse. For Browne, the network of writing’s impact on the construction of slaves as commodities is substantiated through shipping manifests and branding.  Echoing Harper’s abstracting rubric, Browne uses Foucault’s model of the disciplinary society and its technique of producing massified individuals through the exam “as it is ‘accompanied at the same time by a system of intense registration and of documentary accumulation.’” While Foucault does not theorize trauma per se, Browne finds his articulation of the body as social technology useful for demonstrating “how acts of making the black body legible as property were put to work in the production of the slave as vendable object to be bought, sold, and traded.”50 Complementing Harper, who posits “the regime of property” as the “locus of black people’s deleterious social-political abstraction,”51 Browne’s focus on legibility supplies the praxis by which the theorization of abstraction is realized according to the impulse supplied by economic relations. 

Legibility is preceded by visibility, however, and an archaic example provided by Browne of a surveillance apparatus that is at once as illuminating as it is racializing is that of the colonial era lantern laws.52 These laws mandated that every slave or small group of slaves out after dark were to carry lanterns with them for the sake of quick identification and tracking of movement. Drawing connections between these historical antecedents and modern technological methods of matching, sorting, and filtering humans is the core of Browne’s intervention of “critical biometrics.”53 This term is coined in recognition of her perspective that biometric technology has a long history irreducibly connected to technologies of slavery.54 Lanterns and branding were part of a scopic regime designed to make blackness legible as property, and whiteness as “prototypical.”55 These are the preconditions for “epidermalization” which is “literally the inscription of race on the skin.”56 Browne frames epidermalization as “the disassociation between the black ‘body and the world’ that sees this body denied its specificity, dissected, fixed, imprisoned by the white gaze” and that is “deafened” by a host of stereotypes and caricatures.57 Again, “inscription” here is the key operative phrase for how the hegemonic abstractionism Harper identifies as responsible for caricature and massification is applied in a way that implies writing and legibility.

            Digital epidermalization is not only the digitization of epidermalization, but is characterized by Browne specifically as “the exercise of power cast by the disembodied gaze of certain surveillance technologies.”58 Instead of literally branded bodies, today this takes the form of people’s transformation into data objects or “data doubles” with mutable and immutable qualities, to be trafficked in networks and databases as the commodity form of the information age. Constructed based on observable features and modes of interaction within webs of signification, these data doubles are host to rote lists of defining characteristics both demographic and consumptive. This is how the analytic fragmentation of surveillance and marketing systems can be recalled, demonstrating a newfound fluidity, or even porosity, between text and image. Image classifiers on social media platforms perform this function as well, developing lists of elements and qualities detected in an image. Contained within these lists is an implied narrative that allows for increasingly nuanced automated categorization of the viewers, uploaders, and subjects of these images.

My own intervention is to posit that, just as “digital epidermalization” has a functional legacy in common with the practice of branding, image classifiers which produce an implied narrative through their descriptive acts have their own functional legacy with other historic examples of racialized surveillance Browne provides in the forms of advertisements for the capture of runaway slaves, and the Book of Negroes. Advertisements would give simple lists of physical descriptors of the runaway, such as the type and color of their clothes, and the location of any conspicuous scars or brands. These lead Browne herself to speculate on the implied narrative of some of these escaped slaves based on their descriptions, noting how their brands may have jeopardized escaped slaves, and wondering if they were able to escape, as further ads and documentation seem to imply.59 Much like image classifiers also purport to be able to detect emotions, with some AI researchers even claiming their facial recognition models can detect a propensity towards queerness or criminal behavior, some of these ads also listed aspects of character. For instance, one of them described the runaway as a “sly artful fellow.”60 Those compiling the Book of Negroes, an 18th century ledger, sought to determine which self-emancipated slaves would be allowed to leave New York as part of the post-revolutionary British evacuation and which ones would be returned to their American owners. Contained in the ledger are “crude annotations” which mechanically link attributes of gender, race, labor, disability, and apparent marks or character traits.61 Browne describes the ledger as follows: “With its crude inscriptions, such as ‘scar in his forehead’ and ‘stout with 3 scars in each cheek,’ the Book of Negroes is an early imprint of how the body comes to be understood as a means of identification and tracking by the state.”62 However, what is of even more keen interest for the purposes of this article are the public arbitrations to decide who would be included in the ledger and given passage. These arbitrations were held at Fraunces Tavern, a makeshift court, and are characterized by Browne as the sites of “decommodificatory narrative acts.”63 Browne elaborates later: “The making of the Book of Negroes was a historical moment where, particularly in the arbitration hearings held at Fraunces Tavern, black women, men, and children found creative ways to narrate their own freedom within a system that sought for them to remain unfree.”64 This is a striking characterization, and recalls the Verfremdungseffekt Harper championed as a productive mode for reevaluating the abstracted subjectivities imposed upon black people. Browne sees The Book of Negroes as a call to “be alert to the occasions when racialized subjects not only step into the recognitions given to them by others but provide intuitions of a future in which relations of subjugation will (could) be transformed,” recognizing narration as a catalyst for racialized subjects considering the “creative, contested, contradictory and laborious work of constructing racial identities in narrative acts.”65

Operational images are the site at which the implied narrative of text descriptors are made semantically opaque but syntactically transparent in their endless deferral of meaning and equivalence in geometries and number scores. It is the site at which racialization or epidermalization constitutes an incontestable search for an unambiguous empirical fact. Browne charts how digital epidermalization has actually been an explicit goal, motivating research in facial recognition technology with mixed results that (re)produce “pseudo-scientific racist and sexist discourse that sought to define racial and gendered categories and order humans in a linear fashion to regulate those artificial boundaries that could never be fully maintained.”66 This avenue of research was recognized by Bledsoe himself, who in 1965 wrote a proposal to ARPA requesting support in studying the possibility of using facial measurements to determine racial background, even citing the collective effort of anthropologists gathering measurements from people all over the world, whose work had yet to be “properly exploited.”67 Nonetheless, Bledsoe’s methods for making the human face legible to a computer closely follow those techniques pioneered in 1879 by Alphonse Bertillon, the French criminologist and inventor of the mugshot who believed everyone could be uniquely identified through enough relative measurements of their body, such as the length from their elbow to middle finger.68 Browne cites researchers of race-based facial recognition technology as stating, “the difference of Races is obvious, and it is the core field of research of anthropology. Anthropometry is a key technique to find out this difference and abstract the regulation from this difference.”69 This is highly reminiscent of Harper’s definition of abstraction as “withdrawal” and in this sense certain procedural, functional, even formal commonalities can be analogously drawn between these principles, modernist abstraction, and operational images.

Such a conceptual affinity between principles of geometrically reductive figuration and modernist abstraction is noted by Harper in his discussion of Mondrian’s figural obliteration via refinement through the grid, steadily breaking down forms, assimilating, cancelling, and annihilating the figure.70 While Mondrian did not see the grid as wholly detached from nature in terms of process and form, Harper connects Mondrian’s use of it to social reality with its use as a device of planning, prescription, and assimilation, evacuating specificity and privileging its plotter’s agency through its own form as content. Like the grid which produces the thing it orders, operational images and Rokeby’s work structurally function within a network of writing, by analytically breaking down, abstracting, reorganizing, and circulating their objects’ data as the object itself. While Sorting Daemon is an artwork, operational images are not works of art per se, but still take advantage of art’s powers of aestheticizing strangeness and formalist autonomy. Although in this case indexically tied to the human body, and in a sense tracking onto and representing it, they cannot be said to be existing at a distance from the social reality they reflect, as Harper charges of all representational artwork under the rubric of “abstractionism.”71 As defamiliarizing and unnaturalistic as Rokeby’s depiction of the human form is, very much “calling attention to its constructed or artificial character,” as Harper claims of “abstractionist” work, it does not emphasize its distance from reality, but instead exposes a spectrum of representation wherein a clear binary distinction between realistic and abstract cannot be sustained. Harper challenges this binary, claiming that all visual art is necessarily bound up in abstractionism insofar as it is irreducibly a representation at distance from social reality, but operational images, as demonstrated by Rokeby’s work, show how realism and abstractionism are bound up in each other in composing social reality. In this sense, calling attention to the constructed character of reductive figuration using an imaging technique of socialized technology in fact closes that distance to social reality. As Rokeby himself puts it:

I am fascinated by the way we transform the raw impressions streaming in through our senses into a coherent mental picture of reality. So I create artworks that look and listen, and try to make sense of what they see and hear. I am caught in the daily clash between the logical world of the computer and the embodied experience of living. So I bring these two worlds into closer dialogue to see what fails and what resolves.72

Harper highlights this clash between an imposed logic and embodied reality through the examples he provides in Vanessa Beecroft’s art practice of posing hyper-racialized subjects standing in a grid as tableaux. As the models tire, their grid formation undergoes entropy as they begin to sit or fall out of formation, highlighting the inherent incommensurability of prescriptive aesthetic elegance with the reality of bodies in the world. A diachronic force immanent to systems including living beings, entropy is one factor which frustrates synchronic structural elegance so often the product of logical ordering. It is this disappointment wrought by the disjuncture between the procedural logic of abstractionist planning and the obstinate friction of reality that characterizes the shortcomings of computer vision.          

Rokeby’s work follows that model of art production as a form of exposure of internal principles to demonstrate how an aesthetic effect is achieved: a technique politicized by Brecht’s Verfremdungseffekt and an instrument with emancipatory potential for abstractionist aesthetics as described by Harper. However, whereas Shklovsky and Brecht’s principle of making the methods of production visible “adds” pleasure and aesthetic value, Ngai insists that the same technique in the gimmick “directly detracts from both our enjoyment and esteem.”73 Ngai formulates the gimmick as a convention embedded in capitalist labor relations and host to an “operational aesthetic.”74 While revealing a device’s methods of production normally leads to satisfaction on the part of the viewer, the gimmick frustrates because its seductive facade of reducing labor and expanding value becomes unmasked when its procedural operations are revealed.75 Ngai’s elaboration of the gimmick becomes highly instructive when applied to operational images and computer vision as the gap suffered between ambition and reality. For Ngai, this detraction hinges on the gimmick’s duplicitous nature around its associations with labor, while for Harper, the shortcoming of such a distancing effect is due to the negative baggage of racial iconicity which cannot be reliably separated by the audience of visual art. The nature of both explanations has nothing to do with the “here and now” experience of the aesthetic object but instead relies on a “there and then” which has already occurred, and comes from a different order of abstraction: that of language. The linguistic sign in abstract works of art is usually present in the title, which helps to locate the work’s content and ambition but is incommensurable with the work’s iconic or indexical nature.

As much is noted by Harper, who recalls the outrage caused by Duchamp’s 1912 Nude Descending a Staircase No. 2, which refuses an easy “reading” despite the promise of its title.76 Yet as he points out, contemporary audiences have had over a century of accumulated training in reading and, as such, have grown accustomed to abstract compositions, obviating any potential for the defamiliarizing technique to break through and cause a rupture. Add to this the troubled history that has existed between black populations and abstraction per se, resistance against which negates any potential for visual abstractionist aesthetics to sustain social critique as the forms are interpreted in a realist register.77 It is for these reasons that Harper characterizes visual art as an unreliable avenue for a productively critical abstractionist effect, and instead forwards narrative, that is, any storytelling device whose operative logic is “succession and causality,” as being the most promising avenue for eliciting abstractionist alienation.78 This is because of the linear unfolding of most syntax, a convention of representation that is not so saturated and overdetermined with defamiliarization or abstractionism as is visual art, since it is tied to the operative logic of language itself. Operational images are nothing if not an expression of a linear syntax, millions of lines of code which guide a procedural logic seeking affirmation through correspondence. The linguistic presence in Rokeby’s work is in the procedural aesthetic of operational imagery. Programming language evades defamiliarization because it is already so unfamiliar, and does not allow for any change from its conventional syntax since the entire architecture of the program will simply shut down, making it incapable of communicating anything, aesthetic or otherwise. Yet Rokeby’s piece does not have a linear output, nor does it organize the components of subjects it captures according to the format of the database; rather it displays them in a simultaneous all-over presentation format, a simply organized “pile” made according to very rudimentary associations. By any measure such a presentation, as well as the procedure to achieve it, are defamiliarizing to witness in spite of operational imagery’s increasing constitution of the quotidian. Rokeby forwards an aesthetic strategy identified and articulated by Ngai that sustains a defamiliarization which forces its audience to think in new ways, that of “the heap.”

The heap is an aesthetic mechanism identified by Ngai which, rather than following an operational or procedural aesthetic unveiling, defines itself through simultaneity or an all-at-once presentation.79 The heap can exist as a literary, visual, or conceptual instrument, but it proffers no affect towards saving labor, instead frustrating simple coherence conducive to satisfactory reading in favor of enervating the viewer or reader, thereby eliciting a state of “stuplimity,” an affective state which combines boredom with astonishment.80 Evoking the “sublime” only to differentiate itself, stuplimity does not allow either affective state to supersede the other, but holds them in tension. Ngai’s examples of the heap in the visual arts are instructive, and consistently evoke Sorting Daemon’s imagery, especially in reference to a strategy of “agglutination” defined as “the mass adhesion or coagulation of data particles or signifying units.”81 The tedium of the process is sustained by a “stupendous proliferation of discrete quanta held together by a fairly simple syntax or organizing principle.”82 This corresponds to a “logic, less mosaic than congealaic,” which for Ngai recalls the artist Ann Hamilton’s installations consisting of thousands of teeth or pennies ordered by such a nondescript logic, leading to an “accumulation of visual ‘data’” which frustrates any observer’s efforts to read or “metabolize” information.

Perhaps the most pertinent example Ngai cites of artwork operating within the discourse of the heap is another work based on the procedural and recursive logic of the computer: Janet Zweig’s Recursive Apology (1993), a computer network and printer installation wherein a program installed amongst four computers would print out a stream of randomly generated apologies “in the smallest possible type.”83 Although Sorting Daemon produces no physical output, there are some telling concurrences between the two works, demonstrating the unique aptitude of computer algorithms to produce the heap as an aesthetic device. Printing continuously for two weeks straight, no two sheets are the same in Recursive Apology because of the randomized function of the algorithm which, much like that in Sorting Daemon, relies on inscrutable but logical selections for its composition. Sorting Daemon externalizes this selection into the world but is nonetheless reliant on its programming language rather than a sensual body moving through that world in order to make sense of it. Ngai cites Recursive Apology as calling “attention to language as the site where subject and system intersect” while also commenting on the “feminization of apologetic speech acts” as it stages “the convergence of gendered subject and machine.”84 Sorting Daemon stages its own convergence of the racial subject and machine, and even though it does not output a traditionally readable linear text, it is just as bound in that same intersection of subject and language, as its display is nothing other than the expression of its own programmed parameters as syntactic language. The language Ngai uses to evoke commonalities between Zweig and Gertrude Stein’s The Making of Americans is strikingly literal to the workings of Sorting Daemon:

Stein similarly demonstrates through her own vast combinatory of human types—a text in which new ‘kinds’ or models of humans are made through the rhetorically staged acts of enumerating, ‘grouping,’ ‘mixing,’ and above all repeating. For both Stein and Zweig, where system and subject converge is more specifically where language piles up and becomes ‘dense.’85

Having much in common with the conceit of surveillance apparatuses and abstraction itself as expounded by Browne and Harper, Stein’s The Making of Americans is concerned with a procedural taxonomy for the construction of human “kinds” or types of subjectivities.86 What her work has in common with Ngai’s cited art installations is that they all operate as information processing systems “classifying and ordering seemingly banal bits of stuff . . . To encounter the vastness of Stein’s system is to encounter the vast combinatory of language, where particulars ‘thicken’ to produce new individualities.”87 It could be said that while particulars “thicken” in Sorting Daemon, the final resolution of “new individualities” never arrives and instead remains in a sustained congealed mass which seems to deny any available coherence. Yet for Stein, “coherence” is located more in the “process” of creating form, with an emphasis on indeterminate possibility, potentially generative of “unforeseen kinds” which may spring forth from “a vast combinatory, in which new ‘consistencies’ are produced through the ‘mixing’ of others.”88 Further recalling Sorting Daemon, Ngai details how, “For Stein, the work of ‘telling’ or ‘making’ history is inseparable from the labor of making subjects (‘kinds of men and women’), which itself entails the tedious labor of enumerating, differentiating, describing, dividing and sorting, and mixing within the chosen limits of a particular system.”89

That this indeterminacy exists in tandem with an organizing form such as the pile is characteristic of the heap. The heap is neither prescriptive nor mimetic, and as such is categorically other to either the grid or figural portrayal. While operational imagery already undermines the integrity of the distinction between realism and abstractionism through its practical determinacy, the heap evades the binary entirely in its very indeterminacy or “heapness.” Ngai details how stupefaction’s “negative experience” is born of the alienating effect of encounter with “the different,” raising the question of how one responds “before a value has been assigned to it or before it becomes qualified—as ‘sexual’ or ‘racial’ difference, for instance. We are used to encountering and recognizing differences assigned concepts or values.”90 She qualifies further: “Stein’s writing asks us to ask how we negotiate our encounters with difference when these qualifications have not yet been made.”91 It is this sustained indeterminacy in language and concept that motivates Ngai’s selection of examples in art and literature, as well as Rokeby’s art practice.

            Although he never articulates it as such, it is these heap-like qualities which prompt Harper to reference Stein as a positive example for espousing “literary abstractionism’s potential for enacting African Americanist Critique.”92 As Harper argues, narrative defaults as a realist format insofar as “consequentiality is . . . the defining characteristic of narrative as such,” meaning narrative comprehensively integrates “both succession and causality” in a way familiar to our daily experience or our reporting of it.93 This is reflected in a linear syntax and a relatively stable semantics, both of which are frustrated in Stein’s works. The 1906 short story Melanctha features a turn of phrase: “covering over” which, through its repetition in detailing the struggles of the eponymous protagonist, comes to “signif[y] four different things: the (unachieved) hiding of vice from public view; forgiveness of an acquaintance’s offenses; repudiation of one’s erstwhile companions and social activities; and the frustration of one’s potential for accomplishment.”94 This constant need for reevaluation on the reader’s part not only frustrates “the narrative’s syntactical progress, it short-circuits the narration’s semantic coherence.”95 This stands in contrast to the stable linear coherence required by computers to make sense of the world through images and the implied narrative found in lists of terms they generate. These are reminiscent of the highly formalist convention of using lists as an unelaborated implied narrative found in books such as The Things They Carried, The Lovely Bones, and House of Leaves. Another example cited by Harper which frustrates comprehension of succession and causality is the novel Push by Sapphire which served as the source material for the 2009 film Precious. Although he describes both, Harper emphasizes the novel’s unique difficulty over the film in communicating a chronological sequence from the protagonist’s flash backs of major events in her life, doing so in a way evocative of an all-at-once, heap-like presentation where different dates and events move through, behind, and into each other, which Harper finds more conducive to an abstractionist aesthetics.

Ngai’s descriptions of the heap as a literary device are also quite telling. In the cited work of Nathanael West and Edgar Allen Poe, she describes in their staged scenes of “analytic stupor” a degree of superimposition where elements do not follow a linear order but appear behind each other or simultaneously. In a very literal way this can cause a problem for operational imagery, as their algorithms are programmed to recognize discrete forms, not a multiplicity of every possible combination of forms. Such was the case in a high profile fatal 2019 car crash involving a self-driving Uber. The Uber struck and killed a woman crossing the street as she was walking her bicycle. While the NTSB and news reports indicate the ultimate cause of the crash to be the AI’s inability to recognize a pedestrian who is not crossing at a designated crosswalk, also mentioned was the AI’s classificatory confusion. While the AI was programmed to recognize cyclists or pedestrians, a woman walking a bicycle caused it to switch its interpretation to "vehicle, bicycle, and an other" before hitting her almost full speed.96 In a very real way, the superimposition of forms as described in the heap, lead to an analogous state of “stupefaction” in the onboard AI.

Ngai characterizes Recursive Apology’s output as “a surprisingly ‘flat’ or boring display,” due to the disappointment wrought by betrayed expectations of a “fashionable cyborg figure.”97 Sorting Daemon is similarly clinical, and its output could certainly be characterized as “flat” or even “boring,” but what goes otherwise unarticulated by Ngai is her own expectations of computer art, which again much like the negative interpretations of Walker’s work as advancing negative black stereotypes, has nothing to do with the “here and now” experience of an art object, but is bound up in its own separate discourse of trained expectations through exposure. This is related to that waning ability of abstraction to defamiliarize, as cited by Harper in reference to Duchamp’s Nude Descending a Staircase. This same work was taken up by Rokeby in his piece Cheap Imitation (2002), which translates each shard comprising the figure into a line of code, animating the figure when a viewer is detected in order to create a “kaleidoscope effect of cascading movement.”98 The display and movement of the figure is entirely contingent on the presence and movement of a viewer. Rokeby seems prescient to operational images as being the defining aesthetic of the digital age, and seeks to explore the “ambiguous and irresolvable” relation between humanity and the apparent logic of computers he characterizes as “the absolutely arbitrary space of binary information.”99 Binary logic, which is the language of the computer, can only determine truth defined through correspondence and is categorically incapable of accommodating an openness to truth as disclosure via ambiguous presence in the world.

“Dark sousveillance”is a concept introduced by Browne as a foil to racializing surveillance that resists overdetermined existence in the world through technologies of correspondence; critically interrogating, challenging, co-opting, and repurposing technologies of surveillance.100 This could include singing as a method to warn or pass on information about escape routes, or setting up traps for slave catchers. However, it is also generative, “plot[ing] imaginaries that are oppositional and that are hopeful for another way of being.”101 This comes across as strikingly reminiscent of the heap and Rokeby’s working method, with Browne herself acknowledging that acts of “dark sousveillance are not strictly enacted by those who fall under the category of blackness.”102 The qualities of the heap are made even more explicit in a specific form of resistance Browne cites known as “Computer Vision (CV) Dazzle.103

CV Dazzle is a form of camouflage against recognition algorithms using hair and makeup pioneered by Adam Harvey. Taking inspiration from Dazzle camouflage, it fragments the form in order to stupefy any observer by presenting the wearer or wearers as an undifferentiated mass or heap, as seen with zebra herds. Harvey is insistent on CV Dazzle being a process or set of principles rather than any specific fixed pattern, as technologies are constantly adapting.104 Newer forms of camouflage have had to contend with machine learning algorithms which seek to understand the semantic context of images, leading to new forms of semantic play recalling synthetic cubism. Cap_able is an Italian clothing manufacturer whose line tricks the system into recognizing dogs, zebras, giraffes, or small knitted people inside the fabric through its own “heap-like” all-at-once presentation of congealed fragments which render the wearer as background noise.105 A proliferation of forms of camouflage are being developed and tested by computer science departments, with many different techniques used, but a common approach is to print out and wear an “adversarial patch” which consists of ambiguous, massified forms whose presentation bears a resemblance to the congealed forms of Sorting Daemon’s mosaic.106

That the formal principles of disrupting legibility which exist in art and literature as theorized by Ngai have the potential to disrupt a purely formal digitized translation of the same hegemonic narrative tools of quantifying and sorting human beings is profound. Even having undergone technological evolution, much of these mechanisms of control have revealed themselves to only ever have been a Mechanical Turk. The engineers of these systems have walked themselves into an ontological corner by seeking stability of meaning and control in a system whose actual transcendental signified is not the digitized eidetic model of the subject, however racialized and produced, abstracted from yet corresponding to the real world, but language itself. The disembodied gaze is not only visible, but made legible through such acts of sousveillance and abstractionist aesthetics. Operational images are the visualizing strategy employed by computers, but the images shown on Sorting Daemon’s screen, or any other, are not in themselves operational images but merely representations of the process of calculating and ordering being performed by the computer. Making the process presentable to humans is external to the functioning of the system which could just as easily represent its determinations and decision-making process through text as demonstrated in image classifiers. Operational images are part of a system of language without an ontology necessary to evolve that system through meaningful embodied utterances, and as such, will always be susceptible to forms of syntactic and semantic countersurveillance deployed by embodied subjects.

Justin Barski holds a MA in Critical and Curatorial Studies from the University of British Columbia. He is currently a PhD student in the Visual and Cultural Studies program at the University of Rochester. His research spans modern and contemporary art from the mid-nineteenth century to the present day, with an emphasis on art of the avant-garde and machine art.