Carnegie, T. A. (2009). Interface as Exordium: The rhetoric of interactivity. Computers and Composition, 26(3), 164-173.
Where Lemay’s “Pattern Language” (2007) took a development-centered approach to understanding component frameworks for interactivity in rhetorical media, Carnegie’s “Interface as Exordium” (2009) instead creates a more broadly cultural and reception-centered answer to the question of how interfaces and interactivity function to engage audiences.
Carnegie separates the rhetorical modes of interfaces into three categories–multi-directionality, manipulability, and presence. I will briefly address each of these modes in a moment, but first want to touch on the ideas and definitions Carnegie is building from in order to contextualize the significance of these modes.
I – What it means to view interface as rhetorical
For Carnegie, the interface is not only literal, but also figurative and symbolic. The interface, she argues, is capable of being as rudimentary as the physical page of a book, or any point of interactive contact between two objects, two subjects, or a subject and a object–“the interface is a place of interaction, whether the interactions are between user and computer, user and software, computer and software, user and content, software and content, user and culture, and the user and other users” (165). From this rudimentary model, she turns back to computer-mediated communication and argues that we have a traditional sense of the interface as including the “physical arrangements and ergonomic configuration of computer systems, user operation of programs, and how the user interacts with the content to solve a task or to learn material” (Marra, 1996, p. 115 as cited in Carnegie, 165).
The core feature of these various definitions, then, is their inseparability, that there is no singular “layer” of interface which exists between user-as-whole and computer-as-whole. This lends a rhetorical interpretation significant power, as the layers of interfacing “function rhetorically by creating interactivity. In other words, the modes of interactivity are the rhetorical modes of the interface” (166, emphasis added). In fostering an especially active relationship between audience and content (thought this might be debated–any book wonk will tell you that they prefer books over computers for reading in part due to the added active relationship between the material and the human user over the digital), “new media requires action” that “involves and engages the user in using, playing, exploring, experimenting, discovering, and sharing” which effectively creates bodily and spatial rhetorics, while increased engagement produces “higher levels of acceptance, making the user more disposed to persuasion” (166).
II – Multi-directionality
Carnegie explores the suasive nature of heightened interaction through multi-directionality, which she defines through hypertextuality and networked practice (see Castells, Nielsen, Landow for more on this). For Carnegie, intertextuality is hypertextuality, or at least a form of it, in which “users gain more control over how they discover, view, and connect the discrete units available in the network, [creating] their own paths and organizational structures” (167).
Essential to this is an understanding of the difference between hypertextuality (as referent) and multi-directionality – the latter “applies not just to the roles users can play in the network (receiver, sender, or both) but also to the messages the users communicate” (167). And so multi-directionality is a combination of this commutative role and the hyper/intertext which creat “a feedback model” that allows the user to participate in systems in multiple forms, while the “referential nature of messages is limited”–referentialism being key to increasing (at least apparent) interactivity and thus increased rhetoricity (166-67).
III – Manipulability
However, this feedback does not function as a fully rhetorical form without a belief–or evidence held–on the part of the user that their interactions have a hand in shaping or influencing the interfaced space (imagine, for instance, the rhetorical agency lost in a discussion board which permits comment submissions, but one where you can only see your own comments which are not distributed to other users).
This function of mutability is labelled by Carnegie as manipulability–and does not apply solely to the human interface level of interaction. She also notes that the very fact that “objects such as images, sound, text [can] become units of numerical code” which are “separated from physical forms” is an expression of mutability, indicating their ability to be duplicated, deleted, combined, scaled, compressed, and so on (168).
Carnegie sees the rhetorical force of manipulability most present, however, in the user’s capability to customize their working/playing/learning environment in digital interfaces. By adding, deleting, hiding, or moving content, the user exerts their will over the workspace, and as such has a suasive investment in its value, use, and well-suited nature. That these choices are likely deeply pre-determined by the interests of the designer doesn’t necessarily matter so much as the appearance of choice which suasively functions to enlist users in investment behaviors which makes them “feel empowered and engaged [because] the user is given limited power to construct him- or herself as a user” (168).
Additionally, Carnegie notes that while customization is a nominally low-level manipulability, the ability to add or modify content creates a remarkably high-level manipulability which correlates with an extremely suasive environment centered on interactivity (168-69).
IV – Presence
While the first two modes of interactivity/rhetoricity function as products of networks themselves, Carnegie notes that presence “is a product of the integration of system attributes with user perceptions” (emphasis added), attributes including speed, possible actions, controls, responsiveness, which present the use of the platform “in a natural and predictable manner” which might manifest successfully when users “perceive others as present in the environment created by the medium” and the ability to “gather appropriate and personal information to adequately interpret situations and gain a sense of being connected with others and the system” (169-171).
This equivalence of systems and apparent human connectivity is an essential facet of rhetorical presence within interfaces, and will shape much of my study moving forward. I will speak more to this notion with my next source for annotation, Alexander Galloway’s The Interface Effect (2012).
V – Interface as Exordium
I will return to these components further moving forward. However, I would like to end this annotation with a brief quote from Carnegie’s conclusion:
“the shape and design of the interface is not natural and inevitable. The design of the interface is a design of human experience, and, as such, the interface becomes a locus of power. The modes of interactivity it deploys are capable of both enabling empowerment and enacting patterns of control. To see the interface, we must see how it functions rhetorically through modes of interactivity to prepare the user/audience to accept particular world views and constructions of relationships” (172).
I will take this notion, that the interactivity and spatiality of interface and systems functions suasively to prepare users to integrate world views and philosophies/constructions of relationships, much further in exploring my object of study in the coming annotated bibliographic entry.
Carnegie, T. A. (2009). Interface as exordium: The rhetoric of interactivity. Computers and Composition, 26(3), 164-173.
Galloway, A. R. (2006). Gaming: Essays on algorithmic culture (Vol. 18). U of Minnesota Press.
Galloway, A. R. (2012). The Interface Effect. Polity.
What constitutes visual argument?
Arguments, made with or through visuals.
Is it possible?
What elements does it require?
Visuals. An argument. That’s it.
Can it be done without linguistic reference?
The term “contextual reference” is so loaded I had to save 80% of my word count to address this. There is no such thing as non-contextual reference, because to be referential an object or argument must contain a referent. That referent is contextualized by its juxtaposition. Also, there’s no form of argument that is not referential, if at least to the actant(s) of that argument.
So, not only can visual arguments not be made without contextual reference, nothing can be made without contextual reference. Not a verbal argument, not a written one, not a bicycle. Not a banana (P.S. any object which could be argued to have a Platonic ideal–whether you buy into the Platonic ideal or not–is a contextualized argument).
As is so often the problem with rhetoric, the question isn’t category, but scope of inquiry. If we instead ask “can visual argument be done without immediate or direct contextual reference?” then the answer is still “yep.”
My Artifact – “A kind of intellectual catastrophe” (The Witness)
“A cage awaits you – this is it, this is the enlightenment you were promised. You lock yourself in the cage. But The Witness played you for a fool. – you were wrong. You were the one snared in the net of panels and wires. There is no enlightenment here. The cage slowly returns you to the very beginning of your long journey, as the island resets. The puzzles un-solve themselves. Where you started is no different from where you ended up, because the experience has left you unchanged. You have achieved nothing. You have learned nothing.”
“The key motif of The Witness is the circle and line, but it has seeped out of the panels into the fabric of the island. You see the motif everywhere–the path of a stream, the patterns in the ivy, it can even be found in the weather. A player who spends too much time with Tetris can be infected with it–they see the shapes everywhere, and have to resist the urge to plot fictional trajectories. It feels like The Witness wants to do this with you. Soon enough, you start to notice the same motif in the real world. Finally, you have the thought: ‘wouldn’t it be funny,’ you think, If I could just click the–‘”
As always, I’ll be looking at The Witness – this time in terms of an analysis on the YouTube channel Electron Dance. Connecting the visuality and designed structure of The Witness to the concepts of mindfulness, enlightenment, and rebirth through sacred texts and public intellectuals, the video questions what the role of the self is within The Witness.
More on that later, but for now, I would encourage folks (who have NO INTEREST in playing the game) to watch the video (massive spoilers, all the spoilers).
Based on some of the initial coding from my subject testing, I decided to look at Zettl and aesthetics for this component, and found it… wanting in its abbreviated form in the Handbook. So I went and got an instructor edition of Zettl’s Sight, Sound, Motion (6th ed. – it’s in 8th now). And I found that perhaps the most useful analytic mechanism in Zettl is the pedagogical framework he lays out for building a curriculum around aesthetics, including a quite nice little week-by-week breakdown of teaching from the text (Fig 1).
So, the thing I find most fascinating here is that Zettl views aesthetics not only as component-based, but also iterative, building upon first principles towards broad aesthetic narratives. Zettl argues, effectively, that you can teach aesthetics, teach with aesthetics, and teach through aesthetics, and this structure works across all three forms of pedagogy.
In the end, what Zettl is talking about in “Aesthetics Theory” is visualizing aesthetics through cinematographic lens–no surprise based on what his interests are–and this is what drove me to go more afield in his work to explore how spatial rhetorics, sound, motion, and light build towards an aesthetic syntax. I’m also going to be doing a lot with Gilles Deleuze here, but for this assignment I had to choose one of these readings, so Zettl it is.
So, here’s a question – the video artifact I posted is not a montage, but as an edited narrative of play and philosophy, it is certainly montage, and it contains montage(s). Is gameplay itself, even in continuity and without montage, montage? Is it an assemblage of actions, a juxtaposition of “various seemingly unrelated events” into a broader narrative, a “specific meta-message” of “induced meaning”?
Let’s get to the questions.
Why this method for this artifact?
“Everything smells of *long sniff* significance.”
The Electron Dance video is fascinating because it makes a parallel argument to that of The Witness itself – namely, that participating with it will draw the user/viewer/player towards a comprehension of philosophy and a narrativized “truth” about the self. That is to say, that it moves from noise to sound, from light to form, from abstract to concrete. It builds a narrative syntax through editing of aesthetic continuities.
These aesthetic continuities are reflected in the philosophical narrative syntaxes the game itself explores, and the video interpolates. Content and aesthetics are, effectively, one and the same.
What did it reveal?
“The flight simulator is now the only way to visit Meiggs’ Field. Technology empowers us to visit places that do not exist. Places that cannot exist. But we do not celebrate this enough. Critics and players often denigrate virtual environments with demands for purpose. The developer-god must corrupt places with mechanics, poison them with meaning–proof of intelligent design must be demonstrated through challenges and collectibles.
The journey itself is never enough.”
It revealed that the viewer is the open, empty, luminous presence of awareness.
For Zettl, the space of visuality is delineated and limited by aesthetic forms and choices. And this limited frame is useful for what it manages to isolate in scope. The Electron Dance video touches on each component of Zettl’s cinematography: light, color, form, field, area, contrast, contained forces, depth, volume, sound, motion, and effect. It uses these components to build on the narrative structures which are drawn from this; however, it also isolates the experience of the viewer-participant (“audience” is absolutely a falsehood here, but it’s hard to articulate why). It assumes that the space is given to the viewer-participant, and the viewer-participant simply occupies it.
I enjoy this conflict. It’s unresolvable, and that is the curse of cinematic language. It gives us tools and terms of production and distribution, but even at its best (Deleuze?), it misapprehends the mindfulness of reception. Aesthetics are mediated, but they are unilateral. Visual rhetoric makes arguments, but those arguments are always resolved by another party who either sees or does not see. Who pursues truth, but cannot apprehend it.
What did it miss that you think might be important?
The most merciful thing in the world, I think, is the inability of the human mind to correlate all its contents. We live on a placid island of ignorance in the midst of black seas of infinity, and it was not meant that we should voyage far. The sciences, each straining in its own direction, have hitherto harmed us little; but some day the piecing together of dissociated knowledge will open up such terrifying vistas of reality, and of our frightful position therein, that we shall either go mad from the revelation or flee from the deadly light into the peace and safety of a new dark age.
Zettl offers us a cinematographic language and palette which we can use to understand how visuality constructs tonality and argument. What is fascinating is to consider how it fails to do the things necessary to address the narrative choice which exists outside of the cinematographic form–the narrative selection of the viewer, whom Zettl’s aesthetics is inflicted upon.
I’ve more than exceeded the length I planned to dedicate to this at this point, so for now… Part Three?
Lemay, P. (2007). Developing a pattern language for flow experiences in video games. In Situated Play, Proceedings of DiGRA 2007 Conference (pp. 449-455). University of Tokyo, Tokyo.
Lemay’s “Developing a Pattern Language for Flow Experiences in Video Games” (2007) explores the structure of play experience surrounding designed pattern languages in order to create a tool for gamespace designers to create efficiently structured and paced play experiences which encourage and maintain flow. From this, Lemay attempts to create a dimensional model of play experience.
In order to accomplish this model, the author performs a metacritical survey of existing pattern languages in the understanding of play, and attempts to create a supplement or synthesis for these approaches which might offer an “extended corpus” which includes “all facets and dimension of the flow model” for play (454).
Examining Bjork and Holopainen’s (2005) component framework (Figure 1), Lemay considers the categories of play and game systems (Boundary, Holistic, Structural, and Temporal) and finds a lack in the experiential elements of play and interaction–and argues that while the pattern “encompasses the wide variety of games found today,” it requires a “complementary language” in order to create a more coherent structure(449-50).
From this, the author moves on to Csikszentmihalyi’s model of “flow” and optimal experience (1988), which Lemay argues offers sensory and cognitive structures for understanding how individuals interact with spaces and media through a locus of experienced challenge and expressed competence (Figure 2). Extending this argument into interactive spaces, Csikszentmihalyi’s conditions for flow demand expressed or clearly-presented goals, instant or immediate feedback, and an elevated degree of challenge matched to a specific degree of expressed competency (451).
From these two frames of understanding game mechanic experiences, Lemay then attempts to structure a pattern language of experiential flow (Figure 3 ) and the experiential/design relationship (Figure 4 ) in concordance with previous scholars who applied flow experience to game spaces. In doing so, Lemay is able to further articulate the research questions at the core of the investigation, including:
- What elements may help generate or maintain flow experiences?
- What elements would hinder the emergence of flow experiences?
In so doing, Lemay also establishes the notion of an “anti-pattern” language, one which functions against or subverts flow experiences. (452) At this point, the author creates a multifaceted structural metric for coding play experience according to five core categories (sensation, emotion, cognition, behavior, and social) and offers three different pattern examples which might fit various flow experiences and medium types (453-54).
In his conclusion, Lemay argues that these comprehensive flow patterns might be viewed as a guiding set of consequences of design for games which function effectively and communicate flow-capable mechanics.
However, I want to push back, briefly, and argue that I don’t think these metrics will inherently create flow experiences or encourage the maintenance of preexisting flow behaviors. This is to say that the design process is inherently too individualized and specific to theme or project purposes to allow for a heuristic understanding of flow within design procedures. However, this pattern language may allow for studying flow experiences under coding protocols which attempt to locate flow by examining the emotional and performative dimensions examined here.
I’ll return to this more with the methodology and coding sections of my current study at a later point in the semester. However, for now I will note that–while Lemay may not be generally useful for the study of visual rhetoric–this work offers very viable tools for exploring the experiential nature of visuality in interactive spaces.
I’m always very intrigued by the concept of systemic affordances, especially those so macroscopic in nature as to be rendered invisible by sheer scale. To this effect, I have selected to look at the Jefferson Grid for this week’s artifact.
The Jefferson Grid is, effectively, invisible. You’ll probably never see it in your life. And yet it comprises approximately 60% of the continental United States. And it has shaped our entire logistical environment: roads, farms, utilities, public services, ownership of land… which in turn creates a cascade of other, resultant systemic features (the parceling system for subdivisions, the difference in scale and fairway composition between American and European golf courses, even the most popular length of segmented boom arm scaffolds for center pivot irrigation systems).
Developed by Thomas Jefferson in the 18th century as an effecient and accurate method for surveying newly acquired or unmapped federal lands, it has since divided everything west of the Mississippi, and much of the territory east until the original thirteen colonies, into a series of perfectly aligned, one-mile-square spaces.
It’s the shape of our world, both perfectly regimented and completely arbitrary. For some reason, this strikes me as particularly fitting in the New American Century.
Norman argues that we should understand affordances in part through a felt sense of wrongness, “the queasy or knotted feelings in your gut” (12) which tell you about the lack of affordance present in objects without intentional design.
I grew up on a grid farm, in a grid town, in the heavily gridded western Ohio region of the Black Swamp. For me, the existence of this grid defined my life and childhood. It was how I understood (and saw) the world–and yet to define this space to others as formative or essential to my knowledge of space is challenging (perhaps impossible).
To live here, in Virginia, where spaces do not adhere to a logic of place (even when not limited by prevailing topography or natural features) is to exist in a constant state of unease.
For part of my final project for this object of study, I’m going to be considering the visual language of the 2016 video game, The Witness, by Jonathan Blow.
Pinterest Album (LINK)
For me, The Witness is a natural object of study for the visual rhetoric and visually linguistic forms of composition and communication within it. A game that exists almost entirely without language or instruction, the game nonetheless encourages players to develop deep knowledge of an arcane ruleset through trial and error, exemplification, and (and here will be the primary focus of my investigation) designed environmental contexts.
This Pinterest collection, composed of screenshots, response videos from players and critics, materials from the game’s development phase, interviews with Blow, and conference presentation videos by Blow about game design, are all part of a larger effort to built a broad understanding of how the game’s forms extend from a design process–and how that design process might have been informed by rhetorical intents or desires.
As the semester moves forward, the content of the game will be largely “spoiled” in my posts. However, since I’m still going to try to recruit a fellow student to actually play the game for feedback, I’ll avoid saying more than that for now.
In general, the process of building this collection was straightforward and unsurprising – having followed the game from near the beginning of development, many of these resources are ones I’d already consumed or been aware of. Moving forward, I plan to supplement this collection with additional scholarship from game studies, instructional design theory, linguistics and philosophy (this part will make sense later), etc. in order to expand the scope of this collection and begin to hone in on a final research question.
My thesis will be structured by testing, so it is weak right now. Essentially, I’m guessing that I’ll be discussing the roles that doubt, exposure, and epiphany play in the development of knowledge in visual media – connecting pattern recognition practices and cognitive science to the visual design of play spaces. I’m especially interested in how visual design encourages or inhibits flow experiences in interactive spaces. Beyond that, I am interested in how visual spatial designs can communicate philosophy, both of design and interactive intent, but also humanist practices within technologies.
Theoretical perspective or methodology
I am using think-aloud protocols during live-play sessions with a single subject playing the video game The Witness (2016) with no prior knowledge of the title or its content. This testing is currently underway. I am maintaining separate A/V archives of the playspace and the player in-session, as well as notes and A/V content on inter-session Q&A periods. These testing sessions will progress until a natural stopping point to be determined by the subject. The theoretical foundations of the research structure are based in Mannell et al. (1988), Squire (2008), and Ulrich et al. (2014), especially as these sources pertain to inciting flow within interactive spaces.
Significance of project
This project lies at the nexus of multiple interesting conversations happening in the areas of game studies, digital pedagogy, technical communication, instructional design, visual rhetoric, and UI/UX. I am hoping that a successful approach to synthesizing these various approaches to interactivity can help to illuminate the core learning practices of visuality as separated from traditional literacy foundations used previously to study flow and pattern recognition in skills acquisition and competency.
I have a lot more than 2-3 at this point, but I’ll be tuning downwards as I move forward. I’m still looking for the “killer” source in VisRhet for addressing epiphany, doubt, learning, and interactivity (though Burbules “Aporias” and “Rethinking the Virtual” combined are very close to what I need).
Burbules, N. C. (2000). Aporias, webs, and passages: Doubt as an opportunity to learn. Curriculum Inquiry, 30(2), 171-187.
Burbules, N. C. (2006). Rethinking the virtual. In The international handbook of virtual learning environments (pp. 37-58). Springer Netherlands.
Calleja, G. (2010). Digital games and escapism. Games and Culture, 5(4), 335-353.
Carnegie, T. A. (2009). Interface as exordium: The rhetoric of interactivity. Computers and Composition, 26(3), 164-173.
Cogburn, J., & Silcox, M. (2009). Philosophy through video games. Routledge.
El-Nasr, M. S., & Yan, S. (2006, June). Visual attention in 3D video games. In Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology (p. 22). ACM.
Jones, P. C. (2014). The Receiver is the Message?. Semiotics and Visual Communication: Concepts and Practices, 269.
Kang, H., Lee, C. W., & Jung, K. (2004). Recognition-based gesture spotting in video games. Pattern Recognition Letters, 25(15), 1701-1714.
Klimmt, C., & Hartmann, T. (2006). Effectance, self-efficacy, and the motivation to play video games. Playing video games: Motives, responses, and consequences, 133-145.
Kline, S., Dyer-Witheford, N., & De Peuter, G. (2003). Digital play: The interaction of technology, culture, and marketing. McGill-Queen’s Press-MQUP.
Lemay, P. (2007). Developing a pattern language for flow experiences in video games. In Situated Play, Proceedings of DiGRA 2007 Conference (pp. 449-455). University of Tokyo, Tokyo.
Mannell, R. C., Zuzanek, J., & Larson, R. (1988). Leisure states and” flow” experiences: Testing perceived freedom and intrinsic motivation hypotheses. Journal of Leisure Research, 20(4), 289.
Myers, D. (2006). Signs, symbols, games, and play. Games and Culture, 1(1), 47-51.
Squire, K. (2008). Educating the fighter: buttonmashing, seeing, being. Beyond Fun, 27-42.
Squire, K. (2008). Video-game literacy: A literacy of expertise. Handbook of research on new literacies, 635-670.
Ulrich, M., Keller, J., Hoenig, K., Waller, C., & Grön, G. (2014). Neural correlates of experimentally induced flow experiences. Neuroimage, 86, 194-202.
Wolf, M. J. (2003). Abstraction in the video game. The video game theory reader, 1, 47-65.
Visual Media Included:
The final project will include copies of sample play session footage, as well as illustrative content from the game space, documentation of the design process from the developers of The Witness, and other A/V content as appropriate to illustrate concepts of cognitive science and so on (E.G. Lemay’s (2007) illustrations of pattern language.
Today we will be recording an extended testing session for a visual rhetoric analysis of The Witness, a 2016 puzzle-centric video game produced by Thekla, Inc. and game designer Jonathan Blow. The session will take the form of five consecutive 30 minute play sessions, each followed by a brief break during which there will be resets for the testing hardware and questions about the play experience for the previous session. These sessions will be played consecutively, with no restart of game progress or changing of play situations. All sessions will resume where the previous session ended. The participant is free to request a break from play and/or questioning at any point, and may choose to end the testing session for any reason and at any time.
Testing will follow a think aloud protocol, during which the participant will be asked to vocalize thoughts on the game, game space, mechanics, inputs, design, and themes as well as actively comment on their decision-making and learning processes as they play. In order to assure accurate coding and successful recording, the participant is asked to project loudly, clearly, and with strong enunciation whenever speaking aloud. During active play, the tester will not interrupt, provide guidance, or otherwise interfere with play. However, the participant is welcome to ask non-directive questions for troubleshooting or other purposes.
This testing will address components of input and user interface experiences; however, this is not a usability test. The participant is asked to focus primarily on the thematic, experiential, mechanical, and intellectual engagement they have with the game, with special focus on what the puzzles in the game communicate to them, how they understand the puzzles, and why they make specific decisions in solving the puzzles.
The subject’s participation will be recorded in three ways during the process. All recordings will be available for the participant upon request.
- Gameplay will be recorded directly from the play device (Sony Playstation 4) to a secure cloud storage (hosted on Twitch.tv), from which gameplay videos will be downloaded to the tester’s personal computer. These files will not be available for public consumption after recording, and will contain only on-screen play recordings. No in-room audio, video, or other data will be stored in this form. All play footage will be anonymized, and will not include any identifying information about the participant. All play footage will be removed from the cloud immediately after downloading.
- The participant’s play session will be recorded in the room via video camera for each session, including both audio and visual recordings. These recordings will also be stored on the tester’s personal computer, and will be deleted after the research period is complete. This content will be held in the strictest confidence, and any coding resulting from this recording will be anonymized prior to release.
- The tester will take notes on participant behaviors and actions throughout the play process, as well as notes on the participant’s responses during the pre-, inter-, and post-play segments. These notes will serve as a supplement to the previous two recordings, which will be synchronized for coding.
Before we begin, please let me know if you have any questions about this testing protocol, or any concerns or proposed amendments.