Creativity and Virtual Reality

Each time I have put on the stereoscopic headset, I underwent many different experiences: I floated through a postapocalyptic environment where nature and animals prevailed (All Unsaved Progress Will Be Lost, Courtinat 2022); I explored the apartment of a couple whose home slowly underwent a change in gravitational force, witnessing their fight among flying objects (Marco & Polo Go Round, Steiger-Levine 2022); I solved puzzles and enigmas in order to open boxes and doors so as to discover a new kind of matter (The Room VR: A Dark Matter, Fireproof Games 2020); I drew colourful 3D-shapes around which I could walk (Tilt Brush, Google 2016). These experiences were special: I felt completely absorbed by the virtual spaces, time flew by, and I occasionally needed to remember to be mindful of the real-world coffee table or the actual people around me. In these virtual worlds, I also had aesthetic experiences, and I searched, and found, solutions to certain puzzles. Were these experiences occurrences in which I was being creative? Intuitively, it seems to be the case, at least when solving enigmas and drawing colourful shapes. I will argue that I was creative in all these situations, but in different ways. I will start by discussing what is distinctive of virtual reality experiences. I will then discuss the philosophy of creativity and explain how Virtual Reality (VR) offers interesting environments for being creative.

There is one aspect that scholars put forward when speaking about what is distinctive in the experience of VR: the ‘sense of presence’ or ‘feeling of realness’ they provide (Tavinor 2021, 108). David Chalmers has given a definition of VR-environments, which he conceives as ‘roughly synonymous’ with ‘virtual reality’ (Chalmers 2017, 313). According to him, VR-environments are “immersive, interactive, computer-generated” (Chalmers 2017, 312). For Chalmers, immersion means that the VR-user has a ‘perspective within’ the VR, that she is surrounded by the VR, which is typically three-dimensional (Chalmers 2017, 312); an interactive environment is one in which a user can act, and these actions make a difference on the environment (Chalmers 2017, 312); finally, the criterion of being computer-generated enables us to distinguish VR from reality, which is also immersive and interactive (Declos 2022, 3). He adds that it is the immersiveness and interactivity of VR(-environments) that render VR-experiences “at least akin to ordinary reality” (Chalmers 2017, 312).

Grant Tavinor (2021) defines VR-experiences in a similar way to Chalmers. He emphasises that what we usually associate with VR – the headset – is a media (i.e. a ‘middle ground’  (Tavinor 2021, 16, his emphasis)) for accessing the virtual world, just like books, film, photography, or paintings are means for accessing fictional worlds.[1] What is distinctive for VR, according to Tavinor, is what he calls ‘egocentric picturing’. Egocentric picturing means that the virtual world appears to the user in a similar fashion as in real life: it surrounds the user, is seen from the user’s point of view, and is interactive.[2] The egocentric picturing, in turn, creates a ‘feeling of presence’ – a ‘sensation of ‘being there’’ (Bailenson 2018, 19) ­– which I will call here ’feeling of realness’. The feeling of realness is due to the similitude of the sensorimotor experience in VR to real life (Tavinor 2021, 117–18; see also Bailenson 2018, 19). In other words: ‘in VR, computational media give users the impression that they perceive and interact with alternative worlds displayed on screens before them’ (Tavinor 2021, 19).        

A very significant aspect of VR-experiences is thus this ‘feeling of realness’, the similitude in the experience of virtual reality and of the real world. We perceive VR-environments from our own point of view, they are three-dimensional, surround us, and are interactive. What is more, when we are wearing a headset, our attention to the outside world is limited (hence the need to remember to be aware of the coffee table) (Tavinor 2021, 118). However, it would be a step too far to say that we confuse VR with reality: VR is not an illusion. As Tavinor (2021, 125–32) rightly points out, current VR-media only focus on two senses: hearing and seeing; and the imitation of these senses, although imitating our real experiences more realistically than other media, still face (technical) problems. Moreover, the currently developed VR-videogames and -movies do not seem to be solely focussed on creating a sense of reality – it is clear to the player, when pointing one’s virtual saber to the rectangular shapes in Beat Saber (Beat Games 2019), that she has not been transported into a real arcade; idem for walking through the apartment of Marco and Polo (Steiger-Levine 2022): I am aware that the events are not really taking place, that I am rather watching a movie. However, it seems to be one aim of VR-media to imitate our sensory input (at least the visual and acoustic ones) in a realistic way, perhaps in order to increase the special impact VR has on us: this feeling of realness.

Recent research on creativity was greatly impacted by Margaret Boden, who defines creativity as “the ability to come up with ideas or artefacts that are new, surprising, and valuable” (Boden 2010, 29, emphasis in original). Although she defines creativity as a human ability, Boden puts the emphasis on the result (see also Hills and Bird 2019). If the painting I drew of a childhood memory is new, surprising and valuable, I was creative according to Boden[3]. This is common for most discussions of creativity: they generally take the creative person to be someone who generates creative objects. Note here that creative objects can, but must not be physical: they can be artefacts, ideas, performances (see Boden 2010; Gaut 2003).  

Lately, creativity-researchers have found a new focus of interest: the creative process. According to Julia Langkau (2022), it is the process that determines whether a result is creative or not. A mechanical or nonintentional process that leads to a new, surprising and valuable object would not make the object creative. For example, we usually do not speak of nature as being creative, and we do not consider painting by numbers – where you fill out areas with certain colours according to the indicated numbers – creative. In this context, we can ask two questions: (1) What does a creative process consist in?, and: (2) What makes a process creative?

We can find an answer to the first question in Dustin Stokes and Elliot Paul (2023). According to them, the creative process consists of five stages: (a) preparation (acquiring of understanding, knowledge, skill, etc. in a particular field), (b) generation (of ideas), (c) insight (‘aha’-experience with one particular idea), (d) evaluation (of your ideas), (e) externalisation (making the idea communicable). Let’s take an example. The process of me painting a childhood memory starts with acquiring painting-skills, and maybe also with my knowledge about different painting-styles (which I may aim for) (a). My mind then wonders, I daydream, but I may also start (consciously) remembering things about my childhood (b). One particular event strikes me (c). Consequently, I start evaluating my insight more thoroughly: is it really an important event, what exactly is relevant to me in this event, can I find a way to represent it through a painting? (d) And finally, I externalise it, maybe first through sketches, coming back to the former steps (a)-(d), modifying my project, making my idea more precise, until I arrive at the final result (e). Note that the result might also be non-physical, it might just be a (communicable) idea. Stokes’ and Paul’s five-step model offers an engaging description of what problem-solving creative processes are like: we formulate an issue that we solve with our creative object. Note that according to Stokes and Paul, all creative processes are of the problem-solving kind.

If, however, we consider, as Langkau (2022; manuscript) does, that engaging with artworks can be creative, conceiving the creative process in terms of the five-step model starts to look implausible. According to Langkau, we creatively engage with artworks by (1) imaginatively ‘attend[ing] to or reflect[ing] on what is being presented to us or represented in one way or another’, and then (2) ‘intentionally manipulat[ing] what is being presented or represented’ (in manuscript). One can, for instance, read The Magic Mountain (Thomas Mann 1924) creatively by forming rich mental images, associations, by daydreaming and having new ideas; in Langkau’s opinion, one is already creative when simply following one’s imagination when reading. This is because literary fictions offer only partial plots, and leave a lot of informational gaps. The reader who engages in the plot fills the gaps with her own creative imagination. Hence, one can engage in a creative process – be creative – without necessarily externalising something. Langkau adds that, when attending to artworks, we are guided by the artworks themselves, and also by our values. The Magic Mountain guides my experience; but a lot is left up to me, too. I fill the gaps left open by the fiction, and make connections, anticipations, etc. by following what I value. What makes the reading process creative is, according to Langkau, that we engage with and intentionally manipulate ideas in our imagination, and that we do it following our values, i.e. according to our personal interests, values etc.

These two models of creative processes offer a different idea of what an artistic creative process exactly consists in. Stokes and Paul would say artistic creative processes are of a problem-solving kind. One finds (if need be, generates) a problem to solve, e.g. ‘I want to communicate the feeling I had when I ate my first ice-cream as a child’. Langkau would argue that it consists in combining various elements with one another, following what we value: I remember the particular taste of the pistachio ice-cream, and the noises around me in my memory; I find the colour ‘sage’ to represent the taste well, and combine this colour with shapes and colours that I find represent the noises well. All the elements in the painting – I choose them according to what I value: what I like, what interests me, what moves me, etc.

While the problem-solving kind of creativity seems to me to be one possible way in which one can artistically create, I am not convinced that it fits every artistic creative process. I think there are certain key elements that come at play in artistic creative processes, which are present both in Stokes’ and Paul’s account and in Langkau’s, such as skill, values, imagination, but I think creative processes oftentimes involve also play and intentions. But this question does not need to be solved here.

In the previous section, I have discussed creative processes. Coming back to our engaging with virtual reality: are we being creative when engaging with a VR-environment? And if so, in which of the two ways I described above?

In what follows, I will sometimes distinguish gameplay and storyline. In most interactive, virtual environments, one can separate the two. The storyline is the narrative behind the environment: for the Zelda videogames, it would be all that concerns Princess Zelda being kidnapped. Oftentimes in videogames, the storyline is revealed in non-interactive, movie-like moments. ‘Gameplay’ designates all the rest, all the aspects that make interactive environments interactive: the mechanics of the game, the puzzles to solve, the coins one needs to gather in order to acquire more lives, etc. (see also Seddon 2017).

    3.1 Active and guided creative processes

In the introduction, I mentioned four different VR-experiences: two movies and two games. When watching a VR-movie, there is no interactivity: the viewer cannot impact the content of the story in any way. Things happen all around the viewer, things that she can observe by turning her head and looking around. Some movies may enable the viewer to move around in the virtual environment (Marco & Polo Go Round, for example). Following Chalmers’ definition, movies do not count as VR, because of the lack of interactivity. I disagree with this. To me, movies that are created to be viewed with a headset do count as VR, if they present the storyline with egocentric picturing.[4] If one can look at the movie from different angles, is immersed in it, and is even able to move around, then the movie is a VR-environment. For in this case, one discovers the world as if one were ‘in it’ – i.e., surrounded by the environment and the events – and sees the plot unfold from an ‘insider’ point of view. Although one cannot make any change to the plot, being immersed in the plot in this way gives the impression of being involved in it, e.g., like a ghost or like rewatching a memory. In other words, even without interactivity, the egocentric picturing confers a feeling of realness.

Are the viewers of these movies creative in any way? It might seem intuitive to consider a person to be creative only when she actively generates something. I think that asking whether one is creative when watching a VR-movie is akin to asking whether one is creative when reading a novel, watching a 2D-movie, or listening to a rap song. As discussed in section 2, one is creative when reading a novel; one might, but must not, be creative when watching a movie (see Langkau manuscript). Let me explain. When reading a novel, the reader must ‘fill the gaps’ left open by the novel (of which there are necessarily), by forming mental images, connections, interpretations etc. (see Langkau and Balcerak Jackson 2022). This, the reader does creatively, following what is valuable to her. It seems possible that the movie-watcher of, say, a James Bond movie is not being creative: she just enjoys what she sees, without using her imagination in any way. She might, however, engage creatively with the movie, by using her imagination to build connections, by anticipating certain happenings, or by discovering a deeper meaning in a certain scene or event. The kind of creativity at play when engaging in fiction is guided by the work of fiction. The same goes for VR-movies. One can just look at the pictures and think noting, not engaging with the movies creatively; one can also wonder what happened before, what is currently happening, who survived, how this virtual world works. By filling the gaps of the movie-narrative, one follows one’s own values (Langkau manuscript). Thus, the viewer is admittedly guided, but being guided can still mean being creative.

When playing games in VR, the actions of the player impact the game. In many VR-games, one must solve puzzles – that is, one must overcome different obstacles: find objects in order to become stronger, find keys to open doors, kill bosses to access new levels, etc. By solving the puzzles, the player makes the plot move forward and enables new puzzles to present themselves. The interactivity makes it so that the player must do something; otherwise, she is not engaging in the game at all. Evidently, the kind of thing the player does is engage in creative processes. These creative processes are active, rather than guided, in the sense that the gameplay and storyline depend upon human agency to unfold. The creativity at play in the case of VR-puzzle-videogames is of the problem-solving kind. The five-step model can be used for explaining what takes place for each new puzzle: (a) preparation: we put on the headset, and we also may have certain skills for puzzle-solving, (b) generation: we imagine possible solutions to the puzzles, (c) insight: we think we may have found out how to solve the puzzle, (d) evaluation: will this be the right way to solve the puzzle? (e) externalisation: we solve the puzzle. Many interactive VR games have a storyline. For example, in The Room VR, the player must unveil the secrets around a new kind of matter, which was hidden by a scientist and gets revealed through the puzzles. The storyline in interactive VR also offers the possibility for guided creative processes, which is the kind of creative processes involved when engaging with movies or books. One can treat the storyline of a videogame as one would treat any other narrative: creatively, by filling the gaps and drawing connections and inferences.

There are VR-games where no puzzles need to be solved. This is the case in Tilt Brush, where the player can draw whatever she likes, within certain limitations, such as available colours and ‘brush’-variants. The creativity of the player is given free reins: if she acquires the skill of drawing in 3D – which is not such an easy task – she can create any kind of picture. Of course, the result will only be perceivable with a stereoscopic headset. As discussed in Section 2, artistic creative processes are manifold. The player might construe a problem she wants to solve, thus using a problem-solving kind of creative process. For example, she states the following problem: ‘I would like to paint a 3D-lion’. She might also let herself draw freely and see what comes out of it. In any case, the kind of creative process at play in the drawing game does not seem to be drastically different from any other artistic undertaking, such as painting on a canvas, drawing on a piece of paper, composing music etc.

One the one hand, we can see that there does not seem to be a considerable difference between the kinds of creative processes involved in VR, and the kinds involved in real-life. However, when I played The Room in VR, the experience felt quite different from playing The Room on any other platform. While remaining pretty neutral to the storyline in the 2D-versions of the game, playing it in VR caused me to feel fear on several occasions: Will something appear behind me? What are those noises? I hope nothing (alive) is hidden in that coffin ... These were some of the thoughts I had.

    3.2 What is special about creativity in VR?

So I am wondering: What is the difference between my creativity in VR and my creativity when reading a novel or drawing a picture? The difference seems not to lay in functional aspects, but rather in the phenomenology of the experience. Watching a movie or playing a game in VR feels more intense than watching a 2D-movie or playing on a videogame console. One reason for this is that one is partially occluded from the outside world: when I put one the headset, I only see the virtual environment, and I cannot look elsewhere, e.g., on my phone. Moreover, my attention is captured by the VR-environment. Disengaging from a VR-environments means to stop the game or the movie completely, and to withdraw from the virtual world entirely.

Another special feature of VR is that it renders two kinds of creative engagements possible. On the one hand, VR makes active creative processes possible, just like in a real-life creative process. Painting in the 3D-VR-environment feels akin to being the painter of a childhood memory in real-life. This is because of the egocentric picturing: the player is the agent and sees the virtual environment from her own point of view. On the other hand, VR involves guided creativity, just like when engaging with 2D-artworks and -games. VR thus offers a whole new kind of creative experience, where one enjoys games and movies, thus being guided by the environment in which one is immersed, but in a more realistic way than with any other media. One may wonder what impact this has on the creativity of the VR-user…

I have discussed four types of VR-environments and wondered whether the user of these environments was creative. I have argued that in all the cases, the user was creative, but we need to distinguish between an active and a guided kind of creative process. I also drew parallels between the VR-environments and other media. I said that the creative processes involved in VR-environment are of the same kind as with any other media. I ended by discussing the question of what the difference between our creative engagement with VR and two-dimensional media is. Although the creative processes are the same in VR and real-life, it seems that there is a difference in the phenomenology of the experience.

Yet, this answer sparks up many more questions. What is so special about the phenomenology of VR? Does the feeling of realness impact the artistic creative process? Does the awareness of being in a virtual environment modify the creative process? If so, what impact does the feeling of realness, and the awareness of being in a virtual environment, have on our creativity? Are we differently creative when engaging with VR-fiction rather than books and 2D-movies?

These are only some of the questions I have and that I will try to find answers to during the Summer School Reality+ at USI in Lugano, Switzerland. I hope to come back with some insight!

[1] I leave the question whether the virtual environments are real or fictional aside here.

[2] Interactivity is thus part of the egocentric picturing in Tavinor. The term picks out roughly the same phenomenon as ‘immersion’. Tavinor chooses to avoid the notion of ‘immersion’, because he finds it confusingly metaphorical (Tavinor 2021, 117–18).

[3] Boden also distinguishes between new to a person and new altogether, which she defines as psychological creativity (P-Creativity) and historical creativity (H-Creativity) (Boden 2010, 30).

[4] Note that Tavinor understands ‘egocentric picturing’ to include interactivity; I take the interactivity, as non-necessary for egocentric picturing.

Bailenson, Jeremy. 2018. Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do. W. W. Norton & Company.

Beat Games. 2019. ‘Beat Saber’. https://store.steampowered.com/app/620980/Beat_Saber/.

Boden, Margaret A. 2010. Creativity and Art: Three Roads to Surprise. Oxford; New York: Oxford University Press.

Chalmers, David J. 2017. ‘The Virtual and the Real’. Disputatio 9 (46): 309–52. https://doi.org/10.1515/disp-2017-0009.

Courtinat, Mélanie, dir. 2022. All Unsaved Progress Will Be Lost. Images de synthèse - VR 360°. Animation. https://www.unifrance.org/film/55234/all-unsaved-progress-will-be-lost.

Declos, Alexandre. 2022. ‘L’ontologie Du Virtuel’. Klēsis Revue Philosophique 52: 1–25.

Fireproof Games. 2020. ‘The Room VR: A Dark Matter’. https://store.steampowered.com/app/1104380/The_Room_VR_A_Dark_Matter/.

Gaut, Berys. 2003. ‘Creativity and Imagination’. In The Creation of Art: New Essays in Philosophical Aesthetics, edited by Berys Nigel Gaut and Paisley Livingston, 268–93. New York, NY: Cambridge University Press.

Google. 2016. ‘Tilt Brush’. https://store.steampowered.com/app/327140/Tilt_Brush/.

Hills, Alison, and Alexander Bird. 2019. ‘Against Creativity’. Philosophy and Phenomenological Research 99 (3): 694–713. https://doi.org/10.1111/phpr.12511.

Langkau, Julia. 2022. ‘Two Notions of Creativity’. In Wittgenstein and Beyond, by Christoph C. Pfisterer, Nicole Rathgeb, and Eva Schmidt, 1st ed., 254–71. New York: Routledge. https://doi.org/10.4324/9781003202929-18.

——————. manuscript. ‘What Is Creative Imagining?’

Langkau, Julia, and Magdalena Balcerak Jackson. 2022. ‘Literary Fiction and Imagination’. In The Philosophy of Fiction: Imagination and Cognition, edited by Patrik Engisch and Julia Langkau, Routledge.

Mann, Thomas. 1924. Der Zauberberg: Roman. Frankfurt am Main: S. Fischer.

Paul, Elliot Samuel, and Dustin Stokes. 2023. ‘Creativity’. In The Stanford Encyclopedia of Philosophy, edited by Edward N. Zalta and Uri Nodelman, Spring 2023. Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/spr2023/entries/creativity/.

Seddon, Robert. 2017. ‘Video Games and Virtual Reality’. In Macmillan Interdisciplinary Handbooks: Philosophy: Technology, edited by Anthony F. Beavers, 191–216. Macmillan Reference USA.

Steiger-Levine, Benjamin, dir. 2022. Marco & Polo Go Round. Animation. ARTE Cinema. https://www.arte.tv/fr/videos/107846-000-A/marco-polo-go-round/.

Tavinor, Grant. 2021. The Aesthetics of Virtual Reality. 1st ed. New York: Routledge. https://doi.org/10.4324/9781003107644.