# Curating Simulated Storyworlds — Chapter 12: Looking Ahead: Sheldon County > NOTE: THIS IS A MARKDOWN CONVERSION INTENDED FOR LLM CONSUMPTION. > IF YOU ARE A HUMAN BEING, READ THE PDF INSTEAD: > https://jamesryan.computer/papers/2018_ryan_curating_simulated_storyworlds.pdf > > The PDF is the canonical document, typeset with every figure and table. This file > is a plain-text conversion of the LaTeX source, published HERE so that software agents > can read the thesis without parsing a PDF. It is lossy and is not meant for humans. > > This file is chapter 12 of 14 only: Looking Ahead: Sheldon County (~11,000 words). > See below for links to markdown conversions of the individual chapters. > > **Structure.** Sections are `##`/`###`/`####` headings. Cross-references are resolved to > their numbers ("Section 3.1.1", "Chapter 5"), so a heading can be searched for directly. > Citations are inline bracketed keys such as `[ryan2017grimes]`; every key used in the text > has an entry in the bibliography, formatted as `- **[key]** Author. Title. Venue, Year.` > Footnotes appear inline as `[Note: ...]`. > > **Conversion limits.** Figures and most complex tables appear as `[Figure omitted]` or > `[Table omitted]`. Code fences carry no language tag. A few diacritics in author names > were simplified from the Latin-1 source. > > **All files.** > > Index (this list, plus the abstract; small): https://jamesryan.computer/thesis/index.md > Whole thesis in one file (large, ~1.6 MB): https://jamesryan.computer/thesis/full.md > > By chapter: > - 01. Introduction (~4,600 words, ~30 KB) — https://jamesryan.computer/thesis/01-introduction.md > - 02. Defining Emergent Narrative (~6,900 words, ~47 KB) — https://jamesryan.computer/thesis/02-defining-emergent-narrative.md > - 03. The Pleasure of Emergent Narrative (~21,000 words, ~135 KB) — https://jamesryan.computer/thesis/03-the-pleasure-of-emergent-narrative.md > - 04. The Pain of Emergent Narrative (~41,000 words, ~268 KB) — https://jamesryan.computer/thesis/04-the-pain-of-emergent-narrative.md > - 05. Refining Emergent Narrative (~11,000 words, ~73 KB) — https://jamesryan.computer/thesis/05-refining-emergent-narrative.md > - 06. James Ryan Generator (~2,800 words, ~16 KB) — https://jamesryan.computer/thesis/06-james-ryan-generator.md > - 07. World (~20,000 words, ~129 KB) — https://jamesryan.computer/thesis/07-world.md > - 08. Case Study: Diol/Diel/Dial (~9,300 words, ~59 KB) — https://jamesryan.computer/thesis/08-case-study-diol-diel-dial.md > - 09. Talk of the Town (~25,000 words, ~162 KB) — https://jamesryan.computer/thesis/09-talk-of-the-town.md > - 10. Case Study: Bad News (~22,000 words, ~132 KB) — https://jamesryan.computer/thesis/10-case-study-bad-news.md > - 11. Hennepin (~32,000 words, ~209 KB) — https://jamesryan.computer/thesis/11-hennepin.md > - 12. Looking Ahead: Sheldon County (~11,000 words, ~75 KB) — https://jamesryan.computer/thesis/12-looking-ahead-sheldon-county.md > - 13. Conclusion (~2,700 words, ~17 KB) — https://jamesryan.computer/thesis/13-conclusion.md > - 14. Bibliography (~29,000 words, ~233 KB) — https://jamesryan.computer/thesis/14-bibliography.md > *The goals we set for the system are heroic, and demand heroic test cases.* — Sheldon Klein In the fall of 2016, Amazon announced the Alexa Prize [ram2017conversational], a competition that called for university research labs to develop AI technology for the Alexa conversational platform. Amazon was to admit ten entrant teams, each of which would receive a $100,000 grant, Alexa-enabled devices, and free access to Amazon Web Services, a cloud-computing platform. At the time, my coadvisors, Michael Mateas and Noah Wardrip-Fruin, and I were between two rejected grant proposals that each pitched work related to a planned extension to *Talk of the Town*. Michael heard about the Alexa Prize from Ashwin Ram, his former George Tech colleague who was running the competition at Amazon, and he proposed a meeting with Noah and me to brainstorm about a possible entry. [Note: Ashwin Ram is another one of Roger Schank's students, like Jim Meehan and Natalie Dehn and others who have appeared at various points in this dissertation, including my Michael's coadvisor Jaime Carbonell. Ram's own dissertation project was a story understanding system called *Aqua* [ram1989question]. It appeared toward the end of the Yale AI Project, just as Schank was heading to Northwestern University to found the Institute for the Learning Sciences.] I was working as a teaching assistant, and so a funded project sounded quite appealing. In Ram's call for entries, he explained that the specific task of the competition was to develop a speech-enabled bot capable of conversing on “popular topics” for an extended period of time: > Today, we are pleased to announce the Alexa Prize, a $2.5 million university competition to advance conversational AI through voice. [...] The challenge is to create a socialbot, an Alexa skill that converses coherently and engagingly with humans on popular topics for 20 minutes. We challenge teams to invent an Alexa socialbot smart enough to engage in a fun, high quality conversation on popular topics for 20 minutes. [...] Participating teams will advance several areas of conversational AI including knowledge acquisition, natural language understanding, natural language generation, context modeling, commonsense reasoning and dialog planning. Alexa users will experience truly novel, engaging conversational interactions. [ram2016are, n.p.] Adam Summerville and I had recently explored a technical approach to natural language understanding that made use of my authoring tool *Expressionist* [summerville2016cfgs,ryan2016translating,ryan2016expressionist], which I had already been using to drive natural language generation in a testbed extension to *Talk of the Town* [ryan2016aiide]. In turn, I had developed a dialogue manager that could integrate both systems [ryan2016lightweight], and so we felt that we had a good technological substrate in place for a proposal. A key feature of this technology was its *authorability*—the framework was designed to allow human authors to inscribe artistic intent into (and exert authorial control over) a generative system for procedural conversation. As such, while we figured most entrants would propose less authorial systems—likely ones that would automatically mine knowledge about popular topics to then render that knowledge in natural language—our spin would be to leverage our authorial technology to enable compelling interactions driven by artistic and aesthetic considerations. At the Expressive Intelligence Studio, we develop novel AI technologies, but we do so in the service of *media works*—this is why ‘studio' is in the name of our group, and it is why our department is called ‘computational media'. Having this predilection (and mission), our first step was to frame the Alexa Prize call in terms of a prospective work of media, which led us to ask ourselves: what kind of compelling media experience would center on conversation on popular topics? The more we thought about it, however, the less sense the whole competition made to us. What are these ‘popular topics' and why would anyone want to discuss them with a computer? The media experience that this framing suggested seemed to us to be either harrowing feats of endurance—*Talk About The Kardashians With A Computer For Twenty Minutes Straight*—or, at best, redundant undertakings. One who wishes to talk about the Kardashians will have little problem doing so, since there are plenty of humans who gladly provide the service. Moreover, humans are almost certainly better than computers at talking about the Kardashians. What is the point of computationalizing a banal everyday experience that has already been mastered by humans? Of course, it could be interesting to subvert that experience or to computationalize it with a peculiar procedural aesthetics, [Note: I can for instance imagine experimental game designer Pippin Barr answering the Alexa Prize call in an unsettling way.] but this was clearly not what Amazon was after and we struggled to excite ourselves about the idea of procedural conversation on popular topics. [Note: To be fair, this is a decidedly mediacentric appraisal of a competition with primarily technological aims. More on this in footnote Chapter 6.] Instead of abandoning the prospect, we decided to brainstorm about how we could build an actual media experience that would be novel, compelling, and uniquely suited to the Alexa platform. How would one work? First, the experience would be strictly audio-based, with no screens. In consideration of this, our conversation shifted to a discussion about how we each engage with audio already. Someone mentioned the experience of listening to the radio in the car, and the specific pleasure of serendipitously encountering a radio play on a road trip. I explained that I enjoy podcasts but only listen to them in two recurring contexts: doing dishes and working out. In each of these contexts, I am not free to engage with screen-based media, because I cannot maintain eye contact due to either looking at what I am doing with my hands (dishes) or turbulent body movement (I do a hill workout). Moreover, while doing dishes, my hands are not free to control any kind of input device, but I could use my voice. How could we design an experience for dishwashers? Somewhere in the discussion, Michael had a brilliant idea for a computational audio experience that would leverage *Talk of the Town*: a procedural episodic radio play in the style of *A Prairie Home Companion* (1974–2016) [larson1987prairie,larson1992evoked]. Rather than constructing stories about an invented town of fictional characters, a system could recount stories that have actually emerged in a *Talk of the Town* town. Moreover, every listener could have her own personal storyworld, each being driven by a distinct simulation instance, which would suggest an ontology in which the town lives on her Alexa device. It would be like an ant colony, but with orders of magnitude more intrigue. How would a system know what stories to tell? As Michael noted, one prospect would be to automatize the kind of narrative excavation that I carry out in my capacity as *Bad News* wizard—this would likely result in the technical approach that I demonstrated at the beginning of the previous chapter. In fact, this was the task of ‘story recognition' that we had already identified in our paper from the previous year, “Open Design Challenges for Interactive Emergent Narrative” [ryan2015open]. [Note: As I explained in Section 5.5, under the influence of Hayden White, starting with this dissertation I have decided to use the term ‘story sifting' instead of ‘story recognition'. The reason for this is that stories do not exist fully intact in (storyworld) chronicles, including simulation traces. To assume otherwise is to commit the crime of ‘narrativism' (Espen Aarseth's term), as I explained at length in Section 4.2.1.] Once the material was excavated, a natural language account could be generated using the technology that we had already developed, namely *Expressionist*. What we were now brainstorming in this meeting was an archetypal work of *curationist emergent narrative*: an automatic *story sifter* would excavate promising material from the accumulated history of a simulated storyworld, and a *narrativizer* would then construct a narrative artifacts out of that material, which would themselves be mounted into a media experience taking the form of a radio play. Our working title was *Procedural Wobegon*. [Note: *A Prairie Home Companion*'s storyworld is a fictional Minnesotan small town called Lake Wobegon, which is also the setting of Garrison Keillor's spin-off novel *Lake Wobegon Days* [keillor1990lake].] This idea was exciting, but as initially formulated, it would not feature the procedural conversation that was at the heart of the Alexa Prize call (and that is the paradigmatic Alexa interaction). Of course, the way to leverage this interactivity would be to make the radio play interactive. Maybe the listener could interrupt the narrator to ask questions—for instance, an inquiry as to the story of a background character in a scene. Alternatively, or additionally, there could be choice points within or between episodes about what or whom the listener would like to hear about next. In this way, the resulting narrational style would likely become a hybrid between that of a radio play and of conversational storytelling, perhaps with some elements of *branching narrative* [salter2014your,moriarty2015sing]. We were excited by our new idea, but it had obviously deviated from the spirit of competition. What we had conjured was not centered on conversation around popular topics—it in no way pertained to the Kardashians—but it seemed like a strong vision for an experience that would truly leverage the Alexa technology. Michael and Noah figured that Amazon might be interested in supporting the idea to some extent, perhaps by providing hardware and a development kit, even if it was too far off to be accepted as an entrant in the actual Alexa Prize competition. Michael wrote to Ashwin Ram directly to pitch *Procedural Wobegon*, but Ram expressed that it was definitely too removed from the call to be an entrant, and moreover Amazon would not be able to support the project in any other way. [Note: Incidentally, UC Santa Cruz would produce an Alexa Prize entrant in the form of the *SlugBot* team [bowden2018slugbot], led by student Kevin Bowden and composed also of other members of Marilyn Walker's Natural Language and Dialogue Systems lab. To be clear, I do think that this work is valuable and that the Alexa Prize is a good initiative for conversational AI. My qualm is with the notion that procedural conversation around popular topics is a compelling utilization of the Alexa platform from the standpoint of producing actual media experiences. As such, I believe that this work is advancing the cutting edge of (task-driven) conversational AI, but not computational media, and the usage of terms like ‘engaging' and ‘experience' (in the call [ram2016are] and in a recent competition report [ram2017conversational]) suggest an unexamined conflation of these concerns.] The idea faded and eventually we attempted to submit a revised form of an earlier National Science Foundation grant proposal, but as I mentioned at the beginning of this chapter, that would ultimately be rejected. Luckily, Noah found another funding opportunity that would cover the remainder of my PhD tenure, but it was one that was adjacent to my research. A year or so later, I began exploring the integration of fine-grained character actions into *Talk of the Town*, a journey I explained in the previous chapter. This was the first stage of my original dissertation project, which concerned the computational modeling of *autobiographic storytelling* and *persuasive storytelling*—this meant generating the kinds of life stories that people construct to make sense of themselves, and also stories that people tell as a way to change the world (particularly to persuade others). My plan was to have simulated characters accumulate knowledge about what has happened in a storyworld and then curate that knowledge to tell stories about what has happened. In procedurally generating either an autobiographic story or a persuasive one, the storyteller would pursue a goal by targeting a particular *recipient response*—such as someone deciding not to trust someone else—and then chain backward from that response to sift out relevant story material and narrativize it in support of the storytelling goal. While I was highly intrigued by the notion of modeling this storytelling phenomena, which has not been done before in the history of story generation, I did not have any ideas about a media experience that would be enabled thereby. I could imagine *Procedural Wobegon* characters telling stories to one another, but what I was aiming at in this project was an extremely deep dive into the phenomenon, which would take me far beyond the modeling that would be necessary to support the kind of diegetic storytelling (a type of *embedded narrative* [ryan1986embedded]) that one encounters in media. One prospect was to develop a videogame based on this phenomenon—a fully computational *Bad News*-like, perhaps—but that seemed out of scope given my targeted graduation date. After some soul searching, in which I almost changed my topic to an historical endeavor, I decided that I wanted my final UC Santa Cruz project to be mediacentric, meaning I would proceed from a vision for a media experience, rather than from a technology that could conceivably drive a nebulously defined media experience. [Note: If I pivoted to the historical project, my dissertation was going to be titled *Inside Computational Media: Five Histories Plus Reimplementations*. This would riff on Roger Schank and Chris Riesbeck's 1981 classic *Inside Computer Understanding: Five Programs Plus Miniatures* [schank1981inside], which its authors later described in this way: “That book contained two kinds of material. First, there were substantial chapters summarizing doctoral work on various models [...] developed at Yale. These chapters were written by the original researchers and described both the theoretical aspects of their work and the practice details. Second, each chapter was accompanied by miniature or ‘micro' versions of each AI program discussed. These Lisp programs ranged from 5 to 15 pages long, including substantial commentary [...] The intent was that the programs would capture the bare essentials of the doctoral work, while still be short enough and simple enough to be easily understood and modified by anyone with only a cursory introduction to AI programming” [riesbeck2013inside, p. xxi]. One of these miniatures was *Micro Tale-Spin*, and indeed it was through this book chapter that Jim Meehan's system became the most famous early story generator (personal communication with Meehan, June 9, 2017). Later, in 1992, Warren Sack translated this version into a Common Lisp program [sack1992micro] that still runs today. Sack has recently written about this experience and about the phenomenon of the *miniature* [sack2018hox], and another recent paper by Nick Montfort and Natalia Fedorova draws on the topic [montfort2012small]. My plan for this alternative dissertation was to provide historical case studies and coupled *reimplementations* for five forgotten systems in the history of computational media. In his paper that I have just referenced, Sack argues that reimplementation is a productive cultural form in the sense of humanities-style translation: “Whether we call these iterations ‘rational reconstructions,' ‘critical reimplementations,' ‘translations,' or just ‘rewrites,' it became clear to me why Donald Knuth employed the phrase ‘the art of computer programming.' In programming, just as in art, copying is both the highest compliment and one of the most important forms of production” [sack2018hox, p. 14]. Recently, Montfort and collaborators have taken up the *Renderings* project, which takes on the challenge of coupled reimplementation *and* (natural-language) translation of obscure non-English historical computational works [marecki2017renderings]. One output of this project [montfort2018exquisite] was featured as part of the exhibition at least year's Workshop on the History of Expressive Systems [ryan2017workshop], which I co-organized with Mark Nelson; the exhibition also featured other reimplementations by Liza Daly [daly2018saga] and Hugo van Kemenade [vankemenade2018love]. I hope to see growing interest in critical reimplementation, and in the history of computational media, more broadly. Maybe I will still write that book.] This move occurred in tandem with my transition at UC Santa Cruz into the computational media department, from computer science, which reflected this emphasis on proceeding from media design to technology design, rather than vice versa. In the process of attempting to devise a final media project to cap off my time in graduate school, I ended up revisiting the *Procedural Wobegon* design, which had now been shelved for over a year. This idea had stuck around in my head and was still very exciting to me, but it seemed that an experience with speech interaction was now out of scope. I decided to undertake the project, but I would reduce the scope to mere generative audio drama, not *interactive* generative audio drama. [Note: Note: ‘audio drama' is a more generalized term for what is often called ‘radio drama', reflecting the emergence of examples that exist outside of the medium of radio, for instance, as podcasts [hand2011radio]. Likewise, the term ‘audio play' can be used to refer more generally to an individual instance of the form.] In any event, no one has ever tried either, to my knowledge, so the project would still be ambitious (and novel). [Note: A few projects have explored adjacent territory. Max Bense and Ludwig Harig's 1968 radio play *Der Monolog der Terry Jo* [bense1968monolog], an inspiration that I discuss below, incorporates procedurally generated text and synthesized speech (using a vocoder). *Breaking Out*, a 2012 audio play by BBC R&D, alters its content based on the time and geographic location of the listening session [churnside2016object,forrester2012creation]. To express such variation, a synthetic voice delivers some of the generated lines. While certainly related, neither of these antecedents generates the actual narrative of the audio play. To stretch a bit further, other related work includes *When in Rome* [sensible2018when] and *Word Mine* [sensible2018word], two voice-augmented tabletop games in Sensible Object's *Voice Originals* series, and Earplay's growing collection of voice-driven works of choice-based interactive storytelling: *Earplay Demo* [grossman2016earplay], *Mr. Robot Daily Five Nine* [grossman2016robot], *Jurassic World Revealed* [earplay2018jurassic], *You and the Beanstalk* [earplay2018you], *Office Hours* [earplay2018office], *The Orpheus Device* [earplay2018orpheus], *Codename Cygnus Reactivated* [earplay2018codename], and the upcoming *Pugmire* [webb2019pugmire]. Though not efforts in generative audio drama, these are examples of audio-based computational media.] With this plan in place, I began my exploration of integrating fine-grained character actions in *Talk of the Town*, which would eventually result in the *Hennepin* framework. I now had a simulation engine that could drive the generative audio drama—it would recount life in a *Hennepin* county—but I needed to design out the latter. My original plan for the audio drama materialized after a conversation with my labmate Jacob Garbe. He had been advising me on prospective technical solutions for *speech synthesis* [kleijn1995speech], which would be a central concern in the project. An initial challenge was that most of the synthetic voices at my disposal sounded robotic, and so I was considering how to lean into those aesthetics to design around them. Jacob suggested a brilliant idea, in the form of what Ian Horswill calls a *narrative alibi*. In his paper “Game Design for Classical AI”, Horswill identifies design patterns that afford a clever circumnavigation, or even embrace, of the otherwise undesirable quirks inherent in procedural systems. One of these involves the deliberate crafting of a narrative framing that allows for the quirks: > Narrative alibis[:] Use the game narrative to explain away the dysfunctional behavior. For example, make the AI characters be zombies, children, or kooky aliens. [horswill2014game, p. 29] Jacob's idea was to cleverly house the audio plays in a diegetic frame: in the distant future, a troupe of robot actors attempt to put on an episodic audio production concerning human life in an American county. This was a great narrative alibi, because it could explain away a number of potential quirks. If the voices sound robotic, it is because the actors are robots; if the procedurally generated language is awkward, it is because robots wrote it; if the events recounted do not make sense, it is because they express a robotical view of human life. [Note: The earlier procedural audio play *Breaking Out*, mentioned above, utilized this very conceit [hand2011radio, pp. 104–105, 113].] I decided to run with this idea, with the working title *The Procedural Players Present*. Each episode would begin with the robot actors, the titular Procedural Players, being introduced with regard to the *Hennepin* characters they would be portraying in the episode—for example, “This week, we have *Matthew* as... Edward Runyan, the hard-working, hard-drinking mayor of Runyan, USA”. [Note: Matthew is one of the Amazon *Polly* voices. My plan was to utilize all the English-trained voices with their names intact, and together they would compose the troupe of robot actors.] After introducing the full cast, a narrator would commence the episode with a phrase like, “And now, the Procedural Players present: ‘A Nothing Place'{”}. I excitedly began to sketch out some text snippets, which I synthesized using the various voices that are included in Amazon's *Polly* framework [polly]. To my dismay, the voices were far too robotic, even with the narrative alibi in place, and character dialogue was simply not aesthetically viable. I considered abandoning the project at this point, because I was not willing to make such severe aesthetic concessions, but then I began experimenting with a particular *Polly* voice, called *Justin*. While *Justin* is designed to be coded as the voice of a male child, to me it sounded like a young woman. In particular, it reminded me of Joanna Newsom's evocative voice-over narration in Paul Thomas Anderson's *Inherent Vice* (2014) [anderson2014inherent]. [Note: Gimlet Creative producer Katelyn Bogucki independently made the same observation when she heard the voice later on (personal communication, May 25, 2018).] Working with a synthetic voice feels distinctly like a collaboration—craft text, or build a text generator, and literally hear the result—and this one got off to a good start. I began experimenting with prospective aesthetic styles, and eventually I converged on a lyrical approach to narration that was inspired by Nate DiMeo's delivery in the historiographic storytelling podcast *The Memory Palace* [memorypalace,larson2018memory]. By this point, I had converged on a workable aesthetic, and a journey from *Procedural Wobegon* to *The Procedural Players Present* to this ultimate form had culminated. *Sheldon County* is a generative podcast that recounts the emergent characters and events of a a *Hennepin* simulated storyworld. More specifically, the project concerns the procedural generation of a multitude of podcast series, each of which recounts a unique storyworld for a specific listener's listening pleasure. To produce a given podcast series, a system called *Sheldon* sifts through the accumulated history of a *Hennepin* county to excavate promising material that it then narrativizes to construct episode scripts, which are then mounted in a podcast series through the utilization of speech synthesis and algorithmic music composition. As such, it is an archetypal work of *curationist emergent narrative*, and moreover one that features fully automatic curation. Unlike *Diol/Diel/Dial* and *Bad News*, the subjects of my earlier case studies, *Sheldon County* has not been completed. The project is currently in an early stage of development, but its entire architecture is fully developed and I have produced an end-to-end proof of concept in the form of two example generated episodes that have been released online. [Note: These are available online at https://soundcloud.com/james-ryan-887346009.] In this chapter, I will describe the design of *Sheldon County* and the architecture instantiated in its curation system, *Sheldon*. Because the project is still in development, it is too early to provide a full case study, which means this chapter will lack the ‘pleasure' and ‘pain' sections that characterized the structure of Chapters 8 and Chapter 10. However, it is already possible to discuss how the project, through its design and architecture, instantiates the curationist framework that is the subject of this dissertation. As such, I will discuss the project with regard to those concerns. Finally, though I have talked about the project in media appearances, this chapter represents its first official reporting. ## Vision *Sheldon County* is a generative podcast about life in a *Hennepin* simulated storyworld. More specifically, it is a collection of podcast series, each of which is procedurally generated, by a system named *Sheldon*, for a particular listener's listening pleasure. Both *Sheldon* and the eponymous county are named after Sheldon Klein, the unheralded pioneer of expressive artificial intelligence who featured heavily in Chapter 4. In this section, I will provide a brief description of how I imagine the experience itself working once it has been completed; a description of the actual architecture undergirding the experience appears in Section 12.3. #### Inspiration: *Der Monolog der Terry Jo * Before describing my vision for the *Sheldon County* experience, I would like to discuss an historical project that has inspired me in this effort. I mentioned above that there has been essentially no prior exploration into generative audio drama, but two projects have come close: while its narrative is not procedurally generated, BBC R&D's *Breaking Out* (2012), which I learned about only very recently, alters some of its content according to the geographic location of the listener and the time and date at which she listens. As cocreator Anthony Churnside explains in his dissertation on the project, the play's writer defined templates whose gaps would be filled in different ways depending on this data: > The personalised variables identified are listed below: - Town where the story is set. - Three well known places located nearby where the story is set. - The weather at the time and place the listener is listening. - Films being shown at a cinema near to the listener?s location. - The date the listener is listening. - The news on the date the listener is listening. - The social network used by the listener. The variables which could be changed needed to be identified, and the writer needed to determine what the possible variables could be based on the location where the audience is listening. [churnside2016object, p. 105] The earlier example that has been particularly inspirational to me is *Der Monolog der Terry Jo* [bense1968monolog], a 1968 radio play by Max Bense—a pioneer of computer poetry and computational art more broadly [beals2018new, uhl2018einleitung]—and Ludwig Harig. The piece recounts the true story of a young girl, Terry Jo Duperrault, who in 1961 was rescued after what initially appeared to be a yachting accident, but was in fact the scene of a puzzling multiple homicide [duperrault2010alone]. For some time after being rescued, Terry Jo spoke in a highly peculiar way, as an effect of the trauma that she had endured aboard the sailing vessel. Eventually, she came out of this state and returned to speaking as she normally would, and this was the impetus for integrating computation into *Der Monolog der Terry Jo*. In his own dissertation, which in large part concerns this project, Kurt Beals explains its peculiar utilization of technology in this way: > the radio play begins with a radical foray into the mechanical realm: when Terry Jo's voice is first heard, it is represented by a Vocoder, which reads sequences of letters or words randomly generated by a computer. Nine of these sequences are heard, with each successive sequence moving one step closer to natural language. In the first sequence, letters are selected completely at random, producing combinations such as “fyuiomgesevvrhykfds” [...]. In the second, letters are chosen using a stochastic process based on the letter frequency in a set of source texts: “h-rahhueber–sh-dfnupz.” By the fourth, the frequency of three-letter combinations is taken into account, and already the text begins to look more like standard German: “has-wirklieb-stion-und-fusse-etwas.” Finally, the ninth approximation uses words and not letters as the basic units, and takes into account the probability of any given three-word sequence, yielding phrases such as “das was weiss ist die sich niemals mit vater sagte er weg gehen.” [beals2013dada, p. 95] [Note: As Beals notes [beals2013dada, pp. 79–83], this method evokes that which is employed in Claude Shannon's famous 1948 paper “A Mathematical Theory of Communication” [shannon1948mathematical]. In this founding document of *information theory* [gleick2011information], Shannon procedurally generates language using a statistical approach and mechanical means. Here, his first synthetic word is ‘xfoml‘, which I have adopted as my Twitter handle.] The radio play was later adapted to the English language by Robert Goss, as *Monologue: Terry Jo*, and a production aired on Pacifica Radio in the Bay Area in 1984 [bense1984monologue]. I have heard both versions, and they are fantastic. What particularly inspires me is its creators' remarkably keen sense as to the aesthetic value of procedural text and speech synthesis (in this case, vocoder speech). Bense and Harig did not incorporate these computational oddities merely because the result would sound strange and futuristic, but rather they took this approach to serve a larger artistic vision: in real life, Terry Jo fell into a state in which she could only speak in an alien cadence, but this gradually dissipated, and this is striking. In turn, Bense and Harig utilize procedural elements, and gradually reduce those elements over the course of the play, to parallel Terry Jo's linguistic fugue. So what I take away from this example is a particular emphasis on using procedural text and speech synthesis—more broadly, procedural generation—not merely because the result is alien and strange and futuristic, but to satisfy particular artistic aims to which the technology is well-suited. In my case, I am trying to tell stories about computer characters living out little computer lives, and so the aesthetics of procedural text and speech synthesis meets my expressive aims. If I were to instead hand-author the episode text for voicing by a human narrator, there could be an aesthetic mismatch—the narration would sound too natural for the strange abstract world that it recounts. Incidentally, I have no choice but to generate text and use speech synthesis, since it would be impossible to handcraft narration that fits every possible world, but the fact that it is even worth doing in the first place is rooted in the concord between generative worlds and generative aesthetics. This is what I have learned from *Der Monolog der Terry Jo*. #### Requesting a Series A prospective listener navigates to the *Sheldon County* website to request that a unique podcast series be generated for her, which she does by entering in a *storyworld seed*. As I explained in Section 11.1.3, *Hennepin*'s procedure for world generation proceeds from a *random seed* [marsaglia2003random], which means each possible storyworld is indexed by a unique number. In this way, in entering a particular storyworld seed, the prospective listener is in effect requesting the particular storyworld that is indexed by that number in *Hennepin*'s possibility space. [Note: It is interesting to reconceptualize seeded generation as indexed retrieval. Rather than constructing an output by using the seed, the system simply retrieves it, fully intact, from the corresponding index in the possibility space.] In addition to the random seed, the interface also requests contact information for the prospective listener (for example, an email address). Once she has entered both of these items, the interface expresses that her series will be ready in a day or so, at which point she will be contacted with information on how to access it for listening. #### Creating a World Once a new series has been requested, a cloud-based instance of the *Hennepin* framework kicks into gear to simulate the history of the requested storyworld, which it does by setting its storyworld seed to the one that has been requested. From here, the simulation proceeds according to all the procedures that were outlined in the previous chapter, resulting in a huge accumulation of data pertaining to that world and its history—in curationist terms, this is a *chronicle*. This chronicle will include something on the order of a few million transpired character actions, as well as miscellaneous data representing every entity that has ever existed and the relations among these entities (such as social networks). #### Building a Series From here, the system *Sheldon* proceeds to sift through this accumulation to excavate material pertaining to characters and events that could be recounted in the podcast series; this is *story sifting*. As I will explain in more detail in Section 12.3, *Sheldon* also does *narrativization* by using packages of the excavated material to construct actual episode scripts, for which narrational speech is synthesized using Amazon's *Polly* service [polly]. Additionally, *Sheldon* generates a musical score for each episode using the Google *Magenta* framework [eck2016welcome]. The result of this procedure is a set of audio files, each corresponding to one episode of the series. I am not sure yet how many episodes each series will comprise, but in any event *Sheldon* will generate an entire series all at once. #### Packaging Up a Podcast Once the episode files have been generated, as a final step *Sheldon* must package them up into an actual podcast. I have not yet developed an approach for this, but I imagine the files being uploaded to a server, where they would be associated with an *RSS feed* [hammersley2003content] that is created for the particular podcast series at hand. Each individual series will have a unique title that incorporates the storyworld seed—for example, *Sheldon County #1515459035*. At this point, a full-fledged podcast has been created, but my goal is to have the series be accessible from conventional podcast syndication platforms, such as the Apple *Podcasts* app. As I argued in Section 4.2.5, it is critical to mount procedurally generated content in actual full-fledged media experiences. While I am targeting the genre of audio drama, I am more specifically targeting the medium of the podcast. We are in the “golden age of the podcast” [bech2014this,hempel2015podcasts], and I want *Sheldon County* to work just like other podcasts work today—that means being available on the common platforms that listeners use to access them. Unfortunately, this could be challenging, since the syndication platforms make automatic targeting difficult (as a form of spam prevention and quality control). As a fallback approach, I could develop a standalone *Sheldon County* app that delivers the generated audio, but this would not quite meet my vision of mounting the curated stories in actual podcasts. #### Listening Finally, once the podcast becomes accessible, ideally via the popular syndication platforms, *Sheldon* contacts the listener, using the contact information that she provided in her request, to provide information on where and how to access her new podcast. Now she may listen to it, and potentially she could send her seed to a friend as a way of sharing her storyworld and its stories. ## Background: *Expressionist * In this section, I will provide a brief overview of the technology that I use in this project to drive the generation of episode scripts. *Expressionist* is a tool for building text generators that may be used in works of computational media, especially in cases where content should reflect dynamic aspects of an underlying system state. I have been developing the tool over the last few years with the help of several undergraduate collaborators: Taylor Owen-Milner, Max Fisher, Ethan Seither, and Tyler Brothers, the latter of whom is now leading its development. *Expressionist* has been used in a few videogame projects to date, as outlined in a series of case studies reported in an earlier paper [ryan2016expressionist]. More recently, Jonathan Lessard and his *LabLabLab* collective have used it in *Hammurabi* (2017) [lessard2017hammurabi], an ambitious videogame featuring what is to my knowledge the most extensive use of text generation in the medium to date. In a recent paper, Lessard and his collaborators outline their usage of *Expressionist* in the project [lessard2017striving]. Finally, the general approach here has been adapted for Spirit AI's *Character Engine* middleware technology, which to date has been used to produce a pair of in-house demonstration experiences [samuel2018playable, p. 276]. #### Building a Grammar Using *Expressionist*, an author defines a *context-free grammar* [bundy1984context] whose nonterminal symbols may be attributed author-defined tags. As symbols are rewritten in the generation process, their tags are accumulated, and the resulting generated content comes packaged with all the tags of all the symbols that were rewritten. As such, rather than text alone, a *content bundle* coupling text and tags is produced. The use of tags is the tool's calling card and what distinguishes it from Kate Compton's *Tracery* [compton2014tracery,compton2015tracery], a related tool that influenced its design. This tagging scheme works similarly to an *attribute grammar* [knuth1968semantics,knuth1990genesis], except that tags are attached to nonterminal symbols rather than production rules. #### Targeted Generation and Content Understanding When text generation operates over a tagged grammar, two critical patterns are enabled: *targeted generation* and *content understanding*. While text generation using a context-free grammar conventionally entails a random sampling of the grammar's possibility space, the addition of tags provides for targeted generation: a system forms a *content request* that specifies the tags that should and should not be bundled with the generated content. [Note: I call these, respectively, the *desired tags* and the *prohibited tags*.] Moreover, because the approach entails the generation of content bundles, rather than text alone, a system can actually be made to “understand” the content. This is done by specifying system-side logic that processes the author-defined tags as a kind of structured data. For example, in my *Talk of the Town* dialogue generator [ryan2016aiide], a content bundle coupling generated text with the tags ‘move:greeting‘ and ‘tone:rude‘ is understood by the system as being a rude greeting. When content understanding is possible, a system can update its state, in meaningful ways, in response to particular instances of generated content. #### *Productionist * To be clear, *Expressionist* is a web-based tool with which an author may define the source data that a text generator will operate over—in this case, a tagged grammar—but it is not itself a text generator. In the parlance of this project, a *Productionist* module is a text generator that one develops to meet the needs of a particular work of computational media. These modules vary across projects, because each puts into operation application-specific concerns that are captured in the tagset of the corresponding *Expressionist* grammar. Specifically, the job of a *Productionist* module is to satisfy content requests that are submitted to it by some other system, such as a dialogue manager (or *Sheldon*). #### *Reductionist * Additionally, there is another module that is built into the tool, called the *Reductionist*. This module is tasked with reducing the massive possibility space of a tagged grammar—where limited authoring may lead to trillions or quadrillions or more generable outputs—to a much smaller space of *expressible meanings*. An expressible meaning bundles a unique collection of grammar tags with all the *recipes* for generating outputs that come packaged with exactly that set of tags. The idea is that while a grammar may have a quadrillion paths—consider that each possible output equates to a unique path through the grammar—not all of the corresponding outputs will be meaningfully different with regard to the concerns of the project. For instance, subtle variations in sentence structure or lexical choice (such as those produced by synonym replacement) or even punctuation may create variant outputs that are technically distinct strings, but that really have the same essential meaning. In *Expressionist*, this distinction is operational, since content packages that have the exact same tags will be understood as being the same by a system that requests them: by the pattern of content understanding, a system understands the generated content by reasoning over its bundled tags, since otherwise natural language processing would be required to understand the surface content. As such, each unique possible set of tags (that may appear in a content bundle) represents an individual meaning that a grammar can produce—hence ‘expressible meaning'. Once a set of expressible meanings has been produced by *Reductionist*, targeted generation becomes a trivial procedure: retrieve an expressible meaning associated with all of a content request's desired tags (and none of its prohibited tags), and then target any one of its recipes (since each produces the same tags). This is remarkable, because it recasts search over a space of $n$-illion grammar paths as search through on the order of a few thousand expressible meanings. While in an earlier project Joe Osborn and I explored the use of *software verification* techniques to carry out this work [osborn2017analyzing], I have since developed a different technique that is used in a *Reductionist* module that is built into *Expressionist* itself. [Note: The term ‘Reductionist' is Joe's coinage, which he came up with while exploring a technique for *reducing* tagged grammars to *symbolic visibly pushdown automata* [osborn2017analyzing].] In addition to exporting a grammar, the tool also exports a file containing all of the expressible meanings and their associated recipes (again, these are grammar paths that produce text outputs that manifest the associated meanings). #### Precondition Tags and Effect Tags To express the power of *Expressionist* and to foreshadow aspects of the specific approach to text generation employed in *Sheldon County*, I would like to introduce a few critical authoring patterns that I have utilized in my own *Productionist* modules. By the first pattern, I define *precondition tags* that take the form of snippets of Python code that are evaluated by *Productionist*, at generation time, by applying them against the state of a storyworld. If a nonterminal symbol's precondition tags do not hold, then the symbol cannot be rewritten at that time (because the content it would produce would not be appropriate given the current state of the storyworld). Relatedly, I have also used precondition tags to gate symbol rewriting according to generation decisions that have already been made, as I will explain momentarily. The second pattern concerns *effect tags*, which are specified as code snippets that update *Productionist*'s working memory when executed. In the course of a generation instance, the *Productionist* utilized by *Sheldon* maintains a *working memory*, in the style of a production system [davis1975overview], which is a Python data structure that binds variables to data or to storyworld entities. For instance, the effect tag ‘m.side_character = m.arsonist.best_friend‘ would cause the working memory (bound to a variable ‘m‘) to bind, to its variable ‘side_character‘, the best friend of the character already bound to its variable ‘arsonist‘. In this way, *Productionist* has all of a storyworld at its disposal. The working memory also includes information about decisions that have already been made in the generation instance, including a copy of the partially generated episode script. This allows for the coordination of decisions within a generation instance—for instance, those pertaining to lexical choices—according to constraints that propagate from earlier symbol rewriting to guide later rewriting. In this way, backtracking is also supported, since a failed precondition on a later decision causes backtracking to an earlier choice point, where an alternative decision will be made instead. This authoring pattern relates to an interesting recent project by Eric Butler, Kristin Siu, and Alex Zook, in which they generate videogame boss behaviors by hacking generative grammars with similar authorial affordances [butler2017program]. Another related framework is Aarne Uotila's *Orcus* language [uotila2018procedural], with which an author can build a text generator that is driven by context-free grammars that maintain state within and across generation instances. More broadly, grammar-based text generation with backtracking is associated with the *chart generation* technique [kay1996chart]. To be clear, by using precondition and effect tags, the attribute grammars of *Expressionist* are hacked to resemble something more akin to a production system. #### Runtime Expressions Finally, a third authoring pattern pertains to the use of what I call *runtime expressions*. When generating text for works of computational media, there will likely be cases where certain pieces of content cannot be known at authoring time, which means they cannot be included in an authored *Expressionist* grammar. For example, if an author defines a greeting generator for a roguelike game with generated characters, the names of the characters who will be greeted cannot be known at authoring time. As such, in these cases generated outputs will include gaps that must be filled in at runtime, where the unknown concerns will have been resolved. As such, a runtime expression is simply a snippet of code that can be evaluated to a string at runtime, as the content is being generated. Here is an example greeting with a runtime expression (enclosed in square brackets): “Hi, [interlocutor.name].'‘. Once this greeting is generated by a *Productionist*, the module can fill it in by evaluating the runtime expression, which will of course require the appropriate character to be bound to the variable ‘interlocutor‘ in the namespace of the generation procedure. When runtime expressions are used, the generable instances in an *Expressionist* grammar's possibility space are themselves *templated*, in the style of the authored procedural content in projects like *Prom Week* [mccoy2013prom], *Versu* [evans2014versu], and Curveship [montfort2007generating]. ## Podcast Generation In this section, I will explain the operation of *Sheldon*, which is the system that handles podcast generation in the *Sheldon County* project. As I noted in Section 12.1, podcast generation is triggered when a prospective listener requests a podcast series to be generated for her, according to a particular storyworld seed. While I have produced two proof-of-concept episodes that were generated by an end-to-end prototype of this architecture (save for music generation), my approach is bound to change as the project developers further, and there is considerable authoring work that still needs to be undertaken. Namely, this work will entail the authoring of character actions in *Hennepin* and *episode spaces* in *Sheldon*, a notion that I will explain below. As such, to be clear, all of the forthcoming description is subject to change. ### Storyworld Upon a new series being requested, an instance of the *Hennepin* framework kicks into gear to simulate the history of a storyworld, which it does by setting its storyworld seed to the one associated with the request. In the version of *Hennepin* that I am using for this project, every county is hardcoded to have the name ‘Sheldon County', the idea being that each is like an alternative version of the same essential setting. From here, the simulation proceeds according to all the procedures that were outlined in the previous chapter. Once the project is released, this computation will likely occur in the cloud, since that will be easier to set up than a dedicated server. When the storyworld reaches a preset termination date—for example, October 18, 1987—*Hennepin* ceases its simulation of history. I imagine this will take several hours, and perhaps an entire day. At this point, a huge accumulation of data pertaining to that world and its history has been produced: on the order of a few million transpired character actions, as well as miscellaneous data representing every storyworld entity that has existed and various relations among these entities (such as social networks). In curationist terms—see Chapter 5—this data accumulation is a *chronicle*. ### Nuggets With the chronicle still in working memory, *Sheldon* commences with the curationist task of *story sifting*. In this project, that means sifting through the accumulated history of a *Hennepin* storyworld, as captured in the chronicle. The aim here is to excavate narratively potent material pertaining to characters and events that could be recounted in the podcast. Generally, this entails the employment of *sifting patterns*, in the style of the technical approach that I outlined in the opening of the Chapter 11. Of course, the point of excavating material is to construct a narrative artifact, which means that the patterns should also encode knowledge about how to recount the extracted material. #### Narration Moves and Nuggets For this reason, *Sheldon*'s sifting patterns are each associated with ideas for how the material may be used to build narrative artifacts. I call these descriptor tags *narration moves*, and each represents a high-level goal for a prospective episode that would be constructed out of that material (to recount the characters and events captured by the material); an example is ‘recount-arson-revenge‘. Narration moves are similar to, for instance, the descriptors for plot fragments in *Universe* [lebowitz1985story,lebowitz1987planning] and the rhetorical goals of *Terminal Time* [mateas1999terminal,mateas2000generation,domike2003recombinant]. When a sifting pattern matches against material contained in the chronicle, it packages the extracted material into a *nugget* that also includes the narration moves associated with the sifting pattern. The aim here is to package the material in the nugget in a way that will make the narrativization task (about which more soon) more feasible. #### An Example Let us illustrate this with an example. In the current *Sheldon* codebase, a nugget is instantiated as an object of a class whose definition contains both the sifting pattern and a procedure for recording the extracted material in a structured way. As an example, here is a definition for a nugget class corresponding to the ‘arson-revenge‘ sifting pattern, which matches when a ‘set-fire‘ action has a causal ancestor—an earlier action on its causal chain—that is a ‘hatch-revenge-scheme‘ action with the same initiator: ``` class ArsonRevenge(object): """A sifting pattern for excavating events pertaining to revenge via arson.""" def __init__(self, candidate): """Initialize an ArsonRevenge object.""" self.name = "arson-revenge" self.narration_moves = [ "recount-arson-revenge", "recount-revenge" ] # Apply the pattern to the candidate to # find out whether there's a match self.match = ( candidate.name == "set-fire" and candidate.find_ancestor( name="hatch-revenge-scheme", initiator=candidate.initiator ) ) if self.match: # Set nugget components self.set_fire = candidate # Retrieve "hatch-revenge-scheme" action # that precipitated the arson self.hatch_scheme = ( candidate.find_ancestor( name="hatch-revenge-scheme", initiator=self.set_fire.initiator ) ) # Retrieve the arsonist self.arsonist = ( self.hatch_scheme.binding("arsonist") ) # Retrieve the character who was targeted # by the arson act self.target = ( self.hatch_scheme.binding("target") ) # Retrieve the scene of the arson self.scene = ( self.set_fire.binding("building") ) # Retrieve anyone killed in the fire self.casualties = ( self.set_fire.all_bindings("casualty") ) # Retrieve all actions that caused the # hatching of the revenge scheme self.lead_up = [ a for a in self.hatch_scheme.ancestors if a.initiator is self.target ] # Construct additional narration moves self.narration_moves += [ "explore-relationship(a, t)".format( a=self.arsonist.id, t=self.target.id ), "explore-relationship(t, a)".format( t=self.target.id, a=self.arsonist.id ) ] ``` This nugget class is associated with an ‘arson-revenge‘ sifting pattern and four narration moves: ‘recount-arson-revenge‘, ‘recount-revenge‘, and the two ‘explore- relationship‘ variants, which I will explain momentarily. [Note: In the case of ‘explore-relationship‘, the narration move accepts arguments (pertaining to the characters involved in the relationship).] The sifting pattern itself is captured by this snippet: ``` Apply the pattern to the candidate to # find out whether there's a match self.match = ( candidate.name == "set-fire" and candidate.find_ancestor( name="hatch-revenge-scheme", initiator=candidate.initiator ) ) ``` By this code, a match occurs when the candidate is a ‘set-fire‘ action that has a causal ancestor that is a ‘hatch-revenge-scheme‘ action with the same initiator. If the candidate triggers a match, then an actual structured nugget is built using the code listed above. This entails retrieving the arsonist character, the target of her arson attempt, the scene of the crime, and any casualties who may have died in the fire. Additionally, the lead-up to the hatching of the scheme is retrieved. These will be a series of actions that were taken by the target and precipitated the revenge attempt by the arsonist, such as the ‘ridicule‘ actions shown in Figure 11.3. This kind of backward chaining from a ‘most reportable event' (the ‘set-fire‘ action) to its causal ancestors is an operationalization of William Labov's procedure for *narrative pre-construction* [labov2006narrative], which I outlined in Section 4.2.1. It is important to note that *Hennepin*'s extensive causal bookkeeping, reported in Section 11.2.7, makes story sifting feasible to do automatically. ### Episode Spaces Once a nugget has been extracted, it can be used to drive the curationist task of *narrativization*, which *Sheldon* also carries out. In this project, narrativization means constructing an episode script that recounts the material contained in the nugget. Such a script can be said to *perform* the narration moves that are associated with the nugget—thus, an ‘arson-revenge‘ may be used to construct an episode script that performs the narration moves ‘recount-arson-revenge‘, ‘recount-revenge‘, and the two ‘explore-relationship‘ variants. #### Performing Narration Moves Nuggets may have multiple narration moves as a way to provide multiple hooks into the chronicle for *Sheldon* to exploit. As I will explain shortly, in the process of generating a series of episodes, the system may propose narration moves to perform in future episodes, given what has been recounted in earlier ones. For example, a generated script may lead to a proposal for a future episode to explore the relationship of two characters that were mentioned in that script. Such a narration move would take a form like ‘explore-relationship(121, 309)‘, where ‘121‘ and ‘309‘ are identifiers for the characters whose relationship would be explored. If a case of arson by revenge had transpired between these characters, that would certainly be an interesting way to explore their relationship, and this is why those narration moves are included in the example nugget definition above. In this way, nuggets can serve different kinds of storytelling goals, potentially simultaneously, which provides leverage to *Sheldon* and leads to diverse uses of the same patterns across different series. For example, an arson story could be the central event in one series and an interesting background episode in another. *Sheldon*'s narrativization procedure is structured as a series of attempts to perform particular narration moves. Once the system has sifted out as many nuggets as it can from the chronicle, given the sifting patterns that have been authored and the history of the particular storyworld at hand, it will have an initial set of candidate narration moves. As episode scripts are generated, additional narration moves may also be proposed, as I will explain momentarily. To perform a given narration move, a corresponding nugget must be at hand. If one is available, the move can be attempted by targeting a corresponding *episode space*. An episode space is an *Expressionist* grammar that makes use of the two authoring patterns that I described in Section 12.2: precondition tags and effect tags. To be precise, *Sheldon* employs a *Productionist* module that operates over the grammar to procedurally generate an episode script. When working memory is used in a *Productionist*, the procedure works more like executing a computer program than generating from a context-free grammar, and so I conceive of the episode spaces as computer programs for producing particular kinds of episodes. #### Generating a Script To generate a script out of an episode space, *Sheldon* forms a content request that includes (in its desired tags) a tag representing the targeted narration move. In my *Expressionist* grammar, I attach *move tags* to certain nonterminal symbols to express that the content produced by expanding them will lead to episode scripts that perform those narration moves. This allows for coordination between story sifting and narrativization: sifting patterns, nuggets, and episode spaces are all associated with narration moves, as a kind of *lingua franca*, which means sifting patterns can be used to extract nuggets that can be recounted by sampling an episode space. When the *Productionist* module receives a content request with the targeted narration move, it retrieves all of the expressible meanings associated with the corresponding move tag; these are called the *satisficing meanings*. Next, the module begins targeting recipes for those meanings, one by one. As I noted above, a recipe for an expressible meaning is a grammar path that may be traversed to produce content that expresses the meaning (as instantiated by the set of tags bundled with the content). To target a recipe, the *Productionist* adds the contents of the nugget at hand into its working memory. Additionally, the working memory includes a pointer to the storyworld and to the entire chronicle, so that it may evaluate any precondition tags attached to nonterminal symbols in the episode space (since preconditions may reference anything about the storyworld or its history). Next, the module works through the grammar path specified in the recipe. If on that path the module encounters a nonterminal symbol with effect tags, it carries out the corresponding updates to its working memory. Alternatively, if it encounters a nonterminal symbol with precondition tags, it evaluates the preconditions against the working memory. If the preconditions fail, *Productionist* abandons the recipe and moves on to the next one, reverting the working memory to its earlier state; this is essentially a form of backtracking. If all of an expressible meaning's recipes are exhausted, it moves on to the next satisficing meaning. If eventually it abandons the last satisficing meaning, then the targeted narration move cannot be performed given the nugget and storyworld at hand. As the *Productionist* is working through the recipe, it also updates the working memory to contain the partially generated episode script and a list of nonterminal symbols that have already been expanded. With this information recorded in the working memory, the system can coordinate decisions within a generation instance, so that certain later decisions cannot be made if incompatible ones were made earlier on. If at any point a recipe is successfully targeted (all of the preconditions hold for all of its nonterminal symbols), then the resulting generated output will be a complete episode script. #### Title Proposals When an episode script is generated, it comes bundled with three additional kinds of tags: *title proposals*, *narration proposals*, and *bible updates*. To be clear, these are simply examples of other kinds of tags that I attach to nonterminal symbols in authoring episode spaces. Each title proposal contains a prospective title for the episode, along with a *score* representing the utility of that title. Thus, title proposals accumulate as a byproduct of symbol rewriting, and some titles may be better to use than others. For example, if rewriting a particular nonterminal causes a very specific chunk of content to be included in the episode, a salient phrase in that chunk might make for the perfect title. In this case, the score in the corresponding title-proposal tag would be very high. Once a script has been generated, *Sheldon* selects a title probabilistically according to the distribution of scores across the set of proposals. #### Narration Proposals A narration proposal has the same structure, except that instead of titles it proposes narration moves for upcoming episodes. In this way, the specific content in a generated episode may suggest ideas for upcoming episodes, and so forth. For example, if a rivalry between two historical figures in the county's history was mentioned in one episode, it might be worthwhile to explore that rivalry in a follow-up episode. In this way, a ‘revenge-arson‘ nugget like the one above may be used to perform the ‘explore-relationship‘ narration move. More precisely, *Sheldon* would have already extracted the nugget (since the system extracts every possible nugget before generating the first episode), so it would simply be retrieved from an existing collection of extracted nuggets. #### Bible Updates Finally, *Sheldon* also maintains a *show bible*—a term adopted from television writing [espenson2008how,wells2014sorry]—that contains critical information about characters and events that have already been recounted in previous episodes. [Note: A recent paper by Eugenio Concepción and collaborators proposes to operationalize the notion of a show bible in a brilliant shared evaluation task: a variety of story generators each operate from the same shared knowledgebase, meaning they produce stories about the same characters and events [concepcion2016challenge]. In this way, the outputs of different story generators would truly be comparable. (This kind of integration of heterogenous story generators reminds me of the *Slant* blackboard architecture [montfort2013slant] that I mentioned earlier on.) A curationist spin on this task would be to use, as the story bible, a chronicle that records the characters and events that actually emerged in a simulated storyworld. I hope their idea catches on.] The bible-update tags are structured like effect tags, but they instead cause updates to the bible (and only after an episode script has successfully been generated). Precondition tags may also reference the contents of the show bible, which allows for coordination across particular authoring decisions made in each episode. ### Audio Once an entire series worth of episode scripts has been generated, *Sheldon* must produce audio files for each of them. As I have mentioned above, speech synthesis in this project is handled by the Amazon *Polly* framework. This is a cloud-based service that handles requests and sends back audio files (in MP3 format). Requests take the form of the raw text that is to be voiced, but that text may also include expressions in the *Speech Synthesis Markup Language* (SSML) [taylor1997ssml,baggia2010speech,amazonssml], which allows for control over dimensions including the rate and pitch of the synthesized speech. As such, SSML tags can be utilized in authoring an episode space. Because *Polly* has a fairly low character limit, *Sheldon* breaks an episode script into smaller chunks, makes requests for each, and then stitches the resulting audio files together into a single file. #### Score Generation Finally, *Sheldon* will generate a musical score for each episode by utilizing the Google *Magenta* framework [eck2016welcome]. To be clear, this aspect of the architecture is only planned, not implemented. While my proof-of-concept episodes that are already available online did not feature generated scores—but rather music by the human composer Lee Rosevere—I have experimented with *Magenta* fairly extensively and reached a level of satisfaction with some of my trained models. *Magenta* is a essentially a wrapper around the *TensorFlow* framework [abadi2016tensorflow] for *deep learning* [lecun2015deep] that is intended to fuel efforts in procedural generation (and also *creativity support tools* [shneiderman2002creativity]) in the domains of music and visual art. The models that I have been exploring apply *sequence-to-sequence* techniques [sutskever2014sequence] to *MIDI* data [briot2017deep]. MIDI is not audio, but instructions for producing a sequence of sounds, and so a critical final step for *Sheldon* will involve the synthesis of actual audio from the generated MIDI data. In my experimentation phase, I have been doing this manually using software instruments in an audio workstation, which means that I still need to settle on a technical solution for doing this automatically. [Note: While some do not appreciate *Magenta*'s bottom-up, knowledge-poor approach to music generation, I find it to be conceptually intriguing in the tradition of the *aleatoric music* [roig2008understanding,jensen2009john], and earlier methods for *dice music* [hedges1978dice], as well as *chance art* [molderings2010duchamp] more broadly.] #### Backup: Procedural Score Selection If this approach to musical score generation does not pan out either technologically or aesthetically, another approach could entail the automatic selection of background music from a corpus of tagged audio files. This could be driven by *Expressionist* tags, for instance, with a special tagset that works like the title proposals, but instead proposes the mood of a background score according to particular decisions that have been made with regard to the content of an episode script. In this way, the accumulated mood proposals for a script could be used to programmatically retrieve an audio file for background music with the corresponding mood tags. #### Letting *Bardo Compose* A variant approach could follow recent work by Rafael Padovani and collaborators, who developed a system, called *Bardo*, that automatically scores a tabletop roleplaying game session by using speech recognition to sample the sentiment of the players, which it attempts to match by retrieving and playing an audio file with the corresponding mood [padovani2017bardo]. Padovani's collaborator, Lucas Ferreira, is a fellow UC Santa Cruz PhD student who is now further developing *Bardo* as his dissertation project. His ultimate vision for the system is to algorithmically score real-time oral storytelling [ferreira2018advancement]. It would be interesting to apply this system to *Sheldon County*'s synthesized episode narration. Generally, it would be feasible to also include, in the actual content of an episode space, special markers (tags salting the content) that could drive the operation of a musical score generator. This would be in the style of the SSML tags that modulate the synthesized speech, but instead these tags would send control signals to a score generator, for instance enabling musical transitions that are coupled with particular speech events in the generated narration audio. Indeed, this aspect of the project could easily become its own massive undertaking, and this is why I will likely take the easiest approach available, which at this time appears to be *Magenta*'s bottom-up method. ### Delivery Finally, as I have already explained in Section 12.1, *Sheldon*'s last task is to upload the generated audio files somewhere, associate them with a dedicated RSS feed, and contact the listener who requested the series to let her know that it has been created. Like I noted above, this aspect of the project has not been deeply explored at this early stage. ## Curationism and *Sheldon County * [Figure omitted] In this section, I will briefly summarize how *Bad News* instantiates the framework for curationist emergent narrative that I presented in Chapter 5. While my earlier collaborative project *Bad News* is itself a full-fledged curationist project, *Sheldon County* represents a first attempt at fully automatic curation. As I did in my earlier case studies on *Diol/Diel/Dial*, namely in Section 8.5, and *Bad News*, specifically in Section 10.6, I will now proceed to answer the following question: for each curationist architectural component, who or what in this project acts as that component? Figure 12.1 illustrates these concerns with regard to *Sheldon County*, which I will also outline in prose, component by component: - **Experiencer.** *Sheldon County*'s *experiencer* is the podcast listener. Even though each series is generated specifically for one person, there could be potentially many listeners (and thus experiencers). This would occur when someone listens with friends, provides a seed to someone else, or shares audio files or instructions for accessing her podcast. In any case, a *Sheldon County* podcast is not an interactive experience, as Figure 12.1 shows. - **Simulated storyworld.** A *Hennepin* simulation instance is the *simulated storyworld*. It is the procedurally generated American county that is recounted in a generated podcast series. - **Chronicler.** As with *Diol/Diel/Dial* and *Bad News*, there is no distinct *chronicler* module at work in this project, but rather the associated functionality is built into the *Hennepin* codebase. To simplify, we can say that *Hennepin* is its own chronicler. - **Chronicle.** A *chronicle* in this project is composed primarily of millions of character actions that have transpired in the course of a *Hennepin* storyworld's simulated history. Additionally, the record will contain other data pertaining to every entity that has existed during the span of that history, as well relations between those entities. For example, character social networks will be captured in the chronicle. - **Story sifter.** Like *Bad News*, and unlike *Diol/Diel/Dial*, this project features *story sifting*, but critically this task is carried out automatically by the *Sheldon* system. As I explained in Section 12.3, *Sheldon* makes use of *sifting patterns* (first described in Section 5.4) to retrieve nuggets that bundle extracted material and associated narration moves. This aspect of *Sheldon* loosely computationalizes the manual story sifting that I carry out in my capacity as *Bad News* wizard, a process that I elucidated in Section 10.3. - **Material.** As I just mentioned, *Sheldon*'s sifted *material* is packaged into *nuggets* that couple retrieved data with associated *narration moves*. These nuggets express the narrational value of the associated material in terms of what kind of stories could be told to recount it. This material takes the form of data stored in memory. - **Narrativizer.** As in the case of story sifting, *Sheldon County* actually carries out *narrativization*—like *Bad News* and unlike *Diol/Diel/Dial*—but critically this is fully automatic. Narrativization in this system involves the utilization of *episode spaces*, which are authored *Expressionist* grammars with precondition tags that hook into the extracted material of nuggets, which allows for generated text that is tailored to the specific concerns captured therein. *Sheldon* is the narrativizer in this project, and the result of this procedure is a complete episode script. - **Story.** In *Sheldon County*, narrativization results in a constructed narrative artifact—a *story*—that takes the form of a textual episode script. As I have argued at several points, namely in Section 4.2.5, such an artifact is not a full-fledged media experience. By the curationist framework, constructed narrative artifacts should be mounted in actual media experiences. - **Media experience.** In curationism, the term ‘media experience' specifically denotes the experience that constructed narrative artifacts (each being a ‘story') are mounted into for human encounter. In *Sheldon County*, actual audio files (integrating speech and a musical score) are generated using the episode scripts, and the resulting collection of files is uploaded online and associated with an RSS feed. The result of that process is a podcast that has been procedurally generated for a particular listener's listening pleasure. This unique podcast series—something like *Sheldon County #1515459035*—is in curationist terms a *media experience*. #### Summary To summarize, *Sheldon County* is a full-fledged work of curationist emergent narrative, and moreover one that makes use of fully automatic curation. Specifically, this project employs the *feedforward curation* variant of the curationist architecture (see Section 5.3.2), since it mounts stories that are constructed out of *Hennepin*'s emergent material into a second-order experience, a *Sheldon County* podcast. In its instantiation of this pattern, the system *Sheldon* works as a story sifter to excavate narratively potent raw material, which it then narrativizes to build stories that are ultimately mounted in procedurally generated podcast series. I am excited to develop the project further, and I hope to see other works that likewise explore automation of the curationist pipeline.