Guess the original literature classics – The Distant Shout of the Great Storm

Tower of Doom Distant Shout of the Great Storm

Can you identify the original story author used to prompt (start off) the generated text below? The title is also generated.

After, please visit Episode 2: The Bloody Tale.

The generated text is followed by the classic story which was used as the prompt for the computer generation. For technical details see end Notes.

Character names are changed if they appear from the original text. Only paragraph breaks etc. added for clarity.

Made on my creativity app Story Live


The Distant Shout of the Great Storm

It seems that I didn’t remember much, but my memory has been erased as much as any of the others. This was really quite disorienting. I just remember feeling that “something” was happening.

“Aaah… This… It feels right to be in this place… That sounds horrible… Do I really have to die? My thoughts are starting to run away from me, and I’m beginning to feel as if something is wrong with me, and I can’t be sure I know where I am… I can’t be sure of it but… I think I’m getting close, and maybe I can keep moving forward for just a little longer… ”

With that last part, I started moving my body. After moving for a moment, it looked like I was going to try to run away, but I got stuck. There was no way that I was going to pull myself back onto my feet. I needed to get back to my feet as quickly as possible. Maybe I couldn’t run anymore.

The Count, seeing his life gone away with himself, and his brother-in-arms and his friends at the foot of the steps to the entrance of the tower, seized the rapier, and plunged it in the back of his throat; a blow of his thumb, by which the blood rolled upon the floor like a thin sheet, sent the blade through his shoulder and through his spine, and severed his heart.

“Ah, this be the death of all the good! Now be thou slain, let thy spirit go in peace, and myself be dead.”

But as he made an attempt at speech, he heard in the distance the distant shout of the great storm. He saw the pallid faces of many of the men who had fallen of his hand; and with each one raised his dead hands with their dead fingers they raised it again before the black, black, black face of the death, while he staggered away from the tower, pale-faced and pale.

Meanwhile, through the crowd of the dead, a faint flame rose up behind the white curtain. And when the red flame blazed, from the tower, on the walls, all the dead went down behind the curtain.

And now the darkness of ages flowed out from one of the steps.

“For me now to make my home in the ashes of dead men,” said an old man that night in all his glory, on his grave, “if I would. I will see me now, in the sun of a new day, as I remember my old days; as they were, to me they were; and on this grave, and with this pale face, and in these hands, as they should be in my old hands, no more, to be, and to live, in these dead men.

“This world shall be dead, as is the world that came before it, when the world came to be in man’s body.

Then the prince went alone into the black apartment—and with an irresistible vigour, without fear, without regret, came to the last step of the staircase. He went there with a quickness, that neither his feet, nor his arm, nor the arm of any of the party, made a noise in motion, and with suddenness, that neither his feet nor his face, nor his body, or the last step of the staircase, could move, and for a moment the light of the candle, which was not disturbed, glittered.

At last he was called down the stairway; and as he sat down he perceived the presence of the death, and said aloud aloud:—

“Hail! O good man! What a night—and all things that are born of a man are, in the dark hours of his day, as dead as those which lie before him: and he in the dark days of his day, though all things that are born of him should be, in the light and sunshine of his day be, as they were before him, but all things that were before him were dead, in the dark hours of his day.”


Original


The Masque of the Red Death

Edgar Allan Poe 1842

The “Red Death” had long devastated the country. No pestilence had ever been so fatal, or so hideous. Blood was its Avatar and its seal—the redness and the horror of blood. There were sharp pains, and sudden dizziness, and then profuse bleeding at the pores, with dissolution. The scarlet stains upon the body and especially upon the face of the victim, were the pest ban which shut him out from the aid and from the sympathy of his fellow-men. And the whole seizure, progress and termination of the disease, were the incidents of half an hour.

But the Prince Prospero was happy and dauntless and sagacious. When his dominions were half depopulated, he summoned to his presence a thousand hale and light-hearted friends from among the knights and dames of his court, and with these retired to the deep seclusion of one of his castellated abbeys. This was an extensive and magnificent structure, the creation of the prince’s own eccentric yet august taste. A strong and lofty wall girdled it in. This wall had gates of iron. The courtiers, having entered, brought furnaces and massy hammers and welded the bolts. They resolved to leave means neither of ingress nor egress to the sudden impulses of despair or of frenzy from within. The abbey was amply provisioned. With such precautions the courtiers might bid defiance to contagion. The external world could take care of itself. In the meantime it was folly to grieve, or to think. The prince had provided all the appliances of pleasure. There were buffoons, there were improvisatori, there were ballet-dancers, there were musicians, there was Beauty, there was wine. All these and security were within. Without was the “Red Death”.

It was towards the close of the fifth or sixth month of his seclusion, and while the pestilence raged most furiously abroad, that the Prince Prospero entertained his thousand friends at a masked ball of the most unusual magnificence.

 

Notes

Prompt was 10-20 words from the original. GPT-2 system by Fabrice Bellard – see the site below for credits etc.
Made on my creativity app Story Live – please visit.

Text from Gutenburg free classic ebooks

Original 323 words from 2417. Generated is 712 words.

A certain amount of cherry-picking of the most grammatical and interesting generated parts, but no actual editing (moving around of words or new words). Identifying names are changed.

More of these will be posted, see the index.

 

Fake News – what do real writers think about it? AI research

I examined in August 2020 research (UAL London see credits) what happens when writers use a computer text generator to write articles, giving them only an image prompt. There is a summary of the Study in the Index.

Go to Index of AI research

Fake News – (or ‘fake news’) what do real writers think about it?

Summary 

  • People who have a lower general opinion of text generation, and lower wordcounts for feedback, don’t think text generation is relevant to ‘fake news’.

The converse, that people who are more engaged with text generation (more feedback, more positive) can see that it is more related to fake text generation.

So, if people are educated on the topic, they will realise there is a relationship between advances in computer text simulation and fake news.

Definition of fake news

Even defining fake news is not that simple, since if you look for examples in the USA, or Syria, or anywhere else, both sides accuse the other of using it. Depends on there being no consensus as Barak Obama mentioned recently. See the bottom of the blog for some discussion.

Fake News

One of the feedback questions in the Study was about whether computer AI text generation was relevant to ‘fake news’ (false content for spam, propaganda etc.).

Q6-2: If you have heard of ‘fake news’ do you think this is relevant?

This had neutral phrasing in order not to influence replies. It did not say ‘text generator can make fake news’ as this presupposes a technical knowledge that might not be present, even after doing the experiments. Given the sometimes bizarre output of the generator, it might affect replies too specifically.

18 of the 82 respondents – 22% – replied to the question. It was the last question after a long session so perhaps some fatigue had set in. It was also a question not specifically to do with the experiment. About a quarter is still quite a high response rate.

Analysis of responses

I will be examining the actual text answers in another blog. This just looks at some basic relationships.

Several comparisons of the data were calculated and some are worth discussing here (the others are in reference material).

Comparing answer word count with positive – negative ratings:

The actual Q6 answer text was assessed for Positive – Neutral – Negative response to the ‘relevant to fake news’ question.

For instance, the text answer ‘No’ was 2 for negative, word count 1. ‘I stay out of these type debates.’ Is 1 for neutral, word count 7.

This was quite easy to do. I did not use a computer sentiment analyser as the samples were too small. Anything a bit ambiguous or with conflicting statements was rated Neutral.

Pos – neg is 0 (positive), 1 (neutral), 2 (negative) – these are scaled up so can be seen against the word count values. So low red columns are positive (yes, relevant to ‘fake news’) , high red is negative (not relevant to ‘fake news’). This is an ordinal scale.

fake news vs word count.png

Above: Pos-neg is zero for positive for ‘relevance’ to ‘fake news’. The red Pos-neg 0-1-2 value is scaled to make it easier to see. So a tall red line means negative for relevance to fake news. (There is a discussion of different statistical calculations in a forthcoming blog. The tests used in the study are Mann Whitney U, T-Tests, ANOVAS, and Pearsons, along with various bar charts and boxplots.)

At first glance this shows that higher answer word counts are associated with positive for ‘relevance to fake news’.
Low word counts are associated with negative for ‘relevance to fake news’.
There is only one negative out of the top half of the responses (1-9 on the chart above, 2-8 are pos=0).
The highest word count (column 1 on chart) was from an answer that discussed an actual example of real ‘fake news’, or propaganda in the Syrian War, so was a longer than usual negative response. It could be an outlier. After ruminating they decided it was too hard for a text generator to do, writing ‘news of any kind has to be created by humans…’ which is a value judgement (humans are best).

Statistics

Mann Whitney U test

The z-score is 4.25539. The p-value is < .00001. The result is significant at p < .05.

  • People who used the least words thought text generation was less relevant to ‘fake news’.
  • People who used more words in their answer thought it was more relevant to ‘fake news’.

Discussion

In other work, computer Sentiment Analysis revealed that all feedback answers had high scores for ‘Tentative’ and very low (no score) for ‘Confidence’. These results could be because of a general lack of experience of text generation (only 11% had previous experience).
There is a forthcoming blog on the Sentiment Analysis, there is already a blog on Sentiment Emotions (Joy etc.).

When making a positive claim (‘there is relevance’) with no ‘Confidence’ and feeling very ‘Tentative’, discursive answers are to be expected.

In the positive answers, the relevance of ‘fake news’ produced more discussion (‘Yes, and here’s why…’). This could be because the respondent felt it was relevant, but had little or no confidence in their feelings and supplied a tentative, more wordy, answer.

Whereas with a negative response, text generation is not relevant to ‘fake news’, and can be easily dismissed (‘No’ – 1 word).

Ranked Likert scores over the Study vs Pos-neg

Fake-ranked-Likert-Pos-neg.png
Mann-Whitney U Test
The z-score is 5.10963. The p-value is < .00001. The result is significant at p < .05.

This is not visually very clear until you sum the raw (before scaling for graph) scores from columns 1-8 (6) and 2-18 (10).

  • This shows that lower Likert scores (which means positive response to experiments and later questions) relates to lower Pos-neg scores (which means positive relevance to ‘fake news’). This is what would be expected.
  • People who have a lower general opinion of text generation, don’t think text generation is relevant to ‘fake news’ (and possibly anything else of practical use).

Amateur – Professional

People were asked to select Amateur/Professional (or both, scored as Pro) next to Occupations. Most people had more than one occupation (Poet and Journalist etc.) so it is not a fixed job allocation, but provides an indication that the writer either is or would like to be paid for at least some of their work.

This did not provide any significance at the ‘fake news’; response level.

There is a forthcoming blog on Amateur/Professional differences.

Comments

Is there a connection between text generation and ‘fake news’?

1/ Human pride: anthropocentrism: people that dismiss the use of text generation and its relevance to ‘fake news’, say ‘news of any kind has to be created by humans…’.

2/ The idea that a computer could not be used to write news, ‘fake’ or otherwise, might be due to the rather random output of the mid-level (GPT-2) text generator used in the Study. People might think it is producing readable nonsense, and brusquely dismiss any practical uses. This is shown in the relation between overall negative scores (Likerts) and low relevance for ‘fake news’.

3/ Perhaps dismissal of a text generation tool in regard to ‘fake news’ is indicative of not believing there is ‘fake news’ (or that it is not a problem), shown in the low word counts of their short negative responses. Further research required.

Notes

1/ Answers

If you have heard of ‘fake news’ do you think this is relevant?

Replies went from the four shortest (1 or 2 words)

If you have heard of ‘fake news’ do you think this is relevant?

No (scored 2, neg)

No! (scored 2, neg)

Sure (scored 0, pos)

Not really (scored 2, neg)

to the four longest (89, 85, 76, 50 words; 2,2,0,0)

If you have heard of ‘fake news’ do you think this is relevant?

“Unless the AI used to generate the text is extremely advanced, I think any news text generated wouldn’t be very effective — fake or otherwise. I think, at this point in our technology, news of any kind has to be created by humans to effectively educate, manipulate, or make humans react like chickens with their [thinking] heads cut off. If a text generator is going to be effective at writing propaganda, it’s skills are going to have to increase a heck of a lot. Ask again in five years.” (scored 2, neg)

“Very difficult to unpack – particularly as the different sides each claim the other side is distributing fake news. Having followed the Syrian War closely, I know for a fact our major news channels spouted fake news. Could something like this be used in social media to reply with biased opinions? Perhaps, but the existing troll farms used by all sides are often quite sophisticated, with the least sophisticated being the least effective. So, this type of low level autogeneration would not be very effective.” (scored 2, neg)

“Oh yes! Too many people are not analyzing the sources. Maybe that is too big a job or most and maybe many do not know to do this. My son repeats this claptrap to me as if it is gospel and he does not understand me when I tell him that his source is untrustworthy. He thinks if he hears something from dozens of people it must be true. Mass media is powerful in that way. “ (scored 0, pos)

“It is an enabler of fake news (eg allowing much faster fake news to be created – especially if fed in the analytics re effectiveness of dissemination etc.) but could also make REAL news reporting more efficient and effective especially if it supports optimum use of human vs AI input.” (scored 0, pos)

The text samples were too small to use computer sentiment analysis, so I scored the replies myself (human accuracy is still the best for nuanced material). Much training data for sentiment analysis is annotated by humans before use. See references.

Positive – is relevant – 0

Neutral – don’t know, don’t care or not sure – 1

Negative – is not relevant (not good enough, unsuitable, not required) – 2

How done

Sum Likerts (inverting the sixth Q (Q3/of 6 at end), the third after the 3 experiments), rank, then compare to texts, which are scored

0 pos (relevant to fake news), 1 neutral, 2 neg (not relevant to fake news).

Note: Q3/6: Do you feel that you have used somebody else’s work?

This Likert scored the other way, ie, the first option (score 1) was the most negative (is somebody else’s, ie plagiarism).

So in the calculations this score was inverted (1 become 5 etc.).

Other Calculations

1/
Overall Likert score (low positive affect) vs Fake Pos-neg relevant – neutral – not relevant

fake news vs likert.png

There is no significance to this relationship.

2/

vs. word count vs Fake news Pos-neg relevant – neutral – not relevant

fake news vs word count.png

References

Sentiment analysis papers:

https://www.kdnuggets.com/2020/06/5-essential-papers-sentiment-analysis.html

What is fake news?

Important researcher: Xinyi Zhou
These are the most recent papers on fake news:

A Survey of Fake News:

https://dl.acm.org/doi/10.1145/3395046

Fake News Early Detection: 

https://dl.acm.org/doi/10.1145/3377478

Generally, fake news is false or distorted information used as propaganda for immediate or long-term political gain. This definition separates it from advertising, which has similar approaches but is for brand promotion rather than life or death matters. False can be anything from an opinion to a website which appears to be proper news but is actually run to spread false information.

This has led to ‘reality checks’ where claims are checked against reality (which still exists) but the problem is that fake news spreads very quickly (because it appeals to the emotions, often fear or hate) while corrections take time to process and are very dull, so hardly anyone reads them, as they have no emotional content. Certainly the people that react to the fake news (if it is proven so) have moved on already, to the next item.

 

 

 

 

 

 

Computer-Human Hybrid AI Writing and Creative Ethics

Introduction

This blog is about my 2020 research into computer text generation and the effects on professional ands amateur writers. I am working on this topic at the University of the Arts London (UAL CCI, Dir. Mick Grierson).

No-one has asked creatives or writers what they think of the new ‘AI’ systems that generate readable text and so directly threaten their jobs, and could change the way people work forever (or don’t work forever). This is a topic that directly impinges on self-worth and financial worth in more ways than anyone can imagine, although plenty are worrying.

STUDY – ONLINE EXPERIMENT
August-October 2020

I devised an online experiment about this topic, allowing respondents to experiment with creating hybrid stories using a text generator. The people were all professional or serious amateurs (and a couple of small students) invited from my own creative writing software mailing list, a couple of writing forums, and a publisher’s writers’ forum, plus friends and relatives who generally use writing in their work. Credits are at the bottom.

Text generation

You might have heard of Google OpenAI’s GPT-2 and GPT-3. My experiment uses a generating system (Fabrice Bellard’s Text Synth, with permission)  based on GPT-2, that anyone can use. GPT-2 was used here as the model works well for idea generation and is more generally available at the time than GPT-3, which is much larger.

Note: The text generation and editing system is now a free online tool (creativity support tool or CST) at

Story Live writing with AI free online

The experimental results will feed into this blog (see Index for different aspects) and later an academic paper, and also a new book for the general public on the whole subject of computers, creativity and writing.

Please sign up for news and notifications – there’s a form on this page.

Brief description of the Study

Below is a graphic of the entire online study. Each block is a page and journey was left to right from top to bottom. The three text generation and editing experiments used a similar set up to the Story Live tool.

Each writing experiment – Caption, News and Fiction – had a question afterwards, then there were more questions after the experiments (see diagram below). All this will be addressed in blogs here, along with other discussions.

The image writing prompt was the same for each experiment and for all respondents for uniformity (there is a blog on the man and dog here).

Prompt image man and dog
Prompt image man and dog
Flowchart of Study

Geoff Davis

The computer support tool (CST) from this study is Story Live writing with AI free online

My other creativity tools are Notes Story Board and Story Lite from my Story Software. For my other activities please see the home page of this site.

Study

This study was devised and the site programmed by Geoff Davis for post-graduate research at University of London Creative Computing Institute UAL CCI 2020. The Supervisor is Professor Mick Grierson, Research Leader, UAL Creative Computing Institute.

Text Synth

Text Synth, by Fabrice Bellard, is a publicly available text generator, was used as this is the sort of system people might use outside of the study. It was also not practical to recreate (program, train, fine-tune, host) a large scale text generation system for this usability pre-study. Permission was granted to use Text Synth in the study by Fabrice Bellard Jul 7 2020.

Fabrice Bellard, coder of Text Synth.
Fabrice is an all-round genius and writes a lot of OS. Text Synth was built using the GPT-2 language model released by Google OpenAI. It is a neural network of 1.5 billion parameters based on the Transformer architecture.