A Mountain of Pent Up Tears – children’s stories generated texts

Children’s stories this is part of the classic literature series on this blog, experiments in hybrid fiction. 

I turned out so fast, I barely looked back. (I turned to look, only to find the figure had gone.) What has gone before was the dark room.

In the far left corner lay a pile of books. The ones on the page, all full of life’s lessons; the black ones, I thought of the colorless, the dark books, my soul had never seen red in my life. These weren’t books of old stories. These were children’s stories told by old schoolboys, and children’s books have been around ever since:

This was written in the 1840s when young, red dwarfs lived in the forest. They wore their clothes brown and brown and purple. At the close of the story, the author makes the child a dwarf, the woods were too far for them to go, so they are made brown. His book is told like many children’s stories.

“This is where the forest is.”

“The forest, where do I go then?”

“In the woods. In the woods, as you said.”

“I, too, am brown.”

“Yes, I am. I am in the forest. It is the way of the forest.”

“And me?”

“Yes, too, me too.”

“And you too?”

“Yes, too.”

The forest is the only place in which these children can know what things are true. They see the forest as their only home. If they go away, then they see them as an enemy who must be conquered. But these children have all travelled over, and gone past. They’ve seen it all over again.

They know what the forest is; how it works, how things go. They can see everything. And for some reason they know that they are all in the same boat, that they can never cross over to other paths. So they know, even in that boat, what the forest means, and why it is there.

What’s behind the green, silver-tinted door. Where’s the white door? That’s not the forest, is it? The white door has never been there. What was behind that door, the door to the land?

They do not know why they were born into the woods. They think that the forest is God’s gift for them. But what’s there to be gifted for? They are the only ones, after all.

When asked how the sun comes out, and when asked why the sun comes out.

Answer: There is no one here who knows.

And they need not have seen the sun.

 


GD: the first 68 words of the generated text (above) was next used as the prompt to generate the text (below). The 68 words are shown in italics.


I turned out so fast, I barely looked back. (I turned to look, only to find the figure had gone.) What has gone before was the dark room.

In the far left corner lay a pile of books. The ones on the page, all full of life’s lessons; the black ones, I thought of the colorless, the dark books, my soul had never seen red in my life.  

There, I thought, was the door leading to what was yet to come. But in the darkness, an unseen figure, one I couldn’t yet recognize, looked from the left to the right, slowly. As it followed, my eyes scanned the walls, but I couldn’t find the same room. I reached out to touch the door, but the figure stayed, a blank shadow across the room, it never left the door. It seemed to vanish into nothingness.

Then came an accelerating sound, as the figure dropped, and it moved slowly back into the room.

“What have you done?” I cried, the words begins to rise up like a mountain of pent up tears. There, I found, was the door. It turned, and my body seemed to lift it, before the figure itself left to my side.

A smile appeared on my face as I looked at the room, the books filled a giant library with life and lessons, the black ones I had hoped would be my own in my own time, the colorless ones, the ones I’ve already read, the ones I’ve been looking for. They were to tell me what I would need to know, how to survive this night, to know the unknown. They were a path, a map.

In the darkest hall, the room I saw the figure enter, a pale shadow in the light. The place was dark, it was cold. He took me in, and  I felt he was going to kill me, he was taking me inside. The sound woke me, and I opened my eyes to find another shadow, a figure standing behind me. I didn’t know what he was, but I was in my own home.

That man walked towards me, eyes glow green, his eyes were gloomy, the healers he wore, his hair was matted with grey, it was grey that faded in between them. He was big, at least, if a little tall. At least I had grown the right size for this thing.  I don’t think his arms were big, so that is probably just his arms, or perhaps his face was too short for his eyes. His fingers were lank, the seated fingers were black, and  in the dimmest corner of his head was a hood, with no lights.

It was very dark, there was an open door beyond it. He held my hand like a…well, for I was small. I could tell the size of his hand. He had long grey hair in his hands. He held my hand, and I saw red, it was black, as if it was a red colour.

I thought, and in that moment, my skin was soft and shiny. In between my knees. So, I moved over and looked, in my hands, and there was a dark green hand, with red hands, I knew this was a shadow, but that doesn’t mean anything to him.

The first hand, the first hand was big, its face was black, it was large, for I was small. The black was green, its color was darker than red. I touched it, and it was cold.

The second hand was big, but the hands were dark, and had big fingers, the first hand was long and dark green, it was like a purse, I didn’t know how it was, but it was cold, in that moment, I froze.

I could only watch.

 


You can guess the original story from the image (below).

Little Red Riding Hood and Wolf

 

Photo credit https://www.publicdomainpictures.net/

Sentiment Analysis of Caption, News and Fiction text generation experiments

From: text generation editor research (UAL London 2020 see credits).
What happens when writers use a computer text generator to write articles, giving them only an image prompt.

Go to Index of AI research

Sentiment Analysis

The Study had three text generation and editing tasks, to make a Caption,  a News article, and a Fiction story, using the same image prompt of a dog and man.

Summary

This shows differences in Positive sentiment affect between the three different experiments. Most positive at the start, by the end, a balanced neutral view had become evident.

Fear was shown in the first Caption experiment, but was lost in the second News and third Fiction, showing acclimatisation by experience. Generally Negative scores were low.

By Fiction experiment, the third and final, only Sadness was left.

Comment

This shows a learning process during the three experiments.

All results are cautioned by the strong ‘Tentative’ score 0.91 and lack of any ‘Confidence’ scores over 0.5.

Details – Method

For overall sentiment of responses, all text feedback was summed into a total text field per respondent.

Text comment fields were:

Caption, News and Fiction Experiments (3 fields);

Questions 1 (2 fields), 2-6a (5 fields), 6b Fake news (1 field).

This gives 11 feedback text samples per respondent (not all were filled). These are summed vertically in Excel to give the overall text block per column/field.

Texts were also summed per respondent, horizontally in Excel.

NLP and IBM Tone Analyser

NLP (natural language processing) allows computer analysis of text blocks. For this volume of text, the online IBM Tone Analyser (See References at bottom) was used. Writing a custom analyser was outside the scope of the study. Human grading was not possible due to the size of texts when totalled, however human analysis is used for the summary of texts.

IBM Tone Analysis gives a rating for:

Anger Fear Sadness Joy Analytical Confident Tentative

(Coded on data as Ang, A, F, S, J, A, C, T.)

The “most prevalent tones that are detected for each utterance” are shown at a document level, and sentence level.

The document level analysis has scores, and the sentence level (which shows lower occurrences) was added in brackets. This gives a results for example, where the detected tones have a numeric score over 0.5. Scores: <.5 None; =>.5 – .75 Mid; >.75 Strong

At document level, each tone if found, has a score .5-.1.0

Lower graded tones (placed in brackets in data) are placed at the 0.25 level

Examples

J,S,A,T(C,F) – Joy, Sadness, Analytic and Tentative have scores > .5 (and C Confidence, F Fear have lower occurrences only, between >0 and < 0.5)

Ang,F,S() – Ang is Anger, Fear and Sadness have scores > .5 (no others over 0)

F(S) –Fear scores > .5 (Sadness between >0 and < 0.5)

In my data display, a ‘lower occurrence’ (bracketed) is scored at 0.25

Confidence and Tentative are general attitudes shown in the text.

Results

All respondents summed

Caption experiment – all feedback comments

Graph - IMB TA Caprion.png

With motivated stakeholders as respondents, there are high scores for ‘Analytical’ and ‘Tentative’. ‘Confidence’ did not appear at all as a document level tone in all 82 people, and occasionally as a sentence level tone.

Using the highest score amongst ‘Anger’, ‘Fear’ and ‘Sadness’ as Negative, and using ‘Joy’ as Positive, shows a higher degree of positive response.

Fear is evident, but at a low level. Sadness is the strongest of the negative reactions.

Positive is about 20% more than Negative. (Significance.)

News experiment – all feedback comments

Graph - IMB TA News.png

More overall positive result than Caption experiment.

Fear has gone, low levels for Anger and Sadness Negative tones. Positive is about 42% more than Negative.

Fiction experiment – all feedback comments

Graph - IMB TA Fiction.png

Sadness at its highest level. Not Positive or Negative. Anger and Fear do not appear.

Summary

This shows differences in Positive sentiment affect between the three different experiments. Most positive at the start, by the end, a balanced neutral view had become evident.

Fear was shown in the first Caption experiment, but was lost in the second News and third Fiction, showing acclimatisation by experience. Generally Negative scores were low.

By Fiction experiment, the third and final, only Sadness was left.

Comment

This shows a learning process during the three experiments.

All results are cautioned by the strong ‘Tentative’ score 0.91 and lack of any ‘Confidence’ scores over 0.5.

References

IBM Tone Analyser

Please see full Report  for further statistics (tba).

All the Rubbish of a Great City – classic literature vs text generation

All the Rubbish of a Great City

 

From the series of classic literature vs the future…

All The Rubbish of a Great City

Part 1: No Sun Ever Since That Day

 

Dear Farewell

You may be glad to see your letter of 6th April last from me. You are still in the good humour of the last time, and I believe that the people will be kind and kind to you in your letters.

I hope you will send me the following on the 15th of June: 1st: to my brother, a messenger, and my wife; 2nd: my daughter; and to my sister; then my brother; and my sister to my husband; and my brother to my son; and my father to my son; and my daughter to my husband and sons; and to my daughter’s father.

It is impossible for me to perceive a trace of the abominable scenes which I have experienced. There are but a few houses destroyed, and most of them, that is, very pretty ones, and all are occupied with families, and a few shops: one street is covered with all the rubbish of a great city; its streets are of a grey nature; and the streets are all closed off by some heavy iron gate.

I take a seat in a corner of the square; and after taking food from a little table and taking a seat beside a shopkeeper, or at least an old man with a hat on, I look about me. Nothing remarkable is seen or seen; there is no ruin or any signs of destruction; you might as well go near the ruins of the pyramids. The women seem to be quite contented, and are making the little fires that are burning. We are very anxious, we are very sad, to see them.

As I was about to arrive at the coast, I was suddenly knocked out by the wind; my companions, who were not surprised, rushed to me. I lay in their arms, and told them I could not recover myself, and said it was time for my expedition to proceed.

I begged them to give me some of the best part of the sea to make my recovery. They assented, and then threw out with me a large quantity of salt that I could not bring home to myself, and I fell dead.

They gave me some of their gold as I lay dying.

I never heard of such an occurrence; I should have been ashamed to have done so; but I had too much hope in the fortune of the sea; for if I were lost by such misfortunes, one does not have what has been given him; for if the fortune be bad, man do have a right to hope.

They put me into the boat with their captain, Sir Richard, and took me back to the ship where we were sitting. I lay there several days; I thought I had done well, but they told me the fortune was the same; the sun set over the mountains that night, and gave no sun ever since that day.

My mind was so troubled about my condition, that I could not bear the noise of the ship, so I cried, and fell into a terrible state of sleep, and then lay, my head, and neck, and legs, down upon a bed.

[These words, which do not make any impression on the ear, should appear to prove the correctness of the saying.]

 


Original

Frankenstein  or The Modern Prometheus

Mary Wollstonecraft (Godwin) Shelley 1818

Letter 1

To Mrs. Saville, England.

St. Petersburgh, Dec. 11th, 17—.

You will rejoice to hear that no disaster has accompanied the commencement of an enterprise which you have regarded with such evil forebodings. I arrived here yesterday, and my first task is to assure my dear sister of my welfare and increasing confidence in the success of my undertaking.

I am already far north of London, and as I walk in the streets of Petersburgh, I feel a cold northern breeze play upon my cheeks, which braces my nerves and fills me with delight. Do you understand this feeling? This breeze, which has travelled from the regions towards which I am advancing, gives me a foretaste of those icy climes. Inspirited by this wind of promise, my daydreams become more fervent and vivid. I try in vain to be persuaded that the pole is the seat of frost and desolation; it ever presents itself to my imagination as the region of beauty and delight.

There, Margaret, the sun is for ever visible, its broad disk just skirting the horizon and diffusing a perpetual splendour. There—for with your leave, my sister, I will put some trust in preceding navigators—there snow and frost are banished; and, sailing over a calm sea, we may be wafted to a land surpassing in wonders and in beauty every region hitherto discovered on the habitable globe. Its productions and features may be without example, as the phenomena of the heavenly bodies undoubtedly are in those undiscovered solitudes. What may not be expected in a country of eternal light?

I may there discover the wondrous power which attracts the needle and may regulate a thousand celestial observations that require only this voyage to render their seeming eccentricities consistent for ever. I shall satiate my ardent curiosity with the sight of a part of the world never before visited, and may tread a land never before imprinted by the foot of man.

These are my enticements, and they are sufficient to conquer all fear of danger or death and to induce me to commence this laborious voyage with the joy a child feels when he embarks in a little boat, with his holiday mates, on an expedition of discovery up his native river. But supposing all these conjectures to be false, you cannot contest the inestimable benefit which I shall confer on all mankind, to the last generation, by discovering a passage near the pole to those countries, to reach which at present so many months are requisite; or by ascertaining the secret of the magnet, which, if at all possible, can only be effected by an undertaking such as mine.

…Etc…

Your affectionate brother,
R. Walton

Stanley Lying in his Lap – classic literature

Stanley Geoff Davis

 

Classic literature vs artificial intelligence – try and guess classic source.


 

Stanley Lying in his Lap

His face was almost as pale as his fur and the top part of his ears were slightly raised above the shoulder blades.

“Oh, God, what’s happened to me?” he said. “How on Earth have they left me here?”

He saw Stanley lying in his lap. He was standing there naked and looking at nothing. This seemed too strange and too inhuman. Stanley was dead. The whole floor was shaking with the force of the shaking.

“I don’t know,” he thought. “Something’s happened to me. My father… how? God, how have they… done it? What has gone wrong? Where are they?”

And then he heard Stanley’s voice. “Hello, Stanley!”

He turned and saw what looked like him, though his face looked a little too pale to be him, sitting in the chair next to the boy. His hair was in a mess and the fur hat had come loose. He was looking very, very… unwell.

“Why?” he asked.

“I said: ‘Why can you not see me?’ ”

“But…”

Stanley looked up at him. “Your mother… her eyes! Her… look!”

He smiled very happily. “They were mine.”

Stanley looked at the boy and then at him. “I see you. I see the world. I’m a fool. I’m too good for the world. But, in the end, what is this? The world that you can see? Where my father will come when he wakes up in this, this bed, on the other side of the wall, on the other side of this wall, and I won’t have to come here. Where will I find someone? Who? Where will I find him?”

It was the most beautiful and sweet thing he’d ever heard Stanley say.

“Where will I find him?”

Stanley tried to keep his eyelids closed, which was much too hard when the strange visage that appeared before him was so familiar. The picture seemed to say: I am your true self. Stanley imagined himself the girl. An alien being, a human girl at first glance but it should be said he certainly looked like a human girl. Even his eyes, that were always open, did not seem to be aware of his reality. He had no eyes.

The whole room. The whole world, that was. There was only a black box that Stanley had brought with him, a small metal cylinder with three small black valves, and there were four smaller tubes with levers that the girl would let loose with her nose, and she would let them press the small black buttons up and down, all of which were pressed with a certain sort of subtlety and even less subtlety than the valves, a certain kind of subtlety that Stanley supposed was just a thing he did like to do. Stanley was trying to help with the manipulation, but what needed to be done wasn’t to press the levers; it was to open the box, to allow the room to breathe.

His mind had become fragmented as Stanley was trying to find what seemed to be an easy solution and even some understanding of Stanley’s intentions. But even then he couldn’t see the girl from his point of view, and only saw the strange visage.

“I’ll send you an owl,” he said.

He wasn’t exactly sure whether his voice was hoarse and distant or how close to death he was. It was his first time talking to anyone in a foreign language.

“Do it,” said a small voice.

Stanley stood up, looked around and couldn’t believe that his bed was the only place the floor was covered. It was the first morning. But he remembered everything. It was the first real thing he had lost before he went in the hospital. He knew what it was.

“You did,” said the woman. “You did so, and I thought you had passed into a nightmare but something happened next thing and I came back.”

“Oh, you did,” said Stanley. “I guess I got over it.”

“I mean I thought you’d gotten over it but this time you didn’t,” the woman said to herself.


Metamorphosis
Franz Kafka 1915

One morning, when Gregor Samsa woke from troubled dreams, he found himself transformed in his bed into a horrible vermin. He lay on his armour-like back, and if he lifted his head a little he could see his brown belly, slightly domed and divided by arches into stiff sections. The bedding was hardly able to cover it and seemed ready to slide off any moment. His many legs, pitifully thin compared with the size of the rest of him, waved about helplessly as he looked.

“What’s happened to me?” he thought. It wasn’t a dream. His room, a proper human room although a little too small, lay peacefully between its four familiar walls. A collection of textile samples lay spread out on the table – Samsa was a travelling salesman – and above it there hung a picture that he had recently cut out of an illustrated magazine and housed in a nice, gilded frame. It showed a lady fitted out with a fur hat and fur boa who sat upright, raising a heavy fur muff that covered the whole of her lower arm towards the viewer.


See the other classic manglings in this blog.

Why Don’t Cannibals Eat Clowns?

Part of the text generation series – see blog index.

Why don’t cannibals eat clowns?

Because cannibalism doesn’t produce the same health benefits as killing clowns, you will have only about 60% of your body weight of body fat left, leaving you with less body fat and no protection against the other 50%.

By choosing a suitable size clown, you can take advantage of that. If I am in a small town and in need of a good pair of shoes, I’ll eat an ogre and I’ll have about 100% body fat and 70% of my body weight in a pair of shoes.

What the hell am I supposed to do, I am going to starve?!

In that case I would just have to get one of those, because those are probably my only options.

Also, by choosing a size bigger or smaller than I am currently supposed to be, you can take advantage of that as well. I’m going to give up on these shoes and use the boots instead. I mean that literally.

If I were in a small town where there is limited supplies, and the only food available is a variety of meats and fruit that I’m supposed to eat (such as a spider or something), I would eat some squirrel.

But if there’s too much food and no other choice, then I will instead go for the more dangerous monster I can get.

Now the issue of choice is just that these are all options. You don’t know what your favourite will be unless you decide to look into it.

For example, while you can choose a size small enough to be your “smallest” and go for the larger you’re going to be the bigger your body fat will be, it just won’t work like that. The bigger you get, the bigger your body fat will be so it’s not going to be as easy to “do the right thing.”

What you want is for your body fat to be close to your own body weight, which is probably going to be under 0 (the ideal is probably between 10 and 12).

Also what you want is to weigh on average only about 70-75% of your body weight (about 70lbs for one person, or roughly a fat person for another).

That’s what makes you a normal person.

2

Why don’t cannibals like clowns?

If they’re not being eaten by the clown, it’s because they’re so fat they don’t have a stomach, so they get crushed by the horse. In the real world, people’s appetite changes over time as well. A hungry hippo’s hunger will rise over time so an old horse gets eaten by a zombie, and so on.

In fact, a lot of people have to eat a lot in order to keep the hunger in check. And there are also the many social cues that cause our appetite to rise and fall over time. Food is good for us, because it tastes good and is a good source of calories. And when it’s too much, we have a panic attack. When it’s just right, we can just be happy to just eat a bag of popcorn. And once we’re just right, we can eat the popcorn and forget about it.

Why aren’t hippos getting eaten by horses? Because they have a digestive system that has to move in to the horse’s stomach first, or there will be too much food in the horse’s stomach. The idea that this food goes in straight through the horse’s digestive system is just silly.

The only way I’ve seen this food in the wild is that it was consumed by some people who were infected with a fungal life-form and got it into their digestive system and their appetite was just too great.

The best part about all this is that it’s simple enough to understand. The only thing that we can do is stop eating, to do that you just wait till your health gets better. If you could just get rid of all these stupid foods for a week or so and let your appetite return to the normal state, even better. It’ll be like the beginning of autumn in the wild, there’s just not enough sun to cook food like it used to be and it’ll feel like no one’s eating.

As you have experienced, I’ve never.


This from a joke in Geoff Davis’s new novel The Circular River which will be serialised on this website next week

Photo credit – the Solomon Islanders, feasting trough.

Fake News – what do real writers think about it? AI research

I examined in August 2020 research (UAL London see credits) what happens when writers use a computer text generator to write articles, giving them only an image prompt. There is a summary of the Study in the Index.

Go to Index of AI research

Fake News – (or ‘fake news’) what do real writers think about it?

Summary 

  • People who have a lower general opinion of text generation, and lower wordcounts for feedback, don’t think text generation is relevant to ‘fake news’.

The converse, that people who are more engaged with text generation (more feedback, more positive) can see that it is more related to fake text generation.

So, if people are educated on the topic, they will realise there is a relationship between advances in computer text simulation and fake news.

Definition of fake news

Even defining fake news is not that simple, since if you look for examples in the USA, or Syria, or anywhere else, both sides accuse the other of using it. Depends on there being no consensus as Barak Obama mentioned recently. See the bottom of the blog for some discussion.

Fake News

One of the feedback questions in the Study was about whether computer AI text generation was relevant to ‘fake news’ (false content for spam, propaganda etc.).

Q6-2: If you have heard of ‘fake news’ do you think this is relevant?

This had neutral phrasing in order not to influence replies. It did not say ‘text generator can make fake news’ as this presupposes a technical knowledge that might not be present, even after doing the experiments. Given the sometimes bizarre output of the generator, it might affect replies too specifically.

18 of the 82 respondents – 22% – replied to the question. It was the last question after a long session so perhaps some fatigue had set in. It was also a question not specifically to do with the experiment. About a quarter is still quite a high response rate.

Analysis of responses

I will be examining the actual text answers in another blog. This just looks at some basic relationships.

Several comparisons of the data were calculated and some are worth discussing here (the others are in reference material).

Comparing answer word count with positive – negative ratings:

The actual Q6 answer text was assessed for Positive – Neutral – Negative response to the ‘relevant to fake news’ question.

For instance, the text answer ‘No’ was 2 for negative, word count 1. ‘I stay out of these type debates.’ Is 1 for neutral, word count 7.

This was quite easy to do. I did not use a computer sentiment analyser as the samples were too small. Anything a bit ambiguous or with conflicting statements was rated Neutral.

Pos – neg is 0 (positive), 1 (neutral), 2 (negative) – these are scaled up so can be seen against the word count values. So low red columns are positive (yes, relevant to ‘fake news’) , high red is negative (not relevant to ‘fake news’). This is an ordinal scale.

fake news vs word count.png

Above: Pos-neg is zero for positive for ‘relevance’ to ‘fake news’. The red Pos-neg 0-1-2 value is scaled to make it easier to see. So a tall red line means negative for relevance to fake news. (There is a discussion of different statistical calculations in a forthcoming blog. The tests used in the study are Mann Whitney U, T-Tests, ANOVAS, and Pearsons, along with various bar charts and boxplots.)

At first glance this shows that higher answer word counts are associated with positive for ‘relevance to fake news’.
Low word counts are associated with negative for ‘relevance to fake news’.
There is only one negative out of the top half of the responses (1-9 on the chart above, 2-8 are pos=0).
The highest word count (column 1 on chart) was from an answer that discussed an actual example of real ‘fake news’, or propaganda in the Syrian War, so was a longer than usual negative response. It could be an outlier. After ruminating they decided it was too hard for a text generator to do, writing ‘news of any kind has to be created by humans…’ which is a value judgement (humans are best).

Statistics

Mann Whitney U test

The z-score is 4.25539. The p-value is < .00001. The result is significant at p < .05.

  • People who used the least words thought text generation was less relevant to ‘fake news’.
  • People who used more words in their answer thought it was more relevant to ‘fake news’.

Discussion

In other work, computer Sentiment Analysis revealed that all feedback answers had high scores for ‘Tentative’ and very low (no score) for ‘Confidence’. These results could be because of a general lack of experience of text generation (only 11% had previous experience).
There is a forthcoming blog on the Sentiment Analysis, there is already a blog on Sentiment Emotions (Joy etc.).

When making a positive claim (‘there is relevance’) with no ‘Confidence’ and feeling very ‘Tentative’, discursive answers are to be expected.

In the positive answers, the relevance of ‘fake news’ produced more discussion (‘Yes, and here’s why…’). This could be because the respondent felt it was relevant, but had little or no confidence in their feelings and supplied a tentative, more wordy, answer.

Whereas with a negative response, text generation is not relevant to ‘fake news’, and can be easily dismissed (‘No’ – 1 word).

Ranked Likert scores over the Study vs Pos-neg

Fake-ranked-Likert-Pos-neg.png
Mann-Whitney U Test
The z-score is 5.10963. The p-value is < .00001. The result is significant at p < .05.

This is not visually very clear until you sum the raw (before scaling for graph) scores from columns 1-8 (6) and 2-18 (10).

  • This shows that lower Likert scores (which means positive response to experiments and later questions) relates to lower Pos-neg scores (which means positive relevance to ‘fake news’). This is what would be expected.
  • People who have a lower general opinion of text generation, don’t think text generation is relevant to ‘fake news’ (and possibly anything else of practical use).

Amateur – Professional

People were asked to select Amateur/Professional (or both, scored as Pro) next to Occupations. Most people had more than one occupation (Poet and Journalist etc.) so it is not a fixed job allocation, but provides an indication that the writer either is or would like to be paid for at least some of their work.

This did not provide any significance at the ‘fake news’; response level.

There is a forthcoming blog on Amateur/Professional differences.

Comments

Is there a connection between text generation and ‘fake news’?

1/ Human pride: anthropocentrism: people that dismiss the use of text generation and its relevance to ‘fake news’, say ‘news of any kind has to be created by humans…’.

2/ The idea that a computer could not be used to write news, ‘fake’ or otherwise, might be due to the rather random output of the mid-level (GPT-2) text generator used in the Study. People might think it is producing readable nonsense, and brusquely dismiss any practical uses. This is shown in the relation between overall negative scores (Likerts) and low relevance for ‘fake news’.

3/ Perhaps dismissal of a text generation tool in regard to ‘fake news’ is indicative of not believing there is ‘fake news’ (or that it is not a problem), shown in the low word counts of their short negative responses. Further research required.

Notes

1/ Answers

If you have heard of ‘fake news’ do you think this is relevant?

Replies went from the four shortest (1 or 2 words)

If you have heard of ‘fake news’ do you think this is relevant?

No (scored 2, neg)

No! (scored 2, neg)

Sure (scored 0, pos)

Not really (scored 2, neg)

to the four longest (89, 85, 76, 50 words; 2,2,0,0)

If you have heard of ‘fake news’ do you think this is relevant?

“Unless the AI used to generate the text is extremely advanced, I think any news text generated wouldn’t be very effective — fake or otherwise. I think, at this point in our technology, news of any kind has to be created by humans to effectively educate, manipulate, or make humans react like chickens with their [thinking] heads cut off. If a text generator is going to be effective at writing propaganda, it’s skills are going to have to increase a heck of a lot. Ask again in five years.” (scored 2, neg)

“Very difficult to unpack – particularly as the different sides each claim the other side is distributing fake news. Having followed the Syrian War closely, I know for a fact our major news channels spouted fake news. Could something like this be used in social media to reply with biased opinions? Perhaps, but the existing troll farms used by all sides are often quite sophisticated, with the least sophisticated being the least effective. So, this type of low level autogeneration would not be very effective.” (scored 2, neg)

“Oh yes! Too many people are not analyzing the sources. Maybe that is too big a job or most and maybe many do not know to do this. My son repeats this claptrap to me as if it is gospel and he does not understand me when I tell him that his source is untrustworthy. He thinks if he hears something from dozens of people it must be true. Mass media is powerful in that way. “ (scored 0, pos)

“It is an enabler of fake news (eg allowing much faster fake news to be created – especially if fed in the analytics re effectiveness of dissemination etc.) but could also make REAL news reporting more efficient and effective especially if it supports optimum use of human vs AI input.” (scored 0, pos)

The text samples were too small to use computer sentiment analysis, so I scored the replies myself (human accuracy is still the best for nuanced material). Much training data for sentiment analysis is annotated by humans before use. See references.

Positive – is relevant – 0

Neutral – don’t know, don’t care or not sure – 1

Negative – is not relevant (not good enough, unsuitable, not required) – 2

How done

Sum Likerts (inverting the sixth Q (Q3/of 6 at end), the third after the 3 experiments), rank, then compare to texts, which are scored

0 pos (relevant to fake news), 1 neutral, 2 neg (not relevant to fake news).

Note: Q3/6: Do you feel that you have used somebody else’s work?

This Likert scored the other way, ie, the first option (score 1) was the most negative (is somebody else’s, ie plagiarism).

So in the calculations this score was inverted (1 become 5 etc.).

Other Calculations

1/
Overall Likert score (low positive affect) vs Fake Pos-neg relevant – neutral – not relevant

fake news vs likert.png

There is no significance to this relationship.

2/

vs. word count vs Fake news Pos-neg relevant – neutral – not relevant

fake news vs word count.png

References

Sentiment analysis papers:

https://www.kdnuggets.com/2020/06/5-essential-papers-sentiment-analysis.html

What is fake news?

Important researcher: Xinyi Zhou
These are the most recent papers on fake news:

A Survey of Fake News:

https://dl.acm.org/doi/10.1145/3395046

Fake News Early Detection: 

https://dl.acm.org/doi/10.1145/3377478

Generally, fake news is false or distorted information used as propaganda for immediate or long-term political gain. This definition separates it from advertising, which has similar approaches but is for brand promotion rather than life or death matters. False can be anything from an opinion to a website which appears to be proper news but is actually run to spread false information.

This has led to ‘reality checks’ where claims are checked against reality (which still exists) but the problem is that fake news spreads very quickly (because it appeals to the emotions, often fear or hate) while corrections take time to process and are very dull, so hardly anyone reads them, as they have no emotional content. Certainly the people that react to the fake news (if it is proven so) have moved on already, to the next item.

 

 

 

 

 

 

Festival UK 2020 for Creative Computing Institute University Arts London

CCI UAL Camberwell London UK
CCI UAL Camberwell London UK
There’s just been an announcement about the exciting #FestivalUK2022. Professor Grierson (my research director) is part of this, with Creative Computing Institute, CCI UAL. 30 creative teams have been selected to take part in the forthcoming festival of creativity in the R&D Project phase. #STEAM – science, technology, engineering, arts, mathematics.

More news on as this story develops.

The tweet is at
https://twitter.com/ual_cci/status/1328292551967854594

Stats explanation – boxplots

BOXPLOTS or ‘box-and-whisker’ plots

Go to Index of AI research

I will try to explain what the boxplot, a visual summary, or graphic visualisation of data, means by showing actual data.

Occupation vs Tentative

Occupation vs TentativeThis plots data for Occupation (eg, Other, Student, Scribbler, etc) against Sentiment scores for the emotion Tentative (using text analysis of their written feedback).

Looking at all of the different things on the boxplot, you can see:

    • a coloured (usually)

box

    • or

rectangle

    • a

vertical line

    • somewhere in the middle of the box

 

    • horizontal

lines

    • (but not always) coming out left and right

dot

    • or dots (but not always) on the same level but not on the lines

 

    • the data might show as just

one vertical line

    .

Boxplots show:

    • many features of the raw data in a simple way, and

 

    the distribution of a continuous variable.

The box edges to left and right are also called hinges. The vertical line in the middle is the median value (middle value of all the numbers). The horizontal lines are also called whiskers. (This is the Tukey method, see references at bottom.)

The boxplot shows five summary statistics:

    • the median

 

    • two hinges or edges of the box, the quartiles

 

    • with up to two lines or whiskers, showing the other quartiles

 

    • and all outlying (outlier) points individually as dots

 

    any consequently, skewing of data from a symmetrical normal distribution

Example
Now we will look at how one of these graphics is made from the raw data.

If you look at one of the horizontal graphics for occupation – Poet (sixth down):

Poet data Tentative boxplot
Poet data Tentative boxplot

Raw data and graphic explanation
First the numbers are sorted (ranked). Look at this breakdown, below, of where the numbers are in relation to the median (middle) value, and then how this related to the boxplot.
The data is the score on the sentiment analyser for tentative-related words, higher means more, score can be 0 to 1.0.

Data poet tentative explanation
Data poet tentative explanation

Other cases

The one below has just a single line instead of a box, because there is only one data point (0.87) – so you can get a gappy-looking boxplot, that is OK.

tentative-scientist


Notes

A boxplot helps to visualise the distribution of the data by quartile and show any outliers.

The plot above visualises five summary statistics, the median, two hinges or edges, and two whiskers or lines, and all outlier points individually as dots.

The box (coloured rectangle) always extends from the 25th to 75th percentiles. These sometimes called the ‘hinges’ of the plot.

The line in the middle of the box is plotted at the median.

Quartile: a type of quantile which divides the number of data points into four more or less equal parts, or quarters.

Quantile: in statistics and probability, quantiles are cut points dividing the range of a probability distribution into continuous intervals with equal probabilities, or dividing the observations in a sample in the same way.

Outliers: examination of the data for observations that are far removed from the mass of data (which could be for unrelated or distracting issues, or not).

Practical note: In the boxplot above, the data (which is from the experiment, saved as CSV files, and then imported into Excel for data cleaning (tidying up gaps etc. from the CSV format). From Excel it is then used in R statistical package.

References

General statistics calculators (great sites)

https://www.socscistatistics.com/tests/mannwhitney/

https://goodcalculators.com/statistics-calculators/

Boxplots (this is the best introduction)

Box Plot Explained: Interpretation, Examples, & Comparison

Wiki
https://en.wikipedia.org/wiki/Box_plot
R and boxplots
https://www.statmethods.net/graphs/boxplot.html

The box and whiskers plot was first introduced in 1970 by John Tukey, who later published on the subject in 1977.
John W. Tukey (1977). Exploratory Data Analysis. Addison-Wesley.

Writing occupation and emotions in text generation

In August 2020 research (UAL, see credits) I examined what would happen if and when writers use a computer text generator to write articles, giving them only an image prompt. The idea was to only use professional or serious amateur writers.

Go to Index of AI research

Joy, Fear, Anger, Sadness – emotion charts are after this introduction.

Can text generation help the human writing process? What do actual writers (the study respondents) think of it all?

The research examines creative and ethical concerns around the use of advanced systems, and how they will (or already do) affect stakeholders, both professional writers and serious amateurs.
Here’s the prompt image:

Prompt image man and dog
Prompt image man and dog

The results are in but I am still writing it up. So I am now dropping a few things on this blog. These are not the final results as many qualifiers need to be added, statistical definitions, significance, etc. There are over 50 charts, which is why the report is taking a long time.

More about boxplots: This is a blog about some study results boxplots. if you are not sure what it all means, please look at this first.

One question asked was whether they’d used a text generator before, someone replied ‘my unconscious’. 89% had never used a text generator before.

82 respondents from my own creativity writing app list (see below), and various professional bodies.

These are Occupation (type of writer eg, Student, Poet, Journalist etc. – see the left axis);
plotted against amount of Emotion (joy, anger etc.) in their written feedback to all the questions (summed, then scored using a sentiment analyser). (Amateur and Professional are not attached to the actual occupation, so they are on here too.)

Increased emotion values towards the right side of the chart. These plots show ranges so they only give a general visualisation.

Joy

So in the boxplot below, the most joy in responses came from Copywriters.

Perhaps they see a fantastic tool to very quickly make more copy.

Joy vs Occupation
Joy vs Occupation

Fear

The most fear in responses came from Poets and Fiction writers. Perhaps fear of losing their respect as creators of strange new worlds were no one has gone before. Or they see a fantastic tool to very quickly make them unemployed. Other and Scribbler also score on this emotion.

Fear vs Occupation
Fear vs Occupation

Anger

Would appear that Others and Scribblers are somewhat angry about something or other. More research needed! Poet and Fiction also score highly, one each here (a line).

Anger vs Occupation
Anger vs Occupation

Sadness

Perhaps poets know more sad words.

Sadness vs Occupation
Sadness vs Occupation

There’s lots more charts but that will do for today. The actual stats with significance, etc., are for future viewing.

One of the simple charts:
Time Average on Study by Occupation

Graph- Time Rank Occupations
Graph- Time Rank Occupations

Game writers had 2 outliers, one person was on it for hours. Perhaps text generation is familiar to games content writers as some games have generated scenarios. Or they have a lot of spare time – to play games.

(Possibly) confirms rumour that songs are written quickly, and that lyricists and poets have flashes of inspiration quickly recorded (and so do copywriters and scientists). Or they were in a hurry to get away…
Game and Songs, Lyrics were added by people within Other definition.

Next blog – the text generation itself.
In the experiment, people were advised to use the generator to make completed works. Several people put my name in the generator, so I became the protagonist in the stories. What!

Such as this Fiction entry:
“It was nice to hear from Geoff again. He is a reminder that life is like an ant’s journey on a blade of grass across a puddle. There is no other side to reach, because the ant is surrounded on all sides. Like an ant, like all of us, Geoff has strategies for paddling. One admires only the paddling, and not especially the termination of the journey. And perhaps that’s what should be the focus of our lives: the paddling. Not journey, not the conclusion, but the sheer determination of the paddling. With a surfer, this analogy would not work, but thinking about it, ants can’t surf.”


People used the OpenAI GPT-2 text generator in a two panel design. I’m releasing this setup as a free AI text editor soon. The generator version is Text Synth by Fabrice Bellard, who is very helpful.

University of the Arts London: my tutor at UAL CCI is Professor Mick Grierson. See Credits (new window). My app is Notes Story Board, an image and text zooming canvas.