From: text generation editor research (UAL London 2020 see credits).
What happens when writers use a computer text generator to write articles, giving them only an image prompt.
Sentiment Analysis
The Study had three text generation and editing tasks, to make a Caption, a News article, and a Fiction story, using the same image prompt of a dog and man.
Summary
This shows differences in Positive sentiment affect between the three different experiments. Most positive at the start, by the end, a balanced neutral view had become evident.
Fear was shown in the first Caption experiment, but was lost in the second News and third Fiction, showing acclimatisation by experience. Generally Negative scores were low.
By Fiction experiment, the third and final, only Sadness was left.
Comment
This shows a learning process during the three experiments.
All results are cautioned by the strong ‘Tentative’ score 0.91 and lack of any ‘Confidence’ scores over 0.5.
Details – Method
For overall sentiment of responses, all text feedback was summed into a total text field per respondent.
Text comment fields were:
Caption, News and Fiction Experiments (3 fields);
Questions 1 (2 fields), 2-6a (5 fields), 6b Fake news (1 field).
This gives 11 feedback text samples per respondent (not all were filled). These are summed vertically in Excel to give the overall text block per column/field.
Texts were also summed per respondent, horizontally in Excel.
NLP and IBM Tone Analyser
NLP (natural language processing) allows computer analysis of text blocks. For this volume of text, the online IBM Tone Analyser (See References at bottom) was used. Writing a custom analyser was outside the scope of the study. Human grading was not possible due to the size of texts when totalled, however human analysis is used for the summary of texts.
IBM Tone Analysis gives a rating for:
Anger Fear Sadness Joy Analytical Confident Tentative
(Coded on data as Ang, A, F, S, J, A, C, T.)
The “most prevalent tones that are detected for each utterance” are shown at a document level, and sentence level.
The document level analysis has scores, and the sentence level (which shows lower occurrences) was added in brackets. This gives a results for example, where the detected tones have a numeric score over 0.5. Scores: <.5 None; =>.5 – .75 Mid; >.75 Strong
At document level, each tone if found, has a score .5-.1.0
Lower graded tones (placed in brackets in data) are placed at the 0.25 level
Examples
J,S,A,T(C,F) – Joy, Sadness, Analytic and Tentative have scores > .5 (and C Confidence, F Fear have lower occurrences only, between >0 and < 0.5)
Ang,F,S() – Ang is Anger, Fear and Sadness have scores > .5 (no others over 0)
F(S) –Fear scores > .5 (Sadness between >0 and < 0.5)
In my data display, a ‘lower occurrence’ (bracketed) is scored at 0.25
Confidence and Tentative are general attitudes shown in the text.
Results
All respondents summed
Caption experiment – all feedback comments
With motivated stakeholders as respondents, there are high scores for ‘Analytical’ and ‘Tentative’. ‘Confidence’ did not appear at all as a document level tone in all 82 people, and occasionally as a sentence level tone.
Using the highest score amongst ‘Anger’, ‘Fear’ and ‘Sadness’ as Negative, and using ‘Joy’ as Positive, shows a higher degree of positive response.
Fear is evident, but at a low level. Sadness is the strongest of the negative reactions.
Positive is about 20% more than Negative. (Significance.)
News experiment – all feedback comments
More overall positive result than Caption experiment.
Fear has gone, low levels for Anger and Sadness Negative tones. Positive is about 42% more than Negative.
Fiction experiment – all feedback comments
Sadness at its highest level. Not Positive or Negative. Anger and Fear do not appear.
Summary
This shows differences in Positive sentiment affect between the three different experiments. Most positive at the start, by the end, a balanced neutral view had become evident.
Fear was shown in the first Caption experiment, but was lost in the second News and third Fiction, showing acclimatisation by experience. Generally Negative scores were low.
By Fiction experiment, the third and final, only Sadness was left.
Comment
This shows a learning process during the three experiments.
All results are cautioned by the strong ‘Tentative’ score 0.91 and lack of any ‘Confidence’ scores over 0.5.
References
Please see full Report for further statistics (tba).