Opportunities and lessons in applying text analysis in your workplace as the DJ Trump - Cambridge Analytica - Facebook collision fades into our rear view mirrors
Early in 2018 reports that the data of 50 million Facebook profiles had been harvested by Cambridge Analytica surfaced. The scandal unfolded and revealed the firm used Text Analysis (TA), Natural Language Processing (NLP) and other data science techniques to build profiles of these users. Then, the Trump campaign targeted persuasively tailored messages to these social media users. Many believe these ads were a swing factor in the surprise Trump presidential victory a few months earlier..
Image credit: Facebook
Regardless of your feelings about DJ Trump, and of the gross breaches of privacy the story reveals, it is a powerful reminder of the important and increasing role of TA in society today. So, let’s put aside politics and examine some illustrative examples of how text analysis is being applied in the workplace for positive outcomes.
Text Analysis: Worlpace Examples
LenddoEFL and other companies are utilising non-traditional data to enable credit worthiness assessments of the 2 billion underbanked people in the world who currently cannot access credit. Lenddo applicants grant permission for textual and other data from their smartphone to be utilised to assess their credit worthiness. As well as providing a new gateway to financial inclusion, Lenddo is lowering the cost of credit assessments and at times reducing defaults.
Sentifi sweeps and analyses text-data from 8 million tweets and 800k news and blog sites each day. It provides insights to investors on the metrics such as the sentiment (positive, negative, neutral) on 50k individual stocks. These tools provide an additional information stream that can assist investors to better match their portfolio’s risk profile to their risk appetite and reduce unhelpful market volatility.
Textio is an augmented writing app that relies on a range of text analysis and artificial intelligence techniques to assist people improve the efficacy of their written communication. In the screenshot below, the tool is guiding the writer of a job advertisement.
Image credit: Textio
In this further screenshot it is informing the writer how terms may appeal to different age demographics. The tool extends the same type of advice in regard to gender and other demographic dimensions as well.
Image credit: Textio
Other workplace settings where text analysis tools are being used to good effect include;
Improving brand experiences and customer engagement with automated monitoring of social media content,
Improving risk management in similar ways to that achieved by the Fitbit customer complaints team. They analysed 33,000 twitter posts about their product in a 6 month window, then isolated and acted on specific scale issues to improve the product.
Amazon did a similar thing with text data from product reviews and forums in the sub $150 speaker market. They used their insights to build a product that better targeted key needs and preferences
Text Analysis: Cautionary Tales
All rainbows and unicorns in the land of text analysis then? Not exactly. There are no concrete solutions yet to resolve aspects of written linguistics such as tone, sarcasm, polysemy and homonyms, or importantly the meaning of words in context. Consider how current text analysis might help our friend the ‘cookie’ monster?
Text analysis, like innovations generally, has a shadow side. The Cambridge Analytica scandal is an example. 64% of people explain they would rather talk with a real person than a chatbot – even if a chatbot can solve their problem. Amazon’s hiring algorithms inadvertently but explicitly discriminating against women is another famous fail, and an important case that raises the spectre of algorithm bias in text analysis. It likely contributed to the gender imbalances depicted below.
Image credit: graphics.rueters.com
Text Analysis: Future Trends
Turning to future trends in text analysis, what can be expected?
Further innovative uses of new tools and data sources, often ‘big’ data sources. The internet, contemporary text mining applications and social media are all new. Further innovative applications will emerge in this space for many years, searches for prior art in the patent corpus, psychological analysis of dream content, and better prediction of adverse vaccine reactions – topical in the covid and post-covid eras – are all examples.
The democratisation of text analytics as tools become increasingly user-friendly, less code dependent and more ‘click and drag’.
· The increased application of Deep Neural Networks to text analytics problems where early efforts show promise that may surpass incumbent techniques like Support Vector Machines.
Advances in the sophistication of chatbots and in the quality of consequent customer experiences. Perhaps some of the intelligence of deep fakes can be applied outside of pornography and fake news, where it is estimated 96% of the current applications of this technology now reside, to improve chatbots and make exchanges like the following one less familiar.
Image credit: chatbotslife.com
Let’s in closing, try a quick experiment that might help you consider the workplace uses of text analysis. Look over the following output from IBM-Watson’s NLP demo, then scroll to the end of this blog to understand its context.
Image Credit: www.ibm.com/watson
This is IBM-Watson’s text analysis of this blog article - a balanced (51% positive sentiment) account of ‘Text Analysis’ (keyword areas) with a predominantly ‘Joyful’ emotional tone. You can get the same outputs with any piece of text pasted into the free demo.
To my mind, text analysis is one of the most exciting and useful developments in data science today. The text analysis industry is predicted to grow by a factor of three times in the six years ending 2026. There is a plethora of useful materials on the internet and this Sydney University resource guide is just one, so get started and see what miracles you can create in your workplace.