So the UK has just given itself a national headache. Whether you think the Brexit was the right decision or a dangerous and unmitigated botched up excercise, the consequences of the referendum will be non-trivial and take years to complete. But the mechanics of the UK exiting the European Union aside, the question of how people now feel about the Brexit is interesting, and what ripples and damage control it will bring in the years to come, is interesting to analyze.

Data Sciences vs Discernment: The Brexit Prediction Incongruity

The world, and financial markets in particular, were stunned by Britain’s vote to exit the EU. The markets were quite confident that Britain would stay in the EU. The S&P 500 rose sharply towards the end of the trading day before the Brexit vote, which some analysts claimed as a vote by investors against the Brexit. Earlier that morning (at market Open), a unique model that was developed by multiple researches using media sentiment data made the opposite prediction and expected a sharp decline in markets that we saw overnight.

Modern machine learning algorithms can be used to identify complex and deep patterns in noisy datasets – patterns that may be very difficult to detect with less sophisticated algorithms or human intuition. The media coverage of Brexit is one such noisy event: identifying the nuances in the information flow that is likely to sway voters is one promise of such machine learning approaches.

Over the course of time, several machine learning systems have been developed (like Pythia) that predicts financial markets using media sentiment data sets. Once such data set is the Thomson Reuters MarketPsych Indices (TRMI), which quantifies granular sentiments such as trust, uncertainty, and optimism in thousands of global news and social media sources. Pythia can identify the impactful global media sentiments in the TRMI, and then use these characteristics in an ensemble to predict daily movements of the markets with a high degree of accuracy.

The morning of the referendum, most investors were confident that the “Remain” vote would prevail. In line with this, the S&P 500 rose sharply towards the end of the trading day. However, earlier that morning (at market Open), but the Fintech Machine Learning systems made the opposite prediction and expected a sharp decline in markets that we saw overnight.

#Brexit Result Analysis on Twitter using Machine Learning

Being a historic event, we thought it would be interesting to analyze what people were talking about the Brexit result. This could be achieved by leveraging a python library like tweepy to connect to the Twitter stream and get more than 450,000 tweets that used the hashtag #Brexit.

Afterwards, the tweets need to be filtered by language. Using a language classifier could help you filter this out and keep only those that were in English (around 250,000 tweets). Then, analyzing these tweets with some public, pre-trained and ready-to-use machine learning models would commence. Sentiment Analysis could be performed on these tweets to understand if people talking were talking positively, negatively or neutrally about the Brexit.

Finally, to better understand the different point of views, one could perform keyword extraction on the tweets of the different sentiments analyzed to know the words or phrases people were using to get a better picture and more context.

The Divided Opinions

A huge divide in people’s opinions regarding the brexit result has been confirmed using similar type of analysis stated above. 63,024 of tweets talking positively and 70,581 talking negatively were found.

Most relevant keywords for positive tweets with #brexit.

People tweeting with a positive sentiment used these keywords and terms:

% of the most relevant keywords for positive tweets with #brexit.

Many of the positive tweets were thankful of the result and proclaimed that the result was a ‘good thing’. Some people even celebrated the new ‘independence of the UK’. Some ‘positive’ tweets were actually sarcastic, wishing good luck to the UK or mentioning Donald Trump.

The way ahead: Post-Brexit Damage Control Plan for the CXOs

Most CEOs of large firms have seen and navigated quite a few global economic and macro environment shock moments through their managerial careers and several in this century already – such as 9/11, the $145 oil spike and the Arab Spring.  CEOs will rely on patterns and experiences from those prior episodes to help find their way through the unfolding repercussions of the Brexit vote. However, from a technology-related viewpoint, one thing makes Brexit rather different, this time we have Data Science. So what can a chief executive really do with data science?  How can these complex tools help right now with the problem of navigating the Brexit crisis?  I can think of three things CEOs can call for, from their teams – if they have invested in the capabilities:

1. Social Listening– analysis of public social media can provide insights about trends in societal debates, markets and political mood shifts that can help inform business leaders about key transition points between different business scenarios.
2. Impact similarity comparisons. If you have historic data from prior market insecurity episodes such as the financial crisis, you could run detailed comparisons. Are people reacting the same way or differently? Are the dips and recoveries – in ordering behavior or sales cycle times – sharper or shallower? Are they trading up and down between price points and products faster or slower?  Where are the correlating factors and how can you predict recovery tipping points better?
3. Machine data analysis. What can connected machine data tell you about what’s really going on?  Urban public transport usage patterns, equipment idle time, shipping movements and many other kinds of sensor data from the connected machine world could provide critical trade flow and economic activity insights.

Conclusion

We are not really very far into the data science age. Techniques are raw, experts are in short supply and the methods of today will no doubt look quaint a decade from now. But Brexit is the first big business test of a new emerging data-driven strategic management science.  This time, smart CEOs will rely on more than just the accumulated human experience and standard commercial transaction data gathering from a cabal of their closest colleagues. If you have invested in data science capabilities in recent years – this is the moment to give them a real test.