Ecosystem View: One way to think about data

(A guest post from Andrew Brice. Data analytics and visualisation is Andrew's passion, turning data into insights. Enjoy some fascinating insights here: http://www.zyan.co.nz/)

The calls were stacking up. They weren’t meant to be, this was just a normal call centre day. Call agents were racing, management hearts were racing, but no one was quite sure where the finish line was. Or why there was even a race.

Afterward, the vast swathes of data collected by the call system was able to be analysed and it started to become clear as to what had occurred and why. The needle in the haystack was found.

The thing is, it was all preventable. The needle could have been tracked if only they had been analysing their own data, ideally with an ecosystem perspective. If only.

OK, so how would that have worked?

Example dashboard for call flow monitoring

Two parts of the organisation already knew this was not a normal call centre day. It was an end-of-year filing date for a subset of clients (yes, it’s a Government agency). Not a familiar filing date, but a statutory one for these clients. The business knew and so did the call centre. Every year, on the same date, a spike occurred. But the call centre didn’t analyse call patterns at a deep enough granularity so they hadn’t picked this up. They were busy with “real” call volumes… The data was collected but it was never sufficiently deployed through to analytics. Of course, you kind of hope the business might have reminded the call centre. Sigh.

As the calls arrived, the call notes were not being augmented with robust, pre-determined, keywords which would have enabled rapid and clear analysis of that entered text on a near-real-time basis. The same effect could be achieved by simply asking call agents what all the calls were about, but (to be fair), the call agents were rather busy.

That keyword data might have shown that the callers were all identifying as “Directors” and most were asking (in quite elongated conversations), about how to log-in to one of the organisation’s systems. Which, just about then, went splat. That’s a technical IT term which means it stopped working for no particular reason. Now the keyword flow changed from password issues to “the system won’t work” issues.

Luckily, the super-heroes in IT noticed a red-flashing light (I jest) and rebooted the server. The system returned. However, the rest of the morning it kept splatting, being rebooted, … You get the idea.

The data that IT had could have painted a pretty clear picture that this was going to happen. It’s just that they didn’t really care too much about that one little server, they had lots of other more important ones. But their data (for the naughty server) showed annual usage growing at around 4% a year over the last few years and that, at its peak (which oddly only seemed to occur once a year), last year was at 88%. Do the maths, the poor server was over-run. But each month, IT drew beautiful charts showing all the organisations’ servers were paragons of good health. Sometimes, they even shared these charts with the business (not the call centre though).

The final piece of the jigsaw was understanding why logging-in was proving such a problem for this subset of clients and this particular service. Connecting call data to client CRM data quickly showed that this was almost the only interaction these clients had with the organisation each year. The call records then showed that “remembering their password” or “knowing what data to enter” were the key tags to describe the clients’ issues. Eminently common issues and ones that can readily be addressed with better user interface design, education of agents and users, reminder communications, and perhaps adopting external user-validation processes. Of course, the next crisis arrived so these sorts of improvements never actually happened.

So, what lessons do we learn from this? It might be useful to use a simple graphic to show one way of contemplating how to think about getting good data that enables quality analytics.

Data has different owners who, traditionally, impose their own standards, definitions, segmentation, and so on. Adopting co-ordinated governance practices (just like for finance or risk) is a massive enabler of good data.

Data is moving from controlled source systems to all-over-the-place. And there’s so much more of it than there ever used to be. Active and consistent curation is needed to enforce (in a friendly way) an organisation’s data governance model.

Data now lives in many places. Collecting that data together in coherent and consistent ways matters. It’s also quite difficult. This isn’t about data marts, it’s about descriptors and licences and metadata and data packaging. Building confidence in the quality of the data being collected.

Data never seems to arrive in perfect shape. It always needs consolidating or averaging or building into new fields or any one of a myriad number of ways that data gets augmented. It might need to be assembled with other data (IT data and call centre data for example) or shaped a certain way for a particular visual. But there are pretty consistent augmentation requirements and there are pretty consistent ways of standardising and automating such augmentation. Done right, we can join a call with a CRM entry and associate them with a system and then to a server. Now we can see much of the ecosystem. And we can keep rerunning the analysis as we make improvements to see if those improvements really are working.

We also have an endless need to categorise data, to tag data. This is a phone call, it’s a happy person on the line, it’s about this system, it is from that person. These are all ways of adding tags to data. Tags are incredibly useful because that’s how we can start joining things up, clumping them together, showing how processes flow, visualising pathways to outcomes. Tags are exciting. But if we all invent our own tags then value is lost. Governance and curation are how tags are kept moderated.

And, finally, we get to the fun bit. Actually, deploying the data in ways that enable rapid, effective, and (hopefully), elegant visuals that communicate the story inherent in the combined data and which enable audiences to quickly understand and react to complex ecosystems. It might also be deploying into AI systems to read call agents’ notes in real-time and then to tag calls automagically.

Deployment is the real value proposition. But only if you do something about what you’ve learned.

(Andrew Brice works with New Zealand government agencies on the visualisation of business ecosystems using complex, multi-faceted, data.)

To see the future of the West, study New Zealand’s and Zimbabwe's crises

How long can the worlds' gluttony for debt continue? Seemingly forever, until it cannot. That was the experience of New Zealand in the 1970s and early 1980s when government subsidies ruled the economy until the country went broke. 

(Summary: New Zealand radically liberalised the economy, suffered through terrible social and economic pain, and emerged as a modern, vibrant and growing economy. Growing debt cannot last forever, and when a country hits the wall, it can go the way of New Zealand through the pain and recovery, or the way of Zimbabwe to more debt and devaluation, inflation and longer pain with not gain. Europe and the US have this in their future, we just cannot guess when, and we cannot guess which choice they will make.)

Through the first half of the twentieth century, New Zealand’s close ties to England ensured a steady flow of lamb and milk products from the former colony to “blighty”, at economic terms that benefited both the UK and New Zealand. This ensured that the New Zealand agrarian and rural economy continued to grow, benefiting the entire country. This also allowed for subsidies on imported goods, and on good assembled in New Zealand from imported parts (such as automobiles).

That could not and did not last forever. 

When the UK joined the European Common Market, they were required to abandon their Commonwealth trading relationships and imposed the common European trading relations, which included protectionism for European economies. New Zealand suffered. But so did Australia.

After Britain had joined the EEC Australian butter exports dropped by more than 90 percent; the Australian apple trade declined from 86,000 tonnes in 1975 to just 27,000 tonnes in 1990. The economic consequences of Britain's European ambitions for Australia were severe.

New Zealand was hit even harder, with pre-EM exports to the UK accounting for up to 55% of all exports (1958 – 1960), with 90% of milk and butter going to the UK, and over 95% of lamb (and 80% of mutton). This export market had grown New Zealand sheep populations into the 60+ million sheep, or 15 sheep for every Kiwi.

The short story is that with the loss of the UK markets, the New Zealand government and the National Party (the conservative and party of rural and agricultural New Zealand) attempted to hold up farming and rural incomes through subsidies. Up to 40% of the value of a sheep was in subsidies.

The only problem was that the National government (the conservative party) was running deficits like crazy to fund the range of subsidies, and the deficits were doing exactly what should be expected, devaluing the currency and increasing national debt servicing costs. Inflation was high, and a wage and price freeze did nothing to alleviate the problem, and international pressure was undermining the value of the currency.

And they continued to build that debt, and pay the subsidies, until one day the money ran out, or more realistically, until National and the Prime Minister were told by Treasury that the money was going to run out. The crisis had arrived.

So, having kicked the can down the road as long as they could, heaping subsidy on subsidy, hoping that it would all fall apart under the “next” guy’s administration, they ran out of money. It was their problem.

What to do?

Well, Robert Muldoon did what any responsible politician and Prime Minister should do – he got drunk and while drunk, called a snap election, knowing full well that National would lose, and the problem would be Labour’s.

Not surprisingly, National lost, and Labour won. A multi-year devaluation of the currency, ballooning sovereign debt payments, rising unemployment, and a disconnection from urban New Zealand meant it was time for a change.

The only small problem was that the day after Labour won, NZ Treasury went to the new (soon to be installed) government and said “Sorry to tell you this, but there is no money for your programme. In fact, you might not even be able to make the sovereign debt payment that is due soon.”

The can had been kicked as far and as long as possible.

So began years of economic restructuring in New Zealand, with years of associated pain up and down society. With no subsidies, large numbers of farms became financially unsustainable, with bankruptcies and forced sales. There were stories of farmers committing suicide as the auctioneers arrived at the properties.

Automobile assembly plants closed with the loss of jobs. Imports rocketed in price, and taxes increased to nose-bleed levels. I remember 66% income tax over a (fairly low) level.

Labour had the courage to throw away their platform and enact wide-ranging economic reforms. The pain was incredible. 75,000 manufacturing jobs and over 20,000 jobs in the public sector were lost in the five years from 1987 - 1992. With the pain of liberalising the economy, employment began to grow again through the 1990s, and New Zealand became one of the most open economies in the OECD (from a position of being the least open of 24 OECD economies in 1984).

The sale of State Owned Enterprises resulted in both massive pain, exportation of profits from the privatised industries (such as Telecom NZ), but also the modernisation of industries that remained in government hands as businesses (such as NZ Post), most of which became profitable businesses returning an ongoing dividend stream to the Crown (NZ Government).

Unlike New Zealand, when Zimbabwe hit the wall of national debt, they kept printing money and borrowing, resulting in devaluation and inflation, and ultimately a ruined economy (with a little help from property confiscations and destruction of businesses). 

  

Looking at the chart above, national debt exploded to almost 140% of GDP, dropped, then peaked again at 147% of GDP before dropping again. Why did it drop? Without even minimum fiscal discipline, international lenders simply would not buy Zimbabwean national debt at any price, and maturing debt had to be repaid – with printed money. The cycle repeated, and debt to GDP has stabilised around 80%. 

What stopped the international community? When “the inflation rate reached a peak of 89.7 sextillion (10^21) percent” in 2008.

New Zealand, by contrast, managed to keep inflation, while high for a period, relatively under control, and the economic reforms and fiscal discipline provided the comfort required to manage international expectations of the value of the currency. Inflation peaked before Muldoon was forced out (by his own policies) and was brought under control by the Lange government.

The national debt was also brought under control and paid down, and while spending and borrowing have increased, debt to GSP ratio remains well under 30%; healthy by international standards, and simply low by OECD and “First World” standards. 

  

Where to the “West”?

Current debt levels in Europe and the United States are simply unsustainable. And yet the borrowing continues, and balanced budgets (forget about paying down debt) so not exist in any of the major European countries or the US. This cannot continue forever, and the real question is equally simple:

Will the “West” chose the New Zealand route of hard choices and “short term” (3 – 5 years) pain, or the Zimbabwean choice of continued printing of money, devaluation, and hyperinflation?

The following graphic shows the results of the choice made by New Zealand, and the choice made by Zimbabwe. The grey is 1994, blue is 2004, and green is 2014. Zimbabwe’s choice effectively destroyed their economy and they have lost more than a decade. New Zealand’s choice has, after a difficult period in the 1980s and early 1990s, resulted in a consistent and solid growth.

 'C' = Household consumption expenditure, 'G' = General government final consumption expenditure, 'I' = Gross capital formation, 'X' = Exports of goods and services, 'M' = Imports of goods and services

http://zyaneconomics.appspot.com/#/finmodel/

What New Zealand in the 1970s and early 1980s also shows us is that politicians will continue with their profligacy until they cannot. They will keep kicking the can down the road until they cannot. They will keep hoping that their policies can continue until the “next guy” has to deal with it.

We know this because it already happened, in New Zealand and then, to a lesser extent, in 2008, resulting in TARP and bailouts of industries that lasted for years in the US and across Europe. 

Unfortunately, the ammunition to replicate that kind of stimulus probably no longer exists, and as with Zimbabwe, the first period of money printing did not teach politicians that this was a major danger, but seems to have taught them that they can do it again, and again. Now the Fed (in the US) and the ECB (in Europe) face the problem of QT - Quantitative Tightening, a process as fraught with risk as the original QE. Growing economies do not like restrictions on the money supply.

So what choices will the politicians make next time? Whatever the answer, I suspect we will see the results before long.

Stop talking about Austerity as you have no idea what that means

It is disgusting to hear the British and French bleat on about the horrors of Austerity, as if they actually had any idea of what they are talking about.

My friend Andrew Brice in Wellington, New Zealand has produced some simple but effective graphs that showing GDP growth across the world from 1994 to 2014. Looking at the graphs for somewhere like Greece, and you quickly see what Austerity really means.He is graphing World Bank data since 1994 on a range of economic data points for all countries. While not setting out to show "Austerity", the graphical presentation does provide some interesting information. His site can be found here.

I've selected four countries for the chart below: France, Greece, Spain and the United Kingdom.

Notice how the GDP points expand fairly uniformly for France and the UK. Each point on the spider diagram, for the three reference years, shows growth, indicating and reasonably balanced growing of the various key elements of GDP.

Not Greece, in which that growth virtually implodes for 4 of the five factors between 2004 and 2014. Only 'X' - Exports, continues to grow, and that at a slower rate than the previous decade. Household consumption, General government, Gross capital formation and Imports all collapsed. Gross Capital Formation is less than it was in 1994.

Spain looks only marginally better.

Yet for the UK and France, all five indicators continue to expand through the crisis and into the second decade of this century.

GDP growth graphs for http://zyaneconomics.appspot.com/#/finmodel/

In the UK and France, governments have attempted to bring spending under control, and in large measure have failed miserably.

Oxfam's report on Austerity in the UK is a wonderful example of not understanding reality. "Since 2010, austerity – primarily in the form of deep spending cuts with comparatively small increases in tax – has been the UK government’s dominant fiscal policy, with far fewer measures to stimulate the economy. The stated aim of austerity was to reduce the deficit in the UK to give confidence to the markets and therefore deliver growth to the economy. While austerity measures have had some impact on reducing the deficit, they have delivered little growth, and public debt has risen from 56.6 per cent of GDP in July 2009 to 90 per cent of GDP (£1.39 trillion) in 2013."

It is almost as if "Austerity" actually only means "we cannot have everything that we want". Economies just balance what is required to keep the lights on, tax rates that do not disincentive investment, balanced against social programmes that effectively avoid rioting and revolution. In which case Austerity has become the a rejection of a "give me mine" mentality.

Yet contrast that with 10 things the Greeks cannot do (from July 2015 at the height of the crisis). If you want to see real austerity, look at Greece. Could the UK or French governments survive cutting pensions by greater than 50%. Or unemployment higher than 25% (and 50% for under 26 year olds)? What would Oxfam say to 45% of pensioners living below the poverty level, and food consumption dropping by almost 30%?

United Kingdom

Looking at the GPD growth chart for the UK it is almost easy to see the source of discontent. Yet it needs to be remembered that the economy has continued to grow (once over the Global Financial Crisis - GFC - induced great recession) and is now larger then it was in 2008.

UK GDP Growth, 1994 - 2014
Note the continued expansion of all five elements

 
Personal income has (as of 2015) grown to exceed personal income, inflation adjusted, pre-GFC. It took a long to time recover, and certainly the average POME (Prisoner of Mother England, or is that short for Pomme de Terre?) has had a rough ride. But pensions have continued to be paid, the health service has continued to treat patients, and to expand the range of coverage and care provided. The economic effectiveness of that service may be up to question, but that is a factor of quality of provision, not total expenditure in GDP terms.

France

France is not significantly different, with growth across all five data points through the years. Yet France (and the French) are mired in a psychological paradigm that says that they are suffering, oh so horribly, from massive austerity. Each new president is elected on a promise of change, or in the case of Sarkozy, "rupture" with the past. Yet for twenty years, each new president has been met by strikes at the mere hint of market reform legislation, strikes lasting weeks and covering the entire country sometimes. Each president has caved. Even the French military has a better (much) record of refusing to surrender.

France GDP Growth, 1994 - 2014
Not bad for coming through the GFC

Yet looking at the image above, you would think that France has had fairly steady growth, especially when you consider that between 2004 and 2014 there was the GFC knocking their economy into deep recession, and their being in a Europe that has seen lackluster growth at best over the past half decade.

Greece

Turning to Greece, we see a very different graph, in which the only growth has been in exports. The years between 1994 and 2004 showed good growth, in line with the UK and France. Yet with the GFC and their debt crisis, loss of sovereignty and destruction of the social welfare system, the years 2004 and 2014 we can see what austerity really means.

Greek GDP Growth, 1994 - 2014

The collapse in Greek GDP growth has been across the board, with only exports growing past 2004 levels, and that only marginally. The other four indicators have all collapsed, with Gross capital formation falling to below 1994 levels.

Compare that to the GDP performance of Greece's four land-border neighbours; Albania, Bulgaria, Macedonia (well, okay, the Former Yugoslav Republic of Macedoia to give it the official name) and Turkey. All four have experienced consistent and continual GDP growth.

 These countries have come through revolutions, civil wars and military dictatorships, but have then spent 20 years growing. And growing. Meanwhile their Eurozone neighbour has suffered at the hands of creditors and "friendly" governments. "But it's all the Greeks fault, they are perfidious and profligate, and they borrowed the money". All true (well, except the perfidious). Yet looking at the rouges gallery of neighbours, can we really say that the Greeks are any worse?

Greek is in austerity. And this is real austerity; the kind that results from the markets losing faith, and the bankers engaging in as much Moral Hazard as the Greek government itself. Yet when the bill came due, the banks (as effectively representatives of other governments or the ECB and IMF) decided that only one side of the perfidious (and here I mean it) cabal would pay.

Summary

The United Kingdom and France should, to use the English colloquialism, "shut their pie-holes". They are not in austerity, and do not actually know that it means. They are living *slightly* above above their means, but continuing to borrow like drunken sailors.

True Austerity is Greece, and this is in their futures when the markets say "enough". Then we will see real austerity in those two countries, as government debt becomes unavoidable and unsustainable. Greece saw:

– 25%: Fall of gross domestic product
– 28%: Reduction in public sector employees
– 28.5%: Drop in food consumption
– 61%: Drop in average pension (833 euro)
– 45%: Number of pensioners living below the poverty line
– 26%: unemployment (50% at ages under 25)

This is the real face of austerity, and something the UK and France should really fear. Today's weak attempts to controls spending are only a start, and a poor one at that.