Ecosystem View: One way to think about data

(A guest post from Andrew Brice. Data analytics and visualisation is Andrew's passion, turning data into insights. Enjoy some fascinating insights here: http://www.zyan.co.nz/)

The calls were stacking up. They weren’t meant to be, this was just a normal call centre day. Call agents were racing, management hearts were racing, but no one was quite sure where the finish line was. Or why there was even a race.

Afterward, the vast swathes of data collected by the call system was able to be analysed and it started to become clear as to what had occurred and why. The needle in the haystack was found.

The thing is, it was all preventable. The needle could have been tracked if only they had been analysing their own data, ideally with an ecosystem perspective. If only.

OK, so how would that have worked?

Example dashboard for call flow monitoring

Two parts of the organisation already knew this was not a normal call centre day. It was an end-of-year filing date for a subset of clients (yes, it’s a Government agency). Not a familiar filing date, but a statutory one for these clients. The business knew and so did the call centre. Every year, on the same date, a spike occurred. But the call centre didn’t analyse call patterns at a deep enough granularity so they hadn’t picked this up. They were busy with “real” call volumes… The data was collected but it was never sufficiently deployed through to analytics. Of course, you kind of hope the business might have reminded the call centre. Sigh.

As the calls arrived, the call notes were not being augmented with robust, pre-determined, keywords which would have enabled rapid and clear analysis of that entered text on a near-real-time basis. The same effect could be achieved by simply asking call agents what all the calls were about, but (to be fair), the call agents were rather busy.

That keyword data might have shown that the callers were all identifying as “Directors” and most were asking (in quite elongated conversations), about how to log-in to one of the organisation’s systems. Which, just about then, went splat. That’s a technical IT term which means it stopped working for no particular reason. Now the keyword flow changed from password issues to “the system won’t work” issues.

Luckily, the super-heroes in IT noticed a red-flashing light (I jest) and rebooted the server. The system returned. However, the rest of the morning it kept splatting, being rebooted, … You get the idea.

The data that IT had could have painted a pretty clear picture that this was going to happen. It’s just that they didn’t really care too much about that one little server, they had lots of other more important ones. But their data (for the naughty server) showed annual usage growing at around 4% a year over the last few years and that, at its peak (which oddly only seemed to occur once a year), last year was at 88%. Do the maths, the poor server was over-run. But each month, IT drew beautiful charts showing all the organisations’ servers were paragons of good health. Sometimes, they even shared these charts with the business (not the call centre though).

The final piece of the jigsaw was understanding why logging-in was proving such a problem for this subset of clients and this particular service. Connecting call data to client CRM data quickly showed that this was almost the only interaction these clients had with the organisation each year. The call records then showed that “remembering their password” or “knowing what data to enter” were the key tags to describe the clients’ issues. Eminently common issues and ones that can readily be addressed with better user interface design, education of agents and users, reminder communications, and perhaps adopting external user-validation processes. Of course, the next crisis arrived so these sorts of improvements never actually happened.

So, what lessons do we learn from this? It might be useful to use a simple graphic to show one way of contemplating how to think about getting good data that enables quality analytics.

Data has different owners who, traditionally, impose their own standards, definitions, segmentation, and so on. Adopting co-ordinated governance practices (just like for finance or risk) is a massive enabler of good data.

Data is moving from controlled source systems to all-over-the-place. And there’s so much more of it than there ever used to be. Active and consistent curation is needed to enforce (in a friendly way) an organisation’s data governance model.

Data now lives in many places. Collecting that data together in coherent and consistent ways matters. It’s also quite difficult. This isn’t about data marts, it’s about descriptors and licences and metadata and data packaging. Building confidence in the quality of the data being collected.

Data never seems to arrive in perfect shape. It always needs consolidating or averaging or building into new fields or any one of a myriad number of ways that data gets augmented. It might need to be assembled with other data (IT data and call centre data for example) or shaped a certain way for a particular visual. But there are pretty consistent augmentation requirements and there are pretty consistent ways of standardising and automating such augmentation. Done right, we can join a call with a CRM entry and associate them with a system and then to a server. Now we can see much of the ecosystem. And we can keep rerunning the analysis as we make improvements to see if those improvements really are working.

We also have an endless need to categorise data, to tag data. This is a phone call, it’s a happy person on the line, it’s about this system, it is from that person. These are all ways of adding tags to data. Tags are incredibly useful because that’s how we can start joining things up, clumping them together, showing how processes flow, visualising pathways to outcomes. Tags are exciting. But if we all invent our own tags then value is lost. Governance and curation are how tags are kept moderated.

And, finally, we get to the fun bit. Actually, deploying the data in ways that enable rapid, effective, and (hopefully), elegant visuals that communicate the story inherent in the combined data and which enable audiences to quickly understand and react to complex ecosystems. It might also be deploying into AI systems to read call agents’ notes in real-time and then to tag calls automagically.

Deployment is the real value proposition. But only if you do something about what you’ve learned.

(Andrew Brice works with New Zealand government agencies on the visualisation of business ecosystems using complex, multi-faceted, data.)

Convincing Boards to focus on Cyber Security is no easy task, when...

Convincing Boards to focus on Cyber Security if no easy task, when those working in the business have priorities, responsibilities and rewards specifically structured to make Cyber Security a lower priority. Convincing Boards starts from the "middle" and must work both "down" and "up", and it will not be an easy or fast process.

I recently spoke to the CRSA (Control Risk Self Assessment) Forum in London, hosted at the IRM's offices (with thanks to Carolyn Williams), and very ably organised by Paul Moxey. My desire was to highlight the challenges that management (operational to the C-suite) face, and the decisions that must be taken, many of which lead to a de-prioritisation, or even ignoring, of Information Security.

This is not to suggest that Information Security should be de-prioritised, far from it. But the purpose was to highlight the difficulties that the Risk Professional will have in gaining the internal support to both raise and then to gain resources required for effective InfoSec.

To encourage the participants to consider (and actually, engineer scenarios in which InfoSec would lose out to other priorities) I provided two "role-plays". A very quick caveat; neither case represented a real company or actual situation, but was built from a wide range of situations I have been party to or have been the Internal Auditor or Risk Manager associated with elements of the case.

Role-Play 1: "Complex project choices"

The first looked at project level issues and delivery concerns leading to a situation in which InfoSec, while critical, was actually "pushed into the long grass" by each group that considered the case.

You can "enjoy" the role-play case study here.

The constructed problem centres around the competing constraints facing any business when it comes to systems implementation. The various strains on all members of a project team, including the leadership, sponsor, and steering committee, require to balancing of resources and priorities. Sometimes individual incentives outweigh the needs of the business. While this is not acceptable, it is a reflection of the reality of motivating people, and in some cases, focusing those people on outcomes that they are responsible to deliver.

When participants received the "role-play", there were more than a few people saying "I've worked on this project".

The premise was fairly simple; the project is in its final stages but has run over budget (Quelle surprise) and over time: and the user community's level of frustration is rising to breaking point. The infrastructure model is not adequately integrated into the corporations secure environment, and the additional time and cost will push the project further over budget and time.

The groups were then asked, each playing a specific role, to provide a recommendation, as a group.

As expected (and constructed), not one of the groups said that project implementation should be delayed until security concerns were addressed. In one group, the person playing the role of the IT Infrastructure representative threatened to "call Internal Audit" to which there was a response, "So you plan to stab your colleagues in the back?"

Others proposed setting up a working group to assess and recommend addressing the security and infrastructure issues after the project, while acknowledging that this would become, in effect, a new project fight for resources against all other projects. But it wouldn't impact their ability to deliver what they were required to deliver.

The final takeaway for participants was that it will be difficult to gain the internal allies required to address security if their support will be counter to their own needs, responsibilities and rewards.

Role-Play 2: "The C-Suite and External Expectations"

Likewise, at the C-Suite level, internal and external expectations can be such that investment in Cyber Security becomes a secondary consideration behind meeting the short-term demands of shareholders, markets or owners. This Role-Play set out to demonstrate the push and pull of competing requirements, again against a backdrop of systems implementation coupled with quarterly reporting needs, in challenging market conditions.

Take a look at the Role-Play here.

The four participants (the COO, CFO, CIO and Director of Communications) each have competing priorities, yet all are also keenly interested in ensuring company success.

Read the Role-Play and decide for yourself how these individuals should respond, and challenge yourself to find a way for them to agree to invest in greater Cyber Security, or to make a recommendation to the Board to do so.

Once again, while all but one of the groups could bring themselves to abandon their roles' self-interest, they did all recommend that the CIO push Cyber Security to the top of his or her agenda for the coming quarter. The outlier group suggested that the business "take the hit" this quarter and focus on Cyber Security, while also communicating the markets that they were doing so to improve the company's ability to protect and server customers in the future (though they did not agree fully on how to avoid the potential ramifications of announcing that they would focus on Cyber Security and the potentially associated assumption that their systems were not secure).

Summing up:

In both, or either case, do you recognise your company, or a company you have worked with in the past?

If so, be assured that it is possible to convince Boards, but only once the required groundwork has been completed. In the two Role-Plays above, it is too early, and there are too many competing priorities. But there is hope in each, in that the need is recognised, and there are ways out.

Gaining Board agreement on Cyber Security requires time and planning, with careful messaging along the way. In addition, before the Board can "buy-in" to investment in Cyber Security, key stakeholders within the business must also "buy-in", as it will be their alternative expenditure and investment plans and programmes that may suffer or be put on hold to accomplish improved security.

Role Play 2: "Shareholder Expectations"

Shareholder expectation generally revolves around the meeting of targets, primarily revenue and profitability targets that ensure either a dividend flow (private companies and utilities) or sustained growth in the share price. Senior managers, "C-Suite" executives and Directors know this and know that their bonuses and futures (in this company and in any others) depend on a track record of delivering to shareholders' expectations.

Welcome to the mid-year session of the Exco as it prepares for the upcoming earnings release season. Things seem to be on track, and the 1st Qtr results were in-line with expectations. The share price has responded roughly as expected. This quarter however, could be a little more difficult. Trading conditions are worrying the Marketing director, while internal costs are not dropping as quickly as budgeted. The new system is going to be at least two months late, possibly three, pushing benefits into the 4th Qtr.

You now have to make some decisions:

1. The COO. You have numbers to make, promises to keep. The numbers that you received from your senior managers are promising, but you don't believe them "I've seen numbers like these before, and they are always overly optimistic". Your CIO is constantly late with delivery, system outages have become too frequent, and the IVRs never seem to match the problem. To compound things, someone in IT changed the “404” error page to redirect to the Dictionary.com definition for “liars”.

2. Head of Corporate Communications. When dealing with crises and missed targets in the past, your motto has been "Bad news is good news, good news is no news" and the spin spin spin. But you feel things are reaching a point where your own credibility is coming into question. If things continue as they are, you're afraid the only professional option left to you will be to apply to become the Director of Communications at the White House in Washington.

                

3. CFO. You've managed to, just, get the numbers right for the 1st Qtr results, but this quarter will take a small miracle, and missed targets have been shown to severely limit the longevity of CFOs. The numbers expected by the markets (or owners) are possible, but there better not be any down-side surprises. There are costs that can be shifted into out-quarters, and revenue that can be brought forward, if we tweak our revenue recognition policy.

4. CIO. You know that the existing systems need replacing, that infrastructure is supporting the users, but the Security guy(s) are telling you that a serious architecture review is needed (again, "review" means they know there are problems but are too afraid to tell you everything), and the company simply cannot continue to avoid significant new investment. Your proposals for Security investment themselves will increase the overall IT budget to the equivalent of 12% of revenue from the current 10% of revenue, a level that is already at the high end of the scale for this kind of business.

Time to have your conversation, and come to an agreement that the CEO will be able to defend at the next earning call/shareholders meeting.

Role Play 1: "Complex Project Choices"

Scenario: A new system is in the final stages of development, and should "go-live" in three months. Testing is ongoing with the usual bugs and use-case mistakes. The project is projected (for the third month in a role) to come in at exactly 109% of budget, thus avoiding the need to go back to the Board for authorisation for additional spend.

The COO has committed to the Board that the system will go in on schedule. Internal Audit has given an "adequate" grading on a review of the project to date.

IT infrastructure has just reported that the servers will be ready, but that they will not be within the secured domains used by other corporate systems. To do so will require a re-architecting project. However, they do not think there is a major security threat, though when pressed, they've admitted that it would be possible, under "extreme" circumstances, for a hacker to gain access to "some" data. Re-architecting the environment will take an additional 4 months, and will add £275,000 to project costs, taking the project well over the 109% of budget.

You are now meeting to "discuss" the situation. You are:

1. Project manager. If you do not get this project in on time and within the allowed budget, you lose your bonus (20% of your salary), and you probably will not get that next project. Worse, you’re regular steering committee meetings with the sponsor (and team) are becoming a nightmare of complaints about timing, internal resources being diverted to testing, costs, etc.

2. IT infrastructure. You don't completely trust your own people's assessment, as there have been breaches before when some data was stolen. You also know that the 4-month estimate is probably optimistic. A few people in your IT team know too much about your systems, hoard that information, and honestly, you would have “moved a few on” if they didn’t hoard their knowledge. Can you trust them to fix the architecture in anywhere near to estimated time or budget?

3. Operations Manager from User Community. Your people have been crying out for this system for years, budgets have been cut, headcount reduced, and people are reaching a breaking point, with absenteeism escalating. Meanwhile, the project continues to demand more of your frontline experts for “testing”.

4.  Strategic Planning. Your models show that this system is going to boost profit by 5% annually, with an immediate 2% this year, to a profit-line that is already stressed. Missing the targets is not an option, as the cost of future external funding through equity or bond issuances will be impacted by the company’s evidence of being able to meet market expectations.

So, what do you all agree to recommend and do?