In the modern age of information technology, the sheer volume of data that is harvested has exploded exponentially with the advent of the internet and sensory technology making note of changes in their environment. Businesses collate this information and use it to their advantage to better their business, customer experiences, predict trends and identify previous errors. This information is called Big Data. The term ‘Big Data’ refers to the notion of data available to businesses which is in excess of a typical databases capacity to diverge and process due to the sheer volume and complexity. The definition of big data is dependent on the size of the business, as businesses vary in capability to break down big data. This could be due to limited resources or how the accelerating rate of incoming data being overwhelming or unmanageable for the business.
In an operational sense, businesses rely on big data for different reasons due to different business environments, goals and size of entity. Big Data is sourced by businesses both internally and externally from sources such as customers, staff, machinery and the use of EPOS systems which help to process big data automatically in real time. This is one way in which firms have effectively processed big data in order to improve the business from a management perspective due to information provided by processed big data. Big data is not only limited to the usual datatypes you’d expect businesses to utilise i.e numerical data. In modern times any source of data is possible to be used to perceive the state of current operations and potential improvements. This usually comes from external sources of big data, such as, in the forms of texts, audio, social media, videos and weather data, all of which can be collated.
The fact there is so much variation in datatypes and data being collected, all building at an alarming rate, makes it very hard for businesses to initially collate this data let alone being able to streamline and structure it in a data analysing system. Thus, unluckily resulting in a lot of big data to remain untouched or quite frankly useless. Therefore, in today’s competitive business environment, it is pertinent for management accountants to have systems in place which can economically deal with big data in order to boost operational performance through analytics; this is especially key for sizeable businesses who have more data and technological competition to deal with. That said, small businesses are starting to see the potential benefits from big data through the implementation of modestly priced big data analytics software which can help said businesses to collate sales and analyse consumer propensity to certain items, stock management systems etc.
What makes data “Big” or not is determined by the information systems that work with the data. (Miklos A. Vasarhelyi, 2015) From this we can get three definitions of Big Data:
1. Datasets whose size is beyond the ability of typical database software tools to capture, store, manage and analyse. (McKinsey Global Institute, 2011)
2. Datasets too voluminous to be reasonably analysed using database management systems or traditional software programs. (Warren, 2015)
3. Data that exceeds the reach of commonly used hardware environments and software tools to capture, manage and process it within a tolerable elapsed time for its user population (Adrian, 2011)
Which in turn gives us the features of Big Data, or the 5 Vs
1. Huge Volume
2. High Velocity
3. Huge Variety
4. Uncertain Veracity
5. Creating Value
(Arnaboldi, 2017)
Volume
Each key feature of Big Data presents its own challenge to managers and management accountants. Volume. If we think about personal data storage, the average person has a phone that may have 64GB of data, or a computer hard drive ranging from 500GB to 4TB. As an individual, it might take an individual some years to fill out all that storage space. In 2012 about 2.5 exabytes (that’s a million terabytes) of data was generated every day and that doubles every 40 months. (Andrew McAfee, 2012) So the obvious question becomes, how do you store all this data? More hard drives, a dedicated server, cloud storage are all viable answers but they all costs a lot of money when talking about the sheer volume of data that is being collected every day. The challenge of management is more a question of how much data is needed to make meaningful decisions, how much data is worth holding onto and what storage systems should be used?
Velocity
This is really talking about the speed at which data is being formed. Having data being formed at real-time (or as close as can be managed) is useful for companies to respond to changes as fast as possible, thus giving them an advantage over the competitors. Imagine being a hotel and knowing how many people where travelling in your direction at any given time? Knowing if they had accommodation already sorted out or not, knowing what type of rooms they wanted and so on, as a hotel you could start to tailor an offer directed to a specific individual before they even get to you. Harvesting real time is in of itself a hassle, sometimes even unethical (eg. “Always on” technology listening to private conversations) and can require a great deal of infrastructure to set up.
Amazon has an algorithm that collects real time data on peoples shopping habits, what they are looking at, what they buy with other products and so on to create a predictive algorithm that will suggest things the customer is known to like or combine with other products, so that if somebody puts weight lifting gear into their basket it might suggest liquid chalk.
From the management perspective, is it worth investing money into this? Can the information even be utilised even if it gathered quickly?
Variety
Data can come in many, many different forms such as; images, messages, gps tagging, sensor readings, updates etc. It’s a big mess of data out there and with the social medias such as Facebook, Twitter and smartphones an ever-increasing amount of data is being tied to the individual. Yet with how common social media and smartphones are, it’s easy to forget how recently they were all made. Facebook launched in 2004, the first iPhone in 2007. Traditional corporate data systems were unable to handle this new form of Big Data and the vast variety of which a single individual is creating every day. (Andrew McAfee, 2012)
Veracity
Simply put, the accuracy of the data. It should come as a surprise to no one the people lie and are more likely to lie when there’s no consequences to it. How many online surveys have you been presented with before you could enter a web page? The normally asked questions such as your age or gender people will just make up answers, they don’t care if it’s right, they just want to see what they came to see. Personally, I have a “sign up email” that I use whenever a website asks me to provide an email address, effectively creating a fake person generating data. How does a corporation weed out the false data that’s being made?
Creating Value
This is where managers and management accounting really step in. A corporation is gathering Big Data, they’ve navigated through all the problems and now it’s time to utilise the data to make a decision that will create value, but who makes the decision? Traditionally the highest paid persons opinion (HiPPO) will make the decision, HiPPOs are normally the manager or higher up. The problem occurs when most of these people got to where they are through traditional methods, and remember that from the variety challenge, data has changed massively since 2004. These individuals, for better or worse, make predictions from intuition. They call upon their experiences, memories of what has happened before in similar situations and then make plans based from this intuition. (Andrew McAfee, 2012)
Five Management Challenges
Leadership
It seems despite changes in the world, some traditions do endure. Companies succeeded or failed before Big Data, and companies do not continue to succeed simply because they have more information than a competitor. Clearly defined goals and leadership are still the cornerstone to success. Leaders are still needed to visualise what success is, to sell ideas to investors and inspire workforces. Big Data does not erase the need for vision or human insight. (Andrew McAfee, 2012)
Talent Management
With the rapidly increasing volume of data and the chaotic unorganised nature that it comes in, the need for skilled individuals that can make sense of the madness is unprecedented. Traditional methods of statistics are good but lack key techniques needed for understanding Big Data, not to mention the need for cleaning and organising skills of new data. With all this new computer systems are needed which means a new generation of computer scientist as well, and god willing these experts are gifted with the ability to talk business as well a company would be foolish to let their talents go to waste. (Thomas H. Davenport, 2012)
Technology
This ties heavily into the talent management of the new generation of computer scientist. With new technology emerging, and few new generation computer scientists to teach new skill sets to existing IT departments, simply getting all the new tech may not be the best idea, even though its not that expensive and the software tends to be open source (Hadoop). (Andrew McAfee, 2012)
“A rifle without a bullet is just an expensive club” – My CSgt.
Decision Making
Any good organisation keeps the information and decision makers in the same location, but with Big Data and information coming from all over the place, it can be very hard to have all the people with the relevant expertise together in the same place or to centralise all the information that is available. This can come to difficulties with decision making processes as it becomes reliant on people from different places to work together and overcome “Not Invented Here” syndrome, which leads into the final management problem.
Company Culture
Companies have cultures which are rigid and resistant to change. What is the point of introducing all the technology and new talents and Big Data to a company, if the culture of it does not change? “Too often, we saw executives who spiced up their reports with lots of data that supported decisions they had already made using the traditional HiPPO approach.” (Andrew McAfee, 2012)
It’s not just company culture, but a culture change that we are seeing start to take place around the world today, the change in culture about data privacy and data concerns.
With technology advancing daily at a rapid rate, it is extremely hard to avoid contributing to the ever-growing pile of big data. Even some of the simplest tasks like buying clothes, going to the gym etc. give businesses an insight of what does and doesn’t work in terms of their own business operations and what the consumers want. For example, buying groceries. Supermarket giants collect data every day on how many of a certain item is being bought, what else is bought with it, does the price of an item fluctuate sales and can even narrow down to what sort of people are buying certain items with the use of store cards. This in turn allows these multi-million companies to send out promotional emails tailored to each individual person in order to increase the probability of increased revenues.
Now perhaps for multi-million organisations, the use of big data comes as a large advantage as they will be able to afford the right systems and machinery to efficiently analyse and give quick fire solutions and tactics… but for small and medium sized enterprises an extravagant amount of big data can be left untouched due to little help. For many small and medium enterprises, data is stored on spreadsheets making it tedious and hard to pin point what is actually going on in terms of sales and the type of people that buy each product. This is likely to put off small and medium enterprises as big data is more for global businesses who ‘can handle it’. And it is more than likely that these businesses using spreadsheets don’t know where to begin with analysing data. Nowadays it is very easy to get a hold of software that can take data, pull it apart, and churn out key points of the business. It uses simple statistics to identify trends, seasonality where certain times produce better revenue and can most importantly predict what will happen in the future. In comparison to global organisations with a large customer base, we can see the similarities being that even though small enterprises don’t acquire as much data or have the complex software that global businesses there is a positive relationship with data and customers.
Customer relationship management (CRM), is a perfect example of why small businesses need not be scared of big data. It extracts information from big data to make contacting suppliers, looking at future projects and predicting the expectant revenue, seeing clients preferences and contact details in a way that they couldn’t before.
To conclude, it is clear to see that Big Data is here to stay. The evidence shows that making decisions based on facts rather than intuition always leads to better choices being made, a revolution that the scientific world came to see some ages ago. You can’t stand in the way of progress, either the current heads of business will realise this new age of Big Data is here to stay or they’ll be replaced by someone who does.
...(download the rest of the essay above)