Big Data – Data is now an Asset and can even be a Gold Mine – But only so, if Analysed & used timely!

What is all the fuss about Big Data? Everyone is talking about Big Data, Hadoop, Map Reduce, etc


Let me tell you, you can have all the data in the world at your finger tips but it doesn’t mean anything, if all you do is store it. And, that is a costly affair and a big waste of time, money and lost opportunities of owning a gold mine!


There are many examples, of use cases, how many businesses have used their data to make decisions faster (almost real time), smarter (not gut feel, but listening to the data & what it is telling you) and timely (data also has a use by date)!


Using statistical techniques, we are able to run algorithms that identify patterns in the huge data sets. These patterns help businesses identify insights that allow for smarter, faster, timely decisions!. For example, Google used search terms by region in the United States to predict flu outbreaks faster than was possible using hospital data.


Other examples, using sensor data from ships, you are able to build statistical models that will predict more accurately when a ship needs to be serviced or repaired at the right time (instead of calendar days) based on the engine (heat, vibration, sounds, age, engine type, number of gears, gear type, distance travelled, average speed, etc). Statistical models can identify which ships will need to serviced, together with the probability of the engine failing. Using this information, Big Data Analytics will save the business lots of money as the service will be cheaper than actually repairing the ship engine once it fails. By preventing a ship from breaking down, Big Data Analytics will save the business millions of dollars as there will be less delay (repair time is longer and more costly than service time) in doing business as usual. A shipping business can reduce total repair time and reduce costs of repair. 


Building a business intelligence system is the first step in better understanding Big Data. A business intelligence system is typically, a dashboard which displays bar charts, pie charts, trend plots, and these days are highly interactive and can tell a story in a few minutes. Data Visualization is key! It is your starting point to understanding key variables and metrics related to your business profitability and Return on Investment. One thing, whatever you decide to have on your dashboard, make sure it delivers actionable insights. Which means if the “traffic light system is red, the business user needs to know what to do versus if the traffic light system is green”. Many businesses are developing creative ways in visualizing their data and are making it much easier for the  user to see and understand the patterns instead of complex model outputs and formulas.


Big Data Analytics can also help transport related organisations where the next accident is likely to take place and the probability of the accident occurring. This is very useful, as the police, ambulance, fire brigade, re-routing systems can all proactively make decisions based on the information provided by Big Data Analytics. The result being the police, fire brigade, ambulance, etc can strategically position themselves so that they are more likely to be at the right place, at the right time, so that the incident is cleared quicker, the patients are transported more quickly to the hospital and the traffic jam is cleared more quickly. The amount of money that is saved by using Big Data Analytics is massive. The amount of lives that can be saved, because of the use of Big Data Analytics is also big.


So, I can go on and on sharing many examples but I guess the question that needs to be answered is “Where do you start?”

I recommend taking an agile approach. Start with a business problem, one whose solution is  achieveable in a reasonable space of time. Start small, “like a baby just born and take baby steps, with your mother (someone with experience and the expertise, who knows better) to help you along the way. Yes, you may fall as you take your first steps, everyone falls, it is okay. Get up and soon you will be waling confidently, and then running and enjoying the data mining. Like mining gold as you see the benefits reap!


Please let me know if you have any questions on Big Data Analytics….its not easy to cover everything here!


Supply Chain Analytics – How can it help you?

Supply chain analytics helps you to better understand your business processes end to end. Many organisations in the supply chain industry have lots of data but are not making good use of it. By applying statistical techniques and analysing your data you can certainly make better decisions based on what has happened in the past, and based on what is currently happening, and based on what is likely to happen in the future.


By drawing a diagram that footprints your process from the raw ingredient selection  – to the distribution to the machine at the manufacturing factory  – to the employee working at the machine  – to the quality test – to distribution to the inventory warehouse – distribution to the retail store – to the back office – shelf- teller – transport – customer. Easily more than 10 processes.


As a start, visualization is the best starting point. Simple plots such as bar charts, trend charts, box plots and pie charts of the average time, minimum time, maximum time, standard deviation, time between events, can easily help businesses better understand what has happened. 


But the power of supply chain analytics is in what is likely to happen. Knowing this, gives a business the power and time to be proactive and innovatively decide on what to do, based on the forecasts, This will give your business an advantage to differentiate itself from its competition and thereby gain a competitive advantage. 


Forecasting models can easily be automated using statistical modelling techniques and deploying these models into the business operations. But it is important that once these forecasting models are deployed in the operating system, they should be monitored and over time, alerts should also be placed in the operating system when there is a change in the direction of the activity or when the quality of a product is below required standards, the machine operator should be alerted.

It is important that operational analytics are shared among all the stakeholders and departments so that all parties have almost real time information and this will allow the whole company to make proactive decisions always using the latest information. This means that better decisions will be made as they are based on the latest data which is the most accurate information.


Supply chain forecasts also allow a business to plan more effectively. Once the forecasting models are built, what if scenarios may be used to compare and adjust the forecasts based on your business goals and constraints.

Using real time insights businesses will be alerted to when they are overstocking or under-stocking, and thereby help businesses to make adjustments early and avoid loss in sales. These overstocking and under-stocking insights can contribute to tremendous savings for the business, which means increased return on investment. And, that is usually the goal of the business…increased return on investments!


Not sure, if I have convinced you enough that measuring and analysing your supply chain data helps you to make decisions faster, smarter and at the right time. Timing is the essence and the key to improving your supply chain operations.

Of course, there is much more that I can discuss here. But, please let me know if you have any questions.

What do Businesses Do with Their Data?

Collecting Data is easy…but applying analytics to better understand customer needs and wants is not so easy! It all boils down to what valuable information you need to better understand your customer. There is so much data that you earn, accompanied by data available from the outside, so a right balance needs to be struck! You need to find what really works for you, if not, you will possibly be caught in information overloading with Big Data accumulation.


Supermarkets are combining their loyalty card data with social media information to detect and leverage changing buying patterns. For example, it is easy for retailers to predict that a woman is pregnant simply based on the changing buying patterns. This allows them to target pregnant women with promotions for baby related goods.


Grocery Data – Customers usually buy steak, and are now buying hamburgers. They are also now buying with more coupons….This says a lot! Data can be very meaningful if used smartly! Collect data wisely, collect data when you have a good idea why you are collecting it and how you will use it. Using this data and combining insights can be very powerful. Using the above information, a retailer will be aware that the economy is in a down market and this will alert him to allow more promotions on steaks to boost the sales of steaks and to reduce the amount of steaks on the shelf until the economy is up again.


Many businesses collect data and do absolutely nothing with it, except store it! The data does not actually inform the business. Others, spend lots of time collecting the data, cleaning the data, and then produce beautiful charts and summary statistic tables for their regular reports and or dashboards, but that is as far as it goes…  If sales has dropped from one month to another, then the dashboard will display the drop using a bar chart or trend graph. But that is not good enough. The business should ask itself many more questions. Questions such as: Why has the sales dropped? Which products have dropped in sales? Which sales people have dropped in their sales? Which stores have dropped in sales? Which regions have dropped in sales? Which sales channel has dropped in sales?  And most importantly, what should the business do???


By answering the above series of questions, the business will find insights/golden nuggets that will inform the business as to the cause in the drop in sales. Based on these causes, the business will have to take the necessary steps to bring back the sales to an acceptable or increasing level.


Businesses need to develop dashboards that help them in their day to day business decisions. Always let the data speak to you, let it tell you what to do. Gone are the days when decisions were randomly made without any evidence or data!


Do let me know if you have any questions regarding the collection of data. Why do you collect it? And, when and how do you use your data.


What Makes Big Data Challenging?

Almost certainly, data is everywhere…everyday we ourselves are generating huge amounts of information: when we email, or  click on “like” on facebook, or “click on follow” on LinkedIn or tweet when we like a comment on Twitter, etc.

The types of data that we are generating has also changed. We now watch and share more “Youtube” videos on the go, or download and share more music on the go, or post more photos on the go. This results in most data generated being unstructured and also the speed at which we received it is much faster than a few years ago. In fact, 80% of the new data growth is of the form of videos, audio, photos, text, images, etc and millions of it is generated every second.


What makes Big Data even more challenging are the types of communications we use on social media, such as hashtags, colloquial speech, pictures, abbreviations and typos, etc. What do businesses do with the data? Collecting lots of data is easy….but applying analytical techniques to better make sense of the data, is not so easy!


Data is flowing in from all sorts of directions….blogs, websites, sensors, surveys, forums, etc….How do we make sense of all the data??? It seems the best thing to do, is to find out what works for you. What are your business problems or pain points? How can you use just the data that is related to your business problems or pain points? It is all about collecting data smartly, collecting on the data that will add value to your business!

Your Customer is in the Driving Seat….Join them on their Drive

Easy access to data & technology – puts your customer firmly in the driving seat. Your customer now makes decisions based on:
1. Online Reviews
2. Online Ratings
3. Social Media
And it’s Customer Analytics that can drive & enhance your customers’ experience by:
1. Providing insights about your customers.
2. Scoring your customers based on their buying behaviour, online discussions, likes & dislikes.
3. Making targeted recommendations to individual customers instead of Mass Marketing.
4. Providing opportunity for Proactive, Innovative Decision Making

Customer Analytics is the key to improving your customers’ experience with you. And, to enhance your customers’ experience with you, it starts with data!

Customer Analytics is essentially concerned with analyzing data and requires statistical analysis, data mining applications & machine learning algorithms.
By focusing on specific enhancements for your customers’ experience with you, you need to get the right people involved & use only the relevant data.
You also need to ensure your organisation has an online voice, promoting community conversations & keeping your customers engaged.
To keep your customers engaged, you need to provide them with some interesting information & knowledge about your products and services.

Analyse what customers are saying about your products & services. Derive actionable insights from you social media data & creatively decide to ‘WOW’ your
customer & allow your customer to enjoy their experience with you. Measure the impact!

Grow your customer base & strive for real time service, to keep them coming back for more!

But How do we Segment our Customers?

How should you segment your customers? The first question I would ask, what is your business objective?

Do you want to understand when to promote to a customer? If your answer is yes, then I would recommend studying your customer behaviour.

What do I mean by customer behaviour? We have to acknowledge that not all customers are the same. Some customers buy more often than others, some buy more consistently than others, some need a ‘push’ to buy, some will not buy no matter what you do…

So we need to understand our customers in terms of…
1. Which customers will buy
2. Which customers need a ‘push’ to buy
3. Which customers, no matter what you do, will not buy

There are various customer metrics that can help you to understand your customer behaviour. Two top of mind metrics for better understanding customer behaviour is:
1. RFM (Recency, Frequency, Monetary Value)
2. Latency (The Average Time between Purchases)

These two metrics are extremely good as they are actionable metrics…they indicate to you WHEN TO ACT. Very important!
Never compute metrics that do not indicate to you when to ACT or DO NOT assist you in making proactive decisions.

Today, I will speak a little on RFM. RFM allows you to score every customer based on:
1. When last they bought from you (Recency)
2. How often they bought from you (Frequency)
3. How much they have spent from you (Monetary)

Each customer, is scored on a rating from 1-5 for each metric Recency, Frequency, Monetary Value, where ‘5’ is most favourable and ‘1’ is least favourable.

The customers with the highest scores, for example, ‘555’ are your most valuable/profitable customers (They have bought most recent, most frequent and spent the most with you). These customers are likely to buy again on their own and do not need a ‘push’. They are probably your loyal customers. Customers who are already in love with your products. Pampers these customers.

The customers who were scored previously, for example, ‘111’ and now are scored ‘333’ are probably fairly new customers, you need to give them a ‘push’, promote to them through a ‘Cross Sell’ or ‘Up Sell’ Campaign. These customers may not be familiar with all your products as they are fairly new customers so by using ‘Cross-sell’ or ‘Up-Sell’ Campaigns, you are likely to get them to spend more with you & increase their likelihood of becoming loyal customers.

On the other hand, customers who have been previously scored, for example, ‘333’ and are now being scored a ‘111’, it is likely that these customers are about to leave you. Promote to these customers as this is where you are able to get high impact ROI, help these customers to spend more with you before they leave you. Send these customers a campaign, probably an anti-defection campaign.

Customers who have previously been scored ‘111’ whom you have promoted to since, and they still have not bought, these customers are likely to have left you already. Do not promote to these customers.

Does the above make sense? Let me know….If you have any questions, please feel free to ask them

Analysing your Customer Data…

Data is the new ammunition to conquer & prosper!

Every organisation can explore their customer data to determine how best to transform it in a way to create actionable insights that can support and direct the organisation’s marketing strategy and decision making…

Customer segmentation is the starting point as not all customers are the same. Different segments of your customers have different needs and wants. It is important for an organisation to understand their customer’s needs and wants so that they can segment their customers according to their needs and wants and thereby use target marketing, which will improve the return on investment as the response rate is likely to be much higher compared to mass marketing.

Of course, before we even start to segment our customer’s, we must ensure that the data is of good quality as ‘Garbage in, Garbage out’ (GIGO). We also need to know the current status in terms of our customer behaviour, how frequent do they buy, how much money do they usually spend, what is our usual response rates to our mass marketing campaigns, etc. The reason why this is important, is because we need to have a baseline (benchmark) so that once we start analysing our customer data, and using it for decision making, we need to know whether we are improving in response rates, improving in return on investment, customers are buying more often and spending more, etc.

After about 3 months, we need to compare our baseline metrics with our new targeted marketing response results to know whether
our customer segments and data analysis are delivering results. Communicate these quick wins to the rest of the organisation and continue to deliver results every week after that.

Does your organisation analyse its customer data? Do they obtain actionable insights from their customer data analysis? Are the right techniques being used? Share with me…

What is Customer Analytics?

Customer Analytics is about analysing customer data to better understand your customer behaviour, needs & wants. But who exactly are your customers. Are customers who last bought 5 years ago, still your customer? Organisations need to firstly define, what are they talking about, when they talk about ‘customer’ so that the whole organisation is using the same definition and there is consistency within the organisation.

It is also very important for organisations to understand who their profitable customers are. As the rule of thumb ‘20% of customers bring in 80% of your revenue’. So the starting point in customer analytics is to identify who are your profitable & valuable customers. It is not just looking at which customers have spent the most with you, as some customers have been with the organisation for a longer period than others. It’s about identifying for each customer, when last did they buy, how often do they buy, how much have they spent with you in a specified period of time? Based on these results, each customer is scored and the top 20% or so will clearly be identified & confirmed as the organisations profitable & valuable customers.

Do you know your organisations definition of a ‘Customer’?
Will all the employees of your organisation, if asked who are your organisations customers, compute the same answer?
Do you know who are your organisations top 10 profitable & valuable customers?

It’s time to know your customers….this is just the start!

Dr Carol Anne Hargreaves of NUS talk about big data and Trade Index

Dr Carol Anne Hargreaves of NUS talk about big data and Trade Index

The Trade Index which we are developing, will provide quantitative insights into Singapore’s import and export trends. With the Trade Index, we will able to tell, which country is contributing the most to imports; the government will then know which country is driving the export or likewise the import. The government can then make decisions whether it should release more resources, or whether it should  exercise restrain, whether the industry is in good or bad health… The Trade Index will help to make recommendations to drive the market from red to green. When combined with client data through statistically significant quantitative models, these insights will enable local corporations to proactively predict sector specific consumption with greater accuracy. This will lead to significantly improved operational and strategic decisions in key areas such as sourcing, inventory management, distribution and allocation of working capital investments.