What is all the fuss about Big Data? Everyone is talking about Big Data, Hadoop, Map Reduce, etc
Let me tell you, you can have all the data in the world at your finger tips but it doesn’t mean anything, if all you do is store it. And, that is a costly affair and a big waste of time, money and lost opportunities of owning a gold mine!
There are many examples, of use cases, how many businesses have used their data to make decisions faster (almost real time), smarter (not gut feel, but listening to the data & what it is telling you) and timely (data also has a use by date)!
Using statistical techniques, we are able to run algorithms that identify patterns in the huge data sets. These patterns help businesses identify insights that allow for smarter, faster, timely decisions!. For example, Google used search terms by region in the United States to predict flu outbreaks faster than was possible using hospital data.
Other examples, using sensor data from ships, you are able to build statistical models that will predict more accurately when a ship needs to be serviced or repaired at the right time (instead of calendar days) based on the engine (heat, vibration, sounds, age, engine type, number of gears, gear type, distance travelled, average speed, etc). Statistical models can identify which ships will need to serviced, together with the probability of the engine failing. Using this information, Big Data Analytics will save the business lots of money as the service will be cheaper than actually repairing the ship engine once it fails. By preventing a ship from breaking down, Big Data Analytics will save the business millions of dollars as there will be less delay (repair time is longer and more costly than service time) in doing business as usual. A shipping business can reduce total repair time and reduce costs of repair.
Building a business intelligence system is the first step in better understanding Big Data. A business intelligence system is typically, a dashboard which displays bar charts, pie charts, trend plots, and these days are highly interactive and can tell a story in a few minutes. Data Visualization is key! It is your starting point to understanding key variables and metrics related to your business profitability and Return on Investment. One thing, whatever you decide to have on your dashboard, make sure it delivers actionable insights. Which means if the “traffic light system is red, the business user needs to know what to do versus if the traffic light system is green”. Many businesses are developing creative ways in visualizing their data and are making it much easier for the user to see and understand the patterns instead of complex model outputs and formulas.
Big Data Analytics can also help transport related organisations where the next accident is likely to take place and the probability of the accident occurring. This is very useful, as the police, ambulance, fire brigade, re-routing systems can all proactively make decisions based on the information provided by Big Data Analytics. The result being the police, fire brigade, ambulance, etc can strategically position themselves so that they are more likely to be at the right place, at the right time, so that the incident is cleared quicker, the patients are transported more quickly to the hospital and the traffic jam is cleared more quickly. The amount of money that is saved by using Big Data Analytics is massive. The amount of lives that can be saved, because of the use of Big Data Analytics is also big.
So, I can go on and on sharing many examples but I guess the question that needs to be answered is “Where do you start?”
I recommend taking an agile approach. Start with a business problem, one whose solution is achieveable in a reasonable space of time. Start small, “like a baby just born and take baby steps, with your mother (someone with experience and the expertise, who knows better) to help you along the way. Yes, you may fall as you take your first steps, everyone falls, it is okay. Get up and soon you will be waling confidently, and then running and enjoying the data mining. Like mining gold as you see the benefits reap!
Please let me know if you have any questions on Big Data Analytics….its not easy to cover everything here!