Big Data Tips

What makes a Good Analyst?



To be a great analyst you need to be able to think clearly and ensure that you understand the problem at hand well. A great analyst will ask many intelligent questions with regards to the problem and would be able to determine through speaking with a domain expert, which variables will be relevant for solving the problem. A great analyst is a very detailed person who would ask lots of questions and ensure that key information has been considered and collected.

In addition, a great analyst would select the right visual display to demonstrate the results of his/her analysis. Good communication skills are also a key trait as often the statistical results are too complex for the client to understand. So, the analyst needs to present the results in an easy to understand manner. Analysts who present the results in a story like manner with great visuals and good use of technology will be well liked for their presentation skills.

Further, accuracy is very important in the analytical world. So a great analyst will build predictive models as part of their solutions, but will always compare and evaluate the accuracy of their model with a few other models and also ensure that the accuracy is of a reasonable and acceptable level. Further, a great analyst will test their model results by validating the model against unseen data and against the actual data when it becomes available.

In a nut shell, a great analyst is passionate about solving challenging problems and works hard to ensure that the analytical solution while being highly accurate is at the same time making business sense and offers business value for the client. A great analyst will always exceed expectations and deliver more insightful information than the client expects and is paying for.

If you have any questions, please do not hesitate to ask me. You are also most welcome to share your thoughts here, on what makes a good analyst. If you would like to learn more about ‘Analytics’ or how to start ‘Analytics’ in your company, learn more at or contact me.

My First Job in Analytics – A Walk Down Memory Lane….



My first industry job, I worked as a Biostatistician at the National Health & Medical Research Council at the Clinical Trials Centre in Australia. The work was very challenging as I was working with real data, and even more stressful, I was working with patient data.

As a Biostatistician, I had to work closely with the oncologists, surgeons and doctors, and determine the Statistical Analysis Plan for the Clinical Trial. Will the Trial be a Double Blinded or Triple Arm Study, Will a Placebo be used, How many patients do we need on the trial, What are the inclusion and exclusion criteria, When will we stop the trial, etc. All these questions were answered using rigorous statistical techniques based on clinical knowledge and historical study results.

I would often share my solution with my manager, Val Gebski, who too often challenged my solution by throwing a lot of “what if” scenarios and getting me to think on several possible outcomes. We often spoke and debated for hours, sometimes the whole day, about different approaches that could be taken.

I remember some nights, in the middle of my sleep, I would get up to write out a method to use the next day to justify my solution. And, first thing in the morning I would run my method on some data and be very excited when the results agree with the typical method and in this way validates our new innovative approach.

I have many examples that I can share but I would say that problem solving is what I enjoy as it challenges me to think of new methods on how to approach a given problem.

I strongly believe and have seen that statistical/analytical techniques are powerful in producing repeatable and consistent results that are accurate when validated and, in a nut shell, Analytics is all about understanding a problem, understanding the relevant data that can help you solve the problem, asking lots of questions and then applying statistical/analytical techniques to make comparisons, predictions and finally optimizing your solution.

Making the Most of Big Data Now…

When it comes to actually mining Big Data for insights, many companies don’t know where to start or focus on the wrong things and get bogged down…. I say with confidence, Data Visualization is your Key, to making the most of Big Data!

Condensing piles of data to just a few charts is a balancing act of art and science. The visualization should narrate what the next short-term actions should be in order to improve the business outcome.All you need is a few charts with great data visualization – and this is worth 1,000 slides.


The next good tip is –  keep reports easy to understand and don’t forget the actionable steps that need to be taken based on the visualization insights or report insights. Your data visualization should always include recommendations as to what the business user or decision maker need to do.

It’s a 4 step process, ‘DATA – VISUAL INSIGHTS – ACTION – MEASURE’

When it comes to Big Data, it’s important to ask the right questions about what kind of information can empower your business, and your customers.

Identify trends and looking at what people are looking out for during that period. Information can be rendered useless or useful during different periods in your customers’ life.

For example, we found 35 out of 450,000 customers who were at high risk of leaving. That’s a small number, but the loss of those customers would have meant a loss of about $3.75 million dollars.


If the company had waited for a completed data warehouse implementation, this insight would have been missed and the company would have been in danger of closure!