Category: Big Data

From Data to Knowledge to Intelligence | PART TWO: The DATA

Welcome back!

Before we get started I wanted to say that I’m absolutely impressed by the questions and comments that the first and second articles in this series generated. All your feedback seems to indicate that, in spite of all the hype around Big Data, understanding fundamental concepts like data lifecycle, insights, true value of data and so on is still a challenge. Few comments indicate that the complexity is to blame. Steve Jobs once said, “Simple can be harder than complex: You have to work hard to get your thinking clean to make it simple.”

Getting our “thinking clean” is one of the reasons that led me to do this series.

Continue reading “From Data to Knowledge to Intelligence | PART TWO: The DATA”

From Data to Knowledge to Intelligence | PART ONE: The Journey Map

First, I want to Say Thank You for sending comments and sharing the first article of this series. Personally I found your response both humbling and invigorating at the same time. Thank You again!

Today, I’d like to take a closer look at the overall Journey. It’s not that the individual stages are not worth discussing (yet!), but I think we need to spend some time understanding the bigger picture before diving into a lot of details (and some code).

Continue reading “From Data to Knowledge to Intelligence | PART ONE: The Journey Map”

The Data Journey: From Data to Knowledge to Intelligence

Let me start by making a statement that most of us may find common sense: The ultimate goal of any expedition into the Data Cosmos is to make better decisions.

Just like a chess player, we want to not only gain valuable insights from studying and analyzing Data, but also want to be able to simulate different scenarios and pick the most favorable outcomes. Lastly, we want to monitor the results of our decisions and make adjustments as needed.

Continue reading “The Data Journey: From Data to Knowledge to Intelligence”

Scaling Hadoop – Myth vs. Reality

People use Hadoop for storing, processing and analyzing ever-increasing volumes of data. The question is no longer about whether Hadoop can scale to meet the demand. The question is  now at what point the operational cost of scaling Hadoop exceeds the value realized from the data analysis. How much of the Hadoop scale-out story is reality vs. hype?

Read on so you can separate the myth from the reality. Continue reading “Scaling Hadoop – Myth vs. Reality”

Designing for scalability with the Dell | Hadoop solution

Over the last months I’ve been having conversations with a lot of Hadoop users and developers. I’m glad to see that everyone wants to run Hadoop in production. Most of the practitioners also realize that, although Hadoop can scale, there are no clear guidelines that describe how to scale up/out Hadoop from very small to very large. Continue reading “Designing for scalability with the Dell | Hadoop solution”

OSCON 2011 Session: Hadoop – Enterprise Data Warehouse Integration

Last Monday, July 25th, I gave a presentation at OSCON about the integration between Hadoop and the Enterprise Data Warehouse (EDW).

The session was well attended. The dialog and the exchange of opinions was very good. The conversations continued after the session which, to me, shows that Big Data management is top of many practitioners’ mind.

I very much enjoyed the dialog. Also learned few things! Thanks O’Reilly for another successful OSCON! Continue reading “OSCON 2011 Session: Hadoop – Enterprise Data Warehouse Integration”