The Demand Side:

Do a simple test:
Go to Dice.com and search for “Hadoop” jobs. You will find, as of April 17th 2013, 1269 openings all across the US. Yes, the “hadoop” openings are not only in our dear-old, pie-in-the-sky, a-whole -world-by-itself, “bubble-country” Bay Area but ALL across the US. These openings are from places as diverse as Santa Clara, CA and Cary, NC. The openings are not only from startups but also at big companies as the above links will prove.

Thus to recap, across the whole of US, there are 1269 openings for software professionals with Hadoop experiences.

Some more graphs from Indeed.com which demonstrate the surge in job openings in Hadoop areas:

The Supply Side:

Now, do another simple test :
On Dice.com, login as an employer and search for “hadoop” developers. Make sure that the filter is for whole of US. You will find a grand total of 262 candidates!

For those who don’t have an employer account at Dice.com, I have attached a picture of the page.

Thus to recap, across the whole of US, there are a total of 262 software professionals with Hadoop experiences!

It’s been sometime since I have seen such a deviation – actually, the last time I saw such gaps in “demand-supply” was during the hey days of the dot.com rush, circa 97-99. Exploring the reasons for such a deviation, the answers are quick & easy:

• Human-kind never had so much of data since the last five year and so software professionals did not have to deal with so much data.
• Only a handful few lucky ones, who happened to work at data giants like eBay, Walmart, Amazon.com etc. had such exposures – and today they are sitting in companies like Google, Facebook & Twitter, with millions in stock options!
• Companies like Google, Facebook & Twitter are talent magnets! They have been able to lure great talent by the thousands by offering them much more than just annual salaries - lucrative stock options, free lunches, on-premise day care etc. This leaves the rest of the companies trying to fill in their open Hadoop positions in a lurch!
• Earlier, technology was simply not accessible to most companies. The Teradata’s of the world typically cost in the millions for anyone to implement and operate and so only a handful of data giants had the financial muscle to afford them whereas Hadoop based technologies are open-source and free! Every company now can afford them – but still need professionals to implement & operate!

Thus, we are in some sort of a hyper state where the demands from employers have increased substantially while the supply of such Hadoop trained professionals simply does not match up. The only way to solve this problem – creation!

We simply need to create more professionals trained on Hadoop – and the act of creation has already started. They are being created in training centers, employer sponsored class-rooms, universities offering Hadoop classes. Soon, educational institutions across countries like India, China would catch up too!

Thus, the future is not as bleak as it is now. These efforts would soon start churning out qualified professionals to catch up with the demand. Pretty soon the price wars among US based professionals would be fought, then off shoring will happen, then politicians will start complaining jobs are being lost & yada, yada, yada ...

But we all are used to those cycles and heard these things before – haven’t we :)