Do You Really Need New Database Technologies For Big Data?

We briefly covered Big Data a couple of weeks ago with Ralph’s feature article about the subject and his conclusion was that most current DAM requirements were not yet big enough to justify that description, although they might be when combined with data from other sources like website log files.  It seems that even then, you might still need to go to very big data before you need to scrap your current relational database and replace it with something new.

Predictably enough for the IT business, any kind of new buzzword has been exploited by commercial interests to release  or re-brand tools and technologies – some of which might add something significant, others maybe less so.  Big Data also has its own range of high volume products which are ‘must have’ items that self-appointed experts tell us you require if you want to do it properly.

I was therefore encouraged to read an article that summarises some presentations delivered by speakers at the MySQL conference held in San Francisco earlier this month.  Those quoted represent some very large household name web properties such as Facebook, Twitter and PayPal who were all using a more conventional technology, MySQL, as a cornerstone of their application architecture.  The article also referenced this warning from PayPal senior technology architect, Daniel Austin:

“Daniel Austin urged his peers this week not to become “victims of technological fashion” such as NoSQL databases, during an entertaining presentation entitled “Big Data is a Big Scam” at MySQL Connect in San Francisco. “Only use big data solutions when you have a big data problem,” he told attendees. “Too many CIOs feel the need to find a solution to a problem they don’t have.” Defining a big data problem only by the volume of data organisations were faced with capturing and processing was no different to other data issues faced throughout the history of computing, Austin argued.” [Read More]

Big Data wouldn’t be the first solution looking for a problem in the technology sector – and definitely will not be the last.  As with so many new innovations in IT, Big Data might start to become more useful after the gloss comes off and the vendors or consultants who remain are forced to apply their expertise and knowledge to real world problems.

Big Data seems quite similar to the DAM sector about 10-15 years ago when there was an initial rush of excitement about being able to distribute digital images via the web which then died away for a number of years until DAM solutions started to deliver a bit more on their earlier promises and provide users with more useful (and user-friendly) features.

Share this Article: