SciDB – A Science Oriented Large Scale Open Source Database

Most traditional relational databases are oriented towards the needs of business users rather than those of other problem domains, in particular scientists who need to manage data sets that tend to be much larger in scale (hundreds of Petabytes rather than Terabytes).  Scidb.org aims to change all that by providing an open source science-specific database engine that is better equipped to handle the scale and complexity of the needs of scientists:

“SciDB, Inc. is building an open source database technology product designed specifically to satisfy the demands of data-intensive scientific problems. With the advice of the world’s leading scientists across a variety of disciplines including astronomy, biology, physics, oceanography, atmospheric sciences, and climatology, our computer scientists are currently designing and prototyping this technology.” [Read More]

Those involved in preservation or science oriented DAM solutions will no doubt be aware that a typical RDBMS can quickly become slow and difficult to work with. Whether or not SciDB becomes the solution adopted by the industry is less clear cut, issues around interoperability with other RDBMS and (more importantly) skills and expertise to implement systems that use it being other significant challenges.  However, even in business oriented DAMs (e.g. for marketing) the rate at which the range of  metadata is being added to what appear superficially to be standard assets suggests that alternative approaches that can combine the flexibility of XML with the power of SQL will soon need to be found.

Share this Article: