I just finished participating in a discussion that has mirrored many others I have observed in the past: We have a complicated problem with much data before us, and we don’t know where it may evolve or trend. Can we architect a single database schema up front that handles all possible options?
Every programmer or database administrator (DBA) will recommend keeping designs to a single database, schema and vendor. It makes life easier for them.
However, every real world application and community points to the natural outcomes of multiple schemas and databases. This reality, in fact, has what has led to the whole topic of data federation and the various needs to resolve physical, semantic, syntactic and other schema heterogeneities.
Designers can certainly be clever with respect to anticipating growth, changes as seen in the past and so forth. Leaving "open slots" or "generic fields" in schemas are often posited and perhaps may allow for a little bit of growth. Also, perhaps quite a bit of mitigation for schema evolution can be anticipated up front.
But the reality of diversity remains. The semantic Web and proliferation of user-generated metadata will only exacerbate these challenges. Simply talk to the biological or physics communities of what they have seen in finding a single "grand schema." They haven’t been able to, can’t, and it is a chimera.
Thus, smart design does not begin with a naive single database premise. It recognizes that information exists in many forms in many places and in many transmutations from many viewpoints. And what is important today will surely change tomorrow. Explicit recognition of these realities is critical to successful upfront information management design.
Viva la multiple databases!
