The Database Doctor
Musing about Databases

Tag: data‑structures

Cover image for Databases and Disaggregation
Databases and Disaggregation

Iceberg and Parquet, for all their flaws, have shown us a fascinating path forward for the database industry: Disaggregation. Apache Arrow is quickly moving us in the direction of the...

Cover image for Iceberg, The Right Idea - The Wrong Spec - Part 2 of 2: The Spec
Iceberg, The Right Idea - The Wrong Spec - Part 2 of 2: The Spec

Let us finally look at what is so wrong with the Iceberg spec and why this simply isn't a serious attempt at solving the metadata problem of large Data Lakes. In the first part of this I took...

Cover image for Iceberg, The Right Idea - The Wrong Spec - Part 1 of 2: History
Iceberg, The Right Idea - The Wrong Spec - Part 1 of 2: History

Iceberg: The great unifying vision finally allowing us to escape the vendor lock-in of our database engines. One table and metadata format to find them ... And in the darkness bind I the...

Cover image for Why are Databases so Hard to Make? - High Speed DML
Why are Databases so Hard to Make? - High Speed DML

After a brief intermezzo about testing (read about my thoughts here: Testing is Hard and we often use the wrong Incentives) - it is time to continue our journey together to where we will A...