The Database Doctor
Musing about Databases

Tag: se

Cover image for Iceberg, The Right Idea - The Wrong Spec - Part 2 of 2: The Spec
Iceberg, The Right Idea - The Wrong Spec - Part 2 of 2: The Spec

Let us finally look at what is so wrong with the Iceberg spec and why this simply isn't a serious attempt at solving the metadata problem of large Data Lakes. In the first part of this I took...

Cover image for Iceberg, The Right Idea - The Wrong Spec - Part 1 of 2: History
Iceberg, The Right Idea - The Wrong Spec - Part 1 of 2: History

Iceberg: The great unifying vision finally allowing us to escape the vendor lock-in of our database engines. One table and metadata format to find them ... And in the darkness bind I the...

Cover image for Why are Databases so Hard to Make? Part 4 - Digging up Graves
Why are Databases so Hard to Make? Part 4 - Digging up Graves

In my last post about high speed DML, I talked how it is possible to modify tables at the kind of speeds that a modern SSD can deliver. I sketched an outline of an algorithm that can easily us...

Cover image for Why are Databases so Hard to Make? Part 3 - High Speed DML
Why are Databases so Hard to Make? Part 3 - High Speed DML

After a brief intermezzo about testing (read about my thoughts here: Testing is Hard and we often use the wrong Incentives) - it is time to continue our journey together to where we will A...

Cover image for Row or Column based Storage?
Row or Column based Storage?

These days, columnar storage formats are getting a lot more attention in relational databases. Parquet, with its superior compression, is quickly taking over from CSV formats. SAP a...