Great post for two reasons First, the writing quality and the chic formatting aesthetic are nothing short of excellent. The instructions are crisp and complete. The screenshots are all scaled to the same size (nothing zoomed in or out)-and no ugly annotations. Second, the technical detail with every step is worth applauding. It is the type of article for which I come to Medium. Must read for data engineers doing anything on Redshift with dbt.

The core takeaway here is that you should concentrate on creating reference tables for standardization and reducing verbosity in the same way you use something like a utils file/library, full of functions to help you.

Very interesting piece! I haven't come across systems using 6NF. A comparison between Data Vault and 6NF would be interesting. There must be a significant trade-off between agility and organization. From Anchor Modeling's website: The technique incorporates the natural conecpts of objects, attributes and relationships making it easy to use and understand. It is based on the sixth normal form, resulting in a highly decomposed implementation, which avoids many of the pitfalls associated with traditional database modeling.

Writing technical articles is hard with this in mind. You don't want to get things wrong because, if you do, you might lose credibility.

Nice one! 90% cost reduction is amazing. To supplement this read, here's another must read on Iceberg vs. Hive from Rackspace.

This is what a mature dbt deployment should look like. I'm sure it won't be without issues, but things around project structures, agility, CI/CD, etc. have been thought through. A very good read for anyone looking to do something worthwhile with dbt!

@Duy Tran - A reference to the TableAPI you've built brought me here. Absolutely wonderful job with the article, and even a better job with building a scalable data system that enforces standardization. Although you've underscored the importance of having the standardized metadata by way of defining interfaces and APIs rather than custom code, I'll still say one can't stress on that point enough. I'm curious to know about the testing and quality frameworks you've used or have you built your own?