Sign in

I write about tech, Indian classical music, literature, and the workplace among other things. 1x engineer on weekdays. https://kovidrathee.medium.com/membership
Photo by Fotis Fotopoulos on Unsplash

Moving away from desktop-based database clients

Background

From the college days over a decade ago, when my database lab instructor gave me a CD of Oracle 11g database to install on the system, to the current day, when I can spin up a database instance with a simple CLI command, we’ve come a long way. As the…


Photo by Liam Briese on Unsplash

A short introduction to the latest in-memory database service from AWS.

In a recent post in this series, I talked about different options for caching in AWS. I talked about ElastiCache for Redis, as it is one of the most popular caches out there, especially because of its advanced data types like sorted sets which make a developer’s life way easier…


Photo by Alex Kotliarskyi on Unsplash

And the newest TSDBs on the AWS Marketplace

Following from my previous posts on Timeseries databases, where I argued for the case for using timeseries databases along with some hands-on tutorials, in this post, I’ll talk about different timeseries databases and how you can run these databases on AWS.

Running Timeseries Databases on AWS 🏃‍

Hosting and running any open source or enterprise database…


I get where you are coming from, but databases like PostgreSQL and MySQL are extremely good with complex queries. I think a better way to put it would be that relational databases aren't good with handling queries to serve the needs of reporting and business intelligence.

In other words, read scaling is not a problem with relational databases if you're sending the right kind of reads to the database.


Deletes in lakeFS

This feature is super-important. GDPR-compliance in Europe, CDR-compliance in ANZ, and similar privacy regulation elsewhere mandate it. Reading through the documentation, I found that objects can be hard deleted now:

"In order for an object to be hard-deleted, it must be deleted from all branches. You should remove stale branches to prevent them from retaining old objects. For example, consider a branch that has been merged to main and has become stale. An object which is later deleted from main will always be present in the stale branch, preventing it from being hard-deleted."


@Anom - Wouldn't AWS SCT be a good use-case here? Use SCT to convert your MySQL schema to a PostgreSQL schema in a completely lossless fashion. Create the schema in your target PostgreSQL database first and create DMS migration tasks. I have nothing against pgLoader, though :)

Great post!


Here's a nice post on Airflow by @Vinícius Mello! If you're new to Airflow, start here!


Here's a real example of how to use Airflow (with some caveats and suggestions) by @Rachel.

It's unclear if the temporary tables are created in the source database (MySQL) in the pre_mysql step. If that's the case, it might lead us to possible disk space issues. …


Photo by Patrick Pahlke on Unsplash

An overview of managed caches in AWS accompanied with usage notes.

Repeat read access patterns have mandated the implementation of a caching layer in most modern applications. These access patterns might be for different use cases, such as configuration files, images, any file on a CDN. …


Photo by Mehmet Ali Peker on Unsplash

A quick overview of the different specialty databases in AWS and their non-AWS alternatives

AWS probably leads the different cloud platforms in the number and maturity of the different DBaaS offerings. Starting with the most popular database services like RDS, Redshift, and DynamoDB, AWS has started offering many other specialty databases like QLDB, Timestream, Nepture, etc. …

Kovid Rathee

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store