In an ideal world, data engineers are presented with a source of ingesting data (the Extract and Load steps in ELT) from source systems that’s…
The death of the data warehouse, long prophesied, seems to always been on the horizon yet never realized. Much like cold fusion power and fully…
Workflow management platforms are what data engineers use to schedule and coordinate the steps in a data pipeline – an activity sometimes referred to as…
Loading data that’s been stored in an S3 bucket into a Snowflake data warehouse is an incredibly common task for a data engineer. In an…
Data Engineers are a hot commodity in 2020, but it’s surprising how misunderstood they are. Are they a software engineer with a hyped up job…
Last week I had the pleasure of attending and speaking at the MinneBOS 2019 data science and analytics conference in Boston. The folks at MinneAnalytics…
In a previous post, I wrote about using the COPY command to load data from an S3 bucket into a Redshift table. In this post,…
If you don’t yet have a data science team, you may have moments of panic feeling that you’re falling behind. It’s true, data science and…
Data warehouses have been around for decades, but in the last few years you’ve probably heard the term “data lake”. Despite what some believe, a…
MongoDB continues to be a major player in the unstructured data-space, and thus data engineers often confront the challenge of helping organizations make sense of…