“Data Engineering’s Role Is Scaling Beyond Scope”
And That Should Be Celebrated
From Toby Skyes, Global Head of Data Engineering, QuantumBlack.
“There’s certainly more discipline in how analytics solutions are being built, as many are being created with an expectation that the data pipelines eventually scale beyond the initial project scope,” explains Toby, “This means software engineering expertise has become vital, not just in ensuring applications are developed in a robust, scalable way but also in determining speed to development, deployment and faster reuse for future models/data consumption. Traditional software engineering and DevOps have been welcomed into the data science world, enabling a faster path from experimentation to production, more effective reusability of both code and data assets, and more robust, resilient solutions (e.g. DataOps).”
