Big data, data science, and machine learning rose to prominence around a decade ago, and have become cemented in the tech landscape as the size and complexity of data continues to increase. However, many companies are still confused about how to best make use of their data, and try to hire "all-in-one" superstars who can do everything from research to creating efficient ETLs to maintaining machine learning projects in production.
This talk explores how most data science or machine learning work should be a collaboration between dedicated data scientists and data engineers. We'll talk about the core responsibilities of each role, where they can each add the most value to companies, and also where their roles overlap. We'll also discuss where specializations such as machine learning scientists, machine learning engineers, DBAs and ML ops fit in, and whether the future seems to be heading more towards generalization or specialization. We'll end with recommendations on how you can build the best combination of these roles for your company's needs.
This talk was given at Big Data LDN in September 2022.