You Ask, I Answer: Will Automated Machine Learning Make Data Scientists Obsolete?

Sheedeh asks, “Will new advances like automated machine learning make data scientists obsolete?”

Most definitely not, though I can understand why that’s a concern. AI is currently automating a fair number of tasks that data scientists do, but those tasks are relatively low value. I’ve had a chance to test out a bunch of automated machine learning frameworks like IBM’s AutoAI and H2O’s AutoML. The new features are time savers for data scientists, but cannot do what data scientists do. One of the key areas where automated machine learning is, and for the foreseeable future, will fall short is around feature engineering. Watch the video for full details.

Recall that there are 5 key types of feature engineering:

  • Feature extraction – machines can easily do stuff like one-hot encoding or transforming existing variables
  • Feature estimation and selection – machines very easily do variable/predictor importance
  • Feature correction – fixing anomalies and errors which machines can partly do, but may not recognize all the errors (especially bias!)
  • Feature creation – the addition of net new data to the dataset – is still largely a creative task
  • Feature imputation – is knowing what’s missing from a dataset and is far, far away from automation

The last two are nearly impossible for automated machine learning to accomplish. They require vast domain knowledge to accomplish. Will automated machine learning be able to do it? Maybe. But not in a timeline that’s easily foreseen.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.

Pin It on Pinterest

Share This