Sheedeh asks, “Will new advances like automated machine learning make data scientists obsolete?”
Most definitely not, though I can understand why that’s a concern. AI is currently automating a fair number of tasks that data scientists do, but those tasks are relatively low value. I’ve had a chance to test out a bunch of automated machine learning frameworks like IBM’s AutoAI and H2O’s AutoML. The new features are time savers for data scientists, but cannot do what data scientists do. One of the key areas where automated machine learning is, and for the foreseeable future, will fall short is around feature engineering. Watch the video for full details.
Recall that there are 5 key types of feature engineering:
- Feature extraction – machines can easily do stuff like one-hot encoding or transforming existing variables
- Feature estimation and selection – machines very easily do variable/predictor importance
- Feature correction – fixing anomalies and errors which machines can partly do, but may not recognize all the errors (especially bias!)
- Feature creation – the addition of net new data to the dataset – is still largely a creative task
- Feature imputation – is knowing what’s missing from a dataset and is far, far away from automation
The last two are nearly impossible for automated machine learning to accomplish. They require vast domain knowledge to accomplish. Will automated machine learning be able to do it? Maybe. But not in a timeline that’s easily foreseen.
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
- Got a question for You Ask, I’ll Answer? Submit it here!
- Subscribe to my weekly newsletter for more useful marketing tips.
- Find older episodes of You Ask, I Answer on my YouTube channel.
- Need help with your company’s data and analytics? Let me know!
- Join my free Slack group for marketers interested in analytics!
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for watching the video.