In 2012, the Data Scientist was famously coined the "sexiest job of the 21st century" in the Harvard Business Review. They were defined as a rare "unicorn" professional, powerful due to their blend of technical data handling, story-telling to communicate insights to business and scientific curiosity to unveil new insights. This blend was scarce. This is what made them so desirable.
In reality, building got more important than playing the data detective. We saw a fragmentation into various specialized skills (like ML engineer, data engineer, analytics engineer, etc.) The Machine Learning Engineer became arguably the most relevant (and higher paid) role. This happened as companies found out that deploying models into production creates faster business value than discovering new insights - an AI model sitting in a notebook doesn't do anything for the company.
The ML Engineers were also one of the main driving forces in scaling up the large language models that are powering today's AI hype. This was based in the so-called scaling law: the more data, larger AI and bigger compute you have, the better the AI will become. But ML engineers and applied scientists were the plumbers who built the infrastructure to enable this scaling.
But recently, the pendulum has started swinging again.
It's driven by two forces.
- Due to scarcity of latest hardware, AI researchers are creating novel methods to use data more efficiently, to get better yet smaller AI. (Anyone remembering DeepSeek, for example?)
- To create real business value in corporates, you need to specialize LLMs for the concrete problem at hand. Getting this right - accuracy, safety, domain fit - calls for data scientists' expertise.
While the technical engineering roles were needed 10 years ago to drive positive business value, we need more scientific roles today. The pendulum is swinging back. Data scientists are critical again, and reflect the need for scientific and analytical rigor in today's complex tech landscape. Meanwhile, novel genAI tools are automating many routine tasks, allowing the technically capable generalist to execute more in less time.
They might not replace technical workers, but can empower people with a good technical skill basis to realize more in less time. In other words: they help generalists to excel in their tasks.
👉 So do we see a return of the generalist scientist? Or are many technical specialists just irreplaceable? While we saw the pendulum of what the market demands swinging between technical specialist and scientists, an equilibrium might be what we actually need. Both roles are essential for taking AI from a technological novelty to an essential, profitable business tool. Especially when only 1% of companies have reached AI maturity - there's a long road ahead. Which way is the pendulum swinging in your environment - toward generalization or specialization?
References:
- Data Scientist: The Sexiest Job of the 21st Century, published in Harvard Business Review in October 2012
- Superagency in the workplace: Empowering people to unlock AI's full potential, McKinsey, January 2025