Data Engineer to AI Engineer: Why Now Is the Time to Make the Move

Published by Kirill Eremenko

September 1, 2026

Are you a data engineer watching AI do your job in minutes?

Here’s why one engineer I spoke with is moving into AI engineering now — not in three years.

I’ll call him Sameer. Data engineer at a large insurance company, 3+ years of building production pipelines. Python, SQL, Databricks, AWS, Snowflake.

Six months ago, his team was pasting SQL queries into a chat window to speed things up.

Today, an AI agent with access to their data platforms refreshes an entire dashboard from one prompt.

His words: “Claude is doing the work. It’s literally doing my work in minutes.”

And the question he’s asking is the one most data engineers are avoiding: why would companies need this many data engineers?

So I played devil’s advocate on our call.

If AI is automating data engineering, what makes him think AI engineering won’t be automated next?

He didn’t flinch: “It very much could.” An engineer has to be forward-thinking, he said. When the field shifts again, you shift with it.

Then he said the line I keep thinking about:

“I’m not thinking about what’s happening tomorrow. I’m thinking about what’s happening three years from now.”

                 Here’s what I told him.

Data engineering is one of the best starting points for AI engineering.

RAG — the technique behind most enterprise AI systems — is about retrieving the right data and getting it to the model. That is data work.

Vector databases sound exotic. They are another way of storing and querying data. A data engineer can become the vector database expert faster than anyone else in the room.

The pipelines, the orchestration, the data quality instincts — all of it carries over. AI engineering skills stack on top.

Watching AI do your daily work is a signal. Sameer chose to read it three years early.

Kirill

P.S. Feeling AI put pressure on your career? See if you qualify for our AI Engineering Program: https://aiengineeringprogram.com/

Share on

Related Blogs