From SSIS to Databricks: can AI do the heavy lifting?

A lot of companies still run their data pipelines on SSIS. It works, but it is ageing. Moving everything to a modern platform like Databricks is the obvious next step. The catch is that it takes time, people, and patience.

So we asked a simple question. What if AI could take on the boring part of that migration?

We built a proof of concept to find out. Here is what we learned.

The idea: translate, piece by piece

An SSIS package is really just a set of instructions stored in a file. Our accelerator reads that file and rebuilds it for Databricks. The flow looks like this:

1. Read the package. We parse the SSIS file and pull out every component.
2. Split it up. Each component becomes its own small unit.
3. Translate. The AI rewrites each piece as Databricks code.
4. Put it back together. The pieces merge into one clean notebook.

The trick is in step two. Instead of asking the AI to translate a whole package at once, we feed it one small component at a time. Each piece gets its own checkpoint. That keeps the output stable and makes the result easy to follow.

More than a translation: a built-in report

The accelerator does not just hand back code. It also produces a report. The report:

• Maps the package out. You see every component and how they connect.
• Rates the complexity. Each part gets a label, from easy to hard to out of scope.
• Flags the risks. Where logic might need a second look, it says so.

There is a bonus too. The model can suggest small logic improvements along the way. It is still early, but we see this growing into a real feature.

The type of model matters a lot

Not every AI model is up to the job. We tested a few:

• Claude Sonnet 4.6 gave the best results. The code was clean and reliable.
• GPT-4o was usable, but it needed more debugging.
• GPT-5.4-mini was not good enough for this kind of work.

The lesson is clear. The model you pick decides how much value you actually get.

What AI handled well, and what it didn’t

Some things translated smoothly. Others needed a human eye.

• Handled comfortably. Joins and lookups came across without issues. The relational work was solid.
• Translated, but verify. Scripting translated fine, but an engineer should still check it. That is especially true where there are dependencies.
• Out of scope for this POC. Heavy and dynamic pipelines were not part of this round. We will look at them in a next iteration.

The human stays in the loop

The generated code needed almost no debugging. The few tweaks we made were about the environment, not AI mistakes. Think of a serverless setup or a small datatype change.

What the human really does is validate and configure. You confirm the logic, set up the connections, and sign off on quality. The AI does the heavy lifting. The engineer makes sure it lands.

When it’s a fit, and when it isn’t

A few things shape the outcome. Cost scales with package size, since larger packages use more tokens. And because these models are not European, data privacy needs a careful look.

• A good fit. Packages with clear, standard logic. The accelerator shines here.
• Less of a fit. Highly dynamic or sensitive pipelines. These need more thought before you start.

The takeaway

This POC turned a tedious migration into something faster and clearer. It is an accelerator, not a replacement. The technology works. It just needs the right hands around it.

AI does the translation. People do the thinking. That balance is where the real value sits.