Follow every file from discovery to migration

One row per file
How each file is known: its unique ID
What each Discovery run records
How a file's status changes
A migrated file that changes at the source
The status is tracked at the file's destination
The work phases: discover, decide, migrate
Track the progress of the migration
Find the files that are not migrated
Where did this file go
From the report to the run in the app

One row per file

Every run that handles a file records it in the BI tables, and each file has one row there. A later run does not add a second row: it updates the file's own row — its File status, its destination, its result and the run that handled it last.

Discovery Report

Found by a Discovery run, and not moved. The inventory you analyse to decide what to migrate.

Test Migration

Handled by a Test Migration run: tried against its destination, which is recorded, but not moved.

Migration

Moved by a Migration run, with where it landed and its result.

Because each file is counted once, the totals add up: the files in each status are parts of one whole, never the same file twice. The File status filter at the top of the page picks one status or several, and every file also carries its Result — Ok, Warning or Error — with the Message of the last run that handled it.

The report shows each file as it stood at the last refresh

SMART BI.pbix is the record of every file's status at the moment the report was last refreshed. Runs that finish afterwards show once it is refreshed again — refresh it after each wave, and keep a copy of each to see the project's progress over time.

How each file is known: its unique ID

Each row in the BI tables is keyed on the file's unique ID: that is how a later run finds the file's own row and updates it rather than adding a second one. Where the ID comes from depends on the source.

Source	The file's unique ID
SharePoint, SharePoint Online, Teams libraries and OneDrive	The unique ID SharePoint gives every file. It stays with the file when it is renamed or moved inside its library.
File shares	A file on a share has no ID of its own, so SMART Migration makes one: an MD5 hash of the file's name and path.

On a file share, a renamed or moved file is a new file

The ID of a file on a share is made from its name and path, so a file renamed or moved there gets a new ID: the next run records it as a new file, beside the row it had under its old name or folder. Plan renames and reorganisations of a share before its Discovery, or after its migration — and reset the job's From Date for a full scan after one.

What each Discovery run records

The first Discovery run of a source gives every file it finds a row in the BI tables. Every run after it — incremental or full, of the same job or another — records only what is new to the BI data: the files no run has found before, and the files modified since their row was written. A file that has not changed is not written again.

Each Discovery run also leaves its report, an Excel workbook (.xlsx), in the run's folder in Nextcloud Files. The report lists every file the run found, whether or not it is new to the BI tables, and its StatusInBI column says how each one compares with them:

StatusInBI	The file	In the BI tables
`New`	Found for the first time — on the first run, every file	A new row
`Modified`	Found before, and changed since	Its row is updated
`Unchanged`	Found before, and not changed since	Its row is left as it is

So a Discovery run is also the answer to what has changed since the last run: filter the report's StatusInBI column on New and Modified. An incremental run reads only the files created or modified since the last successful run, so its report lists those; a full run lists every file, the unchanged ones included.

Modified files are yellow in the Power BI report

In the Power BI report, the name of a file that has been modified since it was first discovered is shown in yellow — so the files that changed during the project stand out in the File details table.

How a file's status changes

A file's status follows the mode of the last run that handled it. The usual order is discovered, test migrated, migrated — but no step needs the one before it.

Run	The file's File status	What is recorded
Discovery	`Discovery Report`	The file as it is in the source: where it is, its size with versions, its dates and metadata.
Test Migration	`Test Migration`	The destination it was tried against, whether it would be added as new or overwrite a file there, and the result.
Test Migration with Restore on	`Test Restore`	That it was compared with its source, what a restore would do to it, and the result — nothing is copied back; see Restore.
Migration	`Migration`	Where it landed — its Destination Path — and the result.
Migration with Restore on	`Restore`	That it was copied back from Nextcloud to its source, and the result — see Restore.

No Discovery needed first. A Test Migration or Migration job registers the files it handles itself: a file it meets that no Discovery has found gets its row straight away, as Test Migration or Migration. Without a Discovery, though, the report knows only the files that were handled, not what is still to go.
Back to Discovery Report when it changed. A file that is Migration — or Test Migration — goes back to Discovery Report when a later Discovery run finds it again and it has in fact been modified since it was migrated: the copy at the destination is no longer the latest. A file that has not changed keeps its status.
The destination is kept for reference. A file that goes back to Discovery Report keeps the Destination it was migrated or test migrated to, so you can still see where the earlier copy is — and the next migration brings it up to date there.

Rediscover before the final run

Run the wave's Discovery jobs again — with Incremental they read only what changed — just before the final Migration. Every file people changed after it was migrated shows as Discovery Report again, and the incremental migration takes exactly those.

A migrated file that changes at the source

People keep working in the source while the project runs, so a file that is Migration is often changed there afterwards.

A Discovery finds the change
A Discovery run that finds the file modified turns its File status back to Discovery Report. Its Destination stays, so the report still shows that it was migrated, and where to.
The migration takes it again
By default, the next Migration run of the job migrates the changed file again: the rule Add New Overwrite Existing By Newer overwrites the copy at the destination with the newer one, and the file is Migration once more.

To handle changed files another way, open the job, switch on Advanced Settings and pick another rule under Rules for adding and overwriting files, on the Destination step — Add New Only, for example, brings in new files and leaves the files already migrated as they are, even when the source copy changed. There are five rules, and the other advanced options narrow which files a run takes at all.

The status is tracked at the file's destination

The report follows each file to its destination. When the same file is migrated to more than one destination — two jobs over the same library into two team folders, or a job whose destination was changed between runs — only the latest destination is recorded: the row shows the last place the file was migrated or test migrated to, and the earlier copies are not in the report. Their runs, and their logs, stay in Run History.

One destination per file keeps the report true

Give each source one destination. A file migrated twice to different places is counted once, at the last of them — the report then does not show the other copy, though it is there in Nextcloud.

The work phases: discover, decide, migrate

The report is built to be worked in, phase by phase, and the same job goes through every phase — it only changes its Mode:

Discover
Run Discovery jobs on the sources. Every file is in the report as Discovery Report.
Decide in Power BI
Analyse the files interactively in the report — by site, owner, age, file type, size and versions — and decide what to migrate and what to leave out. Set the jobs' filters to match, or export a list of exactly the files to take.
Test
Switch the jobs to Test Migration and run them: the files turn to Test Migration, with the destination each would go to and whether it would be added or overwrite a file there.
Migrate
Switch the same jobs to Migration — for a whole wave at once with Change selected jobs … — and release them. The files turn to Migration; the ones left out on purpose stay Discovery Report, the record of what was left behind.
Rediscover and catch up
Run the Discovery again before the cut-over: files changed since their migration go back to Discovery Report, and a last incremental Migration takes them.

The job dialog's Action step — Action defaults for a Job Import — sums this up under Mode, for the three modes.

Track the progress of the migration

With each file counted once, the report shows the progress of the whole project rather than of one run. Pick a Job group — a migration wave — and the File status filter shows how many of its files, and how much of its volume, are Migration, how many are Test Migration, and how much is still Discovery Report. Pick a site, a OneDrive or a share in Job path for the same figures on one source.

Refresh the report after each wave, and keep a copy: the copies side by side show the project's progress over time.

The Dashboard shows the same count at a glance: its Discovered, Test migrated and Migrated, under Files, jobs and runs, are the files in the BI data in each File status, as SMART Migration last reported them, each file once — Test migrated with Test Migration and Test Restore together, Migrated with Migration and Restore, each apart on its card.

Find the files that are not migrated

Pick Discovery Report in File status: the page shows only the files that were found and have not been migrated — or were migrated and have changed since, which still show their earlier Destination. Combine it with any other filter to find out why — they are older than the cut-off, in a file type left out, or in a library no job has taken yet.

Left out on purpose. Files a job's filters leave out stay Discovery Report. That is the record of what was left behind, and why, for the sign-off.
Still to do. Narrow the table to them, export it, and a job with the source type File List CSV migrates exactly those files.
Failed. Pick Error or Warning in Result for the files a run could not handle as it should, with the run's Message.

Where did this file go

Type part of a file's name in File name, or of its old folder in Path: the File details table shows where the file came from and, for a migrated file, its Destination Path in Nextcloud — the latest one, when it was migrated to more than one. It works the other way round too — type a folder in Nextcloud in Destination Path to see every file that landed there and where each one came from.

That is the question people ask years after a migration, and a saved copy of the report answers it without the server, the database or the app.

Deleting a run takes its files out of the report

The BI tables are filled from Run History. When a Run History row is deleted in the app, SMART Migration removes the files that run recorded from the reporting database, and they are gone from the Power BI report at its next refresh. Delete only test runs and mistakes; archive the others to put them out of everyday view — an archived run stays in the report.

Important: the Dashboard's figures change only once SMART Migration has purged the runs. Refresh the report when the Power BI heading on the Dashboard shows a green Up to date instead of a red mark.

From the report to the run in the app

Every row of the File details table has a Job link. Clicking it opens the SMART Migration app in Nextcloud on that run — the Run History tab, with the run's details — so the report leads straight to the run's log, its report files and its file issues. Nextcloud asks you to sign in first when you are not, and the run opens for a Nextcloud administrator.

See also