How to migrate files from an SFTP server to Amazon S3
Move data from an SFTP/SSH server into an S3 bucket — one-off or on a schedule — with incremental sync and host-key verification.
Moving off an SFTP server
Plenty of data still arrives on SFTP servers — vendor drops, legacy exports, appliance backups. Getting it into S3 usually means a script that pulls files down and pushes them up, re-downloading everything each run because it has no memory of what it already copied, and quietly breaking when a key rotates or a large file blows the box's memory.
How to do it in BucketPilot
- 1.On the Credentials page, open the Servers tab and add the machine (host, port, username, and a private key).
- 2.Add the destination S3 bucket.
- 3.Create a migration job: source = the server and a remote path, destination = the bucket (and optional prefix), plus a schedule if you want it to keep running.
If your server sits behind a firewall, you don't have to open SSH to the whole internet: every BucketPilot job connects out from a single fixed address, 3.111.107.194 — allow that one IP on your SSH port.
The transfer is incremental (rsync-style size + timestamp check), so re-runs copy only what changed. Files are streamed — multi-gigabyte files don't buffer into memory. And the server's SSH host key is pinned on first connect: if it ever changes, the job stops instead of trusting a possibly-impersonated server, and you re-trust it deliberately from the Credentials page.
If something is ever deleted or changed on the S3 side, use Verify & repair on the job to re-scan the bucket and restore anything missing — see Verify & repair.
BucketPilot can also migrate from Azure Blob Storage, and — on the Business plan — write *to* a server: server-to-server replication, or a bucket → server copy that brings a full offline copy of a bucket onto your own machine. Migration jobs into a bucket are a Pro feature; server destinations are Business. See the jobs guide.