Getting Started¶
Linux¶
Download, extract and set permissions
wget -O bp_linux.tar.gz https://github.com/Azure/blobporter/releases/download/v0.6.20/bp_linux.tar.gz
tar -xvf bp_linux.tar.gz linux_amd64/blobporter
chmod +x ~/linux_amd64/blobporter
cd ~/linux_amd64
Set environment variables:
export ACCOUNT_NAME=<STORAGE_ACCOUNT_NAME>
export ACCOUNT_KEY=<STORAGE_ACCOUNT_KEY>
Note
You can also set these values via options
Windows¶
Download BlobPorter.exe
Set environment variables (if using the command prompt):
set ACCOUNT_NAME=<STORAGE_ACCOUNT_NAME>
set ACCOUNT_KEY=<STORAGE_ACCOUNT_KEY>
Set environment variables (if using PowerShell):
$env:ACCOUNT_NAME="<STORAGE_ACCOUNT_NAME>"
$env:ACCOUNT_KEY="<STORAGE_ACCOUNT_KEY>"
Command Options¶
-f, --source_file (string) URL, Azure Blob or S3 Endpoint, file or files (e.g. /data/*.gz) to upload. -c, --container_name (string) Container name (e.g. mycontainer). -n, --blob_name (string) Blob name (e.g. myblob.txt) or prefix for download scenarios. -g, --concurrent_workers (int) Number of go-routines for parallel upload. -r, --concurrent_readers (int) Number of go-routines for parallel reading of the input. -b, --block_size (string) Desired size of each blob block.
Can be specified as an integer byte count or integer suffixed with B, KB or MB.
-a, --account_name (string) Storage account name (e.g. mystorage).
Can also be specified via the ACCOUNT_NAME environment variable.
-k, --account_key (string) Storage account key string.
Can also be specified via the ACCOUNT_KEY environment variable.
-s, --http_timeout (int) HTTP client timeout in seconds. Default value is 600s. -t, --transfer_type (string) Defines the source and target of the transfer.
Must be one of
file-blockblob, file-pageblob, http-blockblob, http-pageblob, blob-file, pageblock-file (alias of blob-file), blockblob-file (alias of blob-file), http-file, blob-pageblob, blob-blockblob, s3-pageblob and s3-blockblob.-m, --compute_blockmd5
- (bool) If set, block level MD5 has will be computed and included
- as a header when the block is sent to blob storage.
Default is false.
-q, --quiet_mode (bool) If set, the progress indicator is not displayed.
The files to transfer, errors, warnings and transfer completion summary is still displayed.
-x, --files_per_transfer (int) Number of files in a batch transfer. Default is 500. -h, --handles_per_file (int) Number of open handles for concurrent reads and writes per file. Default is 2. -i, --remove_directories (bool) If set blobs are downloaded or uploaded without keeping the directory structure of the source.
Not applicable when the source is a HTTP endpoint.
-o, --read_token_exp (int) Expiration in minutes of the read-only access token that will be generated to read from S3 or Azure Blob sources.
Default value: 360.
-l, --transfer_status (string) Transfer status file location. If set, blobporter will use this file to track the status of the transfer.
By referencing the same status file after a failure, the transfer will skip files already transferred.
If the transfer is successful a summary will be appended.
-u, --endpoint_suffix Endpoint suffix to be used for blob sources or targets.
Default is blob.core.windows.net