File Transfer

The File Transfer service utilises the SFTP, SMB and NFS protocols to download and upload files to and from the respective target server in each domain. The available file transfer protocols can be combined freely. For example; the service can be configured to download files in the source domain using SFTP, and then uploading them in the destination domain using SMB.

The service on the Upstream side of the Data Diode continuously monitors the configured server in the source domain for new files. Once the service on the Upstream side detects a file and deems it to be suitable for transfer, it downloads the file and transfers it to the service on the Downstream side of the Data Diode. The service on the Downstream side receives files and uploads them to the configured server in the destination domain.

Once the Upstream service has transferred a file Downstream, it will remove that file from the server in the source domain. However, if the service detects that the file has been modified on the remote server while the service was transferring it, it will not be removed, but instead re-transferred.

As mentioned above, the Upstream service continuously monitors the remote file server. A file is deemed ready for transfer once the service has encountered it, unmodified, a number of consecutive times.

Known limitations

The limitations listed below will be resolved in future versions of the service.

• If the File Transfer service fails to connect to its server, it will result in many reconnection requests.
• In some rare cases when using NFS in File Transfer, the service can freeze for several minutes when simultaneously transferring multiple large files (~10GB each). The file transfer will eventually be carried through, given a few minutes.

Configuration

The Upstream and Downstream parts of this service have most of their configuration in common. Any configuration that is specific to one part of the service is listed in that specific section. Parameters are divided into sections in the same way as they will be when configuring the service using the DD Manager.

Protocol

Parameter	Description	Format
Protocol	The file transfer protocol to use when communicating with the remote file server.	Available options: SFTP, SMB and NFS

SFTP specific

Connection

Parameter	Description	Format
Address	Address of the remote host, either an IP address or a hostname (if DNS has been configured in Local admin)	Internet host name, see RFC 1123, section 2.1
Port	TCP port that the service will use to connect to the remote host.	Integer within range 1-65535
Directory	An absolute path to the directory where files will be either downloaded from or uploaded to.	String
Host key fingerprint	The SHA256 fingerprint of the remote server's public SSH host key. This is used to verify that the service is connecting to the correct server. This parameter is optional, but it is strongly recommended to provide the fingerprint of the server to prevent man-in-the-middle attacks. See Retrieving an SSH host key fingerprint for more information on how to retrieve a server's host key fingerprint.	String representation of the SHA256 fingerprint, with or without the SHA256: prefix
Keep-alive interval	The interval at which the service will send keep-alive packets to the server. This parameter is optional, leaving this field empty disables the sending of keep-alive packets.	Integer within range 1-300 (seconds)

Authentication

Parameter	Description	Format
Type	The method to use when authenticating the configured user towards the remote server.	Available options: None, Password, Key
Username	The username to use when authenticating towards the remote SFTP server.	String
Password	The password to use when authenticating towards the remote SFTP server. This field is only available when the authentication type is set to Password.	String
Private key	The private key file (PEM encoded) to use when authenticating towards the remote SFTP server. This field is only available when the authentication type is set to Key. See SSH public key authentication for more information.	String
Private key password	The password to use when decrypting the provided private key. If the provided private key is NOT encrypted this field shall be left empty. This field is only available when the authentication type is set to Key.	String

SMB specific

Connection

Parameter	Description	Format
Address	Address of the remote host, either an IP address or a hostname.	Internet host name, see RFC 1123, section 2.1
Port	TCP port that the service will use to connect to the remote host.	Integer within range 1-65535
Share	The name of the share to which the service shall connect.	String with alphanumerical characters, - and _ are allowed. Must start and end with an alphanumerical character
Domain	The domain to use when authenticating towards the remote SMB server.	String

Authentication

Parameter	Description	Format
Username	The username to use when authenticating towards the remote SMB server.	String
Password	The password to use when authenticating towards the remote SMB server.	String

NFS specific

Connection

Parameter	Description	Format
Address	Address of the remote host, either an IP address or a hostname.	Internet host name, see RFC 1123, section 2.1
Mount point	An absolute path to the mount point exported by the remote NFS server.	String
Version	Version of the NFS protocol to use.	Available options: NFSv3
User ID	The numerical user ID to be used when interacting with the remote NFS server.	Integer within range 0-4294967295
Group ID	The numerical group ID to be used when interacting with the remote NFS server.	Integer within range 0-4294967295

Advanced

Upstream parameters

Parameter	Description	Format
Poll interval	The interval at which the service will poll the remote server for file updates (added, removed, modified).	Integer within range 1-3600 (seconds)
Stable threshold	The number of consecutive polls a file must be unmodified before being deemed stable enough for download and transfer to the Downstream service.	Integer within range 1-255
Download concurrency	The maximum number of concurrent sessions the service will establish towards the remote server and use for downloading files. Use this parameter with caution, too many concurrent transfers might cause the Downstream service to be overloaded, resulting in data loss.	Integer within range 1-255

Downstream parameters

Parameter	Description	Format
Upload concurrency	The maximum number of concurrent sessions the service will establish towards the remote server and use for uploading files.	Integer within range 1-255

File constraints

Type

Only regular files are currently supported. Symbolic links (and any other non-regular file types) will be silently ignored.

info

When using the SMB protocol, the behavior when handling symbolic links depends on the server implementation/configuration. In some cases symbolic links may be presented as regular files, while in other cases they may not be detected at all by the service.

Size

To prevent loss of data, the Downstream service temporarily store incoming files on disk until it can verify that they have been successfully uploaded to the remote server. Therefore the upper size limit of a file is determined by the amount of disk space available to the Downstream service.

Retrieving an SSH host key fingerprint

The host key fingerprint of a server can be retrieved using the ssh-keygen and ssh-keyscan utility programs. The examples below are from a Linux environment, but the mentioned tools are available for other operating systems as well.

In the examples below SHA256:ipXfG2vYVrCEnY/E+1kcpMRHDmFKyCYpu9vg9TqJ52 is the fingerprint string which shall be entered into the service configuration.

From a remote server

In order to retrieve the host key fingerprint from a remote server, execute the following command in a terminal:

ssh-keyscan <server address> | ssh-keygen -E sha256 -lf -

For example:

ssh-keyscan 192.168.1.100 | ssh-keygen -E sha256 -lf -

Will yield the following result:

256 SHA256:ipXfG2vYVrCEnY/E+1kcpMRHDmFKyCYpu9vg9TqJ52E 192.168.1.100 (ED25519)

From a public key file

In order to retrieve the host key fingerprint given access to the public key file, execute the following command in a terminal:

ssh-keygen -E sha256 -lf <path to public key file>

For example:

ssh-keygen -E sha256 -lf /etc/ssh/ssh_host_key_ed25519_key.pub

Will yield the following result:

256 SHA256:ipXfG2vYVrCEnY/E+1kcpMRHDmFKyCYpu9vg9TqJ52E user@machine (ED25519)

SSH public key authentication

SSH key authentication leverages asymmetric encryption to provide authentication with higher security than traditional passwords. SSH key authentication also enables users to implement single sign-on across multiple SSH servers, as well as passwordless login in automated environments.

An asymmetric key pair consists of a public key and a private key. A user can upload its public key to an SSH server, and if the server considers the key trustworthy that user can in future authenticate itself using its private key.

warning

The private key must remain only with the user, as possession of this key is proof of the users identity.

Generating a new key pair

An SSH key pair can be generated using the ssh-keygen utility program.

It is considered best practice to use the Ed25519 key algorithm, to generate such a key pair execute the following command:

ssh-keygen -t ed25519

If for some reason it is not possible to use the Ed25519 algorithm, you're recommended to use the RSA algorithm with a key length of at least 4096 bits. To generate such a key pair, execute the following command:

ssh-keygen -t rsa -b 4096

Key requirements

The configured private key must be PEM base64-encoded, and in one of the following formats:

• OpenSSH
• PKCS#5
• PKCS#8

The supported key algorithms are the following:

• ssh-rsa
• ssh-ed25519
• rsa-sha2-256
• rsa-sha2-512
• ecdsa-sha2-nistp256
• ecdsa-sha2-nistp384
• ecdsa-sha2-nistp521