Some send modes have options to turn on SSL and TLS mode for sending. What's the difference?
Basically, SSL mode and TLS mode are just short labels to mark the following behaviour:
Early implementations of encrypted protocols used a different TCP port from normal protocol, and expected an encryption negotiation to start immediately, instead of waiting for a special command from the client using the standard port. Such protocol is usually called by adding "s" before or after protocol's name. For example: "FTPS" (or "implicit FTPS", not to be confused with "SFTP"), "HTTPS", "SMTPS" (or "SSMTP"). SMTP uses port 465 for this purpose, FTP uses 990 and HTTP uses 443.
Later implementations of protocols used a different approach. The connection is initially established to unsecured port as with normal protocol. Once a connection is established, the client issues a special command (usually it's a STARTTLS, AUTH SSL or AUTH TLS). If the server accepts this, the client and the server negotiate an encryption mechanism. If the negotiation succeeds, the data that subsequently passes between them is encrypted. Because connection is established as unsecure - the same port (compared to normal protocol) can be used. However, sometimes a different port can be used. Protocol is called "FTPES" (or "explicit FTPS") for FTP, "SMTP AUTH" or "ESMTP" (extended SMTP) for SMTP.
Note: such names (with "SSL" and "TLS") may be a little confusing, because both SSL and TLS are application-layer cryptographic protocols. TLS is just a successor of SSL, i.e. TLS 1.0 is SSL 3.1. TLS is application protocol independent. Higher-level protocols can layer on top of the TLS protocol transparently. The TLS standard, however, does not specify how protocols add security with TLS; the decisions on how to initiate TLS handshaking and how to interpret the authentication certificates exchanged are left to the judgment of the designers and implementors of protocols that run on top of TLS.
So, both so-called "SSL mode" and "TLS mode" will use TLS or SSL protocol (depending on handshake's result) for handling encrypted connections. Words "SSL" and "TLS" are used just as short convenient "labels" for modes, meaning "encrypted protocol, old version, implicit" for "SSL" and "encrypted protocol, new version, explicit" for "TLS".
This is common interpretation to be seen in other software as well.
You can know which mode to select by reading help/FAQ for your server. Usually, it's TLS mode, when available. If you can't figure out the proper mode - try TLS mode first. If it works - keep it, if not - switch to SSL mode.
TLS mode will revert back to plain mode, if issuing special command will be unsuccessful (for example, if server doesn't support secure mode). Such "probing" is not possible for SSL mode, since you must connect in already defined state (secured or unsecured), while TLS mode decides this while negotiating with server. That's why you may keep TLS mode always enabled (unless you need SSL mode only).