Both RAR and ZIP formats support encryption. To encrypt files you need to specify a password before archiving or directly in the Archive name and parameters dialog. In the command line this is done by using switch -p[pwd]. In WinRAR shell, to enter a password you may either press Ctrl+P or select the "Set default password" command in File menu or click on the small icon of a key in the bottom left corner of the WinRAR window. To enter a password in Archive name and parameters dialog press "Set password" button in "Advanced" set of options.
Unlike ZIP, RAR format allows to encrypt not only file data, but also other sensitive archive areas: file names, sizes, attributes, comments and other blocks. If you wish to do it, you need to set "Encrypt file names" option in the password dialog or in the command line mode use the switch -hp[pwd] instead of -p[pwd]. Without a password it is impossible to view even the list of files in archive encrypted in such mode.
Solid RAR archives and archives with encrypted file names can have only one same password for all archived files. Files in non-solid RAR archives without name encryption and in ZIP archives can use different passwords.
Do not forget to remove an entered password, when it is no longer needed, otherwise you may occasionally archive some files using the password without wishing to. To remove a password, enter an empty string instead of a password or close WinRAR and start it again. While a password exists, the icon of key is red, otherwise it will be yellow. Also, when you start an archive operation using a password, title bar of Archive name and parameters dialog flashes twice.
If you enabled "Use for all archives" option in password dialog and entered the empty string as a password, extract and test commands will skip all encrypted files and archives. In this mode the icon of key in WinRAR status bar is not displayed.
You do not need to remove a password if you entered it directly in Archive name and parameters dialog. Unlike other ways such password is valid only for the single archiving operation and automatically removed after its completion.
When extracting encrypted files, it is not necessary to enter the password before starting the operation, though you may do so. If a password was not entered before extraction and WinRAR encounters an encrypted file, the password will be requested from the user.
RAR archives are encrypted with AES-256 algorithm in CBC mode for RAR 5.0 archives and AES-128 in CBC mode for RAR 4.x archives. RAR 5.0 key derivation function is based on PBKDF2 using HMAC-SHA256.
By default ZIP archives use AES-256 algorithm in CTR mode. But such archives can be incompatible with some older unzip tools. You can set "ZIP legacy encryption" option in the password dialog to enable the legacy ZIP encryption, which is less strong than AES, but provides better compatibility with older software.
Even though WinRAR allows to use AES-256 both in RAR and ZIP formats, key derivation function parameters selected in RAR are expected to make RAR encryption implementation more resistant to brute force attack. Also RAR allows to encrypt file names and other file properties. If you need to encrypt sensitive information, it is better to select RAR archive format. For real security use at least 8 character long passwords. Avoid common words in passwords, they make a password weaker. Passwords are case sensitive. Maximum password length is 127 characters. Longer passwords are truncated to this length.
If "Encrypt file names" option is off, file checksums for encrypted RAR 5.0 files are modified using a special password dependent algorithm. It is important, because otherwise it could be possible to guess file contents based on a checksum value only, without knowing a password. Such risk would be especially high for short files or for strong BLAKE2 checksum. So do not expect checksums for encrypted RAR 5.0 files to match actual CRC32 or BLAKE2 values. If "Encrypt file names" option is on, checksums are stored without modification, because they can be accessed only after providing a valid password.
Remember that if you lose your password, you will be unable to retrieve the encrypted files, not even the WinRAR author is able to extract encrypted files.