This is a project I've been working on to encrypt my data and sync it across devices. The idea came out of curiosity, not knowing how end to end encryption works, even though so many apps advertise it these days and wanting to get better at Rust.
It's helped me learn a lot about the cryptography, and file operations. Like the difference between symmetric and asymmetric encryption as well as hybrid encryption where I utilize both.
Symmetric encryption is when data is encrypted with a single key and only that key can decrypt it, like a key generated with a password. This is probably what most people think of when they think of encryption, I know that's what I thought for a long time. What confused me before I started this project was how can something be secure if you still have to come up with some way to send an unencrypted key for someone else to decrypt your message? The answer is asymmetric encryption.
Asymmetric Encryption is when you generate random private keys that can be used to create public keys that can be sent to other people in the open. This one is more interesting than symmetric encryption because the one private key can be used for encryption and signing. The private key can also generate a signing key and verification key, the signing key can sign a hash of a document, then the verification key can be released to the public to verify the document. One drawback though is that asymmetric encryption can only encrypt a small amount of data. So to use asymmetric encryption on a large document requires combining the two kinds of encryption.
Hybrid encryption combines both symmetric and asymmetric by using asymmetric keys to encrypt symmetric keys. This way you can still use asymmetric encryption on large amounts of data, like transferring files securely. This is what I chose to use with this project.
I want to craft a cli app that can take a local path and an S3 path, then bidirectionally sync the locations. This has been more difficult than I originally thought it would be, because there was a lot to learn. For some reason I've had a lot of difficulty with the basic task of checking the directories for changes, deletions and new files, there's a lot more possibilities then I originally expected. I can get a list of files in the directory and the metadata easily using WalkDir, but to actually check things I needed to save the file state for each file. This proved to be difficult because I needed to come up with a system and the first way I tried was saving a large json file, this was very slow since when I loaded it in Rust I had to use Vec which was slow for a lot of data.