Key Takeaways
Anyone handling sensitive user data lives in fear of a data breach. We know that encryption can reduce the negative consequences, but most encryption is relegated to infrastructure-level elements like TLS and VPNs rather than at the application layer. Application-layer and end-to-end encryption can be a powerful tool in our toolkit, but as developers, how can we safely add encryption to our applications without introducing bugs or reducing the utility of the data?
In this article, we discuss the pros and cons of application-layer encryption. We will cover the attack surface of application-layer encryption in the browser, how it is very different from native clients, and how WebCrypto helps.
The reputation, financial, and human impact of breaches can be extremely high. New laws that help protect end-user privacy are an important step forward, but they come with potentially ruinous fines.
Studies showthat encryption is one of the most effective technical security measures to reduce the impact and cost of a data breach. When attackers get encrypted datasets, they either have to attack a different system to get the key or have to settle with metadata and side-channel information instead of the good stuff.
Encryption is typically focused on infrastructure-layer elements, like TLS, VPNs, database encryption flags, and full-disk encryption. These are important tools in our toolbox, but they rely on assumptions about the infrastructure instead of the application code itself.
In fact, if you consider most recent data breaches, at least among established companies, they were certainly using TLS and at-rest database encryption, and yet the leaks happened anyway. For instance,Capital One was recently hackedand sensitive financial information stolen. Google Photosaccidentally gave the wrong users accessto photos and videos from other users. These mistakes could have been prevented, or at least mitigated, by application-layer or end-to-end encryption.
As developers, infrastructure isnt our strength, and sometimes its not our job, so encryption takes a back seat to features. But for those of us who do care about defense in-depth, it makes good sense to add encryption to the application itself. Application-layer encryption can insulate our systems from infrastructure-level failures, known weaknesses of TLS, and some server-side vulnerabilities.
The practice of moving more security, operations, and testing into the development process (known asshift-left) is improving software agility, reliability, and efficiency. It also means that security best practices need to be implemented as part of application developmentnot as an afterthought when things go wrong. However, the vast majority of developers are not security or cryptography experts, and at the same time, the security team has less control over the security posture of IT and development than ever before.
Application-layer encryption, or shift-left cryptography, is part of this trend. It means giving developers more control over what gets encrypted and who gets the keys for decryption. In some cases, the users themselves may be the only parties with the keys. In other cases, application-layer encryption can be an added access control layer on data management, providing defense-in-depth.
As implied by the name, application-layer encryption gets added directly to the codebase of your application, and access to key material is controlled by your application logic. As a result, you can think of the data itself as being encrypted throughout its lifecycle, rather than relying on it being on an encrypted network or disk.
The most widely-understood application-layer encryption is end-to-end encrypted chat like Signal and WhatsApp provide, so lets think through how those applications work. Its a bit over-simplified, but it basically works like this:
End-user action
Access Control Logic (Server)
App-layer Cryptographic Operation (Client)
Add a friend
Create an access control rule where users are allowed to send each-other messages
Trust the friends cryptographic key
Write the friend a message
Create an access control rule where the friend can read the message
Encrypt the message with the friends key (and sign it)
Read a message from a friend
Check for permission to download the message
Decrypt a message with end users key (and check the signature)
In this simple example, we can already see some of the power of application-layer encryption:
Note that this is an example of end-to-end encryption, but not all application-layer encryption is end-to-end. Also, applications like this still need TLS and other infrastructure-layer encryption to enforce things like authentication, prevent replay attacks, and address a host of other issues.
When we think about TLS, we picture data getting encrypted at its source and decrypted on the server. But this over-simplification hides the practical limits of TLS.
The reality of encryption in transit leaves out encryption of data at rest, which impacts the security of both ends of the transmission. It also completely ignores what happens to the data after HTTPS termination which may be further out on the edge of your network than you know; at your load balancer for instance.
So what about encryption at other points in the application? If youre doing an above-average job of crypto, youve written robust, well-tested code in your app to encrypt data at rest, youve used HTTPS and IPSec on your network, and youve enabled transparent database crypto.
Were pretty much encrypting everywhere with this approach, but as the data moves through the system, it gets decrypted and re-encrypted at each step. Each point that touches plain text data is a potential vulnerability, resulting in a large attack surface, and you have to ask yourself, why the heck do these intermediate services need the data in plain text anyway? They probably dont.
Infrastructure-layer encryption also lends itself to gaps in security because unanticipated parts of the infrastructure might get the data. For instance, your database and disk backups might not get encrypted, even if your database is. Or your health monitoring system might be logging sensitive data in plain text, and (horror of horrors) maybe even sending it to a third party. These security gaps happen because different individuals or departments are accountable for security at these various points:
Each one of these solutions uses different ciphers, libraries, and key sizes. Youre counting on a lot of people to get a lot of things right. Thats a problem.
Encryption is about communication; data is written and encrypted by one party, then received and decrypted by another party. The sender and receiver both have to have an application that knows how to do the encryption and decryption, and can be trusted to do it correctly. But that is easier said than done.
What if the encryption code is malicious? What could an attacker do? The simplest attack would be for the application to work exactly as expected, butalsosend the unencrypted messages to the bad guys. More subtle attacks are possible of course; adding hidden vulnerabilities to weaken the encryption, messing with the public keys, etc. But they all amount to the same thing: A bit of code that helps the bad guy get the secret message.
So lets talk about code delivery. For two people communicating using apps on their mobile phones, the trust chain goes something like this: A good programmer writes good encryption code, compiles it into an app, signs the app with a digital signature, and uploads it to an app store via TLS. The user downloads an app over TLS, the operating system checks whether the digital signature is trusted, and the user runs the app to have their encrypted communication. Note that this protocol is itself an application-layer cryptographic data exchange. Systems like Debian Linux have similar protocols for installing and upgrading the server and desktop applications.
There are a number of things that can go wrong with the trusted app download: The user could download a malicious version of the app. The OS vendor could undermine the check of the digital signature on the app. An attacker could trick the user into installing an old and vulnerable version of the app (or not upgrading it). Any of these types of attacks would make the end-to-end encrypted communication suspect. But for the most part, this works well.
Application-level cryptography is typically implemented in native code running on mobile, laptops, or servers, and can use a protocol like this to deliver trustworthy code. But modern applications very often have a major browser-based component, even for critically sensitive information.
The code delivery model on the web looks quite different from an app. When users decide that they want to have a secure conversation, they visit a web page. The browser downloads some JavaScript over TLS on-demand. Beyond warning the user about bad TLS connections, thats the end of the standard protocol for code delivery. It relies completely on TLS. The JavaScript that gets delivered needs to perform the application-layer encryption and tonothave any malicious code that just sends the unencrypted text to the bad guys.
Why is this a problem? Lets say for instance that our security claim is that the data gets encrypted in one browser, decrypted in another browser, and the webserver in between cannot see the data without warning flags and fireworks going off. To undermine this claim, the server simply needs to deliver malicious JavaScript at the application start time. So an attacker that can control the server that delivers code or various aspects of DNS and TLS could pull off this attack without breaking any crypto. The bad code can be sent only to a specific target, making it hard to detect for security researchers.
In fact, with the speed of application updates and continuous integration, similar attacks are possible against mobile apps and desktops. Many modern apps use dynamic code techniques to deliver at least some code to an app in real-time; many desktop apps update their own code at will. This gives attackers the ability to hijack code updates at various points but also gives security teams the ability to patch quickly. That said, the browser-based attacks are a lot better understood.
Some people in the security and cryptography community point to this issue to say that you shouldnt do browser-based encryption, or if you do, you cant claim that its end-to-end secure. Or at the very least, that it creates a false sense of security. We disagree. There are indeed weaknesses, but as developers, we should be doing it anyway, because simply put, people use the web for security-critical purposes.
Despite the code delivery problem, doing application-layer encryption in the browser significantly improves the overall security of any system. The reason for this is that security isnt all-or-nothing. Very rarely in modern server infrastructure is a single browser talking only to a single web server that performs every task; modern systems are just more complex than that.
For instance, lets say your web application uses HTTPS and does browser-based end-to-end encryption, but that it has an SQL injection vulnerability. The nature of this vulnerability is that the attacker tricks the application into tricking the database into dumping out sensitive data (over HTTPS, ironically). But in our example, the data is end-to-end encrypted, so the database only contains encrypted messages. Without application-layer encryption, the bad guy would get something much more sensitive: the plain text messages. Note that with this vulnerability alone, the attacker cannot change the code to inject malicious JavaScript; the browser-based encryption code is still sound.
On the other hand, if the attacker has a remote code execution exploit on the API server, and can modify the JavaScript or inject malicious code into it on the fly, theycanundermine the end-to-end encryption, again by simply adding code that sends the plain text data to themselves.
These are only two examples, one where application-layer encryption can be undermined and one where it cannot, but there are innumerable other attacks that can be prevented with end-to-end encryption: Perhaps you have a too-nosey employee who is looking for the private information on celebrities, but who doesnt have access to the code. Perhaps you backed up your Postgres database to an S3 bucket and accidentally left it open on the web. Perhaps an attacker can undermine TLS, but they only act passively; they can eavesdrop but they cannot do code injection.
As we can see, application-layer encryption in the browser provides defense-in-depth, even though there are challenges to code delivery. In the next section, we will talk about approaches that mitigate those challenges.
There are a number of ways to improve the security of application-layer encryption in the browser. The first line of defense is to use good, trusted code. Modern application development is much faster because we reuse a lot of code we find on the web, but if any of the code that runs in the users browser is malicious or vulnerable, it undermines the encryption significantly.
Protecting the server that delivers the code is also vital. Use the principle of least privilege when assigning access control rights on that server. Use multi-party control for administration and code deployment. This will significantly reduce the risk of insider attacks.
There are also under-used code-delivery settings that instruct the browser to take extra precautions. These arent the default because they somewhat reduce the flexibility of the development and integration process, but the security they provide is worth the work, whether your application does encryption or not:
In addition, there is a relatively new browser API that helps with efficient and secure delivery of cryptographic primitives. The WebCrypto API provides low-level ciphers, hashes, and other encryption components. This helps because you dont have to include those ciphers in your JavaScript. The browser implements them directly and can take advantage of local native execution and even hardware acceleration. It doesnt prevent certain attacks, like just sending an unencrypted copy of the data to the bad guys, but WebCrypto does make browser-based encryption more standard and accessible.
Secure code delivery isnt the only challenge for implementing application-layer encryption. The biggest problem is that most encryption libraries are relatively hard to use securely and difficult to implement consistently in different programming languages and platforms. When you encrypt something in a browser and decrypt it on an app, you probably need three different implementations in different languages (Android, iOS, and JavaScript) that all use the exact same ciphers and modes.
The secure operation of these modes is not very easy to understand. For instance, the well-beloved cipher AES is secure, but pairing it with an insecure mode like ECB (the default mode in Java) is insecure. Pairing AES with GCM is considered a best practice, but even GCM has its flaws; if you encrypt too much data with the same key, or make a mistake with the initialization vector/nonce, you could actually leak key material, which is a flaw that some other modes do not have.
One mistake can make your encrypted data unrecoverable, or even worse, recoverable by a bad guy.
Another challenge is that if you put encrypted data in your database, its no longer as searchable. You have to plan ahead for what kinds of queries and downselects you want the database to do or that you want your application to do. If you encrypt a users home address, for instance, you cant simply SELECT * for all the rows with the string Oregon. If downselecting by state is part of your application workflow, you can instead encrypt the users entire address, but add an unencrypted metadata field with their state so that you can still perform this query. From there, you can potentially use application-layer logic to decrypt the record and perform the rest of the search, but the database wont be of much help.
People I talk to are often concerned about performance for application-layer encryption, but this isnt a significant concern. Encryption is fast, and often hardware accelerated these days. After all, we use HTTPS for streaming entire social networks with photos and videos and dont really notice much of a performance hit. Its similar at the application layer, and you are simply unlikely to find encryption to be a bottleneck.
To be sure, there are still attacks against application-layer encryption. Various governments have made it illegal or legally impractical to operate an encryption service or install an encrypted app. Users selecting weak or reused passwords can completely undermine encryption. Users forgetting passwords is a challenge to address as well; what should happen in that case? Should the user be able to recover their data via a password reset email? That itself weakens the end-to-end encryption argument.
And of course, once the data is decrypted, attackers can attack the end device itself. This happened to WhatsApp in 2019, causing some to wonder if end-to-end encryption is worthwhile or important. But the fact that attackers had to target specific individuals with zero-day attacks against WhatsApp is proof enough to me that end-to-end encryption helps.
When implementing encryption in your application, you will need to consider your specific security goals, any compliance rules you might have to follow, and who you need to have the key material. Cryptography is very specific to your application. A trained cryptographer can help you understand the strengths and weaknesses of your approach, and no magazine article can tell you whats right or wrong. There are, however, a few choices you can make that will get you closer to good cryptography, and you can often safely use them.
First a bit of brief background on the three major cryptographic systemssymmetric, asymmetric, and hashing. Symmetric (shared key) is fast and efficient, these algorithms are usually your baseline for encrypting data. AES is usually what you want. Symmetric encryption suffers from challenges with key management. You need a way to get the shared key to both parties, which is why you need asymmetric encryption. Symmetric multi-block modes vary in their confidentiality and integrity properties, and some work better with different types of data or different system constraints (such as a lack of a random number generator): ECB, GCM, CBC, SIV, etc.
Asymmetric (public/private key) cryptography is slower and more complex than symmetric encryption, these algorithms are typically used for exchanging symmetric keys. RSA is the classic choice here; ECC is more modern and efficient, and almost as widely supported. Roughly speaking, public keys are used for encrypting data and verifying signatures. Private keys are used for decrypting data and generating signatures.
Hashing, cryptographic signatures, and message authentication codes (MACs) provide integrity. Hashing generates a short string that proves the data was either unchanged or in the case of message authentication codes, proves that the person holding a secret key signed the data. Many people think that encryption implies integrity, but it does not. For instance, AES doesnt provide integrity by default. Algorithms like SHA2, Poly1305, and GCM help.
Managing keys is a very big topic in itself, but a few important things to consider:
Beyond key material, there are other elements of randomness or uniqueness that are associated with encrypted messages. Initialization Vector, salt, and nonces fall in this category. These need to be communicated to the decrypting party as well, so they need to be stored or transmitted. Typically, its safe to transmit these unencrypted along with the ciphertext, but you should be careful not to let the attacker modify them.
You also need to pad, encode, serialize, and sign your messages. Believe it or not, even bad padding can undermine the confidentiality of the encrypted message. For signing of structured data like a JSON object or HTTP headers, you need an identical way for both sides to serialize and deserialize the data, or the signatures wont match.
If youve done all of this right, you now have an encrypted and signed message. Its likely at this point that youll want to send this message to another party, who will check the signature and decrypt the message. That means you need to communicate all of your choices: key id, size, cipher, mode, IV, hashing algorithm, etc. This communication itself is a fraught weakness in many cryptography systems. For instance, attackers have been able to trick some symmetric systems into behaving like asymmetric systems and sending their shared key directly to the attacker. Oops.
A few recommendations we have, particularly if you need to or want to stick with the NIST/FIPS-140 ciphers that are sometimes required for compliance in government work or banking:
Encryption is an exceptionally effective way to protect data, but most encryption deployed today is part of the IT infrastructure, and not part of applications. As developers, we have a unique opportunity to improve privacy and security of our users by making application-layer encryption a part of our toolbox. There are challenges to be sure; encrypted data can be harder to manage, and most encryption libraries are very hard to use for untrained developers, but the benefit to our users is worth it!
The following are not the formal definition of these terms, but color commentary to help you understand how these terms and technologies fit into application-layer encryption.
Isaac Potoczny-Jones is the founder and CEO of Tozny, LLC, a privacy and security company specializing in identity management and encryption. Isaacs work in cybersecurity spans open source, the public sector, and commercial companies. His projects have included end-to-end encryption for privacy in human subject research, secure cross-domain collaboration, identity management, anonymous authorization, mobile password-free authentication, anti-forgery in hardware devices, and privacy-preserving authentication. He has worked with agencies including DARPA, the Navy, Air Force Research Laboratory, the Department of Homeland Security, the National Institute of Standards and Technologies, and other elements of the DoD and intelligence communities. Isaac is an active open source developer in the areas of cryptography and programming languages. Education: B.S. in computer science, M.S. in Cybersecurity.
Read the original:
How to Use Encryption for Defense in Depth in Native and Browser Apps - InfoQ.com
