Android R8 & Proguard [Part 1]

Introduction 📖

In this article we will try to demystify the concept of the R8 compiler and the proguard command line tool. We will describe the Android compilation process and some history behind these concepts, to gain a better understanding of how R8 works in Android.

History of Android Compilation Process 📚

Dalvik VM, ART and Proguard

In the past, the Android operating system utilized the Dalvik virtual machine (DVM) to compile Java source code, creating Dalvik bytecode, packaging this bytecode in the apk file and eventually installing in the device. The Dalvik virtual machine is similar to the Java virtual machine but is optimized for Android devices, considering their specific processing capacity, memory, and battery life. So, Android devices always expect Dalvik bytecode to run the apks.

The Dalvik VM uses a JIT (Just In Time) compiler, which means that the translation of Dalvik bytecode to machine language occurs during the execution of the app. This results in lower memory consumption and less physical space, which was great for the older Android devices at that time.

Later in 2014 Google announced a new virtual machine to replace Dalvik which is known as**Android Runtime (ART). The main difference from DVM is that is equipped with an Ahead-of-Time compiler (AOT). This means the translation of dex bytecode into machine language happens during the app's installation. This gives us faster app execution and lower CPU usage, leading to reduced battery drain.

From the very beginning Google introduced Proguard. Proguard is a command line tool that was part of the Android compilation process. The main task of Proguard was to modify Java bytecode to enhance performance while the app is running on the device. The main tasks of Proguard were:

1. Code shrinking: Removes unused code from the APK (project + libraries), resulting in a smaller APK size.

2. Code obfuscation: Rename classes and fields in order to protect the apk from reverse engineering

3. Optimizing code: Inlining functions, removing unreached else branches, etc.

Javac & dex compiler

I think it's time to understand the android compilation process, in other words how our code is converted into dex bytecode, be packed in apk and run through a virtual machine (Dalvik or ART, depends on the Android version of the device)

The Android compilation process has been changed a few times in the past, so I will try to cover and follow all these steps.

First things first I have attached a flow diagram about the compilation process optimized for releases has long looked as follows:

As you can see we have various steps. Let's break it down:

1. In the first step the java compiler (javac) translates our java code (.java) to Java bytecode (.class)

2. In the second step the proguard modifies the Java bytecode to further optimize the .class files. If we have libraries (jar, aar) in the project proguard optimize them too.

3. In the final step the dex compiler takes the Java bytecode and translate it to dalvik bytecode (.dex files). This process is called dexing.

The dexcode is our code translated into machine language that is recognized by Android. Along with resources such as images, XML files, and other assets, they are packaged into an Android package file (APK).

This process is for Java code. For Kotlin, it's a similar process, but instead of the Java compiler (javac), Android uses the Kotlin compiler (kotlinc) to translate the code (.kt files) into Java bytecode (.class files). This also applies to the next diagrams.

Jack & Jill 🧐

In 2014, Google announced the experimental Jack toolchain. The concept was to streamline the build process from two steps (javac + dex) to one. Jack (Java Android Compiler Kit) was created to convert Java source code directly into dex bytecode, aiming to significantly speed up the build process. Additionally, Jack was able to support Java 8 features like lambdas in Android, without the need for the Retrolambda library.

We can see that the process is much smoother now, but Jack faced some issues:

• What we are going to do with tools like proguard that uses Java bytecode? With Jack we don't have bytecode at all. The same issue applies for third party libraries that gives us Java bytecode.

• Another issue was that Jack did not support instant run, which contradicts the purpose of creating Jack in the first place, as it was meant to speed up the build process.

Google, in order to address this gap, integrated Proguard optimization into the Jack toolchain. For third-party libraries, developers were provided with Jill (Jack Intermediate Library Linker) to convert library bytecode into a specific format that Jack could manage (.jack files). Subsequently, Jack can convert the .jack files into dex files. As a result with Jack, it was difficult for the community to use third party libraries. So pretty soon in March 2017 Google announced the deprecation of Jack.

Java 8 and Desugar 🙂

After the deprecation of Jack, Google announced the comeback of java and the dex compiler. In order to support Java 8 features Google introduced a new step in the toolchain called "desugar". The desugaring process is responsible to convert Java 8 bytecode to Java 7 bytecode. The purpose of desugaring is to support the newest java features with backwards compatibility to lower API levels in Android

In this point, it's worth noting that the build process is more complex compared to the build process in 2014.

D8 👏🏼

A few months after the desugar was integrated into the build process, Google introduced the next generation of dex compiler, known as D8. D8 replaced the dex compiler and offers faster build times and smaller .dex files.

Google published some benchmark tests regarding the build time and .dex file size between the dex compiler and D8. I have attached some images showing the differences.

Both images have been taken from official android developers blog

As we can see, with D8, we experience faster build times and a slight improvement in the size of the generated .dex files.

Eventually, Google decided to integrate the desugaring process into the D8 compiler, simplifying the entire build process by eliminating a step from the pipeline. This ensures even faster build times and more optimized code. However, since desugaring occurs in the final step after modifying the Java bytecode, we need to ensure that the code is compatible with Java 8 format.

I have attached a flow diagram of the build toolchain for better understanding.

R8 🔥

In late 2018 Google announced the R8 code shrinker. The R8 occurs in the last step incorporating a replacement for Proguard along with all the functionalities offered by D8. R8 implements a code shrinking by removing all the unused code and resources from the project faster and more optimized than Proguard did.

So now, R8 handles all the code shrinking, desugaring, and dexing in one step, ensuring faster shrinking and dexing build times, as well as optimized output size. Additionally, we have one less step in the build pipeline.

I have attached some comparison data from official Android developers blog

As you can see we have a clear win in terms of build times, and a slight improvement in the size of dex files, leading to a reduction in the size of the apk file.

Ok I think we have a big win here! Right?? 👏🏼 🥳

Last but not least I have attached the simplified build pipeline with R8 to provide a clear picture

Conclusion 🎬

In this article, we discussed the evolution of the Android compilation process over the years. We covered all the theoretical background to prepare for part 2. In part 2, we will delve deeper into the R8 compiler, a tool utilized by every modern Android app. Additionally, we will explore Proguard rules and learn how to effectively use them to optimize our builds.

Send your feedback 💌

Feel free to share your feedback with us and let us know if this article has been helpful to you. If you find this article useful, please press the like button or leave a comment.

Thank you for taking the time to read. Happy coding! 🙏🏼