Deep Dive to Android R8 & Proguard [Part 2]

Introduction 📖

In this second part we will further analyze the R8 compiler and the rules we can apply, in order to feet to our project.

R8 vs Proguard 🥊

As we discussed in the first part R8 is a replacement tool for Proguard. But what are actual the pros and the cons of each one? Since Android Gradle Plugin 3.4.0, Android has used by default the R8 compiler instead of Proguard

• R8 offers better code optimization for Kotlin, while Proguard is more effective for Java.

• R8 builds faster and more efficiently, resulting in smaller APK files and slightly improved runtime speed.

• R8 provides slightly better code optimization, reducing the APK file size by 10%, while Proguard reduces the APK file size by 8.5%.

• R8 is designed to use the same rules from Proguard, so it's very easy to migrate your project from Proguard to R8

• R8 is now part of Android build gradle, so it's easy to start using it in your project

👉🏼 R8 steps 👈🏼

R8 takes your Java bytecode and applies specific optimizations to your code. Let's analyze these steps exactly:

1. Code Shrinking(or tree-shaking): Detects and safely removes all the unused code in your project (classes, fields, methods). Additionally removes unused code from libraries inside your project, this is very helpful when dealing with 64k reference limit.

2. Resource Shrinking: Removes all the unused resources (strings, drawables etc) from project and from library dependencies.

3. Code Optimization: This step is intended to reduce further the apk size and to improve the runtime performance. It extends the code shrinking step and goes deeper to remove more unused code or to rewrite code. The basic rules for optimazation in this step are:

   1. If find an else {} branch that is unreachable, R8 possibly removes the code inside it

   2. If a specific function is called only from a few places in the code, R8 delete this function and inline it to these places

   3. If it finds an class that is not initialised in the code (abstract class) and it includes a subclass then it merge the two classes.

4. Code Obfuscation: R8 renames classes, fields and methods in order to protect the apk from reverse engineering. When you build the project, R8 produce a mappings.txt file. In this file, you can find all the mappings that were made in the project. With this file you can obtain the actual names before obfuscation. This is pretty useful for debugging.

5. Desugaring: This allows us to support newer Java API features without worrying about backward compatibility with older Android APIs.

6. Dexing: This step converts Java bytecode to dex code. Initially, this step was included in the D8 compiler, but it has now been integrated into the R8 compiler.

Enable R8 in project

We wrote before that since Android Gradle Plugin 3.4.0, R8 is the default tool for minify and obfuscate our build code. Instead if we have version prior to 3.4.0 we have to add android.enableR8 = true in the gradle.properties file because proguard is the default option. Let's see how we can enable R8 for release builds

android {
    buildTypes {
        getByName("release") {
            isMinifyEnabled = true
            isShrinkResources = true

            proguardFiles(
                getDefaultProguardFile("proguard-android-optimize.txt"),
                "proguard-rules.pro"
            )
        }
    }
}

This code snippet is inside the build.gradle file of your app module. To start the code shrinking we have to change the "isMinifyEnabled" to true. Additionally if we want to apply resource shrinking we have to set the "isShrinkResources" to true.

For code obfuscating we have the next three lines with proguardFiles

proguardFiles(
    getDefaultProguardFile("proguard-android-optimize.txt"),
    "proguard-rules.pro"
)

We see proguard here because as we said earlier R8 uses the same rules with proguard to obfuscate our code. We see here two different files. The first one is proguard-android-optimize.txt. This file is generated automatically by the Android Gradle Plugin at compile time. It's a default file with a set of basic proguard rules that applied to every Android project.

Most of the time, this file is not sufficient to adequately obfuscate our project. This is where the second file, proguard-rules.pro, comes into play. This file allows us to write custom proguard rules that depend to our specific needs. In this file, we will add all the ProGuard rules necessary to effectively obfuscate the project.

An alternative way to preserve a class or method is by utilizing the @Keep annotation. This annotation is part of the androidx.annotation library and can be applied in your code to prevent removal during the code shrinking process. It is commonly applied to methods and classes that are accessed only via reflection, which might lead the compiler to consider the code as unused.

@Keep
public class TestKeep {

   @Keep
   public void keepMethod() { /*..*/ }
}

Proguard Rules 👨🏼‍🏫

In this section we will discuss about the basic Proguard rules you will have to apply for your project. Usually a proguard rule consists of the rule and the class specification, so let's dive into these concepts.

Keep Rules

In this section we will discuss about the Keep rules. First things first the Keep rule instructs R8 to exclude matching classes and class members (fields + methods) from shrinking, optimization, and renaming. There are various keep options available, which can be applied based on our requirements. Now, let's explore some fundamental keep rules:

• -keep: Exclude matching classes and matching members if specified from shrinking, optimization, and renaming.

• -keepclassmembers: Exclude matching members from matching classes from shrinking, optimization, and renaming.

• -keepclasseswithmembers: Excludes matching classes and matching members from shrinking, optimization, and renaming.

• -keepnames (-keep + allowshrinking): Prevent matching classes and members if specified from being renamed, if they survived first from the shrinking phase

• -keepclassmembernames(-keepclassmembers + allowshrinking): Prevent matching members from being renamed, if they survived first from the shrinking phase

• -keepclasseswithmembernames(-keepclasseswithmembers + allowshrinking): Prevent matching classes and matching members from being renamed, if they survived first from the shrinking phase

-keepclassmembers class com.example.model.** { *; }

General Rules

For the General rules we have commands like these:

• -dontobfuscate: Instruct R8 to not apply renaming

• -dontoptimize: Instruct R8 to not optimize the code

• -dontshrink: Instruct R8 to not shrink the code

• -keepattributes: A useful command for keep some Java attributes. For example it's usual to define this command for debugging informations:

    -keepattributes SourceFile, LineNumberTable
    Instruct the R8 compiler to keep the sourceFiles and the line number tables of all methods.
  

  After R8 version 8.2 the compiler will retain full original source file names in the mapping information without the need to specify -keepattributes SourceFile.

  Also if we target api 26 or above we don't have to specify -keepattributes LineNumberTable as it's enabled by default. An example of source file we can find inside a mapping.txt file is something like this:

    com.example.TestFragment -> com.example.a: # R8 rename TestFragment.kt 
    # {"id":"sourceFile","fileName":"TestFragment.kt"}
  
    com.example.TestFragment$subscribeToLiveData$9 -> com.example.TestFragment.a$n: # R8 rename a liveData inside TestFragment.kt 
    # {"id":"sourceFile","fileName":"TestFragment.kt"}
  

• -printconfiguration [<file>]: Another useful command that prints in a txt file all the rules that have applied to every file. You can set the path of the file like this:

    -printconfiguration "build/outputs/mapping/configuration.txt"
  

• -printseeds [<filename>]: Outputs all the classes, methods, fields that have been affected by the rules we defined

• -printusage [<filename>]: Outputs a list of classes, methods, fields that have been removed after shrinking code

Class Specification

Several rules can take a class specification to instruct R8 on which classes, methods, or fields these rules should apply. We can use Wildcards and special characters to create broader rules.

Wildcards and Special Characters ✶

The most commonly used ones are:

• ?: This character matches a single character. For example, with the rule -keep class T????Fragment, R8 will include Test1Fragment, as well as Test2Fragment, Test3Fragment, and so on.

• *: This character matches any sequence of zero or more characters in file name, excluding the package seperator (.) For example with the rule -keep class com.example.*DataSource R8 will include these files com.example.RemoteDataSource, com.example.LocalDataSource, but it will exclude this one: com.example.parent.GenericDataSource

• **: This character takes again any sequence of zero or more characters in file name, but this time including the package seperator (.) , so in the above example it should include also the com.example.parent.GenericDataSource

Additionally with this rule -keep class com.example.RemoteDataSource { *; }, we instruct R8 to preserve all the members of this specific class

We can use wildcards to specify a particular field like this: -keepclassmembernames class * { long *UUID; } . This rule instructs R8 to not rename every long value field ending with UUID from all classes in the project.

R8 provide us the ability to keep all the fields, methods or init constructor in a specific class like this:

-keepclassmembers class com.example.model.** { # specify all classes inside com.example.model package
   <fields>; # keep every field in the class
   <methods>; # keep every method in the class
   public <init>(); # keep init constructor
}

Instead of class value we can use in the same way these values:

• Interface

• Enum

• @SomeTestAnnotation

-keep enum com.bumptech.glide.load.resource.bitmap.ImageHeaderParser$** {
}

Modifiers

In the Class specification R8 allows us to use modifiers in order to narrow down wildcards.

Let's provide an example here for better understanding:

-keep public class * { # keep all public classes
    public final *; # All public final fields in those classes
    private synchronized *(...); # All private synchronized methods in those classes
    public protected abstract *(...); # All public or protected abstract methods in those classes
}

Finally we have other two powerful functionalities that can be used with class specifications: subtype matching and annotated matching.

• For the subtype matching we can use these formats: extends <class-name> or implements <interface-name>. For example we can define this rule: -keep public class * extends java.lang.Exception.This rule keeps all the classes that extends the java Exception class. And this rule is from Glide: -keep public class * implements com.bumptech.glide.module.GlideModule. With this rule we can keep all the classes that implements the GlideModule interface. Another common rule we use in Android is about parcelable objects: -keepnames class * implements android.os.Parcelable { public static final * CREATOR; }.

• For the annotated matching we can use a rule like this: -keep @com.example.TestAnnotation interface *. This rule keeps all the interfaces that are annotated with @TestAnnotation

Aggressive Shrinking Option

R8 compiler has the ability for a more aggressive optimization proccess. If you have Android Gradle Plugin 8.0 or above this mode is enabled by default. If you have older AGP you can enable this mode with this declaration in gradle.properties file: android.enableR8.fullMode=true

Keep in mind that because of the more aggressive optimization maybe you have to include more rules to properly build your code.

Some additional rules of this non-compat mode optimization includes:

• The default constructor init() is not kept by default when you keep a class

• Default methods are not implicitly kept as abstract methods.

• Attributes (likeSignature, Annotations, etc.) are only retained for matching classes, even if we specify the generic keepattributes for all entities

Debugging R8

When the production code is obfuscated and some crash occurred, you may notice in firebase crashlytics that the stack trace will be something like that:

Caused by androidx.fragment.app.a0: Unable to instantiate fragment yb.d: could not find Fragment constructor
       at androidx.fragment.app.Fragment.instantiate(Fragment.java:687)
       at androidx.fragment.app.FragmentContainer.instantiate(FragmentContainer.java:57)
       at androidx.fragment.app.FragmentManager$3.instantiate(FragmentManager.java:525)

We can see that the stack trace is point to obfuscated code (yb.d). So we have to find a solution on how to read this trace in order to detect the exact point in our code that the crash occurred.

In order to achieve that Android has created a command line tool called Retrace. Retrace is usually present on the following path: Android/sdk/tools/proguard/bin. You may optionally use a GUI-based plugin by Jetbrains, as shown below:

You can de-obfuscate the trace using Retrace tool with this command:

retrace path-to-mapping-file [path-to-stack-trace-file] [options]

So, you need to provide Retrace with the mapping.txt file, which is automatically generated by R8 during the build, and a text file with the obfuscated stack trace. The mapping.txt file can be located in this path in the project: app/build/outputs/mapping/{flavorName}]/{buildType}/mapping.txt. You can generate a mapping file to another path if you add in proguard-rules.pro the rule -printmapping mapping.txt. As for the stack trace file, you can download it from Firebase Crashlytics.

The result from Retrace will be the stack trace de-obfuscated. According to the keep rules you have added to the project you can extract more detailed stack trace. For example if you have added this rule: -keepattributes LineNumberTable, you can extract the trace with the corresponding line numbers inside

Conclusion 🎬

In this article, we have provided a detailed analysis of how R8 works. We have covered almost everything you need to know to start writing your own custom rules in your projects. Of course, there are many other options and commands to build your own rules. I suggest reading this R8 documentation if you want to explore further.

Send your feedback 💌

Feel free to share your feedback with us and let us know if this article has been helpful to you. If you find this article useful, please press the like button or leave a comment.

Thank you for taking the time to read. Happy coding