Booster

Static Analysis

What Can Static Analysis Do?

After a lot of practice, we have found that many problems can actually be detected before the product is released or launched. However, due to the lack of corresponding tools, many problems are hidden and brought to production until they encountered by customers. such as UI jank, crash, security issues, etc. Through static analysis, we can detect these potential issues and risks as early as possible and fix them before release, this is the original intention of the Booster project.

Why not SonarQube?

SonarQube is a famous source code oriented analysis tools, it doesn't support analysis byte code generated by compilers, due the limitation of source code based analysis, Booster was born for bytecode oriented static analysing.

What Problems does Booster's Static Analysis Solve?

booster-task-analyseropen in new window analyse apps by generating the cal graph, and then generate a analysis report, it provides more insights to help developers improve the app quality continously, including but not limited to:

  1. Detecting potential performance issues, such as: API calls that may block or slow down the main/UI thread.
  2. Detecting risky API calls
  3. Analysing class dependencies

What's the Solution?

Standalone Task

Booster's static analysis is based on an independent task but not a transformer, the reason for this design is mainly based on the following considerations:

  1. The frequency of static analsysis of the application might not as frequent as build, so a task is more suitable than a transformer.
  2. The CHA (Class Hierarchy Analysis) need to get all class information, and the transformer is based on pipeline processing, which is not suitable for static analysis.
  3. The static analysis might take a long time, as a transformer, it will slow down the build process, and the app build doesn't depend on the analysis result.

The following figure shown the dependency of Analyser Task:

analyseranalyseanalyseanalyseDebuganalyseDebuganalyse->analyseDebuganalyseReleaseanalyseReleaseanalyse->analyseReleasetransformClassesWithXxxForDebugtransformClassesWithXxxForDebuganalyseDebug->transformClassesWithXxxForDebugtransformClassesWithXxxForReleasetransformClassesWithXxxForReleaseanalyseRelease->transformClassesWithXxxForRelease

Class Hierarchy Analysis

The CHA (Class Hierarchy Analysis) is very important for static analysis, it determines the accuracy of the analysis results.

In transform, CHA is implemented by using ClassLoader, which is easy and simple, about more details, please see KlassPoolopen in new window & Klassopen in new window, it is mainly to solve the problem of how to judge whether the two type have an inheritance relationship.

However, the booster-task-analyseropen in new window is implemented by loading all classes with ClassFileParseropen in new window.

Why there are two implementations of CHA? the main reasons are as follows:

  1. When ClassLoader load a class, although it is not necessary to initialize the class, the ClassLoader will verify the bytecode and may throw a VerifyError which will cause the entire analysis process to fail.
  2. Overhead performance —— The performance of loading class by ClassLoader is far from ASM.
  3. In addition to analyzing the inheritance relationship of the classes, it's also necessary to analyze the fields, methodand annotations, and obtaining information from class by reflection has some limitation, such as runtime-invisible annotations, it's impossible to reflect they from Class.
  4. Compared with transformer, task can be executed independently, if all classes are loaded in the process of Transform, it may cause memory constraints and even OOM.

The Entry Points for Analysing

To analyse an application, the static analyser requires an entry point, for general programs, the main method is the entry point, for Android applications, the Application, Activity, Service, Broadcast and Provider are the entry points, so, the first thing is locating these entry points.

Android Components

For Application, Activity, Service, Broadcast and Provider, they all registered in the AndroidManifest.xml, it's easy to get the final AndroidManifest.xml through the mergedManifestsopen in new window.

Custom Views

The most straightforward way to list up all custom views is to parse the layout xmls, which can be obtained through the mergeResopen in new window, it's just an APC format (AAPT2 Container Format)open in new window file produced by the AAPT2open in new window command-line tool, that's why booster-aapt2open in new window comes out.

According to the benchmark test, we have found that parsing AAPT2 container file is not as efficient as parsing the XML source file, so, the final implementation is a combination of two solutions -- parsing the AAPT container file header by AAPT2 container parser first, and then get the source XML from the file header.

Methods and Classes Annotated with Thread Annotations

The Android SDK already provided the Thread Annotationsopen in new window to help the compiler and lint to improve the inspection accuracy. so, if a class or method annotated with the Thread Annotationsopen in new window, then it can be considered as an entry point.

Considering that some third-party frameworks also have their own thread annotations, such as Event Busopen in new window , so the booster-task-analyseropen in new window supports it by treating the method annotated with @Subscribe(threadMode = MAIN) as an entry point.

Getting Started

Putting the booster-task-analyseropen in new window into the classpath of Gradle build script:

buildscript {
    ext.booster_version = "4.16.3"

    dependencies {
        classpath "com.didiglobal.booster:booster-gradle-plugin:$booster_version"
        classpath "com.didiglobal.booster:booster-task-analyser:$booster_version"
    }
}
1
2
3
4
5
6
7
8
buildscript {
    val booster_version = "4.16.3"

    dependencies {
        classpath("com.didiglobal.booster:booster-gradle-plugin:$booster_version")
        classpath("com.didiglobal.booster:booster-task-analyser:$booster_version")
    }
}
1
2
3
4
5
6
7
8

Then, execute the analyse task by command line:

$ ./gradlew analyse
1

After the execution is successful, a dot format report will be generated in the build/reports/ directory, the .dot files can be converted to .png format by using the dot command line tool provided by Graphvizopen in new window:

$ find build/reports -name '*.dot' | xargs -t -I{} dot -O -Tpng {}
1

White/Black Lists

The white list is used for API ignoring, and the black list is used for API highlighting, Booster already provided a default whiltelist.txtopen in new window and blacklist.txtopen in new window, these are came from the practical experience, it also supports customization.

Specify white/black lists by gradle.properties

booster.task.analyser.whitelist=file:///Users/booster/whitelist.txt
booster.task.analyser.blacklist=file:///Users/booster/blacklist.txt
1
2

Specify white/black lists by command line

$ ./gradlew assembleDebug \
    -Pbooster.task.analyser.whitelist=file:///Users/booster/whitelist.txt \
    -Pbooster.task.analyser.blacklist=file:///Users/booster/blacklist.txt
1
2
3

The white/black list also can be a remote URL, for example:

./gradlew assembleDebug \
    -Pbooster.task.analyser.whitelist=https://booster.johnsonlee.io/analyser/whitelist.txt
1
2