Bytecode Processing Pipeline

Transform API

Starting from Android Gradle Plugin 1.5.0-beta1, to simplify the injection of custom classes, Android provided the Transform APIopen in new window, which allows third-party plugins to modify class files before they are converted to dex. Before this, achieving the same operation could only be done through Hook Task. Since Booster only supports Android Gradle Plugin 3.0.0 and above, it chose to use the Transform APIopen in new window approach, processing class bytecode by registering BoosterTransformopen in new window with the Android Extension.

Transform Pipeline

In the Android Gradle Plugin, Transforms registered through the Transform APIopen in new window are ultimately executed in the form of a Transform Pipeline. The Transform Pipeline consists of a series of TransformStreams and Transforms, and Transforms are mounted to the Gradle task system through TransformTask for execution. Taking Android Gradle Plugin 3.0.0+ as an example, the position of the Transform Pipeline in the overall build process is shown in the following diagram:

Transform Stream

Each Transform Stream has specific types (Content Types) and scopes (Scopes).

Content Types

Content Type	Description
QualifiedContent.DefaultContentType.CLASSES	class or JAR files
QualifiedContent.DefaultContentType.RESOURCES	Java resources
ExtendedContentType.DEX	dex files
ExtendedContentType.NATIVE_LIBS	so native libraries
ExtendedContentType.CLASSES_ENHANCED	class updated by Instant Run
ExtendedContentType.DATA_BINDING	data binding artifacts
ExtendedContentType.DEX_ARCHIVE	dex archive files, each class corresponds to a separate dex file

Since Booster only focuses on bytecode processing, the input content type received by BoosterTransformopen in new window is QualifiedContent.DefaultContentType.CLASSES.

Scopes

Scope	Description
QualifiedContent.Scope.PROJECT	Applies only to the current project
QualifiedContent.Scope.SUB_PROJECTS	Applies to all subprojects
QualifiedContent.Scope.EXTERNAL_LIBRARIES	Applies to external dependency libraries
QualifiedContent.Scope.TESTED_CODE	Applies to test code of the current variant
QualifiedContent.Scope.PROVIDED_ONLY	Applies only to provided dependencies
QualifiedContent.Scope.PROJECT_LOCAL_DEPS	Applies to local dependency packages of the project
QualifiedContent.Scope.SUB_PROJECTS_LOCAL_DEPS	Applies to local dependency packages of all subprojects
InternalScope.MAIN_SPLIT	Applies to main package in Instant Run mode
InternalScope.LOCAL_DEPS	Applies to local dependency libraries
InternalScope.FEATURES	Applies to Dynamic Feature projects

TransformManager defines some commonly used Scopes:

Scope	Description
TransformManager.SCOPE_FULL_PROJECT	Applies to the entire project, including current project, all subprojects, and external dependencies
TransformManager.PROJECT_ONLY	Applies only to the current project, excluding subprojects or external dependencies
TransformManager.SCOPE_FULL_WITH_FEATURES	Applies to the entire project and Dynamic Feature projects
TransformManager.SCOPE_FULL_LIBRARY_WITH_LOCAL_JARS	Applies to the current project and local dependency libraries

For APP projects, the scope of BoosterTransformopen in new window is TransformManager.SCOPE_FULL_PROJECT, which not only processes the bytecode of the current project itself, but also the local libraries or external libraries it depends on; for Library projects, the scope is TransformManager.PROJECT_ONLY ; for dynamic-feature projects, it is TransformManager.SCOPE_FULL_WITH_FEATURES

Original Stream & Intermediate Stream

Based on the origin of the Transform Stream, it is divided into Original Stream and Intermediate Stream. As the name suggests, the Original Stream is created by the Android Gradle Plugin for other Transforms to consume its contents, while the Transform Stream produced after a custom Transform consumes is an Intermediate Stream (non-original).

When we use a custom Transform to process build intermediates, we can focus on two dimensions:

1. Content Types: The types of content that the Transform will process, which can be multiple types;
2. Scopes: Where the content that the Transform will process comes from, which can be multiple scopes;

When a custom Transform is registered through the Transform API, the Transform Manager will select streams from the existing Transform Stream list based on the Content Types and Scopes that the Transform focuses on as input for that Transform. The output of that Transform (Intermediate Stream) serves as the input for the next Transform. Each Transform is connected through the Transform Stream, forming a pipeline. BoosterTransformopen in new window is a node in the Transform Pipeline, as shown in the following diagram:

Booster Transformer Pipeline

To simplify bytecode operations and improve module reusability, Booster internally designed a Transformer Pipeline framework similar to the Android Transform Pipeline to implement streaming bytecode operations. Transformer serves as the bytecode processing unit. Developers can choose JavassistTransformer, AsmTransformer, or create a custom Transformer based on other bytecode manipulation frameworks according to their preference. Inside JavassistTransformer and AsmTransformer, there is another layer of Class Transformer Pipeline, where each ClassTransformer serves as the class processing unit. The structure of the Transformer Pipeline is shown in the following diagram:

Parallel Transformer Pipeline

To maximize build speed, Booster creates an independent Transformer Pipeline for each input (JarInput or DirectoryInput), and all Transformer Pipelines share the same Transformer.

Adjusting Transformer Order in the Pipeline

By default, Booster does not guarantee the execution order of Transformers. The benefit of this is to reduce dependencies between Transformers and lower the complexity of the Transform Pipeline. However, there are always some special cases. To allow adjustment of Transformer order, Booster provides a way to specify priority for Transformers by annotating them with @Priorityopen in new window (lower values indicate higher priority). For example:

@Priority(Int.MAX_VALUE)
@AutoService(ClassTransformer::class)
class HighestPriorityTransformer : ClassTransformer {

    override fun transform(context: TransformContext, klass: ClassNode): ClassNode {
        // do something...
        return super.transform(context, klass)
    }

}

@Priority also applies to adjusting the execution order of VariantProcessor

God's Eye View

At this point, we have understood the internal implementation details of the Transform Pipeline. How can we customize a Transform to only observe the contents of the current Transform Pipeline without modifying them? This question is left for the readers of this book.