Multi-threading Optimization

The Challenges of Thread Management

For developers, thread management has always been one of the most troublesome problems, especially for apps with complex business. Each business module has dozens or even hundreds of threads. Moreover, from the business perspective, they always want that the priority of the business thread is the highest, so that more CPU time slices can absorb during the process scheduling. However, the more competition, the more resources are wasted on thread scheduling.

How to solve the thread management problem mentioned above effectively? Most people may think of "using a unified thread management library." Of course, this is the most ideal solution, but the reality is not always satisfactory. With the rapid iteration of the business, the accumulated technical debt is also increasing. In the face of intricate business logic and historical issues, how can architects deal with it calmly?

According the thread monitoring logs on production, some symptoms have been found:

In some specific scenarios, new threads will be created repeatly, which will eventually lead to OOM
Thread spike (hundreds or even thousands) might be happen in some cases
Even a thread is idle, it has always been WAITING in the pool.

These symptoms will eventually lead to the following problems:

Out Of Memory
Unable to distinguish the module to which the thread belongs, so it's difficult to identify the root cause.

General Solutions

The general solutions for thread management are:

Using a unified thread management library, the advantages and disadvantages of this solution are obvious. The advantage is that it is highly operable and easy to implement. The disadvantage is that the optimization is not complete enough and it is impossible to manage the threads which come from third-party libraries;
Some teams even prohibit the use of new Thread() in the code, the Code Review will be rejected, but HandlerThread is also Thread, which will inevitably be used in Android;

Non-Intrusive Solution

booster-transform-threadopen in new window resolved these problems mentioned above by using a new idea, replacing all the instructions for creating new threads with customized method calls in compile time. After a lot of analysis and verification, we have found that Android creates threads mainly in the following ways:

Thread and its sub classes
TheadPoolExecutor and its sub classes
The classes derived from Executors
The classes derived from ThreadFactory
AsyncTask
Timer and its sub classes

The Ideas of Optimization

Reduce The Number of Threads

Reducing the number of threads doesn't mean that don't use threads or use threads as few as possible, but destroying threads automatically when they are idle. Based on this theory, it's possible to reduce the numer of threads by replacing the method of creating a thread pool, or modifying the parameters of the thread pool directly:

Set the corePoolSize to 0
```
new ThreadPoolExecutor(
    /* corePoolSize*/ 0,
    maxCorePoolSize,
    keepAliveTime,
    timeUnit,
    workQueue,
    threadFactory
);
```
1
2
3
4
5
6
7
8
WARNING
Setting the minPoolSize of ScheduledThreadPoolExecutor to 0 below JDK 9 will cause high CPU loading problem, more details, please see: JDK-8022642open in new window, JDK-8129861open in new window, these two bugs have already been fixed in JDK 9.
Limit the maxPoolSize
Usually, the maxPoolSize is 2 * NCPU or 2 * NCPU + 1, if multiple modules use thread pool in this way, there will be a thread spike in some cases, especially for CachedThreadPoolExecutor, so, we can prevent the thread spike by limiting the maxPoolSize of every single thread pool.
```
new ThreadPoolExecutor(
   /* corePoolSize */ 0,
   /* maxCorePoolSize */ Math.min(maxCorePoolSize, NCPU),
   keepAliveTime,
   timeUnit,
   workQueue,
   threadFactory
);
```
1
2
3
4
5
6
7
8
Allow core threads to be destroyed automaticallly when they are idle
```
executor.allowCoreThreadTimeOut(true)
```
1
Replace HandlerThread with SingleThreadPoolExecutor as much as possible
Why is it not recommended to use HandlerThread? It's undeniable that the HandlerThread simplified the development of multi-threading on Android, the actual situation is that very few developers will destroy HandlerThread actively, if a HandlerThread is used without destroying it, the HandlerThread will always hold the memory, and the stack size of each thread is at least 1040k, if a large number of HandlerThread stays in the WATTING state for a long time, it is an unnecessary waste and will cause Out Of Memory probably, this issue can be prevented by using SingleThreadPoolExecutor and allowing the core threads to be destroyed when they are idle.

More Meaningful Information

Renaming threads or prefixing the thread name with the class name of the caller will be helpful for troubleshooting, especially integrated with the APM system.

Thread Renaming

Taking the Thread class as an example, the thread renaming can be done by replacing the original constructor with the customized method which defined in the ShadowThreadopen in new window.

Thread	ShadowThreadopen in new window
`Thread()`	`newThread(String)`
`Thread(Runnable)`	`newThread(Runnable, String)`
`Thread(ThreadGroup, Runnable)`	`newThread(ThreadGroup, Runnable, String)`
`Thread(String)`	`newThread(String, String)`
`Thread(ThreadGroup, String)`	`newThread(ThreadGroup, String, String)`
`Thread(Runnable, String)`	`newThread(Runnable, String, String)`
`Thread(ThreadGroup, Runnable, String)`	`newThread(ThreadGroup, Runnable, String, String)`
`Thread(ThreadGroup, Runnable, String, long)`	`newThread(ThreadGroup, Runnable, String, String)`

You may find that the parameters of these static methods of the ShadowThread class have one more String type parameter that the constructor of Thread. In fact, this String parameter is the className of the class that calls the Thread constructor, and this class name is obtained in transform phase, and it has been passed to the static methods of the ShadowThread by instrumenting additional JVM instructions, so that, it's able to determine where is the thread came from at runtime.

Let me explain to you with an example, assume that we have a class called MainActivity:

public class MainActivity extends AppCompatActivity {

    public void onCreate(Bundle savedInstanceState) {
        new Thread(new Runnable() {
            public void run() {
                doSomething();
            }
        }).start();
    }

}

1
2
3
4
5
6
7
8
9
10
11

Without renaming, the name of threads created by MainActivity has the same pattern Thread-{N}, in order to make the name collected by APM system become com.didiglobal.booster.demo.MainActivity#Thread-{N}, we need to add a prefix to the thread name as an identifier, this prefix is the origin of the last String parameter of the static methods of ShadowThread. The effect of renaming is shown in the figure below:

Thread Renaming

Thread Pool Optimization

If you have already understood the implementation of thread renaming, it's easy to understand the thread pool parameter optimization.

Replacing static method calls of Executors with the corresponding static method calls of ShadowExecutors.
Replacing constructor calls of ThreadPoolExecutor with the corresponding static method calls of ShadowThreadPoolExecutor.
Calling the ShadowAsyncTask.optimizeAsyncTaskExecutor() in the <clinit>() block of Application to modify the parameters of thread pool which used by AsyncTask.

Take the class Executors as an example:

Executors	ShadowExecutorsopen in new window
`newFixedThreadPool(int)`	`newOptimizedFixedThreadPool(int, String)`
`newFixedThreadPool(int, ThreadFactory)`	`newOptimizedFixedThreadPool(int, ThreadFactory, String)`
`newSingleThreadExecutor()`	`newOptimizedSingleThreadExecutor(String)`
`newSingleThreadExecutor(ThreadFactory)`	`newOptimizedSingleThreadExecutor(ThreadFactory, String)`
`newCachedThreadPool()`	`newOptimizedCachedThreadPool(String)`
`newCachedThreadPool(ThreadFactory)`	`newOptimizedCachedThreadPool(ThreadFactory, String)`
`newSingleThreadScheduledExecutor()`	`newOptimizedSingleThreadScheduledExecutor(String)`
`newSingleThreadScheduledExecutor()`	`newOptimizedSingleThreadScheduledExecutor(String)`
`newSingleThreadScheduledExecutor(ThreadFactory)`	`newOptimizedSingleThreadScheduledExecutor(ThreadFactory, String)`
`newScheduledThreadPool(int)`	`newOptimizedScheduledThreadPool(int, String)`
`newScheduledThreadPool(int, ThreadFactory)`	`newOptimizedScheduledThreadPool(int, ThreadFactory, String)`

Disable Thread Pool Optimization

To avoid unexpected side effect, Booster also supports renaming threads without optimization.

Configuring by gradle.properties

booster.transform.thread.optimization.enabled=false

Configuring by command line

./gradlew assembleDebug -Pbooster.transform.thread.optimization.enabled=false

If the thread optimization is disabled, the method calls of Executors will be replaced with the corresponding static methods of ShadowExecutors:

Executors	ShadowExecutorsopen in new window
`newFixedThreadPool(int)`	`newFixedThreadPool(int, String)`
`newFixedThreadPool(int, ThreadFactory)`	`newFixedThreadPool(int, ThreadFactory, String)`
`newSingleThreadExecutor()`	`newSingleThreadExecutor(String)`
`newSingleThreadExecutor(ThreadFactory)`	`newSingleThreadExecutor(ThreadFactory, String)`
`newCachedThreadPool()`	`newCachedThreadPool(String)`
`newCachedThreadPool(ThreadFactory)`	`newCachedThreadPool(ThreadFactory, String)`
`newSingleThreadScheduledExecutor()`	`newSingleThreadScheduledExecutor(String)`
`newSingleThreadScheduledExecutor()`	`newSingleThreadScheduledExecutor(String)`
`newSingleThreadScheduledExecutor(ThreadFactory)`	`newSingleThreadScheduledExecutor(ThreadFactory, String)`
`newScheduledThreadPool(int)`	`newScheduledThreadPool(int, String)`
`newScheduledThreadPool(int, ThreadFactory)`	`newScheduledThreadPool(int, ThreadFactory, String)`

# Multi-threading Optimization

# The Challenges of Thread Management

# General Solutions

# Non-Intrusive Solution

# The Ideas of Optimization

# Reduce The Number of Threads

# More Meaningful Information

# Thread Renaming

# Thread Pool Optimization

# Disable Thread Pool Optimization

# Configuring by gradle.properties

# Configuring by command line

Multi-threading Optimization

The Challenges of Thread Management

General Solutions

Non-Intrusive Solution

The Ideas of Optimization

Reduce The Number of Threads

More Meaningful Information

Thread Renaming

Thread Pool Optimization

Disable Thread Pool Optimization

Configuring by gradle.properties

Configuring by command line