Notes on a Survey of Automated Data Augmentation Methods

Today I organized a few automated data augmentation papers I have read recently, writing down the ideas and experimental results.

AutoAugment: Searching for Augmentation Policies with Reinforcement Learning

Paper: AutoAugment: Learning Augmentation Policies from Data

The core idea of this work is to use an RNN as a policy controller, paired with reinforcement learning, to optimize the sampling probabilities for data augmentation. Concretely, the workflow is: the controller samples a set of data augmentation policies, trains a child model with this set of policies, and then feeds the accuracy the child model achieves on the validation set back to the controller as a reward, iterating the search continuously. You could call it a textbook case of the “throw a big RNN at it and wonders happen” approach.

As for the experimental results, the numbers reported on ImageNet are top-5 accuracy, while the other datasets use top-1, so be careful to distinguish them when comparing.

RandAugment: Drastically Shrinking the Search Space

Paper: RandAugment: Practical automated data augmentation with a reduced search space

AutoAugment’s search cost is too high. This work takes a more direct approach: instead of searching for the specific probability of each transformation, it uses just two global hyperparameters—the number N of augmentation operations and the transformation magnitude M—and then randomly picks N transformations from the candidate set and applies them in sequence. The search space immediately shrinks by orders of magnitude.

transforms = [
    'Identity', 'AutoContrast', 'Equalize',
    'Rotate', 'Solarize', 'Color', 'Posterize',
    'Contrast', 'Brightness', 'Sharpness',
    'ShearX', 'ShearY', 'TranslateX', 'TranslateY']


def randaugment(N, M):
    """Generate a set of distortions.

    Args:
      N: Number of augmentation transformations to
          apply sequentially.
      M: Magnitude for all the transformations.
    """

    sampled_ops = np.random.choice(transforms, N)
    return [(op, M) for op in sampled_ops]

Judging from the experimental results, Random Augmentation can ultimately converge to very good accuracy too, with a rather small gap from AutoAugment, but at a far lower computational cost.

Fast AutoAugment: Speeding Up the Search by Merging Policies

Paper: Fast AutoAugment

The idea here is: first search out N groups of data augmentation sub-policies that each perform well, then directly merge them into one large policy set for training. Compared with AutoAugment’s end-to-end reinforcement learning search, search efficiency improves noticeably.

Algorithm 1: Fast AutoAugment
Input: (θ, D_train, K, T, B, N)

1: Split D_train into K-fold data D_train^(k) = {(D_M^(k), D_A^(k))}   // stratified shuffling
2: for k ∈ {1, ..., K} do
3:     T*^(k) ← ∅,  (D_M, D_A) ← (D_M^(k), D_A^(k))                    // initialize
4:     Train θ on D_M
5:     for t ∈ {0, ..., T-1} do
6:         B ← BayesOptim(T, L(θ | T(D_A)), B)                        // explore-and-exploit
7:         T_t ← Select top-N policies in B
8:         T*^(k) ← T*^(k) ∪ T_t                                      // merge augmentation policies
9: return T* = ⋃_k T*^(k)

Summary

The thread running through the three works is fairly clear: AutoAugment proved the feasibility of automatically searching for augmentation policies, but at an extremely high search cost; Fast AutoAugment improved on search efficiency, reducing overhead by merging multiple groups of candidate policies; RandAugment took another route entirely, compressing the search space down to the bare minimum and striking a good balance between practicality and final accuracy through random sampling plus two hyperparameters. Today I also incidentally organized the experimental results for model compression, and tried modifying ResNet-18 to adapt it to the Apollon dataset—I’ll record the detailed results later.