Handling Banned Words

During testing, it was found that “…” frequently appeared in conversations. Analysis showed that there were at least 2K similar responses in the script. Although such responses, combined with CG images, can subtly convey a character’s mindset, AIGC tasks require direct responses. Therefore, these characters need to be removed from the script, along with similar symbols such as ***.

...
...

Use the sed command for text file preprocessing, and place the results in the pt_txt folder for pre-training. In addition, the SFT fine-tuning script needs to be re-executed.

1
2
3
4
5
6
sed '/「………」/d' CLANNAD.txt > CLANNAD_NODOT.txt_1
sed '/「……」/d' CLANNAD_NODOT.txt_1 > CLANNAD_NODOT.txt_2
sed '/「…」/d' CLANNAD_NODOT.txt_2 > CLANNAD_NODOT.txt
sed '/………/d' CLANNAD_NODOT.txt > CLANNAD_NODOT.txt_3
sed '/\*\*\*\*\*/d' CLANNAD_NODOT.txt_3 > CLANNAD_NODOT.txt
sed '/嗯…/d' CLANNAD_NODOT.txt > CLANNAD_NODOT.txt_e

Splitting Fine-Tuning Data

Continuing from last time, because the responses from the trained large model suffered from overfitting.

Randomly split the dataset into training and test sets with a ratio of 7:3. Adjust the original code as follows to cluster and randomly split dialogues between different characters.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
public BuildFinetuneFile map() {
    for (FinetuneWithCharacter finetuneWithCharacter : finetuneWithCharacters) {
        String characterKey = finetuneWithCharacter.getInputCharactor() + "_" + finetuneWithCharacter.getOutputCharactor();
        List<Finetune> finetunes = finetuneMap.getOrDefault(characterKey, new ArrayList<>());
        finetunes.add(finetuneWithCharacter.getFinetune());
        finetuneMap.put(characterKey, finetunes);
    }
    return this;
}

public void split(List<Finetune> dataset, List<Finetune> validator) {
    for (String character : finetuneMap.keySet()) {
        List<Finetune> list = finetuneMap.get(character);
        Collections.shuffle(list);
        int splitPoint = (int) (list.size() * 0.7);
        dataset.addAll(list.subList(0, splitPoint));
        validator.addAll(list.subList(splitPoint, list.size()));
    }
}

The startup script updates are as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
public static void main(String[] args) throws IOException {
    Parser parser = new Parser("CLANNAD.txt");
    List<Finetune> dataset = new ArrayList<>();
    List<Finetune> validator = new ArrayList<>();
    new BuildFinetuneFile(parser.parseLines()).prepare().map().split(dataset, validator);

    BufferedWriter finetuneOut = new BufferedWriter(new FileWriter("finetune_json/CLANNAD_LLaMA_finetune.json"));
    BufferedWriter validatorOut = new BufferedWriter(new FileWriter("validator_json/CLANNAD_LLaMA_validator.json"));

    finetuneOut.write(new Gson().toJson(dataset));
    validatorOut.write(new Gson().toJson(validator));

    finetuneOut.close();
    validatorOut.close();
}

Modifying Pre-training and Fine-Tuning Configurations

PT Pre-training Configuration

1
2
3
4
5
6
7
8
pretrained_model=/mnt/workspace/chinese-alpaca-2-7b
chinese_tokenizer_path=/mnt/workspace/chinese-alpaca-2-7b
dataset_dir=/mnt/workspace/CLANNAD_LLaMA/pt_txt
data_cache=/mnt/workspace/cache
per_device_train_batch_size=1
gradient_accumulation_steps=8
block_size=512
output_dir=/mnt/workspace/CLANNAD_LLaMA_model_pt

Set --num_train_epochs to 30 to allow the model to fully learn.

SFT Fine-Tuning Configuration

First round of SFT

1
2
3
4
5
6
7
8
9
pretrained_model=/mnt/workspace/CLANNAD_LLaMA_model_pt_merged
chinese_tokenizer_path=/mnt/workspace/CLANNAD_LLaMA_model_pt_merged
dataset_dir=/mnt/workspace/CLANNAD_LLaMA/finetune_json
per_device_train_batch_size=1
per_device_eval_batch_size=1
gradient_accumulation_steps=8
max_seq_length=512
output_dir=/mnt/workspace/CLANNAD_LLaMA_model_sft
validation_file=/mnt/workspace/CLANNAD_LLaMA/validator_json/CLANNAD_LLaMA_validator.json

Second round of SFT

1
2
3
4
5
6
7
8
9
pretrained_model=/mnt/workspace/CLANNAD_LLaMA_model_sft_merged
chinese_tokenizer_path=/mnt/workspace/CLANNAD_LLaMA_model_sft_merged
dataset_dir=/mnt/workspace/CLANNAD_LLaMA/finetune_json
per_device_train_batch_size=1
per_device_eval_batch_size=1
gradient_accumulation_steps=8
max_seq_length=512
output_dir=/mnt/workspace/CLANNAD_LLaMA_model_sft_2
validation_file=/mnt/workspace/CLANNAD_LLaMA/validator_json/CLANNAD_LLaMA_validator.json

When executing the script, an OOM error occurred (16G V100), so --load_in_kbits was changed to 4.

In addition, the --save_total_limit parameter was changed to 1 to prevent excessive disk usage from terminating the task.

Increase --num_train_epochs to 10 to train for a few more epochs.

Merging Parameters

Merge pre-training parameters

1
2
3
4
5
cd /mnt/workspace/Chinese-LLaMA-Alpaca-2 && python scripts/merge_llama2_with_chinese_lora_low_mem.py \
    --base_model /mnt/workspace/chinese-alpaca-2-7b \
    --lora_model /mnt/workspace/CLANNAD_LLaMA_model_pt \
    --output_type huggingface \
    --output_dir /mnt/workspace/CLANNAD_LLaMA_model_pt_merged

Merge fine-tuning parameters

1
2
3
4
5
cd /mnt/workspace/Chinese-LLaMA-Alpaca-2 && python scripts/merge_llama2_with_chinese_lora_low_mem.py \
    --base_model /mnt/workspace/CLANNAD_LLaMA_model_pt_merged \
    --lora_model /mnt/workspace/CLANNAD_LLaMA_model_sft/checkpoint-5000/sft_lora_model \
    --output_type huggingface \
    --output_dir /mnt/workspace/CLANNAD_LLaMA_model_sft_merged

Test Results

The character can quote dialogues from other characters.

The character can quote dialogues from other characters
The character can quote dialogues from other characters

Kotomi can finally make retorts, and very professionally at that!

Kotomi can finally make retorts
Kotomi can finally make retorts

It fits the character’s basic attributes, and the forms of address are correct.

Matches character attributes
Matches character attributes

The model has acquired reasoning capabilities and fits well with the character’s personality. (According to the original script, to confirm whether Tomoyo is male or female, Sunohara asks in his usual punch-worthy manner by asking if she has a razor.)

Model dialogue reasoning
Model dialogue reasoning

After being beaten up countless times, Sunohara has finally learned his lesson.

Indeed gets sent flying
Indeed gets sent flying