Skip to content

ATEPC训练指标显示不全 #411

Description

@Kaia010

代码:
from pyabsa import DatasetItem, ModelSaveOption, DeviceTypeOption
import warnings

忽略警告信息

warnings.filterwarnings("ignore")

from pyabsa.framework.trainer_class.trainer_template import Trainer
from pyabsa.tasks.AspectTermExtraction.configuration.atepc_configuration import ATEPCConfigManager
from autocuda import auto_cuda

config = ATEPCConfigManager.get_atepc_config_english()
dataset = '137.all'

dataset = '100.CustomDataset'

添加sklearn的评估指标

from sklearn.metrics import precision_score, recall_score, f1_score, accuracy_score
import numpy as np

config.show_metric=True, #启用这个可以展示更多指标?
config.output_dim=3, #据说这个是方面分类的个数?
config.cross_validate_fold = 5 # 5折交叉验证
config.max_seq_length=512
config.batch_size = 8
config.learning_rate = 1e-5 # 降低学习率(当前为2e-5)
config.l2reg = 1e-4 # 增强L2正则化(当前为1e-5)
config.early_stop_by = "ate_f1" # 默认可能监控APC_ACC,改为监控ATE_F1
config.max_grad_norm = 1.0 # 防止梯度爆炸
config.use_crf = True # 改善序列标注的连贯性
config.patience = 2
config.log_step = -1
config.seed = [1]
config.verbose = False # If verbose == True, PyABSA will output the model strcture and seversal processed data examples
config.notice = (
"This is an training example for aspect term extraction" # for memos usage
)
config.checkpoint_save_mode = ModelSaveOption.SAVE_FULL_MODEL # 保存完整模型

trainer = ATEPC.ATEPCTrainer(
config=config,
dataset=dataset,
auto_device=DeviceTypeOption.AUTO, # use cuda if available
checkpoint_save_mode=ModelSaveOption.SAVE_FULL_MODEL, # save state dict only instead of the whole model
load_aug=False, # there are some augmentation dataset for integrated datasets, you use them by setting load_aug=True to improve performance
)

问题:

  1. 指标显示不全,如Epoch: 0| loss_apc:1.3189 | loss_ate:0.0577 |: 100%|█| 613/613 [06:13<00:00, 1.64it/s, APC_ACC: 84.49(max:84.49) | A
    Epoch: 1 | Smooth Loss: 0.7048: 100%|██████████████████████████████████████████████▉| 612/613 [05:39<00:00, 1.78it/s]
    根据教程,每个Epoch都应该显示Epoch: 0| loss_apc:0.0099 | loss_ate:0.0139 |: 100%|██████████| 226/226 [01:02<00:00, 3.59it/s, APC_ACC: 85.96(max:87.21) | APC_F1: 78.04(max:81.65) | ATE_F1: 82.87(max:84.58)]
    但是这里APC_F1和ATE_F1都没有显示出来,如图:
Image
  1. 为什么模型只能输出ATE Classification Report,而没有输出APC Classification Report的表格呢?应该怎样设置?我想知道APC任务的各项指标。如图:
Image
  1. 我曾设置了多个随机种子进行训练,如[1,2,3]但每个随机种子训练后似乎是独立输出指标,也就是说,不管连续训练几次,Std和IQR的值都是0,和教程中的情况不符,请问应该怎样设置,才能使这两个值正常计算!

您的解答将对我非常有帮助,提前感谢!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions