HuggingFace Trainer() cannot report to wandb

Question

I am trying to set trainer with arguments report_to to wandb, refer to this docs
with config:

training_args = TrainingArguments(
    output_dir="test_trainer",
    evaluation_strategy="steps",
    learning_rate=config.learning_rate,
    num_train_epochs=config.epochs,
    weight_decay=config.weight_decay,
    logging_dir=config.logging_dir,
    report_to="wandb",
    save_total_limit=1,
    per_device_train_batch_size=config.batch_size,
    per_device_eval_batch_size=config.batch_size,
    fp16=True,
    load_best_model_at_end=True,
    seed=42
)

yet when I set trainer with:

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    compute_metrics=compute_metrics
)

it shows:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-68-b009351ab52d> in <module>
      4     train_dataset=train_dataset,
      5     eval_dataset=eval_dataset,
----> 6     compute_metrics=compute_metrics
      7 )

~/.virtualenvs/transformers_lab/lib/python3.7/site-packages/transformers/trainer.py in __init__(self, model, args, data_collator, train_dataset, eval_dataset, tokenizer, model_init, compute_metrics, callbacks, optimizers)
    286                 "You should subclass `Trainer` and override the `create_optimizer_and_scheduler` method."
    287             )
--> 288         default_callbacks = DEFAULT_CALLBACKS + get_reporting_integration_callbacks(self.args.report_to)
    289         callbacks = default_callbacks if callbacks is None else default_callbacks + callbacks
    290         self.callback_handler = CallbackHandler(

~/.virtualenvs/transformers_lab/lib/python3.7/site-packages/transformers/integrations.py in get_reporting_integration_callbacks(report_to)
    794         if integration not in INTEGRATION_TO_CALLBACK:
    795             raise ValueError(
--> 796                 f"{integration} is not supported, only {', '.join(INTEGRATION_TO_CALLBACK.keys())} are supported."
    797             )
    798     return [INTEGRATION_TO_CALLBACK[integration] for integration in report_to]

ValueError: w is not supported, only azure_ml, comet_ml, mlflow, tensorboard, wandb are supported.

Have anyone got same error before?

Asked By: Weber Huang

||

Source

Answer 1

Although the documentation states that the report_to parameter can receive both List[str] or str I have always used a list with 1! element for this purpose.

Therefore, even if you report only to wandb, the solution to your problem is to replace:

 report_to = 'wandb'

with

report_to = ["wandb"]

Answered By: Timbus Calin

HuggingFace Trainer() cannot report to wandb

Question:

Answers: