loss issues

[Dlonly904]

I encountered a problem where the loss does not decrease when training with DATASETS="llava_hound". My script is set as follows, and I am using three 3090 GPUs for training. Could you please point out what might be wrong?
#!/bin/bash

Complete QwenVL Training Launch Script with Full Parameter Documentation

======================

Distributed Configuration

======================

CUDA_VISIBLE_DEVICES=1,2,3
MASTER_ADDR="127.0.0.1" # [Required] Master node IP for multi-GPU training
MASTER_PORT=$(shuf -i 20000-29999 -n 1) # Random port to avoid conflicts
NPROC_PER_NODE=3 # Automatically detects available GPUs

======================

Path Configuration

======================

MODEL_PATH="Qwen/Qwen2.5-VL-3B-Instruct" # [ModelArguments] Pretrained model path
GEOMETRY_ENCODER_TYPE="vggt"
GEOMETRY_ENCODER_PATH="facebook/VGGT-1B"
OUTPUT_DIR="PATH_TO_OUTPUT_DIR" # Directory for saving checkpoints
CACHE_DIR="/data2" # [TrainingArguments] Cache directory for models
mkdir -p $OUTPUT_DIR

======================

Model Configuration

======================

#DATASETS="spar_234k,llava_hound_64k" # [DataArguments] Dataset with sampling rate
DATASETS="llava_hound"

======================

Training Hyperparameters

======================

LR=1e-5
total_batch_size=64
#GRADIENT_ACCUMULATION_STEPS=$(($total_batch_size / $NPROC_PER_NODE))
GRADIENT_ACCUMULATION_STEPS=8
torchrun --nproc_per_node=$NPROC_PER_NODE
--master_addr=$MASTER_ADDR
--master_port=$MASTER_PORT
src/qwen_vl/train/qwen_lora.py
--model_name_or_path $MODEL_PATH
--tune_mm_llm True
--tune_mm_vision False
--tune_mm_mlp False
--dataset_use $DATASETS
--output_dir $OUTPUT_DIR
--cache_dir $CACHE_DIR
--bf16
--per_device_train_batch_size 1
--gradient_accumulation_steps $GRADIENT_ACCUMULATION_STEPS
--learning_rate $LR
--mm_projector_lr 1e-5
--vision_tower_lr 1e-6
--optim adamw_torch
--model_max_length 12800
--data_flatten False
--max_pixels $((5762828))
--min_pixels $((162828))
--base_interval 2
--video_max_frames 8
--video_min_frames 4
--video_max_frame_pixels $((16642828))
--video_min_frame_pixels $((2562828))
--num_train_epochs 1
--warmup_ratio 0.05
--lr_scheduler_type "cosine"
--weight_decay 0.01
--logging_steps 50
--save_steps 1000
--save_total_limit 1
--deepspeed "scripts/zero2_opt.json"
--gradient_checkpointing
--dataloader_num_workers 4
--group_by_modality_length true
--seed 0
--report_to "wandb"
--use_geometry_encoder true
--geometry_encoder_type $GEOMETRY_ENCODER_TYPE
--geometry_encoder_path $GEOMETRY_ENCODER_PATH
--feature_fusion_method "add"
> ${OUTPUT_DIR}/train.log 2>&1