Arguments

Finetuning Arguments

Basic Parameters

FinetuningArguments

Parameter Name	Type	Description	Default Value
pure_bf16	bool	Whether to train the model in pure bf16 precision (without using AMP).	False
stage	Literal[“pt”, “sft”, “rm”, “ppo”, “dpo”, “kto”]	Training Stage	sft
finetuning_type	Literal[“lora”, “freeze”, “full”]	Fine-tuning Method	lora
use_llama_pro	bool	Whether to only train parameters in extension blocks (LLaMA Pro mode).	False
use_adam_mini	bool	Whether to use Adam-mini optimizer.	False
freeze_vision_tower	bool	Whether to freeze vision tower during MLLM training.	True
freeze_multi_modal_projector	bool	Whether to freeze multimodal projector during MLLM training.	True
train_mm_proj_only	bool	Whether to train only the multimodal projector.	False
compute_accuracy	bool	Whether to compute token-level accuracy during evaluation.	False
disable_shuffling	bool	Whether to disable shuffling of the training set.	False
plot_loss	bool	Whether to save the loss curve during training.	False
include_effective_tokens_per_second	bool	Whether to compute effective tokens per second.	False

LoRA

LoraArguments

Parameter Name	Type	Description	Default Value
additional_target	Optional[str]	Names of modules besides LoRA layers that are set as trainable and saved in the final checkpoint. Use commas to separate multiple modules.	None
lora_alpha	Optional[int]	LoRA scaling coefficient. Generally lora_rank * 2.	None
lora_dropout	float	Dropout rate in LoRA fine-tuning.	0
lora_rank	int	Intrinsic dimensionality r of LoRA fine-tuning. Larger r means more trainable parameters.	8
lora_target	str	Names of modules to apply LoRA method to. Use commas to separate multiple modules, use all to specify all modules.	all
loraplus_lr_ratio	Optional[float]	LoRA+ learning rate ratio (λ = ηB/ηA). ηA, ηB are the learning rates of adapter matrices A and B respectively.	None
loraplus_lr_embedding	Optional[float]	Learning rate for LoRA+ embedding layer.	1e-6
use_rslora	bool	Whether to use Rank-Stabilized LoRA.	False
use_dora	bool	Whether to use Weight-Decomposed LoRA.	False
pissa_init	bool	Whether to initialize PiSSA adapter.	False
pissa_iter	Optional[int]	Number of iteration steps for FSVD execution in PiSSA. Use -1 to disable it.	16
pissa_convert	bool	Whether to convert PiSSA adapter to normal LoRA adapter.	False
create_new_adapter	bool	Whether to create a new adapter with randomly initialized weights.	False

RLHF

RLHF Training Parameters

Parameter Name	Type	Description	Default Value
pref_beta	float	Beta parameter in preference loss.	0.1
pref_ftx	float	SFT loss coefficient in DPO training.	0.0
pref_loss	Literal[“sigmoid”, “hinge”, “ipo”, “kto_pair”, “orpo”, “simpo”]	Type of preference loss used in DPO training. Available values are: sigmoid, hinge, ipo, kto_pair, orpo, simpo.	sigmoid
dpo_label_smoothing	float	Label smoothing coefficient, range [0,0.5].	0.0
kto_chosen_weight	float	Weight of chosen label loss in KTO training.	1.0
kto_rejected_weight	float	Weight of rejected label loss in KTO training.	1.0
simpo_gamma	float	Reward margin in SimPO loss.	0.5
ppo_buffer_size	int	Mini-batch size in PPO training.	1
ppo_epochs	int	Number of PPO training iterations.	4
ppo_score_norm	bool	Whether to use normalized scores in PPO training.	False
ppo_target	float	Target KL value for adaptive KL control in PPO training.	6.0
ppo_whiten_rewards	bool	Whether to normalize rewards in PPO training.	False
ref_model	Optional[str]	Path to reference model used in PPO or DPO training.	None
ref_model_adapters	Optional[str]	Adapter path for reference model.	None
ref_model_quantization_bit	Optional[int]	Quantization bit width for reference model, supports 4-bit or 8-bit quantization.	None
reward_model	Optional[str]	Path to reward model used in PPO training.	None
reward_model_adapters	Optional[str]	Adapter path for reward model.	None
reward_model_quantization_bit	Optional[int]	Quantization bit width for reward model.	None
reward_model_type	Literal[“lora”, “full”, “api”]	Type of reward model used in PPO training. Available values are: lora, full, api.	lora

Freeze

FreezeArguments

Parameter Name	Type	Description	Default Value
freeze_trainable_layers	int	Number of trainable layers. Positive numbers indicate the last n layers are set as trainable, negative numbers indicate the first n layers are set as trainable.	2
freeze_trainable_modules	str	Names of trainable layers. Use all to specify all modules.	all
freeze_extra_modules	Optional[str]	Names of modules that can be trained besides hidden layers; specified modules will be set as trainable. Use commas to separate multiple modules.	None

Apollo

ApolloArguments

Parameter Name	Type	Description	Default Value
use_apollo	bool	Whether to use APOLLO optimizer.	False
apollo_target	str	Names of modules to apply APOLLO to. Use commas to separate multiple modules, use all to specify all linear modules.	all
apollo_rank	int	Rank of APOLLO gradient.	16
apollo_update_interval	int	Step interval for updating APOLLO projection.	200
apollo_scale	float	APOLLO scaling coefficient.	32.0
apollo_proj	Literal[“svd”, “random”]	APOLLO low-rank projection algorithm type (svd or random).	random
apollo_proj_type	Literal[“std”, “right”, “left”]	APOLLO projection type.	std
apollo_scale_type	Literal[“channel”, “tensor”]	APOLLO scaling type (channel or tensor).	channel
apollo_layerwise	bool	Whether to enable layer-wise updates to further save memory.	False
apollo_scale_front	bool	Whether to use norm growth limiter before gradient scaling.	False

BAdam

BAdamArgument

Parameter Name	Type	Description	Default Value
use_badam	bool	Whether to use BAdam optimizer.	False
badam_mode	Literal	BAdam usage mode, available values are layer or ratio.	layer
badam_start_block	Optional[int]	Starting block index for layer-wise BAdam.	None
badam_switch_mode	Optional[Literal]	Block update strategy in layer-wise BAdam, available values are: ascending, descending, random, fixed.	ascending
badam_switch_interval	Optional[int]	Step interval for block updates in layer-wise BAdam. Use -1 to disable block updates.	50
badam_update_ratio	float	Update ratio in ratio-wise BAdam.	0.05
badam_mask_mode	Literal	Mask mode for BAdam optimizer, available values are adjacent or scatter.	adjacent
badam_verbose	int	Verbose output level for BAdam optimizer, 0 means no output, 1 means output block prefix, 2 means output trainable parameters.	0

GaLore

GaLoreArguments

Parameter Name	Type	Description	Default Value
use_galore	bool	Whether to use GaLore algorithm.	False
galore_target	str	Names of modules to apply GaLore to. Use commas to separate multiple modules, use all to specify all linear modules.	all
galore_rank	int	Rank of GaLore gradient.	16
galore_update_interval	int	Step interval for updating GaLore projection.	200
galore_scale	float	Scaling coefficient for GaLore.	0.25
galore_proj_type	Literal	Type of GaLore projection, available values are: std, reverse_std, right, left, full.	std
galore_layerwise	bool	Whether to enable layer-wise updates to further save memory.	False

Data Arguments

DataArguments

Parameter Name	Type	Description	Default Value
template	Optional[str]	Template for constructing prompts during training and inference.	None
dataset	Optional[str]	Name of dataset(s) for training. Use commas to separate multiple datasets.	None
eval_dataset	Optional[str]	Name of dataset(s) for evaluation. Use commas to separate multiple datasets.	None
eval_on_each_dataset	Optional[bool]	Whether to calculate the loss separately on each evaluation dataset; by default, the losses are concatenated and computed as a whole.	False
dataset_dir	Union[str, Dict[str, Any]]	The path to the folder storing the dataset, which can be a string or a dictionary. If a string, it represents the dataset directory path, for example data ; if a dictionary, it overrides the default behavior of loading from the local dataset_info.json.	data
media_dir	Optional[str]	Folder path where images, videos, or audio are stored. If not specified, defaults to dataset_dir.	None
data_shared_file_system	Optional[bool]	When using multiple nodes and multiple cards, whether the dataset paths on different machines are on a shared file system. If set to true, dataset processing only occurs on the first node; if false, processing is performed on each node.	false
cutoff_len	int	Maximum number of tokens for input, inputs exceeding this length will be truncated.	2048
train_on_prompt	bool	Whether to train on input prompts.	False
mask_history	bool	Whether to train only on the current dialogue turn.	False
streaming	bool	Whether to enable streaming mode.	False
buffer_size	int	Buffer size for randomly selecting samples when streaming is enabled.	16384
mix_strategy	Literal[“concat”, “interleave_under”, “interleave_over”]	Dataset mixing strategy, supports concat, interleave_under, interleave_over.	concat
interleave_probs	Optional[str]	When using interleave strategy, specify the probability of sampling from multiple datasets. Separate probabilities for multiple datasets with commas.	None
overwrite_cache	bool	Whether to overwrite the cached training and evaluation datasets.	False
preprocessing_batch_size	int	Number of examples per batch during preprocessing.	1000
preprocessing_num_workers	Optional[int]	Number of processes to use during preprocessing.	None
max_samples	Optional[int]	Maximum number of samples per dataset: when set, the number of samples from each dataset will be truncated to the specified max_samples.	None
eval_num_beams	Optional[int]	The num_beams parameter during model evaluation.	None
ignore_pad_token_for_loss	bool	Whether to ignore pad tokens when computing loss.	True
val_size	float	Size of validation set relative to the training dataset used. Value should be in [0,1). When streaming is enabled, val_size should be an integer.	0.0
packing	Optional[bool]	Whether to enable sequence packing. Enabled by default during pre-training.	None
neat_packing	bool	Whether to enable sequence packing without using cross-attention.	False
tool_format	Optional[str]	Format used for constructing function calling examples.	None
tokenized_path	Optional[str]	Path to save or load tokenized datasets. If the path exists, existing tokenized datasets will be loaded; if the path does not exist, tokenized datasets will be saved to this path after tokenization.	None

Model Arguments

Basic Parameters

ModelArguments

Parameter Name	Type	Description	Default Value
model_name_or_path	Optional[str]	Model path (local path or Huggingface/ModelScope path).	None
adapter_name_or_path	Optional[str]	Adapter path (local path or Huggingface/ModelScope path). Use commas to separate multiple adapter paths.	None
adapter_folder	Optional[str]	Folder path containing adapter weights.	None
cache_dir	Optional[str]	Local path to save models downloaded from Hugging Face or ModelScope.	None
use_fast_tokenizer	bool	Whether to use fast_tokenizer.	True
resize_vocab	bool	Whether to resize vocabulary and embedding layer.	False
split_special_tokens	bool	Whether to split special tokens during tokenization.	False
new_special_tokens	Optional[str]	Special tokens to add to the tokenizer. Separate multiple special tokens with commas.	None
model_revision	str	The specific model version to use.	main
low_cpu_mem_usage	bool	Whether to use memory-efficient model loading.	True
rope_scaling	Optional[Literal[“linear”, “dynamic”, “yarn”, “llama3”]]	Scaling strategy for RoPE Embedding, supports linear, dynamic, yarn, or llama3.	None
flash_attn	Literal[“auto”, “disabled”, “sdpa”, “fa2”]	Whether to enable FlashAttention to accelerate training and inference. Available values are auto, disabled, sdpa, fa2.	auto
shift_attn	bool	Whether to enable Shift Short Attention (S^2-Attn).	False
mixture_of_depths	Optional[Literal[“convert”, “load”]]	Specify: convert when need to convert model to mixture_of_depths (MoD) model. Specify: load when need to load mixture_of_depths (MoD) model.	None
use_unsloth	bool	Whether to use unsloth to optimize LoRA fine-tuning.	False
use_unsloth_gc	bool	Whether to use unsloth’s gradient checkpointing.	False
enable_liger_kernel	bool	Whether to enable liger kernel to accelerate training.	False
moe_aux_loss_coef	Optional[float]	The aux_loss coefficient in MoE architecture. Larger values lead to more balanced load across experts.	None
disable_gradient_checkpointing	bool	Whether to disable gradient checkpointing.	False
use_reentrant_gc	bool	Whether to enable reentrant gradient checkpointing	True
upcast_layernorm	bool	Whether to upcast layernorm layer weight precision to fp32.	False
upcast_lmhead_output	bool	Whether to upcast lm_head output precision to fp32.	False
train_from_scratch	bool	Whether to randomly initialize model weights.	False
infer_backend	Literal[“huggingface”, “vllm”]	Backend engine used during inference, supports huggingface or vllm.	huggingface
offload_folder	str	Path to offload model weights.	offload
use_cache	bool	Whether to use KV cache during generation.	True
infer_dtype	Literal[“auto”, “float16”, “bfloat16”, “float32”]	Data type of model weights and activation values used during inference. Supports auto, float16, bfloat16, float32.	auto
hf_hub_token	Optional[str]	Authentication token for logging in to HuggingFace.	None
ms_hub_token	Optional[str]	Authentication token for logging in to ModelScope Hub.	None
om_hub_token	Optional[str]	Authentication token for logging in to Modelers Hub.	None
print_param_status	bool	Whether to print the status of model parameters.	False
trust_remote_code	bool	Whether to trust code execution from datasets/models on the Hub.	False
compute_dtype	Optional[torch.dtype]	Data type used for computing model output, no manual specification needed.	None
device_map	Optional[Union[str, Dict[str, Any]]]	Device map for model allocation, no manual specification needed.	None
model_max_length	Optional[int]	Maximum input length for the model, no manual specification needed.	None
block_diag_attn	bool	Whether to use block diagonal attention, no manual specification needed.	False

Multimodal Model

ProcessorArguments

Parameter Name	Type	Description	Default Value
image_max_pixels	int	Maximum number of pixels for image input.	768 x 768
image_min_pixels	int	Minimum number of pixels for image input.	32 x 32
video_max_pixels	int	Maximum number of pixels for video input.	256 x 256
video_min_pixels	int	Minimum number of pixels for video input.	16 x 16
video_fps	float	Sampling frame rate for video input (frames sampled per second).	2.0
video_maxlen	int	Maximum number of sampled frames for video input.	128

vLLM Inference

VllmArguments

Parameter Name	Type	Description	Default Value
vllm_maxlen	int	Maximum sequence length (including input text and generated text).	4096
vllm_gpu_util	float	GPU utilization ratio, range between (0, 1).	0.9
vllm_enforce_eager	bool	Whether to disable CUDA graph in vLLM.	False
vllm_max_lora_rank	int	Maximum allowed LoRA Rank for inference.	32
vllm_config	Optional[Union[dict, str]]	vLLM engine initialization configuration. Input as dictionary or JSON string.	None

Model Quantization

QuantizationArguments

Parameter Name	Type	Description	Default Value
quantization_method	Literal[“bitsandbytes”, “hqq”, “eetq”]	Specify the algorithm for quantization, supports “bitsandbytes”, “hqq” and “eetq”.	bitsandbytes
quantization_bit	Optional[int]	Specify the number of bits used in the quantization process, typically 4-bit, 8-bit, etc.	None
quantization_type	Literal[“fp4”, “nf4”]	Data type used during quantization, supports “fp4” and “nf4”.	nf4
double_quantization	bool	Whether to use double quantization during the quantization process, typically used for “bitsandbytes” int4 quantization training.	True
quantization_device_map	Optional[Literal[“auto”]]	Device map for inference with 4-bit quantized models. Requires “bitsandbytes >= 0.43.0”.	None

Model Export

ExportArguments

Parameter Name	Type	Description	Default Value
export_dir	Optional[str]	Path to the directory where the exported model will be saved.	None
export_size	int	File shard size of the exported model (in GB).	5
export_device	Literal[“cpu”, “auto”]	Device used when exporting model, auto can automatically accelerate export.	cpu
export_quantization_bit	Optional[int]	Number of bits used when quantizing exported model.	None
export_quantization_dataset	Optional[str]	Dataset path or dataset name for quantizing exported model.	None
export_quantization_nsamples	int	Number of samples used during quantization.	128
export_quantization_maxlen	int	Maximum length of model input for quantization.	1024
export_legacy_format	bool	True: save in .bin format. False: save in .safetensors format.	False
export_hub_model_id	Optional[str]	Repository name for uploading model to Huggingface.	None

Eval Arguments

EvalArguments

Parameter Name	Type	Description	Default Value
task	str	Name of evaluation task, available options are mmlu_test, ceval_validation, cmmlu_test	None
task_dir	str	Path to the folder containing evaluation datasets.	evaluation
batch_size	int	Batch size per GPU.	4
seed	int	Random seed for data loader.	42
lang	str	Language for evaluation, available values are en, zh.	en
n_shot	int	Number of few-shot examples.	5
save_dir	str	Path to save evaluation results. If the path already exists, an error will be thrown.	None
download_mode	str	Download mode for evaluation dataset, reuse if dataset already exists, otherwise download.	DownloadMode.REUSE_DATASET_IF_EXISTS

Generating Arguments

GeneratingArguments

Parameter Name	Type	Description	Default Value
do_sample	bool	Whether to use sampling strategy for text generation. If set to False, greedy decoding will be used.	True
temperature	float	Used to adjust the randomness of generated text. Higher temperature makes text more random; lower temperature makes text more deterministic.	0.95
top_p	float	Parameter to control the size of candidate token set during generation. For example: top_p = 0.7 means the model will first select tokens with highest probabilities until their cumulative probability exceeds 0.7, then sample from this set of tokens.	0.7
top_k	int	Parameter to control the size of candidate token set during generation. For example: top_k = 50 means the model will sample from the set of 50 tokens with highest probability.	50
num_beams	int	Beam width for beam_search. Value of 1 means not using beam_search.	1
max_length	int	Maximum text length (including both input and generated text).	1024
max_new_tokens	int	Maximum length of generated text. Setting max_new_tokens will override max_length.	1024
repetition_penalty	float	Penalty coefficient for generating repeated tokens. Generation probability for already generated tokens is multiplied by 1/repetition_penalty. Values less than 1.0 increase probability of repeating tokens, values greater than 1.0 decrease it.	1.0
length_penalty	float	Penalty coefficient for generated text length when using beam_search. length_penalty > 0 encourages model to generate longer sequences, length_penalty < 0 encourages shorter sequences.	1.0
default_system	str	Default system_message, for example: “You are a helpful assistant.”	None
skip_special_tokens	bool	Whether to skip special tokens during decoding.	True

SwanLab Arguments

SwanLabArguments

Parameter Name	Type	Description	Default Value
use_swanlab	bool	Whether to use SwanLab.	False
swanlab_project	str	Project name in SwanLab.	“llamafactory”
swanlab_workspace	str	Workspace name in SwanLab.	None
swanlab_run_name	str	Experiment name in SwanLab.	None
swanlab_mode	Literal[“cloud”, “local”]	Run mode for SwanLab.	cloud
swanlab_api_key	str	API key for SwanLab.	None

Training Arguments

RAY

RayArguments

Parameter Name	Type	Description	Default Value
ray_num_workers	int	Number of worker processes used by Ray training.	1
ray_init_kwargs	Optional[dict, str]	Ray initialization parameters.	None
master_addr	Optional[str]	The IP address of the master node for distributed communication. Defaults to the IP address of the Ray cluster’s master node.	None
master_port	Optional[str]	The port used for listening to connections on the master node. Defaults to a random available port.	None

Environment Variables

Environment Variables

Name	Type	Description
API_HOST	API	The host address the API server listens on
API_PORT	API	The port number the API server listens on
API_KEY	API	The password for accessing the API.
API_MODEL_NAME	API	Specifies the model name to be loaded and used by the API service
API_VERBOSE	API	Controls the verbosity level of API logs
FASTAPI_ROOT_PATH	API	Sets the root path for the FastAPI application
MAX_CONCURRENT	API	The maximum number of concurrent requests for the API.
DISABLE_VERSION_CHECK	General	Whether to disable version check on startup.
FORCE_CHECK_IMPORTS	General	Forces checking of optional imports
ALLOW_EXTRA_ARGS	General	Allows passing extra arguments in the command line
LLAMAFACTORY_VERBOSITY	General	Sets the log level of LLaMA-Factory (“DEBUG”, “INFO”, “WARN”)
USE_MODELSCOPE_HUB	General	Prioritizes using ModelScope to download models/datasets or uses them from the cache path
USE_OPENMIND_HUB	General	Prioritizes using Openmind to download models/datasets or uses them from the cache path
USE_RAY	General	Whether to use Ray for distributed execution or task management.
RECORD_VRAM	General	Whether to record VRAM usage.
OPTIM_TORCH	General	Whether to enable specific PyTorch optimizations.
NPU_JIT_COMPILE	General	Whether to enable JIT compilation for NPU.
CUDA_VISIBLE_DEVICES	General	GPU selection.
ASCEND_RT_VISIBLE_DEVICES	General	NPU selection.
FORCE_TORCHRUN	Torchrun	Whether to force using torchrun to start the script
MASTER_ADDR	Torchrun	The network address of the master node in Torchrun deployment
MASTER_PORT	Torchrun	The port number used for communication by the master node in Torchrun deployment
NNODES	Torchrun	The total number of nodes participating in the distributed deployment
NODE_RANK	Torchrun	The rank of the current node among all nodes, typically from 0 to NNODES-1.
NPROC_PER_NODE	Torchrun	The number of GPUs per node
WANDB_DISABLED	Log	Whether to disable wandb
WANDB_PROJECT	Log	Sets the project name in wandb.
WANDB_API_KEY	Log	The API key for accessing wandb
GRADIO_SHARE	Web UI	Whether to create a shareable Web UI link
GRADIO_SERVER_NAME	Web UI	Sets the Gradio server IP address (e.g., 0.0.0.0)
GRADIO_SERVER_PORT	Web UI	Sets the Gradio server port
GRADIO_ROOT_PATH	Web UI	Sets the root path for the Gradio application
GRADIO_IPV6	Web UI	Enables IPv6 support for the Gradio server
ENABLE_SHORT_CONSOLE	Setting	Supports using lmf as an alias for llamafactory-cli