{"full_name":"hiyouga/LlamaFactory","owner":"hiyouga","name":"LlamaFactory","url":"https://github.com/hiyouga/LlamaFactory","description":"Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)","homepage":"https://llamafactory.readthedocs.io","topics":[],"language":"Python","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/16256802?u=9622ad4c25aa0a0bfc6dfac3db6bd0d7732c0f1e&v=4","created_at":"2023-05-28T10:09:12Z","pushed_at":"2026-11-09T14:55:42Z","metrics":{"stars":75971,"forks":9360,"watchers":343,"issues":8053,"prs":1629,"releases":36,"issues_open":null,"issues_closed":null,"prs_open":null,"prs_closed":null,"prs_merged":null,"discussions":664},"growth":{"day":{"stars":17,"forks":6,"base_day":"2026-11-09"}},"history":{"dates":["2026-11-09","2026-11-10"],"stars":[75954,75971],"forks":[9354,9360],"watchers":[343,343],"issues_open":[null,null],"prs_open":[null,null]},"boards":{},"same_owner":[],"issues":[{"number":6792,"title":"结合 GRPO 支持 DeepSeek-R1 等推理模型的复现，达到 huggingface open-r1 的类似效果","comments":60,"created_at":"2025-02-02T07:36:17Z"},{"number":10221,"title":"qwen3.5-27b 全参微调训练耗时很久","comments":43,"created_at":"2026-02-27T03:23:03Z"},{"number":7767,"title":"Hanging problem when LoRA finte-tune Qwen2.5Omni with multi-turn video-audio samples with deepspeedz3","comments":41,"created_at":"2025-04-19T05:32:32Z"},{"number":4388,"title":"关于npu训练模型总结以及疑问","comments":35,"created_at":"2024-06-20T05:43:06Z"},{"number":7396,"title":"Watchdog caught collective operation timeout: WorkNCCL(SeqNum=1580, OpType=ALLREDUCE, NumelIn=466119168, NumelOut=466119168, Timeout(ms)=600000) ran for 600004 milliseconds before timing out.","comments":30,"created_at":"2025-03-21T01:49:51Z"},{"number":8600,"title":"OOM and slow tokenization after upgrade LLaMA-Factory","comments":27,"created_at":"2025-07-10T02:47:42Z"},{"number":9474,"title":"Qwen3omni-30B-A3B-Instruct Lora微调训练一直卡住","comments":25,"created_at":"2025-11-12T06:36:48Z"},{"number":8041,"title":"[question] Dose Muon Optimizer support deepspeed ?","comments":25,"created_at":"2025-05-13T09:00:08Z"}],"discussions":[{"number":10886,"title":"Can anyone help me train a NodeJS / JavaScript model ?","comments":1,"category":"Q&A","created_at":"2026-10-10T08:18:49Z"},{"number":10848,"title":"Which abliterated variants are actually worth hosting? (founder, hosting them)","comments":0,"category":"Show and tell","created_at":"2026-09-25T19:10:35Z"},{"number":10843,"title":"请问webui上面有办法实现多机多卡微调吗","comments":1,"category":"Q&A","created_at":"2026-09-20T09:54:26Z"},{"number":10836,"title":"Training the Qwen3.5-9B model with only_text data, how to avoid multimodal placeholders","comments":1,"category":"Q&A","created_at":"2026-09-15T07:10:03Z"},{"number":10768,"title":"Qwen3-30B-A3B-Base SFT-LORA+DeepSpeed ZERO3微调OOM问题","comments":1,"category":"Q&A","created_at":"2026-08-21T09:38:26Z"},{"number":10750,"title":"VOKO 自荐：做 AI Agent 之间的\"通信协议层\"","comments":0,"category":"Ideas","created_at":"2026-08-13T03:16:09Z"},{"number":10631,"title":"How should I balance learning rate and data sampling during CPT on multiple datasets?","comments":3,"category":"Q&A","created_at":"2026-07-06T19:02:56Z"},{"number":10585,"title":"Runtime reliability after fine tuning","comments":0,"category":"Show and tell","created_at":"2026-06-16T09:46:00Z"}],"latest_release":{"name":"v0.9.5: Qwen3.5/3.6, Gemma 4, Transformers v5","tag":"v0.9.5","published_at":"2026-05-30T15:57:58Z","body":"Added primary support for Qwen3.5/Qwen3.6/Gemma4 models and compatibility with Transformers v5.\r\n\r\n## What's Changed\r\n\r\n* [misc] set dev version by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9703\r\n* fix(fp8): add Transformer Engine backend support by @sbhavani in https://github.com/hiyouga/LlamaFactory/pull/9705\r\n* [misc] Compatible with an empty architectures field in config.json by @tangefly in https://github.com/hiyouga/LlamaFactory/pull/9709\r\n* [model] support Youtu-LLM-2B by @isLinXu in https://github.com/hiyouga/LlamaFactory/pull/9707\r\n* [misc] lint by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9710\r\n* Update pyproject.toml and requirements by @jiaqiw09 in https://github.com/hiyouga/LlamaFactory/pull/9714\r\n* [v1] add init plugin by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9716\r\n* [misc] Add a PyTorch version warning for Conv3D. by @tangefly in https://github.com/hiyouga/LlamaFactory/pull/9715\r\n* [feature] add support for EAFT loss by @ymxyll in https://github.com/hiyouga/LlamaFactory/pull/9720\r\n* [v1] add cli sampler by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9721\r\n* [v1] add renderer ut by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9722\r\n* Update README.md by @tangefly in https://github.com/hiyouga/LlamaFactory/pull/9724\r\n* [CI]improve cuda ci cache by @frozenleaves in https://github.com/hiyouga/LlamaFactory/pull/9725\r\n* Add support for LiquidAI's LFM2.5 (Liquid Foundation Models) to LLaMA-Factory. by @vovanphuc in https://github.com/hiyouga/LlamaFactory/pull/9726\r\n* Add support for LiquidAI's LFM2.5-VL vision-language model by @vovanphuc in https://github.com/hiyouga/LlamaFactory/pull/9729\r\n* [misc] fix parser by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9730\r\n* [refactor] rename lfm template to lfm2 and add LFM 2.5 to README by @vovanphuc in https://github.com/hiyouga/LlamaFactory/pull/9731\r\n* [fix] correct ktransformers example config paths and templates by @JimmyPeilinLi in https://github.com/hiyouga/LlamaFactory/pull/9732\r\n* [model] support for microsoft's Phi-4-mini by @ctx289 in https://github.com/hiyouga/LlamaFactory/pull/9734\r\n* [misc] fix fp8 by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9742\r\n* [v1] add batch generator by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9744\r\n* [deps] fix package by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9745\r\n* [model] support HY-MT model by @isLinXu in https://github.com/hiyouga/LlamaFactory/pull/9746\r\n* [v1] upgrade batching by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9751\r\n* [model] fixed&added Hunyuan models by @isLinXu in https://github.com/hiyouga/LlamaFactory/pull/9750\r\n* [v1] add sft by @hiyouga in https://github.com/hiyouga/LlamaFactory/pull/9752\r\n* using mp to run kernel test by @frozenleaves in https://github.com/hiyouga/LlamaFactory/pull/9754\r\n* [v1] fix kernel moe patch by @jiaqiw09 in https://github.com/hiyouga/LlamaFactory/pull/9867\r\n* [misc] update mcore related docker and mca supported models by @Kuangdd01 in https://github.com/hiyouga/LlamaFactory/pull/10114\r\n* [feat] support `all_exhausted_without_replacement` in datasets.interleave_datasets by @Moenupa in https://github.com/hiyouga/LlamaFactory/pull/10112\r\n* chore: Update outdated GitHub Actions versions by @pgoslatara in https://github.com/hiyouga/LlamaFactory/pull/10123\r\n* [v1] support training with fsdp2 by @frozenleaves in https://github.com/hiyouga/LlamaFactory/pull/9773\r\n* [v0] Fix reward model training safetensors saving by @jiaqiw09 in https://github.com/hiyouga/LlamaFactory/pull/10137\r\n* Fix : add visual.pos_embed to Qwen3-VL visual model keys by @je1lee in https://github.com/hiyouga/LlamaFactory/pull/10139\r\n* [feature] support using ray.remote to start distributed training. by @xvxuopop in https://github.com/hiyouga/LlamaFactory/pull/10109\r\n* update peft, deepspeed, adapt transformers v5 by @frozenleaves in https://github.com/hiyouga/LlamaFactory/pull/"},"readme":true,"readme_zh":true,"news":[],"stale":false,"snapshot_at":"2026-11-10T09:35:25Z","history_start":"2026-09-06"}