LLM FAQ
约 1 分钟阅读
标签：FAQ
- LLM
- 预测下一个 Token
- 训练不需要标注
- Instruct GPT
- SFT - Supervised fine-tuning
- HF Blog:rlhfHuggingFaceHF Blog:rlhfhuggingface.co · huggingface.co/blog/rlhf
- yizhongw/self-instructGitHubyizhongw/self-instruct
- platform.openai.com/docs/model-index-for-researchers
- Tokenizer
model metrics
- 参数量
- 非Embedding参数量
- GQA - Generalized Question Answering
- Tie Embedding
- Context Window - 上下文长度
- 语言支持度
- Code Switch - 语言转换
- 避免模型直接切换语言
- Safety
- 评测/Benchmark
- MMLU - Massive Multitask Language Understanding
- MMLU-Pro
- GPQA
- TheoremQA
- BBH
- HumanEval
- MBPP
- MultiPL-E
- GSM8K
- MATH
- C-Eval
- CMMLU
- Multi-Exam
- Multi-Understanding
- Multi-Mathematics
- 参考
EF BF BD
- Replacement Character
�\ufffd\xef\xbf\xbd- Azure OpenAI GPT 3.5 turbo
- community.openai.com/t/tokens-are-mangled-for-some-non-english-characters-resolved/74315
- community.openai.com/t/gpt-4-1106-preview-messes-up-function-call-parameters-encoding/478500
format
- GGUF
- llama.cpp, August 21st 2023, 替代 GGML
- GGML
关联信息
反向链接、本文链接的其他页面和外部资料。
参考资料
GitHub2 条
- QwenLM/Qwen/tokenization_note_zh.mdgithub.com/QwenLM/Qwen/blob/main/tokenization_note_zh.md
- yizhongw/self-instructgithub.com/yizhongw/self-instruct
HuggingFace1 条
- HF Blog:rlhfhuggingface.co/blog/rlhf
其他外链4 条
- community.openai.com/t/gpt-4-1106-preview-messes-up-function-call-parameters-encoding/478500community.openai.com/t/gpt-4-1106-preview-messes-up-function-call-parameters-encoding/478500
- community.openai.com/t/tokens-are-mangled-for-some-non-english-characters-resolved/74315community.openai.com/t/tokens-are-mangled-for-some-non-english-characters-resolved/74315
- platform.openai.com/docs/model-index-for-researchersplatform.openai.com/docs/model-index-for-researchers
- qwenlm.github.io/zh/blog/qwen2qwenlm.github.io/zh/blog/qwen2