LLM Awesome

• Engine/Framework/Service/Inference
  • ggerganov/llama.cpp
    • MIT, C++
    • 有非常多的语言绑定
    • 支持最好最广泛使用的推理引擎
  • NVIDIA/TensorRT-LLM
    • Toolbox for Optimized LLM Inference
  • codelion/optillm
    • Optimizing inference proxy for LLMs
  • bklieger-groq/g1
    • Reasoning/CoT/reasoning chain
    • HN
  • mlc-ai/web-llm
    • Apache-2.0, TS, WebGPU
    • 直接在浏览器内运行 模型 inference
  • 商业
    • Perplexity
    • Fireworks AI
    • groq
    • Cloudflare AI Workers
    • Nvidia NIM
  • Models
    • models.litellm.ai
    • openrouter.ai/models
    • app.hyperbolic.xyz/models
    • minthemiddle.github.io/openrouter-model-comparison
  • 价格
    • arc53/llm-price-compass
    • llmprices.dev
    • aichatmakers.com/2024/04/15/llms-pricing-comparison-with-chat-arena-leaderboard-ranking-april-2024
• Runner/Inference
  • vllm-project/vllm
    • vllm.ai
    • vLLM: Easy, Fast, and Cheap LLM Serving
  • jmorganca/ollama
    • MIT, Golang
    • 快速启动任意模型
    • 提供模型管理
    • 提供管理接口
• Application/WebUI/UI/Desktop/Consumer
  • mudler/LocalAI
    • MIT, C++, Go
  • menloresearch/jan
    • AGPLv3, TS
  • LostRuins/koboldcpp
    • AGPLv3, C++
    • Run GGUF models easily with a KoboldAI UI
  • abi/secret-llama
    • Apache-2.0, TS
    • Fully private LLM chatbot that runs entirely with a browser with no server needed. Supports Mistral and LLama 3.
  • Yonom/assistant-ui
    • React Components for AI Chat
  • lobehub/lobe-chat
    • Apache-2.0, TypeScript
  • n4ze3m/page-assist
    • MIT, TS
    • 浏览器插件
    • chrome.google.com/webstore/detail/page-assist/jfgfiigpkhlkbnfnbobbkinehhfdhndo
  • open-webui/open-webui
    • MIT, Svelte, Python
    • WebUI for LLMs
    • open-webui/desktop
      • BSD-3, TS, Svelte, Electron
  • janhq/jan
    • AGPLv3, Typescript
    • alternative to ChatGPT that runs 100% offline
    • Multiple engine - llama.cpp, TensorRT-LLM
  • a16z-infra/llm-app-stack
  • TavernAI
    • MIT, JS
  • SillyTavern
    • AGPLv3, JS
    • fork TavernAI
    • LLM Frontend for Power Users
  • enricoros/big-agi
    • MIT, TS, JS
  • swirlai/swirl-search
    • Apache-2.0, Python
    • AI Search & RAG Without Moving Your Data
  • Pythagora-io/gpt-pilot
    • MIT, Python
    • The first real AI developer
  • All-Hands-AI/OpenHands
    • MIT, Python, TS
• Structure
  • outlines-dev/outlines
    • Apache-2.0, Pythone
    • Structured Text Generation
  • guidance-ai/guidance
    • MIT, Pythone
• Platform/Traing/Playground
  • lm-sys/FastChat
    • Apache-2.0
    • training, serving, and evaluating
• Web/UI/Chat
  • Yidadaa/ChatGPT-Next-Web
  • nluxai/nlux
    • MPL-2.0, TS, React
    • UI for any LLM, supporting LangChain / HuggingFace / Vercel AI,
  • CopilotKit
    • MIT, TS
    • React UI + elegant infrastructure for AI Copilots
    • framework for building custom AI Copilots
    • npm:@copilotkit/react-core,@copilotkit/react-ui,@copilotkit/react-textarea
  • Oneirocom/Magick
    • toolkit for AI builder
  • hubtype/botonic
    • MIT, TS, React
  • yoctol/bottender
    • MIT, TS, React
• RAG/Embedding
  • vanna-ai/vanna
    • MIT
    • Text-to-SQL
  • truefoundry/cognita
    • Apache-2.0, TS, Python
  • EZ-hwh/AutoCrawler
  • marqo-ai/marqo
    • Apache-2.0, Python
    • Unified embedding generation and search engine.
• Service
  • dify
• Agent
  • cloudflare/agents
    • MIT, TS
  • kingjulio8238/memary
    • Longterm Memory for Autonomous Agents
  • a16z-infra/ai-town
    • MIT, TS
  • coder-hxl/x-crawl
    • MIT, TS
    • Node.js AI-assisted crawler library
  • andrewyng/translation-agent
    • Agentic translation using reflection workflow
• Coding
  • replit/ReplitLM
  • albertan017/LLM4Decompile
    • Decompiling Binary Code
  • jehna/humanify
    • Deobfuscate Javascript
    • thejunkland.com/blog/using-llms-to-reverse-javascript-minification.html
  • facebook/llm-compiler
    • Meta LLM Compiler, a family of models built on Meta Code Llama with additional code optimization and compiler capabilities
• Fine-tuning
  • unslothai/unsloth
    • Apache-2.0, Python
    • Finetune Llama 3, Mistral, Phi, Gemma
    • QWen2 unslothai/unsloth#149
  • meta-llama/llama-recipes
  • LoRA
  • Efficiently fine-tune Llama 3 with PyTorch FSDP and Q-Lora
  • HF Dataset:tatsu-lab/alpaca
• Uncensor
  • llm-attacks/llm-attacks
  • HF Blog:mlabonne/abliteration
    • HN
• Promopt
  • langgptai/LangGPT
    • 结构化提示词
  • linexjlin/GPTs
    • leaked prompts of GPTs
• Vision
  • vikhyat/moondream
    • tiny vision language model
• stanfordnlp/dspy
  • DSPy: The framework for programming—not prompting—foundation models
• BloopAI/bloop
  • Answer questions about your code with an LLM agent
• HazyResearch/flash-attention
  • Fast and memory-efficient exact attention
• refuel-ai/autolabel
  • Label, clean and enrich text datasets with LLMs
• [cpacker/MemGPT](- cpacker/MemGPT)
  • Teaching LLMs memory management for unbounded context
• haotian-liu/LLaVA
  • Large Language-and-Vision Assistant
  • 类似 GPT4V
• bionic-gpt/bionic-gpt
• Reading/Explain
  • LLM Visualization
    • bbycroft.net/llm
    • bbycroft/llm-viz
  • intro-llm.github.io
    • 大规模语言模型：从理论到实践
  • spreadsheets-are-all-you-need.ai
• tyxsspa/AnyText
  • by Alibaba
• Hannibal046/Awesome-LLM
• imoneoi/openchat
• What can LLMs never do?
  • HN

Models

• facebookresearch/llama
  • GPLv3
• ymcui/Chinese-LLaMA-Alpaca
  • Apache-2.0
  • 中文大语言模型
• PotatoSpudowski/fastLLaMa
  • MIT, C
• clue-ai/PromptCLUE
• clue-ai/ChatYuan
• cocktailpeanut/dalai
• belladoreai/llama-tokenizer-js
• cogentapps/chat-with-gpt/app/src/core/tokenizer/bpe.ts
• dqbd/tiktoken
• functorism/gpt4-tokenizer-visualizer
• run-llama/llama_index