Skip to main content

Hugging Face

brew install huggingface-cli
pip install huggingface_hub

huggingface-cli download gpt2 config.json model.safetensors # 下载单个文件
huggingface-cli download Qwen/Qwen2.5-VL-7B-Instruct        # 下载模型

# ~/.cache/huggingface/stored_tokens
huggingface-cli login
huggingface-cli whoami
  • --repo-type dataset
  • --repo-type space
  • --revision v1.1

Cache​

  • HF_DATASETS_CACHE
  • ~/.cache/huggingface/hub
  • ~/.cache/huggingface/hub/models--<username>--<modelname>/
    • blobs/
    • refs/
    • snapshots/

Inference​

from huggingface_hub import InferenceClient

client = InferenceClient(
  provider="hyperbolic",
  api_key="hf_",
)

messages = [
  {
    "role": "user",
    "content": [
      {
        "type": "text",
        "text": "Describe this image in one sentence."
      },
      {
        "type": "image_url",
        "image_url": {
          "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
        }
      }
    ]
  }
]

completion = client.chat.completions.create(
  model="Qwen/Qwen2.5-VL-72B-Instruct",
  messages=messages,
  max_tokens=500,
)

FAQ

The model Qwen/Qwen2.5-VL-72B-Instruct is too large to be loaded automatically (146GB > 10GB).​

  • InferenceClient 指定 Provider
  • serverless inference API,

You are trying to access a gated repo.​

  1. 同意 Model 条款
  2. 生成 Token
  3. HF 登录
  4. 下载模型
Please enable access to public gated repositories in your fine-grained token settings to view this repository

We couldn't connect to 'https://huggingface.co' to load this file
  • 调整 Token 权限
from huggingface_hub import login
login()

Jupyter Notebook

from huggingface_hub import notebook_login
notebook_login()