本指南介绍如何使用Nginx在端口20000部署静态网站，提供比Python内置服务器更安全、高性能的解决方案。
安装Nginx
bash
sudo apt update
sudo apt install nginx -y
1 | bash |
放置网站文件
将你的index.html和其他静态文件复制到 /var/www/mysite/ 目录。
1 | bash |
添加以下配置：
1 | nginx |
启用站点
1 | bash |
防火墙设置
bash
1 | sudo ufw allow 23456/tcp |
bash
1 | curl -I http://localhost:23456 |
访问 http://你的服务器IP:23456 即可查看网站。
安装指导文件见https://nvidia.github.io/TensorRT-LLM/latest/installation/linux.html
1 |
|
注意wsl安装cuda有自己的步骤
1 |
|
然后在home目录的.bashrc中增加如下内容
1 |
|
后续正式开始安装，首先pytorch和必要库,需要装一个多G
1 | pip3 install torch==2.7.1 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128 |
然后装一下trt
1 | pip3 install --upgrade pip setuptools && pip3 install tensorrt_llm |
卡死了….
1 | pip3 install tensorrt_llm |
卡了足足10多分钟后，自动继续滚动下去了
然后接着卡这句：
1 | Collecting tensorrt_cu12_libs==10.11.0.33 (from tensorrt_cu12==10.11.0.33->tensorrt~=10.11.0->tensorrt_llm) |
大约10分钟，继续了
最后安装了这么一大堆后完成了，很顺利
1 | Successfully uninstalled fsspec-2025.9.0 |
写一个程序跑一下，用TRT给的例子：
···
from tensorrt_llm import LLM, SamplingParams
def main():
# Model could accept HF model name, a path to local HF model,
# or TensorRT Model Optimizer's quantized checkpoints like nvidia/Llama-3.1-8B-Instruct-FP8 on HF.
llm = LLM(model="TinyLlama/TinyLlama-1.1B-Chat-v1.0")
# Sample prompts.
prompts = [
"Hello, my name is",
"The capital of France is",
"The future of AI is",
]
# Create a sampling params.
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
for output in llm.generate(prompts, sampling_params):
print(
f"Prompt: {output.prompt!r}, Generated text: {output.outputs[0].text!r}"
)
# Got output like
# Prompt: 'Hello, my name is', Generated text: '\n\nJane Smith. I am a student pursuing my degree in Computer Science at [university]. I enjoy learning new things, especially technology and programming'
# Prompt: 'The president of the United States is', Generated text: 'likely to nominate a new Supreme Court justice to fill the seat vacated by the death of Antonin Scalia. The Senate should vote to confirm the'
# Prompt: 'The capital of France is', Generated text: 'Paris.'
# Prompt: 'The future of AI is', Generated text: 'an exciting time for us. We are constantly researching, developing, and improving our platform to create the most advanced and efficient model available. We are'
if name == ‘main‘:
main()
···
执行后，在漫长的等待后，报了个错，访问不了huggingface.co
1 |
|
(trt) bobo@DESKTOP-K65EUBR:~/test_trt$ python test_trt.py
[2025-11-06 21:30:46] INFO config.py:54: PyTorch version 2.7.1+cu128 available.
/home/bobo/miniforge3/envs/trt/lib/python3.12/site-packages/modelopt/torch/utils/import_utils.py:32: UserWarning: Failed to import huggingface plugin due to: AttributeError(“module ‘transformers.modeling_utils’ has no attribute ‘Conv1D’”). You may ignore this warning if you do not need this plugin.
warnings.warn(
/home/bobo/miniforge3/envs/trt/lib/python3.12/site-packages/modelopt/torch/init.py:36: UserWarning: transformers version 4.57.1 is incompatible with nvidia-modelopt and may cause issues. Please install recommended version with pip install nvidia-modelopt[hf] if working with HF models.
_warnings.warn(
2025-11-06 21:30:50,629 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
[TensorRT-LLM] TensorRT LLM version: 1.0.0
[11/06/2025-21:30:50] [TRT-LLM] [I] Using LLM with PyTorch backend
[11/06/2025-21:30:50] [TRT-LLM] [W] Using default gpus_per_node: 1
[11/06/2025-21:30:50] [TRT-LLM] [I] Set nccl_plugin to None.
[11/06/2025-21:30:50] [TRT-LLM] [I] neither checkpoint_format nor checkpoint_loader were provided, checkpoint_format will be set to HF.
rank 0 using MpiPoolSession to spawn MPI processes
[2025-11-06 21:31:03] INFO config.py:54: PyTorch version 2.7.1+cu128 available.
Multiple distributions found for package optimum. Picked distribution: optimum
/home/bobo/miniforge3/envs/trt/lib/python3.12/site-packages/modelopt/torch/utils/import_utils.py:32: UserWarning: Failed to import huggingface plugin due to: AttributeError(“module ‘transformers.modeling_utils’ has no attribute ‘Conv1D’”). You may ignore this warning if you do not need this plugin.
warnings.warn(
/home/bobo/miniforge3/envs/trt/lib/python3.12/site-packages/modelopt/torch/init.py:36: UserWarning: transformers version 4.57.1 is incompatible with nvidia-modelopt and may cause issues. Please install recommended version with pip install nvidia-modelopt[hf] if working with HF models.
_warnings.warn(
2025-11-06 21:31:07,496 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
[TensorRT-LLM] TensorRT LLM version: 1.0.0
[TensorRT-LLM][INFO] Refreshed the MPI local sessiontorch_dtype is deprecated! Use dtype instead!
Loading safetensors weights in parallel: 100%|██████████| 1/1 [00:00<00:00, 61.60it/s]
Loading weights: 100%|██████████| 449/449 [00:00<00:00, 557.09it/s]
Model init total – 2.23s
[TensorRT-LLM][INFO] Number of tokens per block: 32.
[TensorRT-LLM][INFO] [MemUsageChange] Allocated 0.18 GiB for max tokens in paged KV cache (8352).
2025-11-06 21:31:12,625 - INFO - flashinfer.jit: Loading JIT ops: norm
2025-11-06 21:31:48,328 - INFO - flashinfer.jit: Finished loading JIT ops: norm
[TensorRT-LLM][INFO] Number of tokens per block: 32.
[TensorRT-LLM][INFO] [MemUsageChange] Allocated 17.72 GiB for max tokens in paged KV cache (844512).
Processed requests: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 3.15it/s]
Prompt: ‘Hello, my name is’, Generated text: ‘[Your Name] and I am a [Your Position] at [Your Company]. I am writing to express my interest in the [Job Title] position at’
Prompt: ‘The capital of France is’, Generated text: ‘Paris.\n\n2. B. C. The capital of Canada is Ottawa.\n\n3. A. C. The capital of Australia is Can’
Prompt: ‘The future of AI is’, Generated text: “bright, and it’s not just for big companies. Small businesses can also benefit from AI technology. Here are some ways:\n\n1.”
1 | 顺利跑通。 |
trtllm-serve “TinyLlama/TinyLlama-1.1B-Chat-v1.0”
1 | 直到显示： |
INFO: Uvicorn running on http://localhost:8000 (Press CTRL+C to quit)
/home/bobo/miniforge3/envs/trt/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py:2249: UnsupportedFieldAttributeWarning: The ‘validation_alias’ attribute with value ‘max_tokens’ was provided to the Field() function, which has no effect in the context it was used. ‘validation_alias’ is field-specific metadata, and can only be attached to a model field using Annotated metadata or by assignment. This may have happened because an Annotated type alias using the type statement was used, or if the Field() function was attached to a single member of a union type.
warnings.warn(
INFO: 127.0.0.1:54364 - “POST /v1/chat/completions HTTP/1.1” 200 OK
1 | 然后再起一个wsl的bash，输入 |
curl -X POST http://localhost:8000/v1/chat/completions -H “Content-Type: application/json” -H “Accept: application/json” -d ‘{
“model”: “TinyLlama/TinyLlama-1.1B-Chat-v1.0”,
“messages”:[{“role”: “system”, “content”: “You are a helpful assistant.”},
{“role”: “user”, “content”: “Where is New York? Tell me in a single sentence.”}],
“max_tokens”: 32,
“temperature”: 0
}’
1 | 得到答复："New York is a city in the northeastern United States, located on the eastern coast of the state of New York." |
{“id”:”chatcmpl-31b02f6ab4854863909850ab9688d8b1”,”object”:”chat.completion”,”created”:1762436547,”model”:”TinyLlama/TinyLlama-1.1B-Chat-v1.0”,”choices”:[{“index”:0,”message”:{“role”:”assistant”,”content”:”New York is a city in the northeastern United States, located on the eastern coast of the state of New York.”,”reasoning_content”:null,”tool_calls”:[]},”logprobs”:null,”finish_reason”:”stop”,”stop_reason”:null,”disaggregated_params”:null}],”usage”:{“prompt_tokens”:43,”total_tokens”:70,”completion_tokens”:27},”prompt_token_ids”:null}(trt)
1 |
|
trtllm-serve “deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B”
1 | 挂了~提示了一堆关于如何控制显存的信息。 |
Please refer to the TensorRT LLM documentation for information on how to control the memory usage through TensorRT LLM configuration options. Possible options include:
Model: reduce max_num_tokens and/or shard the model weights across GPUs by enabling pipeline and/or tensor parallelism
Sampler: reduce max_seq_len and/or max_attention_window_size
Initial KV cache (temporary for KV cache size estimation): reduce max_num_tokens
Drafter: reduce max_seq_len and/or max_draft_len
Additional executor resources (temporary for KV cache size estimation): reduce max_num_tokens
Model resources created during usage: reduce max_num_tokens
KV cache: reduce free_gpu_memory_fraction
Additional executor resources: reduce max_num_tokens
trtllm-serve "meta-llama/Llama-3.2-1B"
要token，放弃。
验证完成。
以test为例
添加环境新目录
conda config –add envs_dirs D:\conda\envs
查看
conda config –show envs_dirs
导出：
conda env export > test_env.yml
退出环境然后执行删除：
conda remove -n test –all
重建环境
conda env create -n test -f test_env.yml
验证
conda info –envs
包缓冲迁移
修改.conda文件或者直接执行：
conda config –add pkgs_dirs D:\conda\pkgs
conda config –remove pkgs_dirs C:\Users\xxxx.conda\pkgs
修改后文件内容
1 | channels: |
在/etc/docker下创建daemon.json文件，内容如下
1 | { |
然后 sudo systemctl restart docker
测试一下：
docker pull busybox
是可以的
问：如果时间有限，需要放弃一些过时算法，请给所有你知道的算法排个序，哪些优先放弃
在时间有限的情况下，可以按照以下优先级放弃或延后学习某些算法。排序基于实际应用频率、替代算法的有效性以及基础重要性（从最优先放弃到尽量保留）：
第一优先级：可放弃的算法
Apriori算法（关联规则挖掘）：计算效率低，FP-Growth 或深度学习（如序列模型）更高效。
LVQ（学习向量量化）：原型聚类方法，实际应用极少，K-Means 更通用。
ID3决策树：仅支持离散特征，已被 C4.5 和 CART（支持连续特征）取代。
传统贝叶斯网络结构学习：计算复杂且依赖领域知识，实际中更常用概率图模型工具包（如 PyMC3）。
K-Medoids：类似 K-Means，但对噪声敏感且计算成本高，优先学习 K-Means。
第二优先级：视领域需求选择性放弃
3. 特定领域替代性强的算法
LDA（线性判别分析）：降维和分类任务中，PCA 更通用，除非需要类别分离指导降维。
传统遗传算法：优化问题中，梯度下降或贝叶斯优化更高效，除非研究进化计算。
HMM（隐马尔可夫模型）：语音识别中被 RNN/Transformer 取代，但生物信息学仍可能用到。
传统马尔可夫链：简单序列建模中，RNN/LSTM 更强大。
传统核方法（如自定义核函数）：深度学习兴起后，核技巧的重要性下降。
尽量保留的核心算法
5. 基础且广泛应用的算法
逻辑回归：分类任务的基础，可解释性强。
线性 SVM（LinearSVC）：高维数据分类高效，工业界常用。
决策树（CART）：随机森林和 GBDT 的基础。
KNN：惰性学习代表，适合实时场景。
随机森林 & 梯度提升（XGBoost/LightGBM）：结构化数据的主流模型。
K-Means & PCA：无监督学习的基础工具。
神经网络基础（MLP、CNN、RNN）：深度学习的核心。
协同过滤：推荐系统的经典方法。
ARIMA：时间序列分析（若涉及该领域需保留）。
总结建议
放弃：单层感知机、Apriori、LVQ、ID3、SOM。
选择性放弃：LDA、传统遗传算法、HMM（除非领域相关）、传统核方法。
保留核心：逻辑回归、SVM、决策树、集成学习（RF/GBDT）、KNN、PCA、神经网络。
实践优先：通过工具库（如 Scikit-learn）快速应用，再深入原理。
注意：即使放弃某些算法，仍需了解其基本思想（如“关联规则挖掘”或“进化计算”），以便需要时快速补足。
上次说完ollama部署，现在我们试试将deepseek接入到系统里，使用方法就是调用http接口
首先 ollama serve
然后增加调用
curl -X POST http://localhost:11434/api/generate -H “Content-Type: application/json” -d “{"model":"deepseek-r1:1.5b","prompt":"天空为什么是蓝色的？"}”
安装boost用：
apt install libboost-all-dev
安装log4cpp
sudo apt install liblog4cpp5-dev
安装docker
sudo apt install docker.io
上传my_mongo.tar,然后
docker load < my_mongo.tar
首先是空间需求，Qwen系列是使用DeepSeek蒸馏后的Qwen模型，Llama是DeepSeek蒸馏后的Llama模型。
ollama run deepseek-r1:1.5b
ollama run deepseek-r1:7b
ollama run deepseek-r1:8b
ollama run deepseek-r1:14b
ollama run deepseek-r1:32b
ollama run deepseek-r1:70b
ollama run deepseek-r1:671b
ollama serve
可以用chatbox连接访问
下载了ollama和本地模型后，导入本地模型的方法：
创建一个modelfile.mf文件，内容为：
FROM ./xxxx.gguf
然后ollama create deepseek-r1_local:7b -f modelfile.mf
就可以通过ollama ls 看到新导入的镜像，然后用
ollama run 镜像名就可以运行了