feat: 外露推理调优参数并支持 device: auto
把此前只能改代码的参数摊到 config.yaml,交付后按客户机器调整: use_tensorrt / precision / enable_cinn / mkldnn_cache_capacity text_det_input_shape / text_rec_input_shape device 新增 auto:启动时探测可用的 CUDA 设备,有卡用 GPU、没卡用 CPU, 同一份配置在不同机器之间搬运不用改。探测不到卡是预期路径,报 INFO; 显式写 gpu 却用不了是配置问题,报 WARNING——便于区分这两种情况。 设备名校验加了两层(配置层拦拼写错误 + 运行时最后一道防线)。起因是实测 踩到:冻结包里的代码比 config.yaml 旧时,字符串 auto 被原样传给 PaddleX, 抛出的 AssertionError 完全看不出是配置写错了。 text_recognition_batch_size 默认值 8 -> 32:实测 1.png 同分辨率 23.87s -> 19.03s,框数不变、字符数 2240 -> 2241(差异仅英文分词空格与个别低置信字符)。 顺带把参数装配收敛到 OCRService.from_settings():启动、下载脚本、压测脚本 原本各自手工罗列参数,压测脚本已经漏掉了新配置项。 Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
+92
-8
@@ -21,9 +21,66 @@ rec_model_name: PP-OCRv4_mobile_rec
|
||||
# ⚠️ PP-OCRv4_mobile_det 检测不出密集小字:1.png 只检出 46 框(v5 为 339 框),
|
||||
# 字符数只剩约 10%。仅适合文本稀疏的图片(票据/证件/少数字段)。
|
||||
|
||||
# 每个线程会独立占用一份 OCR 模型内存。
|
||||
# ── 交付包预置的模型 ─────────────────────────────────────────
|
||||
# scripts/download_models.py 会把下面八套(v4/v5 × 服务端/移动端 × 检测/识别)
|
||||
# 全部下载到 model_dir/official_models/,共约 398MB。
|
||||
# 交付后断网也能直接改模型名切换,不必重新下载。
|
||||
#
|
||||
# 检测 识别
|
||||
# PP-OCRv5_server_det 85MB PP-OCRv5_server_rec 82MB
|
||||
# PP-OCRv5_mobile_det 5MB PP-OCRv5_mobile_rec 17MB
|
||||
# PP-OCRv4_server_det 109MB PP-OCRv4_server_rec 87MB
|
||||
# PP-OCRv4_mobile_det 5MB PP-OCRv4_mobile_rec 11MB
|
||||
#
|
||||
# 怎么选(按客户机器分档):
|
||||
# 低配 CPU(4 核以下/内存吃紧):PP-OCRv5_mobile_det + PP-OCRv4_mobile_rec
|
||||
# 常规 CPU(本机默认) :PP-OCRv5_server_det + PP-OCRv4_mobile_rec
|
||||
# 有 GPU :PP-OCRv5_server_det + PP-OCRv5_server_rec + use_tensorrt
|
||||
# 要最高精度、不在乎速度 :PP-OCRv5_server_det + PP-OCRv5_server_rec
|
||||
#
|
||||
# ⚠️ PP-OCRv4_mobile_det 在密集文本上会大面积漏检:1.png 只检出 46 框(v5 为 339 框),
|
||||
# 字符数只剩约 10%。预置它是为了完整性,别用在表格/文档类图片上。
|
||||
# ⚠️ 换组合前先用真实图片对比「字符数」,不要只看速度(见 README)。
|
||||
|
||||
# 每个线程会独立占用一份 OCR 模型内存(实测每多一个 worker 约 +105MB)。
|
||||
# 识别推理本身接近单线程,所以并发上传时加大 worker 数可以得到接近线性的吞吐提升。
|
||||
ocr_workers: 1
|
||||
device: cpu
|
||||
|
||||
# ── 推理设备 ──────────────────────────────────────────────────
|
||||
# 取值:cpu / gpu / gpu:0 / auto / npu 等,具体可用性取决于 PaddlePaddle 编译版本。
|
||||
#
|
||||
# auto = 启动时探测有没有可用的 CUDA 设备,有就用 GPU、没有就用 CPU,
|
||||
# 日志里会明确打出实际选了什么(INFO 级)。适合同一份配置在不同客户机
|
||||
# 之间搬运:不用为了换机器改这一行。GPU 版分发包默认就是 auto。
|
||||
# cpu = 固定用 CPU,永远不碰 GPU。
|
||||
# gpu = 固定用 GPU(gpu:1 指定第 2 块卡)。如果这台机器不保证有卡,别用它。
|
||||
#
|
||||
# ⚠️ 光把这里改成 gpu 是启不了 GPU 的。PyPI 上的 paddlepaddle 是纯 CPU 版,
|
||||
# 必须安装与本机 CUDA 匹配的 paddlepaddle-gpu(版本同样要固定 3.2.x,
|
||||
# 原因见文件末尾 enable_mkldnn 的说明)。
|
||||
# 详见 README「CPU / GPU 切换」。
|
||||
#
|
||||
# 兜底行为:配置了 gpu 但 GPU 不可用时,启动会打印中文警告并自动回落 CPU,
|
||||
# 不会让服务起不来(装的是纯 CPU 版,或装了 GPU 版但机器没有可用 N 卡/驱动,
|
||||
# 两种情况都能兜住)。上线前请确认日志里「加载 OCR 引擎」那一行打印的
|
||||
# device 是不是你期望的。
|
||||
#
|
||||
# 交付时 CPU / GPU 是两个产物(原因和打包命令见 README「CPU / GPU 切换」):
|
||||
# CPU 包 -> dist\ ,本项为 cpu
|
||||
# GPU 包 -> dist-gpu\ ,打包脚本已自动把本项写成 gpu
|
||||
device: auto
|
||||
|
||||
# 是否启用 TensorRT(只在 device=gpu 时生效;device=cpu 时会打印警告并忽略)。
|
||||
# 开启后可显著加速,但首次推理会花时间做引擎构建/调优,且换机器要重建。
|
||||
use_tensorrt: false
|
||||
|
||||
# 推理精度:fp32 / fp16。fp16 只在 GPU(尤其 TensorRT)上生效,CPU 下无影响。
|
||||
# fp16 更快、显存占用更低,但精度可能略降,上线前请用真实图片核对字符数。
|
||||
precision: fp32
|
||||
|
||||
# 是否启用 CINN 编译器(面向国产加速卡等场景,普通 CPU/GPU 保持 false 即可)。
|
||||
enable_cinn: false
|
||||
|
||||
lang: ch
|
||||
|
||||
# 识别速度/精度参数。
|
||||
@@ -33,23 +90,47 @@ lang: ch
|
||||
# side=320 -> 检出 0 框 / 0 字符 (完全失效)
|
||||
# 原因:分辨率下采样后小字高度不足,检测模型看不见。这是断崖式失效,不是平滑降精度。
|
||||
# 结论:大图保持 640 或更高;480/512 只适合小尺寸、大字号的图片。
|
||||
#
|
||||
# 往上调则是拿速度换召回(1.png 实测,v5 server det + v4 mobile rec):
|
||||
# side=1024 -> 检 428 框 / 2240 字符,约 19s ← 当前值
|
||||
# side= 768 -> 检 405 框 / 2191 字符,约 16s
|
||||
# side= 640 -> 检 339 框 / 1746 字符,约 11s (字符数少 22%)
|
||||
# 也就是说 1024 比 640 多召回约 28% 的字符,代价是慢 8 秒左右。
|
||||
# 低配客户机可以降到 768 折中;只要速度就 640,但要接受漏字。
|
||||
text_det_limit_side_len: 1024
|
||||
# 检测缩放类型
|
||||
text_det_limit_type: max
|
||||
# 固定检测输入尺寸,形如 [3, 960, 960]。null = 用上面的 limit_side_len 动态缩放。
|
||||
# 一般不用动;只有在需要可复现的固定算力占用时才设。
|
||||
text_det_input_shape: null
|
||||
# 检测阈值
|
||||
text_det_thresh: 0.3
|
||||
text_det_box_thresh: 0.6
|
||||
text_det_unclip_ratio: 1.5
|
||||
# 识别置信度阈值
|
||||
text_rec_score_thresh: 0.0
|
||||
# 识别批量大小
|
||||
text_recognition_batch_size: 8
|
||||
# 固定识别输入尺寸,形如 [3, 48, 320]。null = 按文本框宽高比动态填充(推荐)。
|
||||
text_rec_input_shape: null
|
||||
# 识别批量大小。框越多收益越大,是性价比最高的旋钮。
|
||||
# 1.png(1920x1080) 实测:
|
||||
# side=1024:batch=8 -> 23.87s / 2240 字符,batch=32 -> 19.03s / 2241 字符
|
||||
# side= 640:batch=8 -> 19.87s,batch=32 -> 11.22s(框数同为 339)
|
||||
# 结论:调到 32 就能拿到大部分收益;再往上收益饱和(48 反而略慢,属噪声)。
|
||||
#
|
||||
# ⚠️ 批次大小会轻微影响识别结果(不是逐字节一致)。1.png 上 2240 -> 2241 字符,
|
||||
# 官方样例图上 253 -> 250,差异只出现在英文分词空格和个别低置信字符上。
|
||||
# 框数(segments)完全一致。要切换 batch 请先核对真实图片的字符数。
|
||||
# 批量越大峰值内存越高,低配机器(内存 < 4GB)建议用 16。
|
||||
text_recognition_batch_size: 32
|
||||
|
||||
# CPU 推理线程数(Paddle 的 cpu_math_library_num_threads)。
|
||||
# 取值为正整数,或 max。
|
||||
# max = 吃满所有逻辑核心,例如 20 逻辑核机器上解析为 20(本机即为 20)。
|
||||
# 注意:max 不一定最快。超线程会争抢,超过物理核心数后收益递减,
|
||||
# 而且会挤占同机其它服务。建议用 benchmark_ocr.py 实测后再定。
|
||||
# 取值为正整数,或 max(吃满所有逻辑核心,20 逻辑核机器上解析为 20)。
|
||||
#
|
||||
# 实测结论:开启 oneDNN 之后这个参数几乎不影响耗时。官方样例图(896x528)实测:
|
||||
# 1 -> 7.25s 4 -> 6.82s 8 -> 7.22s 14 -> 7.29s 20 -> 7.25s
|
||||
# 差异都在测量噪声内。也就是说识别推理已经跑在多线程上,再加线程不再获益。
|
||||
# 保持 1 即可,省下来的核留给 ocr_workers 或同机其它服务。
|
||||
# (README 里「建议调到 max」是 oneDNN 之前的结论,已过期。)
|
||||
cpu_threads: 1
|
||||
|
||||
# 是否启用 oneDNN(MKLDNN)CPU 加速。
|
||||
@@ -63,6 +144,9 @@ cpu_threads: 1
|
||||
# (同图同参数:3.3.1 检出 339 框,3.2.2 检出 452 框),上线前请用真实数据核对。
|
||||
enable_mkldnn: true
|
||||
|
||||
# oneDNN 算子缓存容量,只在 enable_mkldnn: true 时生效,一般不用动。
|
||||
mkldnn_cache_capacity: 10
|
||||
|
||||
# 日志级别:DEBUG / INFO / WARNING / ERROR
|
||||
log_level: INFO
|
||||
|
||||
|
||||
+1
-16
@@ -22,22 +22,7 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
field_separators = tuple(settings.extract_separators or (":", ":"))
|
||||
|
||||
def create_ocr() -> OCRService:
|
||||
return OCRService(
|
||||
lang=settings.lang,
|
||||
device=settings.device,
|
||||
model_dir=settings.model_dir,
|
||||
det_model_name=settings.det_model_name,
|
||||
rec_model_name=settings.rec_model_name,
|
||||
text_det_limit_side_len=settings.text_det_limit_side_len,
|
||||
text_det_limit_type=settings.text_det_limit_type,
|
||||
text_det_thresh=settings.text_det_thresh,
|
||||
text_det_box_thresh=settings.text_det_box_thresh,
|
||||
text_det_unclip_ratio=settings.text_det_unclip_ratio,
|
||||
text_rec_score_thresh=settings.text_rec_score_thresh,
|
||||
text_recognition_batch_size=settings.text_recognition_batch_size,
|
||||
cpu_threads=settings.cpu_threads_resolved,
|
||||
enable_mkldnn=settings.enable_mkldnn,
|
||||
)
|
||||
return OCRService.from_settings(settings)
|
||||
|
||||
tasks = OCRTaskQueue(create_ocr, worker_count=settings.ocr_workers)
|
||||
app = FastAPI(title="OCR Server", version="0.1.0")
|
||||
|
||||
@@ -12,6 +12,27 @@ PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
CPU_THREADS_MAX = "max"
|
||||
|
||||
# 设备名形如 cpu / gpu / gpu:0 / npu。取值的合法集合来自 PaddleX 的
|
||||
# paddlex/utils/device.py:SUPPORTED_DEVICE_TYPE,外加本项目自有的 auto。
|
||||
# 在配置层就挡住拼错的值:否则它会一路传到 PaddleX 内部,
|
||||
# 抛出一个没有任何上下文的 AssertionError(实测踩过)。
|
||||
DEVICE_PATTERN = re.compile(r"^[a-z_]+(:\d+)?$")
|
||||
SUPPORTED_DEVICE_TYPES = (
|
||||
"cpu",
|
||||
"gpu",
|
||||
"xpu",
|
||||
"npu",
|
||||
"mlu",
|
||||
"gcu",
|
||||
"dcu",
|
||||
"iluvatar_gpu",
|
||||
"metax_gpu",
|
||||
)
|
||||
AUTO_DEVICE = "auto"
|
||||
|
||||
# 与 paddleocr._constants.SUPPORTED_PRECISION_LIST 保持一致。
|
||||
PRECISION_CHOICES = ("fp32", "fp16")
|
||||
|
||||
|
||||
def _runtime_root() -> Path:
|
||||
if getattr(sys, "frozen", False):
|
||||
@@ -88,22 +109,37 @@ class Settings(BaseModel):
|
||||
port: int = Field(default=8000, ge=1, le=65535)
|
||||
upload_dir: Path = _runtime_root() / "data" / "uploads"
|
||||
model_dir: Path = _runtime_root() / "models"
|
||||
# 下面这些类的默认值与 config.yaml 里的推荐配置保持一致,作为「配置文件缺失」时的兜底。
|
||||
det_model_name: str = "PP-OCRv5_server_det"
|
||||
rec_model_name: str = "PP-OCRv5_server_rec"
|
||||
rec_model_name: str = "PP-OCRv4_mobile_rec"
|
||||
ocr_workers: int = Field(default=1, ge=1, le=32)
|
||||
max_upload_mb: int = Field(default=10, ge=1, le=100)
|
||||
max_image_pixels: int = Field(default=20_000_000, ge=1)
|
||||
# 推理设备。cpu / gpu / gpu:0 / npu 等,具体可用性取决于 PaddlePaddle 编译版本。
|
||||
# `auto` 表示启动时探测可用的 CUDA 设备自行选择,适合同一份配置在有无显卡的
|
||||
# 机器之间搬运。注意:PyPI 上的 paddlepaddle 是纯 CPU 版,只有
|
||||
# paddlepaddle-gpu 才能跑 GPU。
|
||||
device: str = "cpu"
|
||||
lang: str = "ch"
|
||||
text_det_limit_side_len: int = Field(default=960, ge=32, le=4096)
|
||||
text_det_limit_side_len: int = Field(default=1024, ge=32, le=4096)
|
||||
text_det_limit_type: Literal["max", "min"] = "max"
|
||||
text_det_thresh: float = Field(default=0.3, ge=0.0, le=1.0)
|
||||
text_det_box_thresh: float = Field(default=0.6, ge=0.0, le=1.0)
|
||||
text_det_unclip_ratio: float = Field(default=1.5, ge=0.1, le=5.0)
|
||||
# 固定检测输入尺寸,形如 [3, 960, 960]。默认 null 表示用 text_det_limit_side_len 动态缩放。
|
||||
text_det_input_shape: list[int] | None = None
|
||||
text_rec_score_thresh: float = Field(default=0.0, ge=0.0, le=1.0)
|
||||
text_recognition_batch_size: int = Field(default=1, ge=1, le=64)
|
||||
cpu_threads: int | str = 10
|
||||
enable_mkldnn: bool = False
|
||||
# 固定识别输入尺寸,形如 [3, 48, 320]。默认 null 表示按文本框宽高比动态填充。
|
||||
text_rec_input_shape: list[int] | None = None
|
||||
text_recognition_batch_size: int = Field(default=32, ge=1, le=64)
|
||||
cpu_threads: int | str = 1
|
||||
enable_mkldnn: bool = True
|
||||
# oneDNN 算子缓存容量,仅在 enable_mkldnn 打开时生效。
|
||||
mkldnn_cache_capacity: int = Field(default=10, ge=1)
|
||||
# 以下三项只在 GPU 上生效;device 为 cpu 时会被忽略。
|
||||
use_tensorrt: bool = False
|
||||
precision: Literal["fp32", "fp16"] = "fp32"
|
||||
enable_cinn: bool = False
|
||||
extract_fields: list[FieldPreset] = Field(default_factory=list)
|
||||
extract_separators: list[str] = Field(default_factory=lambda: [":", ":"])
|
||||
extract_include_empty: bool = False
|
||||
@@ -130,6 +166,36 @@ class Settings(BaseModel):
|
||||
raise ValueError("cpu_threads 取值范围为 1~256")
|
||||
return value
|
||||
|
||||
@field_validator("device")
|
||||
@classmethod
|
||||
def _validate_device(cls, value: Any) -> str:
|
||||
name = str(value).strip().lower()
|
||||
if not DEVICE_PATTERN.match(name):
|
||||
raise ValueError(
|
||||
f"device 格式无效 {value!r},应为 cpu / gpu / gpu:0 / auto / npu 这类写法"
|
||||
)
|
||||
device_type = name.split(":", 1)[0]
|
||||
if device_type not in (*SUPPORTED_DEVICE_TYPES, AUTO_DEVICE):
|
||||
allowed = "、".join((*SUPPORTED_DEVICE_TYPES, AUTO_DEVICE))
|
||||
raise ValueError(
|
||||
f"device 取值无效 {value!r}。可用值:{allowed};"
|
||||
"也可以带编号,如 gpu:0 表示第 0 块卡。"
|
||||
)
|
||||
return name
|
||||
|
||||
@field_validator("text_det_input_shape", "text_rec_input_shape")
|
||||
@classmethod
|
||||
def _validate_input_shape(cls, value: Any) -> list[int] | None:
|
||||
if value is None:
|
||||
return None
|
||||
shape = list(value)
|
||||
if len(shape) != 3 or any(
|
||||
isinstance(item, bool) or not isinstance(item, int) or item < 1
|
||||
for item in shape
|
||||
):
|
||||
raise ValueError(f"输入尺寸必须是 3 个正整数,例如 [3, 48, 320],收到 {value!r}")
|
||||
return shape
|
||||
|
||||
@property
|
||||
def cpu_threads_resolved(self) -> int:
|
||||
"""实际传给 Paddle 的线程数;`max` 在这里展开成核心数。"""
|
||||
|
||||
+176
-3
@@ -1,4 +1,5 @@
|
||||
import importlib
|
||||
import importlib.util
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
@@ -7,7 +8,11 @@ import threading
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable
|
||||
|
||||
from ocr_server.config import resolve_cpu_threads
|
||||
from ocr_server.config import (
|
||||
AUTO_DEVICE,
|
||||
SUPPORTED_DEVICE_TYPES,
|
||||
resolve_cpu_threads,
|
||||
)
|
||||
|
||||
logger = logging.getLogger("ocr_server.ocr")
|
||||
|
||||
@@ -17,6 +22,24 @@ ProgressCallback = Callable[[int, str], None]
|
||||
# NotImplementedError(ConvertPirAttribute2RuntimeAttribute not support)。
|
||||
KNOWN_MKLDNN_BROKEN_PREFIXES = ("3.3.",)
|
||||
|
||||
# 交付包预置的模型。scripts/download_models.py 会把这几套全部下载到 model_dir,
|
||||
# 之后在客户机器上只改 config.yaml 的模型名即可离线切换,无需联网。
|
||||
# 覆盖 v4/v5 两个代际的服务端与移动端,共 4 检测 + 4 识别。
|
||||
# ⚠️ PP-OCRv4_mobile_det 在密集文本上会大面积漏检(1.png 只检出 46 框,v5 为 339 框),
|
||||
# 预置它是为了完整性,不要用在表格/文档类图片上。
|
||||
STAGED_DET_MODELS = (
|
||||
"PP-OCRv5_server_det",
|
||||
"PP-OCRv5_mobile_det",
|
||||
"PP-OCRv4_server_det",
|
||||
"PP-OCRv4_mobile_det",
|
||||
)
|
||||
STAGED_REC_MODELS = (
|
||||
"PP-OCRv5_server_rec",
|
||||
"PP-OCRv5_mobile_rec",
|
||||
"PP-OCRv4_server_rec",
|
||||
"PP-OCRv4_mobile_rec",
|
||||
)
|
||||
|
||||
|
||||
class OCRNotAvailable(RuntimeError):
|
||||
pass
|
||||
@@ -45,10 +68,16 @@ class OCRService:
|
||||
text_det_thresh: float = 0.3,
|
||||
text_det_box_thresh: float = 0.6,
|
||||
text_det_unclip_ratio: float = 1.5,
|
||||
text_det_input_shape: list[int] | None = None,
|
||||
text_rec_score_thresh: float = 0.0,
|
||||
text_rec_input_shape: list[int] | None = None,
|
||||
text_recognition_batch_size: int = 1,
|
||||
cpu_threads: int | str = 10,
|
||||
enable_mkldnn: bool = False,
|
||||
mkldnn_cache_capacity: int = 10,
|
||||
use_tensorrt: bool = False,
|
||||
precision: str = "fp32",
|
||||
enable_cinn: bool = False,
|
||||
) -> None:
|
||||
self.lang = lang
|
||||
self.device = device
|
||||
@@ -60,16 +89,55 @@ class OCRService:
|
||||
self._text_det_thresh = text_det_thresh
|
||||
self._text_det_box_thresh = text_det_box_thresh
|
||||
self._text_det_unclip_ratio = text_det_unclip_ratio
|
||||
self._text_det_input_shape = text_det_input_shape
|
||||
self._text_rec_score_thresh = text_rec_score_thresh
|
||||
self._text_rec_input_shape = text_rec_input_shape
|
||||
self._text_recognition_batch_size = text_recognition_batch_size
|
||||
self._cpu_threads = resolve_cpu_threads(cpu_threads)
|
||||
self._enable_mkldnn = bool(enable_mkldnn)
|
||||
self._mkldnn_cache_capacity = int(mkldnn_cache_capacity)
|
||||
self._use_tensorrt = bool(use_tensorrt)
|
||||
self._precision = precision
|
||||
self._enable_cinn = bool(enable_cinn)
|
||||
self._effective_device = self.device
|
||||
self._engine: Any | None = None
|
||||
self._lock = threading.Lock()
|
||||
self._inference_lock = threading.Lock()
|
||||
self._progress_callback: ProgressCallback | None = None
|
||||
self._progress_reached = 0
|
||||
|
||||
@classmethod
|
||||
def from_settings(cls, settings: Any, **overrides: Any) -> "OCRService":
|
||||
"""按 Settings 构造实例。
|
||||
|
||||
集中在这里,避免启动、下载脚本、压测脚本各自手工罗列参数时漏掉新配置项。
|
||||
`settings` 只做鸭子类型使用,不引入对 config 模块的硬依赖。
|
||||
"""
|
||||
arguments: dict[str, Any] = {
|
||||
"lang": settings.lang,
|
||||
"device": settings.device,
|
||||
"model_dir": settings.model_dir,
|
||||
"det_model_name": settings.det_model_name,
|
||||
"rec_model_name": settings.rec_model_name,
|
||||
"text_det_limit_side_len": settings.text_det_limit_side_len,
|
||||
"text_det_limit_type": settings.text_det_limit_type,
|
||||
"text_det_thresh": settings.text_det_thresh,
|
||||
"text_det_box_thresh": settings.text_det_box_thresh,
|
||||
"text_det_unclip_ratio": settings.text_det_unclip_ratio,
|
||||
"text_det_input_shape": settings.text_det_input_shape,
|
||||
"text_rec_score_thresh": settings.text_rec_score_thresh,
|
||||
"text_rec_input_shape": settings.text_rec_input_shape,
|
||||
"text_recognition_batch_size": settings.text_recognition_batch_size,
|
||||
"cpu_threads": settings.cpu_threads_resolved,
|
||||
"enable_mkldnn": settings.enable_mkldnn,
|
||||
"mkldnn_cache_capacity": settings.mkldnn_cache_capacity,
|
||||
"use_tensorrt": settings.use_tensorrt,
|
||||
"precision": settings.precision,
|
||||
"enable_cinn": settings.enable_cinn,
|
||||
}
|
||||
arguments.update(overrides)
|
||||
return cls(**arguments)
|
||||
|
||||
def _get_engine(self) -> Any:
|
||||
if self._engine is None:
|
||||
with self._lock:
|
||||
@@ -135,16 +203,26 @@ class OCRService:
|
||||
"DependencyError", (Exception,), {}
|
||||
)
|
||||
except ImportError as exc:
|
||||
# paddle 不在 pyproject 的依赖里(CPU/GPU 互斥、CUDA 通道要按机器选),
|
||||
# 所以「PaddleOCR 未安装」和「paddle 没装」要分开提示,
|
||||
# 否则 `uv sync` 之后报「请先安装项目依赖」会让人找不着北。
|
||||
if importlib.util.find_spec("paddle") is None:
|
||||
raise OCRNotAvailable(
|
||||
"未安装 paddlepaddle。它不在项目依赖里,需要按机器单独安装:"
|
||||
"python scripts/install_paddle.py(自动识别显卡)"
|
||||
",或指定 --cpu / --gpu。详见 README「CPU / GPU 切换」。"
|
||||
) from exc
|
||||
raise OCRNotAvailable(
|
||||
"PaddleOCR 未安装,请先安装项目依赖"
|
||||
) from exc
|
||||
with self._process_model_lock:
|
||||
self._resolve_device()
|
||||
logger.info(
|
||||
"加载 OCR 引擎: det=%s rec=%s device=%s cpu_threads=%d "
|
||||
"enable_mkldnn=%s model_dir=%s",
|
||||
self._det_model_name,
|
||||
self._rec_model_name,
|
||||
self.device,
|
||||
self._effective_device,
|
||||
self._cpu_threads,
|
||||
self._enable_mkldnn,
|
||||
self.model_dir,
|
||||
@@ -152,11 +230,12 @@ class OCRService:
|
||||
self._warn_if_mkldnn_unsupported()
|
||||
self._engine = PaddleOCR(
|
||||
**self._model_arguments(),
|
||||
device=self.device,
|
||||
**self._device_arguments(),
|
||||
use_doc_orientation_classify=False,
|
||||
use_doc_unwarping=False,
|
||||
use_textline_orientation=False,
|
||||
enable_mkldnn=self._enable_mkldnn,
|
||||
mkldnn_cache_capacity=self._mkldnn_cache_capacity,
|
||||
text_det_limit_side_len=self._text_det_limit_side_len,
|
||||
text_det_limit_type=self._text_det_limit_type,
|
||||
text_det_thresh=self._text_det_thresh,
|
||||
@@ -206,6 +285,100 @@ class OCRService:
|
||||
arguments[argument_name] = str(local_path)
|
||||
return arguments
|
||||
|
||||
def _device_arguments(self) -> dict[str, Any]:
|
||||
"""组装设备相关参数。需先调用 _resolve_device()。"""
|
||||
arguments: dict[str, Any] = {
|
||||
"device": self._effective_device,
|
||||
"use_tensorrt": self._use_tensorrt,
|
||||
"precision": self._precision,
|
||||
"enable_cinn": self._enable_cinn,
|
||||
}
|
||||
# None 就是这两项的默认值,不传更稳妥:PaddleOCR 只在非空时才下发到 pipeline。
|
||||
if self._text_det_input_shape is not None:
|
||||
arguments["text_det_input_shape"] = list(self._text_det_input_shape)
|
||||
if self._text_rec_input_shape is not None:
|
||||
arguments["text_rec_input_shape"] = list(self._text_rec_input_shape)
|
||||
return arguments
|
||||
|
||||
@staticmethod
|
||||
def _gpu_unavailable_reason() -> str | None:
|
||||
"""GPU 不可用时返回原因描述,可用时返回 None。
|
||||
|
||||
只查 is_compiled_with_cuda() 不够:那是编译期属性,装了 paddlepaddle-gpu
|
||||
但机器没有 NVIDIA 驱动时会返回 True,直到构造引擎才报错。所以还要探一次
|
||||
真实可见的设备数。
|
||||
"""
|
||||
try:
|
||||
import paddle
|
||||
except ImportError:
|
||||
return "paddlepaddle 未安装"
|
||||
version = getattr(paddle, "__version__", "unknown")
|
||||
if not paddle.device.is_compiled_with_cuda():
|
||||
return f"当前 PaddlePaddle {version} 是纯 CPU 版本"
|
||||
try:
|
||||
count = int(paddle.device.cuda.device_count())
|
||||
except Exception as exc: # CUDA 初始化失败的原因很杂,统一兜住
|
||||
return f"CUDA 初始化失败({exc})"
|
||||
if count < 1:
|
||||
return "PaddlePaddle 支持 CUDA,但系统里看不到可用的 NVIDIA 设备或驱动"
|
||||
return None
|
||||
|
||||
def _resolve_device(self) -> None:
|
||||
"""确定真正可用的设备,避免配置了 GPU 但环境不支持时服务直接起不来。
|
||||
|
||||
`device: auto` 会探测可用的 CUDA 设备自行选择,适合同一份配置在
|
||||
「有卡」和「没卡」的机器之间来回搬的场景。探测失败时的日志是 INFO,
|
||||
因为那是预期行为;而显式写了 `device: gpu` 却用不了是配置问题,报 WARNING。
|
||||
"""
|
||||
device = self.device
|
||||
if device == AUTO_DEVICE:
|
||||
reason = self._gpu_unavailable_reason()
|
||||
if reason is None:
|
||||
device = "gpu"
|
||||
logger.info("device=auto:检测到可用的 CUDA 设备,使用 GPU")
|
||||
else:
|
||||
device = "cpu"
|
||||
logger.info("device=auto:没有可用的 GPU(%s),使用 CPU", reason)
|
||||
elif device.startswith("gpu"):
|
||||
reason = self._gpu_unavailable_reason()
|
||||
if reason:
|
||||
logger.warning(
|
||||
"配置 device=%s,但 GPU 无法启用:%s。"
|
||||
"需安装与本机 CUDA 匹配的 paddlepaddle-gpu(版本同样要固定 3.2.x,"
|
||||
"详见 README「CPU / GPU 切换」)。本次自动回落 CPU 运行。"
|
||||
"如果这台机器本来就不一定有卡,可以改用 device: auto。",
|
||||
device,
|
||||
reason,
|
||||
)
|
||||
device = "cpu"
|
||||
self._effective_device = self._finalize_device(device)
|
||||
if self._use_tensorrt and not self._effective_device.startswith("gpu"):
|
||||
logger.warning(
|
||||
"use_tensorrt 已开启,但生效设备是 %s;"
|
||||
"TensorRT 只在 GPU 上生效,本次忽略该项。",
|
||||
self._effective_device,
|
||||
)
|
||||
self._use_tensorrt = False
|
||||
|
||||
@staticmethod
|
||||
def _finalize_device(device: str) -> str:
|
||||
"""最后一道防线:绝不把 PaddleX 不认识的值传下去。
|
||||
|
||||
配置层已经校验过一次,但**冻结包里的代码可能比 config.yaml 旧**——实测踩过:
|
||||
旧 exe 配上新配置里的 `device: auto`,字符串原样传给 PaddleX,最后在
|
||||
`paddlex/utils/device.py` 里抛 AssertionError,日志里看不出是配置写错了。
|
||||
这里兜住并给出可读的报错。
|
||||
"""
|
||||
if device.split(":", 1)[0] in SUPPORTED_DEVICE_TYPES:
|
||||
return device
|
||||
logger.error(
|
||||
"device=%s 不是 PaddleX 支持的设备类型(可用:%s),本次回落 CPU 运行。"
|
||||
"请检查 config.yaml 的 device 取值。",
|
||||
device,
|
||||
"、".join(SUPPORTED_DEVICE_TYPES),
|
||||
)
|
||||
return "cpu"
|
||||
|
||||
@classmethod
|
||||
def _is_complete_model(cls, model_path: Path) -> bool:
|
||||
return model_path.is_dir() and all(
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
"""Settings 新参数的校验,以及 GPU 不可用时的兜底行为。"""
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from ocr_server.config import Settings
|
||||
from ocr_server.ocr import OCRService
|
||||
|
||||
|
||||
def test_accepts_tuning_overrides() -> None:
|
||||
settings = Settings(
|
||||
use_tensorrt=True,
|
||||
precision="fp16",
|
||||
enable_cinn=False,
|
||||
mkldnn_cache_capacity=32,
|
||||
text_det_input_shape=[3, 960, 960],
|
||||
text_rec_input_shape=[3, 48, 320],
|
||||
)
|
||||
assert settings.use_tensorrt is True
|
||||
assert settings.precision == "fp16"
|
||||
assert settings.mkldnn_cache_capacity == 32
|
||||
assert settings.text_det_input_shape == [3, 960, 960]
|
||||
assert settings.text_rec_input_shape == [3, 48, 320]
|
||||
|
||||
|
||||
def test_optional_shapes_default_to_none() -> None:
|
||||
settings = Settings(text_det_input_shape=None, text_rec_input_shape=None)
|
||||
assert settings.text_det_input_shape is None
|
||||
assert settings.text_rec_input_shape is None
|
||||
|
||||
|
||||
def test_device_is_normalised() -> None:
|
||||
assert Settings(device=" GPU:0 ").device == "gpu:0"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("device", ["", "gpu:", "gpu-0", "0", "cuda device"])
|
||||
def test_rejects_malformed_device(device: str) -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
Settings(device=device)
|
||||
|
||||
|
||||
def test_rejects_unknown_precision() -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
Settings(precision="int8")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shape", [[3, 48], [3, 48, 320, 1], [3, 0, 320], "48x320"])
|
||||
def test_rejects_invalid_input_shape(shape: object) -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
Settings(text_rec_input_shape=shape)
|
||||
|
||||
|
||||
def test_gpu_falls_back_when_cuda_unavailable() -> None:
|
||||
"""配置了 GPU 但当前 PaddlePaddle 不支持 CUDA 时,应回落 CPU 而不是崩溃。"""
|
||||
paddle = pytest.importorskip("paddle")
|
||||
service = OCRService(device="gpu", use_tensorrt=True)
|
||||
service._resolve_device()
|
||||
if paddle.device.is_compiled_with_cuda():
|
||||
assert service._effective_device == "gpu"
|
||||
assert service._use_tensorrt is True
|
||||
else:
|
||||
assert service._effective_device == "cpu"
|
||||
# TensorRT 只在 GPU 上生效,回落 CPU 后必须被关掉。
|
||||
assert service._use_tensorrt is False
|
||||
|
||||
|
||||
def test_tensorrt_ignored_on_cpu() -> None:
|
||||
service = OCRService(device="cpu", use_tensorrt=True)
|
||||
service._resolve_device()
|
||||
assert service._effective_device == "cpu"
|
||||
assert service._use_tensorrt is False
|
||||
|
||||
|
||||
def test_auto_is_accepted_as_device() -> None:
|
||||
assert Settings(device="auto").device == "auto"
|
||||
|
||||
|
||||
def test_rejects_unrecognised_device_name() -> None:
|
||||
"""格式合法但取值不存在的设备名要在配置层就挡住。
|
||||
|
||||
否则它会一路传到 PaddleX 内部抛 AssertionError,日志里看不出是配置写错了。
|
||||
"""
|
||||
with pytest.raises(ValidationError):
|
||||
Settings(device="autoo")
|
||||
with pytest.raises(ValidationError):
|
||||
Settings(device="cuda")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("device", ["cpu", "gpu", "gpu:1", "npu", "xpu"])
|
||||
def test_accepts_supported_devices(device: str) -> None:
|
||||
assert Settings(device=device).device == device
|
||||
|
||||
|
||||
@pytest.mark.parametrize("device", ["auto", "autoo", "cuda"])
|
||||
def test_finalize_device_never_passes_unknown_value(device: str) -> None:
|
||||
"""最后一道防线:冻结包代码比 config.yaml 旧时(实测踩过)也不能把值透传给 PaddleX。"""
|
||||
resolved = OCRService._finalize_device(device)
|
||||
assert resolved.split(":", 1)[0] in {
|
||||
"cpu",
|
||||
"gpu",
|
||||
"xpu",
|
||||
"npu",
|
||||
"mlu",
|
||||
"gcu",
|
||||
"dcu",
|
||||
"iluvatar_gpu",
|
||||
"metax_gpu",
|
||||
}
|
||||
|
||||
|
||||
def test_finalize_device_keeps_valid_values() -> None:
|
||||
assert OCRService._finalize_device("gpu:1") == "gpu:1"
|
||||
assert OCRService._finalize_device("cpu") == "cpu"
|
||||
|
||||
|
||||
def has_usable_gpu() -> bool:
|
||||
paddle = pytest.importorskip("paddle")
|
||||
if not paddle.device.is_compiled_with_cuda():
|
||||
return False
|
||||
try:
|
||||
return int(paddle.device.cuda.device_count()) > 0
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def test_auto_selects_matching_device() -> None:
|
||||
"""device=auto 探测到卡就用 GPU,探测不到就用 CPU——两种环境都要能起来。"""
|
||||
service = OCRService(device="auto", use_tensorrt=True)
|
||||
service._resolve_device()
|
||||
if has_usable_gpu():
|
||||
assert service._effective_device == "gpu"
|
||||
assert service._use_tensorrt is True
|
||||
else:
|
||||
assert service._effective_device == "cpu"
|
||||
assert service._use_tensorrt is False
|
||||
Reference in New Issue
Block a user