aodianyun/ad-pdf-extract 🖼️📝 → 🖼️

start
/tmp/tmpkkr03214sw1.pdf
2024-09-26 09:45:07.994 | INFO     | magic_pdf.libs.pdf_check:detect_invalid_chars:57 - cid_count: 0, text_len: 6, cid_chars_radio: 0.0
2024-09-26 09:45:07.994 | WARNING  | magic_pdf.filter.pdf_classify_by_type:classify:334 - pdf is not classified by area and text_len, by_image_area: False, by_text: False, by_avg_words: False, by_img_num: True, by_text_layout: False, by_img_narrow_strips: True, by_invalid_chars: True
2024-09-26 09:45:15.442 | INFO     | magic_pdf.model.pdf_extract_kit:__init__:180 - DocAnalysis init, this may take some times. apply_layout: True, apply_formula: True, apply_ocr: True, apply_table: False
2024-09-26 09:45:15.442 | INFO     | magic_pdf.model.pdf_extract_kit:__init__:188 - using device: cuda
2024-09-26 09:45:15.442 | INFO     | magic_pdf.model.pdf_extract_kit:__init__:190 - using models_dir: /src/models
CustomVisionEncoderDecoderModel init
CustomMBartForCausalLM init
CustomMBartDecoder init
[09/26 09:45:34 detectron2]: Rank of current process: 0. World size: 1
[09/26 09:45:35 detectron2]: Environment info:
-------------------------------  ------------------------------------------------------------------------------------
sys.platform                     linux
Python                           3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0]
numpy                            1.26.4
detectron2                       0.6 @/root/.pyenv/versions/3.10.15/lib/python3.10/site-packages/detectron2
Compiler                         GCC 11.4
CUDA compiler                    not available
DETECTRON2_ENV_MODULE            <not set>
PyTorch                          2.3.1+cu121 @/root/.pyenv/versions/3.10.15/lib/python3.10/site-packages/torch
PyTorch debug build              False
torch._C._GLIBCXX_USE_CXX11_ABI  False
GPU available                    Yes
GPU 0                            Tesla T4 (arch=7.5)
Driver version                   535.104.12
CUDA_HOME                        /usr/local/cuda
Pillow                           10.4.0
torchvision                      0.18.1+cu121 @/root/.pyenv/versions/3.10.15/lib/python3.10/site-packages/torchvision
torchvision arch flags           5.0, 6.0, 7.0, 7.5, 8.0, 8.6, 9.0
fvcore                           0.1.5.post20221221
iopath                           0.1.9
cv2                              4.6.0
-------------------------------  ------------------------------------------------------------------------------------
PyTorch built with:
- GCC 9.3
- C++ Version: 201703
- Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v3.3.6 (Git Hash 86e6af5974177e513fd3fee58425e1063e7f1361)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 12.1
- NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
- CuDNN 8.9
- Built with CuDNN 8.9.2
- Magma 2.6.1
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.9.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.3.1, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=1, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF,
[09/26 09:45:35 detectron2]: Command line arguments: {'config_file': '/root/.pyenv/versions/3.10.15/lib/python3.10/site-packages/magic_pdf/resources/model_config/layoutlmv3/layoutlmv3_base_inference.yaml', 'resume': False, 'eval_only': False, 'num_gpus': 1, 'num_machines': 1, 'machine_rank': 0, 'dist_url': 'tcp://127.0.0.1:57823', 'opts': ['MODEL.WEIGHTS', '/src/models/Layout/model_final.pth']}
[09/26 09:45:35 detectron2]: Contents of args.config_file=/root/.pyenv/versions/3.10.15/lib/python3.10/site-packages/magic_pdf/resources/model_config/layoutlmv3/layoutlmv3_base_inference.yaml:
AUG:
DETR: true
CACHE_DIR: ~/cache/huggingface
CUDNN_BENCHMARK: false
DATALOADER:
ASPECT_RATIO_GROUPING: true
FILTER_EMPTY_ANNOTATIONS: false
NUM_WORKERS: 4
REPEAT_THRESHOLD: 0.0
SAMPLER_TRAIN: TrainingSampler
DATASETS:
PRECOMPUTED_PROPOSAL_TOPK_TEST: 1000
PRECOMPUTED_PROPOSAL_TOPK_TRAIN: 2000
PROPOSAL_FILES_TEST: []
PROPOSAL_FILES_TRAIN: []
TEST:
- scihub_train
TRAIN:
- scihub_train
GLOBAL:
HACK: 1.0
ICDAR_DATA_DIR_TEST: ''
ICDAR_DATA_DIR_TRAIN: ''
INPUT:
CROP:
ENABLED: true
SIZE:
- 384
- 600
TYPE: absolute_range
FORMAT: RGB
MASK_FORMAT: polygon
MAX_SIZE_TEST: 1333
MAX_SIZE_TRAIN: 1333
MIN_SIZE_TEST: 800
MIN_SIZE_TRAIN:
- 480
- 512
- 544
- 576
- 608
- 640
- 672
- 704
- 736
- 768
- 800
MIN_SIZE_TRAIN_SAMPLING: choice
RANDOM_FLIP: horizontal
MODEL:
ANCHOR_GENERATOR:
ANGLES:
- - -90
- 0
- 90
ASPECT_RATIOS:
- - 0.5
- 1.0
- 2.0
NAME: DefaultAnchorGenerator
OFFSET: 0.0
SIZES:
- - 32
- - 64
- - 128
- - 256
- - 512
BACKBONE:
FREEZE_AT: 2
NAME: build_vit_fpn_backbone
CONFIG_PATH: ''
DEVICE: cuda
FPN:
FUSE_TYPE: sum
IN_FEATURES:
- layer3
- layer5
- layer7
- layer11
NORM: ''
OUT_CHANNELS: 256
IMAGE_ONLY: true
KEYPOINT_ON: false
LOAD_PROPOSALS: false
MASK_ON: true
META_ARCHITECTURE: VLGeneralizedRCNN
PANOPTIC_FPN:
COMBINE:
ENABLED: true
INSTANCES_CONFIDENCE_THRESH: 0.5
OVERLAP_THRESH: 0.5
STUFF_AREA_LIMIT: 4096
INSTANCE_LOSS_WEIGHT: 1.0
PIXEL_MEAN:
- 127.5
- 127.5
- 127.5
PIXEL_STD:
- 127.5
- 127.5
- 127.5
PROPOSAL_GENERATOR:
MIN_SIZE: 0
NAME: RPN
RESNETS:
DEFORM_MODULATED: false
DEFORM_NUM_GROUPS: 1
DEFORM_ON_PER_STAGE:
- false
- false
- false
- false
DEPTH: 50
NORM: FrozenBN
NUM_GROUPS: 1
OUT_FEATURES:
- res4
RES2_OUT_CHANNELS: 256
RES5_DILATION: 1
STEM_OUT_CHANNELS: 64
STRIDE_IN_1X1: true
WIDTH_PER_GROUP: 64
RETINANET:
BBOX_REG_LOSS_TYPE: smooth_l1
BBOX_REG_WEIGHTS:
- 1.0
- 1.0
- 1.0
- 1.0
FOCAL_LOSS_ALPHA: 0.25
FOCAL_LOSS_GAMMA: 2.0
IN_FEATURES:
- p3
- p4
- p5
- p6
- p7
IOU_LABELS:
- 0
- -1
- 1
IOU_THRESHOLDS:
- 0.4
- 0.5
NMS_THRESH_TEST: 0.5
NORM: ''
NUM_CLASSES: 10
NUM_CONVS: 4
PRIOR_PROB: 0.01
SCORE_THRESH_TEST: 0.05
SMOOTH_L1_LOSS_BETA: 0.1
TOPK_CANDIDATES_TEST: 1000
ROI_BOX_CASCADE_HEAD:
BBOX_REG_WEIGHTS:
- - 10.0
- 10.0
- 5.0
- 5.0
- - 20.0
- 20.0
- 10.0
- 10.0
- - 30.0
- 30.0
- 15.0
- 15.0
IOUS:
- 0.5
- 0.6
- 0.7
ROI_BOX_HEAD:
BBOX_REG_LOSS_TYPE: smooth_l1
BBOX_REG_LOSS_WEIGHT: 1.0
BBOX_REG_WEIGHTS:
- 10.0
- 10.0
- 5.0
- 5.0
CLS_AGNOSTIC_BBOX_REG: true
CONV_DIM: 256
FC_DIM: 1024
NAME: FastRCNNConvFCHead
NORM: ''
NUM_CONV: 0
NUM_FC: 2
POOLER_RESOLUTION: 7
POOLER_SAMPLING_RATIO: 0
POOLER_TYPE: ROIAlignV2
SMOOTH_L1_BETA: 0.0
TRAIN_ON_PRED_BOXES: false
ROI_HEADS:
BATCH_SIZE_PER_IMAGE: 512
IN_FEATURES:
- p2
- p3
- p4
- p5
IOU_LABELS:
- 0
- 1
IOU_THRESHOLDS:
- 0.5
NAME: CascadeROIHeads
NMS_THRESH_TEST: 0.5
NUM_CLASSES: 10
POSITIVE_FRACTION: 0.25
PROPOSAL_APPEND_GT: true
SCORE_THRESH_TEST: 0.05
ROI_KEYPOINT_HEAD:
CONV_DIMS:
- 512
- 512
- 512
- 512
- 512
- 512
- 512
- 512
LOSS_WEIGHT: 1.0
MIN_KEYPOINTS_PER_IMAGE: 1
NAME: KRCNNConvDeconvUpsampleHead
NORMALIZE_LOSS_BY_VISIBLE_KEYPOINTS: true
NUM_KEYPOINTS: 17
POOLER_RESOLUTION: 14
POOLER_SAMPLING_RATIO: 0
POOLER_TYPE: ROIAlignV2
ROI_MASK_HEAD:
CLS_AGNOSTIC_MASK: false
CONV_DIM: 256
NAME: MaskRCNNConvUpsampleHead
NORM: ''
NUM_CONV: 4
POOLER_RESOLUTION: 14
POOLER_SAMPLING_RATIO: 0
POOLER_TYPE: ROIAlignV2
RPN:
BATCH_SIZE_PER_IMAGE: 256
BBOX_REG_LOSS_TYPE: smooth_l1
BBOX_REG_LOSS_WEIGHT: 1.0
BBOX_REG_WEIGHTS:
- 1.0
- 1.0
- 1.0
- 1.0
BOUNDARY_THRESH: -1
CONV_DIMS:
- -1
HEAD_NAME: StandardRPNHead
IN_FEATURES:
- p2
- p3
- p4
- p5
- p6
IOU_LABELS:
- 0
- -1
- 1
IOU_THRESHOLDS:
- 0.3
- 0.7
LOSS_WEIGHT: 1.0
NMS_THRESH: 0.7
POSITIVE_FRACTION: 0.5
POST_NMS_TOPK_TEST: 1000
POST_NMS_TOPK_TRAIN: 2000
PRE_NMS_TOPK_TEST: 1000
PRE_NMS_TOPK_TRAIN: 2000
SMOOTH_L1_BETA: 0.0
SEM_SEG_HEAD:
COMMON_STRIDE: 4
CONVS_DIM: 128
IGNORE_VALUE: 255
IN_FEATURES:
- p2
- p3
- p4
- p5
LOSS_WEIGHT: 1.0
NAME: SemSegFPNHead
NORM: GN
NUM_CLASSES: 10
VIT:
DROP_PATH: 0.1
IMG_SIZE:
- 224
- 224
NAME: layoutlmv3_base
OUT_FEATURES:
- layer3
- layer5
- layer7
- layer11
POS_TYPE: abs
WEIGHTS:
OUTPUT_DIR:
SCIHUB_DATA_DIR_TRAIN: ~/publaynet/layout_scihub/train
SEED: 42
SOLVER:
AMP:
ENABLED: true
BACKBONE_MULTIPLIER: 1.0
BASE_LR: 0.0002
BIAS_LR_FACTOR: 1.0
CHECKPOINT_PERIOD: 2000
CLIP_GRADIENTS:
CLIP_TYPE: full_model
CLIP_VALUE: 1.0
ENABLED: true
NORM_TYPE: 2.0
GAMMA: 0.1
GRADIENT_ACCUMULATION_STEPS: 1
IMS_PER_BATCH: 32
LR_SCHEDULER_NAME: WarmupCosineLR
MAX_ITER: 20000
MOMENTUM: 0.9
NESTEROV: false
OPTIMIZER: ADAMW
REFERENCE_WORLD_SIZE: 0
STEPS:
- 10000
WARMUP_FACTOR: 0.01
WARMUP_ITERS: 333
WARMUP_METHOD: linear
WEIGHT_DECAY: 0.05
WEIGHT_DECAY_BIAS: null
WEIGHT_DECAY_NORM: 0.0
TEST:
AUG:
ENABLED: false
FLIP: true
MAX_SIZE: 4000
MIN_SIZES:
- 400
- 500
- 600
- 700
- 800
- 900
- 1000
- 1100
- 1200
DETECTIONS_PER_IMAGE: 100
EVAL_PERIOD: 1000
EXPECTED_RESULTS: []
KEYPOINT_OKS_SIGMAS: []
PRECISE_BN:
ENABLED: false
NUM_ITER: 200
VERSION: 2
VIS_PERIOD: 0
[09/26 09:45:37 d2.checkpoint.detection_checkpoint]: [DetectionCheckpointer] Loading from /src/models/Layout/model_final.pth ...
[09/26 09:45:37 fvcore.common.checkpoint]: [Checkpointer] Loading from /src/models/Layout/model_final.pth ...
download https://paddleocr.bj.bcebos.com/PP-OCRv4/chinese/ch_PP-OCRv4_det_infer.tar to /root/.paddleocr/whl/det/ch/ch_PP-OCRv4_det_infer/ch_PP-OCRv4_det_infer.tar
  0%|          | 0.00/4.89M [00:00<?, ?iB/s]
  0%|          | 3.07k/4.89M [00:00<05:36, 14.5kiB/s]
  1%|          | 35.8k/4.89M [00:00<00:58, 83.1kiB/s]
  1%|          | 52.2k/4.89M [00:00<00:47, 102kiB/s] 
  1%|▏         | 68.6k/4.89M [00:00<00:45, 107kiB/s]
  2%|▏         | 85.0k/4.89M [00:00<00:52, 90.8kiB/s]
  2%|▏         | 118k/4.89M [00:01<00:44, 108kiB/s]  
  3%|▎         | 134k/4.89M [00:01<00:55, 85.2kiB/s]
  3%|▎         | 151k/4.89M [00:01<00:58, 81.1kiB/s]
  3%|▎         | 167k/4.89M [00:01<01:01, 76.9kiB/s]
  4%|▎         | 183k/4.89M [00:02<01:03, 74.1kiB/s]
  4%|▍         | 200k/4.89M [00:02<01:03, 73.4kiB/s]
  4%|▍         | 216k/4.89M [00:02<01:05, 71.7kiB/s]
  5%|▍         | 232k/4.89M [00:02<01:02, 74.2kiB/s]
  5%|▌         | 249k/4.89M [00:03<01:04, 72.2kiB/s]
  5%|▌         | 265k/4.89M [00:03<01:01, 75.5kiB/s]
  6%|▌         | 282k/4.89M [00:03<01:03, 73.0kiB/s]
  6%|▌         | 298k/4.89M [00:03<01:03, 72.7kiB/s]
  6%|▋         | 314k/4.89M [00:04<01:04, 71.2kiB/s]
  7%|▋         | 331k/4.89M [00:04<01:09, 65.6kiB/s]
  7%|▋         | 347k/4.89M [00:04<01:17, 59.0kiB/s]
  7%|▋         | 364k/4.89M [00:04<01:17, 58.7kiB/s]
  8%|▊         | 380k/4.89M [00:05<01:19, 56.9kiB/s]
  8%|▊         | 396k/4.89M [00:05<01:21, 55.2kiB/s]
  8%|▊         | 413k/4.89M [00:05<01:19, 56.3kiB/s]
  9%|▉         | 429k/4.89M [00:06<01:23, 53.6kiB/s]
  9%|▉         | 445k/4.89M [00:06<01:22, 54.1kiB/s]
  9%|▉         | 462k/4.89M [00:06<01:22, 54.0kiB/s]
 10%|▉         | 478k/4.89M [00:07<01:21, 53.9kiB/s]
 10%|█         | 495k/4.89M [00:07<01:23, 53.0kiB/s]
 10%|█         | 511k/4.89M [00:07<01:22, 52.9kiB/s]
 11%|█         | 527k/4.89M [00:08<01:24, 51.4kiB/s]
 11%|█         | 544k/4.89M [00:08<01:24, 51.3kiB/s]
 11%|█▏        | 560k/4.89M [00:08<01:23, 51.7kiB/s]
 12%|█▏        | 577k/4.89M [00:09<01:25, 50.7kiB/s]
 12%|█▏        | 593k/4.89M [00:09<01:24, 51.2kiB/s]
 12%|█▏        | 609k/4.89M [00:10<01:48, 39.6kiB/s]
 13%|█▎        | 626k/4.89M [00:10<01:33, 45.5kiB/s]
 13%|█▎        | 642k/4.89M [00:10<01:25, 49.9kiB/s]
 13%|█▎        | 658k/4.89M [00:10<01:14, 56.9kiB/s]
 14%|█▍        | 675k/4.89M [00:10<01:07, 62.5kiB/s]
 14%|█▍        | 691k/4.89M [00:11<01:00, 69.4kiB/s]
 14%|█▍        | 708k/4.89M [00:11<00:53, 77.6kiB/s]
 15%|█▍        | 724k/4.89M [00:11<00:49, 84.7kiB/s]
 15%|█▌        | 740k/4.89M [00:11<00:44, 94.1kiB/s]
 15%|█▌        | 757k/4.89M [00:11<00:40, 101kiB/s] 
 16%|█▌        | 773k/4.89M [00:11<00:37, 111kiB/s]
 16%|█▌        | 790k/4.89M [00:11<00:34, 119kiB/s]
 17%|█▋        | 822k/4.89M [00:12<00:30, 135kiB/s]
 17%|█▋        | 855k/4.89M [00:12<00:26, 151kiB/s]
 18%|█▊        | 888k/4.89M [00:12<00:24, 167kiB/s]
 19%|█▉        | 921k/4.89M [00:12<00:22, 179kiB/s]
 19%|█▉        | 953k/4.89M [00:12<00:20, 195kiB/s]
 20%|██        | 986k/4.89M [00:12<00:18, 208kiB/s]
 21%|██        | 1.02M/4.89M [00:12<00:17, 226kiB/s]
 21%|██▏       | 1.05M/4.89M [00:13<00:15, 246kiB/s]
 22%|██▏       | 1.08M/4.89M [00:13<00:14, 263kiB/s]
 23%|██▎       | 1.12M/4.89M [00:13<00:13, 277kiB/s]
 24%|██▍       | 1.17M/4.89M [00:13<00:12, 305kiB/s]
 25%|██▍       | 1.22M/4.89M [00:13<00:11, 330kiB/s]
 26%|██▌       | 1.26M/4.89M [00:13<00:10, 353kiB/s]
 27%|██▋       | 1.31M/4.89M [00:13<00:09, 373kiB/s]
 28%|██▊       | 1.36M/4.89M [00:13<00:08, 400kiB/s]
 29%|██▉       | 1.41M/4.89M [00:13<00:08, 425kiB/s]
 30%|███       | 1.48M/4.89M [00:14<00:07, 462kiB/s]
 32%|███▏      | 1.54M/4.89M [00:14<00:06, 491kiB/s]
 33%|███▎      | 1.61M/4.89M [00:14<00:06, 522kiB/s]
 35%|███▍      | 1.69M/4.89M [00:14<00:05, 568kiB/s]
 36%|███▌      | 1.77M/4.89M [00:14<00:05, 605kiB/s]
 38%|███▊      | 1.85M/4.89M [00:14<00:04, 641kiB/s]
 40%|███▉      | 1.94M/4.89M [00:14<00:04, 682kiB/s]
 42%|████▏     | 2.03M/4.89M [00:14<00:03, 733kiB/s]
 44%|████▎     | 2.13M/4.89M [00:14<00:03, 784kiB/s]
 46%|████▌     | 2.23M/4.89M [00:15<00:03, 831kiB/s]
 48%|████▊     | 2.35M/4.89M [00:15<00:02, 895kiB/s]
 50%|█████     | 2.46M/4.89M [00:15<00:02, 948kiB/s]
 53%|█████▎    | 2.59M/4.89M [00:15<00:02, 1.01MiB/s]
 56%|█████▌    | 2.72M/4.89M [00:15<00:02, 1.08MiB/s]
 58%|█████▊    | 2.85M/4.89M [00:15<00:01, 1.14MiB/s]
 61%|██████▏   | 3.00M/4.89M [00:15<00:01, 1.22MiB/s]
 65%|██████▍   | 3.17M/4.89M [00:15<00:01, 1.31MiB/s]
 68%|██████▊   | 3.33M/4.89M [00:15<00:01, 1.39MiB/s]
 72%|███████▏  | 3.51M/4.89M [00:16<00:00, 1.47MiB/s]
 75%|███████▌  | 3.69M/4.89M [00:16<00:00, 1.57MiB/s]
 79%|███████▉  | 3.89M/4.89M [00:16<00:00, 1.67MiB/s]
 84%|████████▎ | 4.10M/4.89M [00:16<00:00, 1.77MiB/s]
 88%|████████▊ | 4.33M/4.89M [00:16<00:00, 1.89MiB/s]
 93%|█████████▎| 4.56M/4.89M [00:16<00:00, 2.00MiB/s]
 98%|█████████▊| 4.80M/4.89M [00:16<00:00, 2.13MiB/s]
100%|██████████| 4.89M/4.89M [00:16<00:00, 293kiB/s]
download https://paddleocr.bj.bcebos.com/PP-OCRv4/chinese/ch_PP-OCRv4_rec_infer.tar to /root/.paddleocr/whl/rec/ch/ch_PP-OCRv4_rec_infer/ch_PP-OCRv4_rec_infer.tar
  0%|          | 0.00/11.0M [00:00<?, ?iB/s]
  0%|          | 16.4k/11.0M [00:00<02:07, 85.7kiB/s]
  1%|          | 65.5k/11.0M [00:00<01:20, 135kiB/s] 
  1%|          | 98.3k/11.0M [00:00<01:21, 133kiB/s]
  1%|          | 131k/11.0M [00:00<01:15, 143kiB/s] 
  2%|▏         | 180k/11.0M [00:01<01:12, 149kiB/s]
  2%|▏         | 213k/11.0M [00:01<01:20, 134kiB/s]
  2%|▏         | 229k/11.0M [00:01<01:19, 135kiB/s]
  2%|▏         | 246k/11.0M [00:01<01:24, 127kiB/s]
  2%|▏         | 262k/11.0M [00:02<01:33, 115kiB/s]
  3%|▎         | 279k/11.0M [00:02<01:28, 121kiB/s]
  3%|▎         | 295k/11.0M [00:02<01:34, 114kiB/s]
  3%|▎         | 311k/11.0M [00:02<01:28, 120kiB/s]
  3%|▎         | 328k/11.0M [00:02<01:38, 109kiB/s]
  3%|▎         | 344k/11.0M [00:02<01:43, 103kiB/s]
  3%|▎         | 377k/11.0M [00:03<01:40, 105kiB/s]
  4%|▎         | 393k/11.0M [00:03<01:32, 114kiB/s]
  4%|▎         | 410k/11.0M [00:03<01:39, 106kiB/s]
  4%|▍         | 426k/11.0M [00:03<01:38, 108kiB/s]
  4%|▍         | 442k/11.0M [00:03<01:40, 105kiB/s]
  4%|▍         | 462k/11.0M [00:03<01:41, 104kiB/s]
  4%|▍         | 478k/11.0M [00:04<01:33, 113kiB/s]
  5%|▍         | 495k/11.0M [00:04<01:44, 101kiB/s]
  5%|▍         | 511k/11.0M [00:04<01:54, 91.7kiB/s]
  5%|▍         | 527k/11.0M [00:04<01:50, 94.4kiB/s]
  5%|▍         | 544k/11.0M [00:04<02:06, 82.4kiB/s]
  5%|▌         | 560k/11.0M [00:05<02:09, 80.3kiB/s]
  5%|▌         | 577k/11.0M [00:05<02:15, 76.7kiB/s]
  5%|▌         | 593k/11.0M [00:05<02:19, 74.2kiB/s]
  6%|▌         | 609k/11.0M [00:05<02:22, 72.5kiB/s]
  6%|▌         | 626k/11.0M [00:05<02:10, 79.5kiB/s]
  6%|▌         | 642k/11.0M [00:06<02:16, 75.8kiB/s]
  6%|▌         | 658k/11.0M [00:06<02:18, 74.4kiB/s]
  6%|▌         | 675k/11.0M [00:06<02:22, 72.2kiB/s]
  6%|▋         | 691k/11.0M [00:06<02:19, 73.7kiB/s]
  6%|▋         | 708k/11.0M [00:07<02:14, 76.6kiB/s]
  7%|▋         | 724k/11.0M [00:07<02:19, 73.6kiB/s]
  7%|▋         | 740k/11.0M [00:07<02:27, 69.3kiB/s]
  7%|▋         | 757k/11.0M [00:07<02:40, 63.5kiB/s]
  7%|▋         | 773k/11.0M [00:08<02:40, 63.7kiB/s]
  7%|▋         | 790k/11.0M [00:08<02:42, 62.7kiB/s]
  7%|▋         | 806k/11.0M [00:08<02:41, 63.1kiB/s]
  7%|▋         | 822k/11.0M [00:08<02:43, 62.0kiB/s]
  8%|▊         | 839k/11.0M [00:09<02:43, 62.2kiB/s]
  8%|▊         | 855k/11.0M [00:09<03:17, 51.3kiB/s]
  8%|▊         | 871k/11.0M [00:10<03:31, 47.9kiB/s]
  8%|▊         | 888k/11.0M [00:10<03:03, 55.1kiB/s]
  8%|▊         | 904k/11.0M [00:10<02:42, 62.0kiB/s]
  8%|▊         | 921k/11.0M [00:10<02:22, 70.8kiB/s]
  9%|▊         | 937k/11.0M [00:10<02:09, 77.6kiB/s]
  9%|▊         | 953k/11.0M [00:10<01:55, 87.2kiB/s]
  9%|▉         | 970k/11.0M [00:11<01:45, 94.4kiB/s]
  9%|▉         | 986k/11.0M [00:11<01:35, 104kiB/s] 
  9%|▉         | 1.00M/11.0M [00:11<01:29, 112kiB/s]
  9%|▉         | 1.02M/11.0M [00:11<01:22, 121kiB/s]
  9%|▉         | 1.04M/11.0M [00:11<01:17, 129kiB/s]
 10%|▉         | 1.07M/11.0M [00:11<01:08, 144kiB/s]
 10%|█         | 1.10M/11.0M [00:11<01:01, 162kiB/s]
 10%|█         | 1.13M/11.0M [00:12<00:57, 172kiB/s]
 11%|█         | 1.17M/11.0M [00:12<00:52, 187kiB/s]
 11%|█         | 1.20M/11.0M [00:12<00:49, 199kiB/s]
 11%|█         | 1.23M/11.0M [00:12<00:44, 220kiB/s]
 12%|█▏        | 1.26M/11.0M [00:12<00:42, 229kiB/s]
 12%|█▏        | 1.30M/11.0M [00:12<00:39, 246kiB/s]
 12%|█▏        | 1.33M/11.0M [00:12<00:37, 260kiB/s]
 13%|█▎        | 1.38M/11.0M [00:12<00:30, 312kiB/s]
 13%|█▎        | 1.41M/11.0M [00:12<00:32, 298kiB/s]
 13%|█▎        | 1.46M/11.0M [00:13<00:29, 318kiB/s]
 14%|█▍        | 1.51M/11.0M [00:13<00:27, 345kiB/s]
 14%|█▍        | 1.56M/11.0M [00:13<00:25, 367kiB/s]
 15%|█▍        | 1.61M/11.0M [00:13<00:23, 394kiB/s]
 15%|█▌        | 1.66M/11.0M [00:13<00:22, 414kiB/s]
 16%|█▌        | 1.72M/11.0M [00:13<00:20, 450kiB/s]
 16%|█▋        | 1.79M/11.0M [00:13<00:19, 478kiB/s]
 17%|█▋        | 1.85M/11.0M [00:13<00:17, 513kiB/s]
 17%|█▋        | 1.92M/11.0M [00:14<00:16, 536kiB/s]
 18%|█▊        | 2.00M/11.0M [00:14<00:15, 579kiB/s]
 19%|█▉        | 2.08M/11.0M [00:14<00:14, 613kiB/s]
 20%|█▉        | 2.16M/11.0M [00:14<00:13, 652kiB/s]
 20%|██        | 2.25M/11.0M [00:14<00:12, 690kiB/s]
 21%|██▏       | 2.34M/11.0M [00:14<00:11, 742kiB/s]
 22%|██▏       | 2.44M/11.0M [00:14<00:10, 786kiB/s]
 23%|██▎       | 2.54M/11.0M [00:14<00:10, 830kiB/s]
 24%|██▍       | 2.66M/11.0M [00:14<00:09, 885kiB/s]
 25%|██▌       | 2.77M/11.0M [00:15<00:08, 943kiB/s]
 26%|██▋       | 2.89M/11.0M [00:15<00:08, 989kiB/s]
 27%|██▋       | 3.02M/11.0M [00:15<00:07, 1.05MiB/s]
 29%|██▊       | 3.15M/11.0M [00:15<00:07, 1.11MiB/s]
 30%|███       | 3.30M/11.0M [00:15<00:06, 1.18MiB/s]
 31%|███▏      | 3.44M/11.0M [00:15<00:06, 1.25MiB/s]
 33%|███▎      | 3.61M/11.0M [00:15<00:05, 1.32MiB/s]
 34%|███▍      | 3.77M/11.0M [00:15<00:05, 1.40MiB/s]
 36%|███▌      | 3.95M/11.0M [00:15<00:04, 1.49MiB/s]
 38%|███▊      | 4.13M/11.0M [00:16<00:04, 1.56MiB/s]
 39%|███▉      | 4.33M/11.0M [00:16<00:03, 1.66MiB/s]
 41%|████      | 4.52M/11.0M [00:16<00:03, 1.74MiB/s]
 43%|████▎     | 4.74M/11.0M [00:16<00:03, 1.85MiB/s]
 45%|████▌     | 4.97M/11.0M [00:16<00:03, 1.96MiB/s]
 47%|████▋     | 5.21M/11.0M [00:16<00:02, 2.08MiB/s]
 50%|████▉     | 5.47M/11.0M [00:16<00:02, 2.24MiB/s]
 52%|█████▏    | 5.72M/11.0M [00:16<00:02, 2.29MiB/s]
 55%|█████▍    | 6.00M/11.0M [00:16<00:02, 2.43MiB/s]
 57%|█████▋    | 6.29M/11.0M [00:16<00:01, 2.57MiB/s]
 60%|██████    | 6.60M/11.0M [00:17<00:01, 2.71MiB/s]
 63%|██████▎   | 6.93M/11.0M [00:17<00:01, 2.86MiB/s]
 66%|██████▋   | 7.28M/11.0M [00:17<00:01, 3.03MiB/s]
 70%|██████▉   | 7.64M/11.0M [00:17<00:01, 3.19MiB/s]
 73%|███████▎  | 8.02M/11.0M [00:17<00:00, 3.38MiB/s]
 77%|███████▋  | 8.41M/11.0M [00:17<00:00, 3.52MiB/s]
 80%|████████  | 8.83M/11.0M [00:17<00:00, 3.72MiB/s]
 84%|████████▍ | 9.28M/11.0M [00:17<00:00, 3.91MiB/s]
 89%|████████▉ | 9.75M/11.0M [00:17<00:00, 4.13MiB/s]
 93%|█████████▎| 10.2M/11.0M [00:17<00:00, 4.31MiB/s]
 98%|█████████▊| 10.7M/11.0M [00:18<00:00, 4.56MiB/s]
100%|██████████| 11.0M/11.0M [00:18<00:00, 607kiB/s]
download https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar to /root/.paddleocr/whl/cls/ch_ppocr_mobile_v2.0_cls_infer/ch_ppocr_mobile_v2.0_cls_infer.tar
  0%|          | 0.00/2.19M [00:00<?, ?iB/s]
  0%|          | 3.07k/2.19M [00:00<02:56, 12.4kiB/s]
  2%|▏         | 35.8k/2.19M [00:00<00:30, 71.0kiB/s]
  2%|▏         | 52.2k/2.19M [00:00<00:26, 80.5kiB/s]
  3%|▎         | 68.6k/2.19M [00:00<00:23, 89.5kiB/s]
  4%|▍         | 85.0k/2.19M [00:01<00:28, 74.9kiB/s]
  5%|▌         | 118k/2.19M [00:01<00:23, 88.3kiB/s] 
  6%|▌         | 134k/2.19M [00:01<00:28, 71.1kiB/s]
  7%|▋         | 151k/2.19M [00:02<00:30, 66.1kiB/s]
  8%|▊         | 167k/2.19M [00:02<00:32, 62.2kiB/s]
  8%|▊         | 183k/2.19M [00:02<00:33, 60.2kiB/s]
  9%|▉         | 200k/2.19M [00:03<00:33, 59.2kiB/s]
 10%|▉         | 216k/2.19M [00:03<00:34, 56.7kiB/s]
 11%|█         | 232k/2.19M [00:03<00:33, 58.8kiB/s]
 11%|█▏        | 249k/2.19M [00:03<00:33, 57.4kiB/s]
 12%|█▏        | 265k/2.19M [00:04<00:31, 61.1kiB/s]
 13%|█▎        | 282k/2.19M [00:04<00:32, 58.5kiB/s]
 14%|█▎        | 298k/2.19M [00:04<00:32, 58.4kiB/s]
 14%|█▍        | 314k/2.19M [00:04<00:32, 57.8kiB/s]
 15%|█▌        | 331k/2.19M [00:05<00:35, 51.8kiB/s]
 16%|█▌        | 347k/2.19M [00:05<00:40, 45.3kiB/s]
 17%|█▋        | 364k/2.19M [00:06<00:40, 45.0kiB/s]
 17%|█▋        | 380k/2.19M [00:06<00:40, 44.4kiB/s]
 18%|█▊        | 396k/2.19M [00:06<00:41, 43.5kiB/s]
 19%|█▉        | 413k/2.19M [00:07<00:40, 43.4kiB/s]
 20%|█▉        | 429k/2.19M [00:07<00:41, 42.8kiB/s]
 20%|██        | 445k/2.19M [00:08<00:39, 43.9kiB/s]
 21%|██        | 462k/2.19M [00:08<00:39, 43.6kiB/s]
 22%|██▏       | 478k/2.19M [00:08<00:38, 44.5kiB/s]
 23%|██▎       | 495k/2.19M [00:09<00:38, 43.5kiB/s]
 23%|██▎       | 511k/2.19M [00:10<00:50, 33.0kiB/s]
 24%|██▍       | 527k/2.19M [00:10<00:46, 36.1kiB/s]
 25%|██▍       | 544k/2.19M [00:10<00:40, 40.5kiB/s]
 26%|██▌       | 560k/2.19M [00:10<00:35, 45.8kiB/s]
 26%|██▋       | 577k/2.19M [00:11<00:31, 50.9kiB/s]
 27%|██▋       | 593k/2.19M [00:11<00:28, 56.8kiB/s]
 28%|██▊       | 609k/2.19M [00:11<00:25, 62.0kiB/s]
 29%|██▊       | 626k/2.19M [00:11<00:22, 68.4kiB/s]
 29%|██▉       | 642k/2.19M [00:11<00:21, 73.4kiB/s]
 30%|███       | 658k/2.19M [00:12<00:19, 80.4kiB/s]
 31%|███       | 675k/2.19M [00:12<00:17, 85.4kiB/s]
 32%|███▏      | 691k/2.19M [00:12<00:16, 90.5kiB/s]
 32%|███▏      | 708k/2.19M [00:12<00:15, 97.8kiB/s]
 33%|███▎      | 724k/2.19M [00:12<00:14, 103kiB/s] 
 34%|███▍      | 740k/2.19M [00:12<00:12, 112kiB/s]
 35%|███▍      | 757k/2.19M [00:12<00:12, 117kiB/s]
 35%|███▌      | 773k/2.19M [00:13<00:11, 126kiB/s]
 36%|███▌      | 790k/2.19M [00:13<00:10, 135kiB/s]
 38%|███▊      | 822k/2.19M [00:13<00:09, 145kiB/s]
 39%|███▉      | 855k/2.19M [00:13<00:08, 158kiB/s]
 41%|████      | 888k/2.19M [00:13<00:07, 169kiB/s]
 42%|████▏     | 921k/2.19M [00:13<00:07, 177kiB/s]
 44%|████▎     | 953k/2.19M [00:14<00:06, 190kiB/s]
 45%|████▌     | 986k/2.19M [00:14<00:06, 200kiB/s]
 47%|████▋     | 1.02M/2.19M [00:14<00:06, 182kiB/s]
 48%|████▊     | 1.05M/2.19M [00:14<00:05, 190kiB/s]
 50%|████▉     | 1.08M/2.19M [00:14<00:05, 198kiB/s]
 51%|█████     | 1.12M/2.19M [00:14<00:05, 201kiB/s]
 53%|█████▎    | 1.15M/2.19M [00:14<00:05, 205kiB/s]
 54%|█████▍    | 1.18M/2.19M [00:15<00:04, 211kiB/s]
 56%|█████▌    | 1.22M/2.19M [00:15<00:04, 213kiB/s]
 57%|█████▋    | 1.25M/2.19M [00:15<00:04, 211kiB/s]
 59%|█████▊    | 1.28M/2.19M [00:15<00:04, 213kiB/s]
 60%|██████    | 1.31M/2.19M [00:15<00:04, 211kiB/s]
 62%|██████▏   | 1.35M/2.19M [00:15<00:03, 216kiB/s]
 63%|██████▎   | 1.38M/2.19M [00:16<00:03, 220kiB/s]
 65%|██████▍   | 1.41M/2.19M [00:16<00:03, 231kiB/s]
 66%|██████▌   | 1.44M/2.19M [00:16<00:03, 243kiB/s]
 68%|██████▊   | 1.48M/2.19M [00:16<00:02, 256kiB/s]
 69%|██████▉   | 1.51M/2.19M [00:16<00:02, 269kiB/s]
 71%|███████   | 1.54M/2.19M [00:16<00:02, 279kiB/s]
 72%|███████▏  | 1.58M/2.19M [00:16<00:02, 282kiB/s]
 73%|███████▎  | 1.61M/2.19M [00:16<00:02, 251kiB/s]
 75%|███████▍  | 1.64M/2.19M [00:16<00:02, 263kiB/s]
 76%|███████▋  | 1.67M/2.19M [00:17<00:01, 276kiB/s]
 78%|███████▊  | 1.71M/2.19M [00:17<00:01, 280kiB/s]
 79%|███████▉  | 1.74M/2.19M [00:17<00:01, 286kiB/s]
 81%|████████  | 1.77M/2.19M [00:17<00:01, 294kiB/s]
 82%|████████▏ | 1.81M/2.19M [00:17<00:01, 296kiB/s]
 84%|████████▍ | 1.84M/2.19M [00:17<00:01, 294kiB/s]
 85%|████████▌ | 1.87M/2.19M [00:17<00:01, 297kiB/s]
 87%|████████▋ | 1.90M/2.19M [00:17<00:00, 295kiB/s]
 88%|████████▊ | 1.94M/2.19M [00:17<00:00, 297kiB/s]
 90%|████████▉ | 1.97M/2.19M [00:18<00:00, 303kiB/s]
 91%|█████████▏| 2.00M/2.19M [00:18<00:00, 303kiB/s]
 93%|█████████▎| 2.03M/2.19M [00:18<00:00, 299kiB/s]
 94%|█████████▍| 2.07M/2.19M [00:18<00:00, 300kiB/s]
 97%|█████████▋| 2.12M/2.19M [00:18<00:00, 319kiB/s]
 99%|█████████▉| 2.17M/2.19M [00:18<00:00, 340kiB/s]
100%|██████████| 2.19M/2.19M [00:18<00:00, 117kiB/s]
2024-09-26 09:46:35.967 | INFO     | magic_pdf.model.pdf_extract_kit:__init__:248 - DocAnalysis init done!
2024-09-26 09:46:35.968 | INFO     | magic_pdf.model.doc_analyze_by_custom_model:custom_model_init:98 - model init cost: 87.97307825088501
2024-09-26 09:46:39.620 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:259 - layout detection cost: 1.91
0: 1888x1504 14 embeddings, 237.6ms
Speed: 21.7ms preprocess, 237.6ms inference, 1.8ms postprocess per image at shape (1, 3, 1888, 1504)
2024-09-26 09:46:41.567 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:289 - formula nums: 14, mfr time: 0.72
2024-09-26 09:47:03.059 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:372 - ocr cost: 21.48
2024-09-26 09:47:04.696 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:259 - layout detection cost: 1.64
0: 1888x1344 12 embeddings, 217.7ms
Speed: 18.7ms preprocess, 217.7ms inference, 1.7ms postprocess per image at shape (1, 3, 1888, 1344)
2024-09-26 09:47:05.967 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:289 - formula nums: 12, mfr time: 0.85
2024-09-26 09:47:28.866 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:372 - ocr cost: 22.88
2024-09-26 09:47:30.793 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:259 - layout detection cost: 1.93
0: 1888x1376 1 embedding, 223.2ms
Speed: 24.1ms preprocess, 223.2ms inference, 1.8ms postprocess per image at shape (1, 3, 1888, 1376)
2024-09-26 09:47:31.647 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:289 - formula nums: 1, mfr time: 0.58
2024-09-26 09:48:00.164 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:372 - ocr cost: 28.5
2024-09-26 09:48:01.953 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:259 - layout detection cost: 1.79
0: 1888x1344 3 embeddings, 215.8ms
Speed: 20.7ms preprocess, 215.8ms inference, 1.4ms postprocess per image at shape (1, 3, 1888, 1344)
2024-09-26 09:48:02.637 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:289 - formula nums: 3, mfr time: 0.39
2024-09-26 09:48:28.050 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:372 - ocr cost: 25.4
2024-09-26 09:48:30.080 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:259 - layout detection cost: 2.03
0: 1888x1280 1 embedding, 204.6ms
Speed: 19.8ms preprocess, 204.6ms inference, 2.0ms postprocess per image at shape (1, 3, 1888, 1280)
2024-09-26 09:48:30.646 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:289 - formula nums: 1, mfr time: 0.32
2024-09-26 09:49:09.381 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:372 - ocr cost: 38.72
2024-09-26 09:49:11.338 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:259 - layout detection cost: 1.96
0: 1888x1408 2 embeddings, 235.1ms
Speed: 23.4ms preprocess, 235.1ms inference, 1.5ms postprocess per image at shape (1, 3, 1888, 1408)
2024-09-26 09:49:12.379 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:289 - formula nums: 2, mfr time: 0.74
2024-09-26 09:49:46.451 | INFO     | magic_pdf.model.pdf_extract_kit:__call__:372 - ocr cost: 34.06
2024-09-26 09:49:46.452 | INFO     | magic_pdf.model.doc_analyze_by_custom_model:doc_analyze:136 - doc analyze cost: 188.74599361419678
2024-09-26 09:49:47.552 | INFO     | magic_pdf.pipe.UNIPipe:pipe_mk_uni_format:48 - uni_pipe mk content list finished
2024-09-26 09:49:47.567 | INFO     | magic_pdf.pipe.UNIPipe:pipe_mk_markdown:53 - uni_pipe mk mm_markdown finished
end