Skip to content

Commit 23036df

Browse files
cau-gitcau-git
committed
Update branch to match final design
Signed-off-by: Christoph Auer <[email protected]>
2 parents cf2ce1f + f084ce7 commit 23036df

File tree

124 files changed

+24678
-8114
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

124 files changed

+24678
-8114
lines changed

‎.github/workflows/checks.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
on:
22
workflow_call:
33

4+
env:
5+
RUN_IN_CI: "1"
6+
47
jobs:
58
run-checks:
69
runs-on: ubuntu-latest

‎.pre-commit-config.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,13 @@ repos:
44
hooks:
55
- id: black
66
name: Black
7-
entry: poetry run black docling_eval docling_eval_next tests docs/examples
7+
entry: poetry run black docling_eval tests docs/examples
88
pass_filenames: false
99
language: system
1010
files: '\.py$'
1111
- id: isort
1212
name: isort
13-
entry: poetry run isort docling_eval docling_eval_next tests docs/examples
13+
entry: poetry run isort docling_eval tests docs/examples
1414
pass_filenames: false
1515
language: system
1616
files: '\.py$'
@@ -22,7 +22,7 @@ repos:
2222
# files: '\.py$'
2323
- id: mypy
2424
name: MyPy
25-
entry: poetry run mypy docling_eval docling_eval_next tests docs/examples
25+
entry: poetry run mypy docling_eval tests docs/examples
2626
pass_filenames: false
2727
language: system
2828
files: '\.py$'

‎README.md

Lines changed: 26 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
<p align="center">
2+
<a href="https://github.com/docling-project/docling-eval">
3+
<img loading="lazy" alt="Docling" src="docs/assets/docling-eval-pic.png" width="40%"/>
4+
</a>
5+
</p>
6+
17
# Docling-eval
28

39

@@ -19,28 +25,23 @@ Evaluate [Docling](https://github.com/DS4SD/docling) on various datasets.
1925

2026
Evaluate docling on various datasets. You can use the cli
2127

22-
```sh
23-
docling-eval % poetry run evaluate --help
24-
25-
Usage: python -m docling_eval.cli.main [OPTIONS]
26-
27-
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
28-
│ * --task -t [create|evaluate|visualize] Evaluation task [default: None] [required] │
29-
│ * --modality -m [end-to-end|layout|table_structure|code_transcription|math_transcription|reading_order|markdown_text|ca Evaluation modality [default: None] [required] │
30-
│ ptioning|bboxes_text] │
31-
│ * --benchmark -b [DPBench|OmniDocBench|WordScape|PubLayNet|DocLayNetV1|DocLayNetV2|FUNSD|Pub1M|PubTabNet|FinTabNet|WikiT Benchmark name [default: None] [required] │
32-
│ abNet] │
33-
│ * --output-dir -o PATH Output directory [default: None] [required] │
34-
│ --input-dir -i PATH Input directory [default: None] │
35-
│ --converter_type -c [Docling|SmolDocling] Type of document converter [default: Docling] │
36-
│ --split -s TEXT Dataset split [default: test] │
37-
│ --artifacts-path -a PATH Load artifacts from local path [default: None] │
38-
│ --begin_index -bi INTEGER Begin converting from the given sample index (inclusive). Zero based. [default: 0] │
39-
│ --end_index -ei INTEGER End converting to the given sample index (exclusive). Zero based. -1 indicates to take all │
40-
│ [default: 1000] │
41-
│ --debug --no-debug Enable debugging [default: no-debug] │
42-
│ --help Show this message and exit. │
43-
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
28+
```shell
29+
terminal %> poetry run docling_eval --help
30+
31+
Usage: docling_eval [OPTIONS] COMMAND [ARGS]...
32+
33+
Docling Evaluation CLI for benchmarking document processing tasks.
34+
35+
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
36+
│ --help Show this message and exit. │
37+
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
38+
╭─ Commands ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
39+
│ create Create both ground truth and evaluation datasets in one step. │
40+
│ create-eval Create evaluation dataset from existing ground truth. │
41+
│ create-gt Create ground truth dataset only. │
42+
│ evaluate Evaluate predictions against ground truth. │
43+
│ visualize Visualize evaluation results. │
44+
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
4445

4546

4647
```
@@ -62,7 +63,9 @@ On our list for next benchmarks:
6263
- [OmniOCR](getomni-ai/ocr-benchmark)
6364
- Hyperscalers
6465
- [CoMix](https://github.com/emanuelevivoli/CoMix/tree/main/docs/datasets)
65-
66+
- [DocVQA](https://huggingface.co/datasets/lmms-lab/DocVQA)
67+
- [rd-tablebench](https://huggingface.co/datasets/reducto/rd-tablebench)
68+
6669
## Contributing
6770

6871
Please read [Contributing to Docling](https://github.com/DS4SD/docling/blob/main/CONTRIBUTING.md) for details.

0 commit comments

Comments
 (0)