feat: Azure prediction provider #50

cau-git · 2025-03-25T18:32:31Z

This branch is a selection of units from branch praveenmidde/table-evaluation. It is in sync with the latest API on main.

Features picked

AzureDocIntelligencePredictionProvider class
New Azure dependencies in pyproject.toml

Features not included

COS source
Table dataset builders (they are already implemented in main)

Signed-off-by: Peter Staar <[email protected]>

Signed-off-by: Christoph Auer <[email protected]>

…ss-design

Signed-off-by: Christoph Auer <[email protected]>

…ss-design

Signed-off-by: Christoph Auer <[email protected]>

Need to work on image dimensions as the visualizations seem to be off

- Save the hyperscaler API jsons - Save the docling document formats - Temporary files are stored as below: - `intermediate_files` -- for parquet files - `microsoft` - Root folder that contains API output files - `docling_document` - docling document output of the MS outputs - `visualizations` - Output of table visualizations

Signed-off-by: Christoph Auer <[email protected]>

Signed-off-by: Nikos Livathinos <[email protected]>

… accept external predictions. And utility adapters. Signed-off-by: Nikos Livathinos <[email protected]>

- New implementation for S3 source - Save the intermediate files - prediction and doclingDocuments for pred+groundTruth for debugging - Implement the run for TEDS metric for (limited) Fintabnet dataset from cos; If the API predictions are available, will use them instead of calling the API - Needed to pin library versions of pydantic, url + s3 library

…etRecord Signed-off-by: Christoph Auer <[email protected]>

…into cau/new-class-design

Signed-off-by: Christoph Auer <[email protected]>

praveenmidde

Will review and add more..

pyproject.toml

1. Cleanup poetry of unwanted libraries for this PR 2. Address passing in the images bytes for Azure API 3. Added log statements for filename; Also moved azure api logs to warning. 4. Remove commented out code as discussed 5. pytest - set "ignore_missing_predictions" to True to skip the missing files and move forward.

praveenmidde

Changes look good. Approving..

Tested the code by running pytest test_tables_azure.py --log-cli-level=INFO from the tests folder;
I could see the results of TEDS for fintabnet dataset for 4 files (Note: end_index was set to 5; and the 3rd file in the dataset has a height of 42 pixels, which the Azure API will fail to process - Refer https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/model-overview?view=doc-intel-4.0.0#input-requirements)
Note: PubTabNetDatasetBuilder has some problem with the conversion - that shall be looked later.

Signed-off-by: Christoph Auer <[email protected]>

…ng-eval into cau/table-evaluation-updates

Signed-off-by: Christoph Auer <[email protected]>

praveenmidde

Ran the test with latest code.. Test is successful

PeterStaar-IBM and others added 30 commits February 12, 2025 12:48

correct mpy

3053b56

Signed-off-by: Peter Staar <[email protected]>

reformatting

a7869e5

Signed-off-by: Peter Staar <[email protected]>

Merge branch 'main' into fix/docling-dpbench

9b03f64

adding the script to make an initial dataset from pdf's

31f7a1a

Signed-off-by: Peter Staar <[email protected]>

before switching to specific docling-core branch

86efb2d

Signed-off-by: Peter Staar <[email protected]>

rebased on kv-items and updated the create script in CVAT

b504372

Signed-off-by: Peter Staar <[email protected]>

fixed the cvat

68f8b1d

Signed-off-by: Peter Staar <[email protected]>

Merge branch 'main' into fix/docling-dpbench

f045f0b

added the annotation description on CVAT

7f59ff2

Signed-off-by: Peter Staar <[email protected]>

added the annotation description on CVAT (2)

3baf9f1

Signed-off-by: Peter Staar <[email protected]>

added the annotation description on CVAT (3)

c45d2e3

Signed-off-by: Peter Staar <[email protected]>

[WIP] Crafting new dataset builder and prediction provider API

d8a8a59

Signed-off-by: Christoph Auer <[email protected]>

Merge from main

b8cb738

Signed-off-by: Christoph Auer <[email protected]>

Restructure to docling_eval_next

23834fb

Signed-off-by: Christoph Auer <[email protected]>

Fix mypy

2206a97

Signed-off-by: Christoph Auer <[email protected]>

Merge branch 'main' of github.com:DS4SD/docling-eval into cau/new-cla…

ea02901

…ss-design

Fix f-strings

a6295bc

Signed-off-by: Christoph Auer <[email protected]>

Skeleton for fintabnet dataset with azure provider - InProgress

a8ba228

Parking initial changes for Fintabnet builder

d9c7281

Merge from main

684fd27

Signed-off-by: Christoph Auer <[email protected]>

Merge branch 'main' of github.com:DS4SD/docling-eval into cau/new-cla…

415d767

…ss-design

Changes for running Azure API

68f48c7

Merge branch 'cau/new-class-design' into praveenmidde/table-evaluation

22e4cd8

Update lock with latest changes

f1b6115

Changes for prediction_provider interface, to support all cases.

3b62bc6

Signed-off-by: Christoph Auer <[email protected]>

Parking changes for Fintabnet databuilder + Azure Predictor

bf20fe5

Working Azure prediction

3dfbac6

Need to work on image dimensions as the visualizations seem to be off

Change file and folder names for readability

6bdc396

Add omnidocbench DatasetBuilder

0555485

Signed-off-by: Christoph Auer <[email protected]>

nikos-livathinos and others added 11 commits March 21, 2025 11:44

chore: Fixing imports

57df7bb

Signed-off-by: Nikos Livathinos <[email protected]>

chore: Update docling-core version

1250f5d

Signed-off-by: Nikos Livathinos <[email protected]>

feat: Introduce new design for Evaluators based on BaseEvaluator that…

51260fd

… accept external predictions. And utility adapters. Signed-off-by: Nikos Livathinos <[email protected]>

Remove unused code

11363ea

Add missing shard_id so that shards do not overwrite themselves

33ead0e

Factor PredictionProvider out of dataset builder, many fixes on Datas…

6812656

…etRecord Signed-off-by: Christoph Auer <[email protected]>

Merge branch 'cau/new-class-design' of github.com:DS4SD/docling-eval …

8df9157

…into cau/new-class-design

Sketch example for file-directory prediction provider

9aed020

Signed-off-by: Christoph Auer <[email protected]>

Update from base branch

091d5ed

Signed-off-by: Christoph Auer <[email protected]>

Updates to FinTabNet dataset builder and Azure DI provider

81e136e

Signed-off-by: Christoph Auer <[email protected]>

cau-git requested a review from praveenmidde March 25, 2025 18:32

Apply formatting

cf2ce1f

Signed-off-by: Christoph Auer <[email protected]>

cau-git changed the base branch from praveenmidde/table-evaluation to main April 7, 2025 15:46

Update branch to match final design

be637c7

Signed-off-by: Christoph Auer <[email protected]>

cau-git force-pushed the cau/table-evaluation-updates branch from 23036df to be637c7 Compare April 7, 2025 18:21

cau-git assigned cau-git and praveenmidde Apr 7, 2025

praveenmidde reviewed Apr 8, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

pyproject.toml Outdated Show resolved Hide resolved

pyproject.toml Outdated Show resolved Hide resolved

cau-git changed the title ~~Updates for table-evaluation branch~~ feat: Azure prediction provider Apr 8, 2025

praveenmidde previously approved these changes Apr 8, 2025

View reviewed changes

cau-git added 3 commits April 8, 2025 13:38

Fixes for Azure provider

49066ba

Signed-off-by: Christoph Auer <[email protected]>

Merge branch 'cau/table-evaluation-updates' of github.com:DS4SD/docli…

f3d0353

…ng-eval into cau/table-evaluation-updates

Cleanup

70e04f9

Signed-off-by: Christoph Auer <[email protected]>

cau-git dismissed praveenmidde’s stale review via 70e04f9 April 8, 2025 11:40

praveenmidde approved these changes Apr 8, 2025

View reviewed changes

cau-git marked this pull request as ready for review April 8, 2025 12:22

cau-git merged commit 683de7a into main Apr 8, 2025
8 checks passed

praveenmidde mentioned this pull request Apr 16, 2025

Add table evaluation for table datasets, and hyperscaler predictors #47

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Azure prediction provider #50

feat: Azure prediction provider #50

Uh oh!

cau-git commented Mar 25, 2025 •

edited

Loading

Uh oh!

praveenmidde left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

praveenmidde left a comment •

edited

Loading

Uh oh!

praveenmidde left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

feat: Azure prediction provider #50

feat: Azure prediction provider #50

Uh oh!

Conversation

cau-git commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Features picked

Features not included

Uh oh!

praveenmidde left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

praveenmidde left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

praveenmidde left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cau-git commented Mar 25, 2025 •

edited

Loading

praveenmidde left a comment •

edited

Loading

praveenmidde left a comment •

edited

Loading