Skip to content

pull-mirror sync seems unnecessarily slow #18352

Closed
@petergardfjall

Description

@petergardfjall

Gitea Version

6c7084c

Git Version

2.25.1

Operating System

Linux (Ubuntu 20.04)

How are you running Gitea?

Built from source, run locally on the host.

Database

PostgreSQL

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Description

A mirror-sync operation not only runs git remote update, but also tries to sync any repo releases with the available repo tags. For large repositories with many tags, this can be a costly operation, both in time and computational resources.

I noticed that when doing a "plain git mirror" (which doesn't include any releases) of a big repo (such as Kubernetes with about 900 tags), the synchronize operation (SyncReleasesWithTags) spent a lot of time (about six minutes) listing/syncing tags with releases.

This appears to be caused by repetitive calls like (one for each repo tag):

git show-ref --tags -- v8.2.4477
git cat-file -t 29ab6ce9f36660cffaad3c8789e71162e5db5d2f
git cat-file -p 29ab6ce9f36660cffaad3c8789e71162e5db5d2f
git rev-list --count 29ab6ce9f36660cffaad3c8789e71162e5db5d2f

In particular git rev-list --count' can be heavy for large repos with many commits.

It seems like there is an opportunity to improve performance and reduce resource use by making this procedure more efficient for pull-mirrors.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions