Skip to content

all: re-implement Windows support #941

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

all: re-implement Windows support #941

wants to merge 1 commit into from

Conversation

keegancsmith
Copy link
Member

This commit re-adds support for building and running Zoekt on Windows platforms. We had previously reverted Windows support in commits 70647ba and 0d03621 due to possible memory mapping issues affecting Linux servers.

The key changes are:

  • Added platform-specific implementations for file handling
  • Re-introduced Windows-specific signal handling
  • Fixed disk usage reporting on Windows via filepath.VolumeName
  • Used OS-specific build tags to keep the page size hardcoded on non-Windows platforms.

We maintain the approach of using a hardcoded page size (4KB) on non-Windows platforms, which was the key reason for the initial support removal, while providing correct Windows implementations.

Test Plan: Successfully builds with GOOS=windows GOARCH=amd64

This commit re-adds support for building and running Zoekt on Windows
platforms. We had previously reverted Windows support in commits
70647ba and 0d03621 due to possible memory mapping issues
affecting Linux servers.

The key changes are:
- Added platform-specific implementations for file handling
- Re-introduced Windows-specific signal handling
- Fixed disk usage reporting on Windows via filepath.VolumeName
- Used OS-specific build tags to keep the page size hardcoded on
  non-Windows platforms.

We maintain the approach of using a hardcoded page size (4KB) on
non-Windows platforms, which was the key reason for the initial support
removal, while providing correct Windows implementations.

Test Plan: Successfully builds with GOOS=windows GOARCH=amd64
@keegancsmith
Copy link
Member Author

AI Prompt

In commit 70647ba and 0d03621 we reverted support for this project building and working on windows. The project has changed quite a bit since, can you re-add support? You can run env GOOS=windows GOARCH=amd64 go build -o /dev/null ./cmd/zoekt-webserver ./cmd/zoekt-git-index to test if the build works.

Note this for why we reverted support for windows initially:

We have a suspicion that when we switched to using the mmap-go library
it contributed to an issue where zoekt-webserver stops responding on
some linux versions. Our suspicion has something to do from us
hardcoding the page size to 4k to asking the kernel and how that
interacts with THP.

So on most platforms can we stick to hardcoding the page size.

Copy link
Contributor

@jtibshirani jtibshirani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tackling this and reverting my revert :)

func mustRegisterDiskMonitor(path string) {
prometheus.MustRegister(prometheus.NewGaugeFunc(prometheus.GaugeOpts{
Name: "src_disk_space_available_bytes",
Help: "Amount of free space disk space.",
ConstLabels: prometheus.Labels{"path": path},
}, func() float64 {
usage, _ := disk.Usage(path)
// I know there is no error handling here, and I don't like it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tiny comment: these comments are unrelated to the change and not super useful (and also use "I" in an inaccurate way :))

@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.

//go:build linux || darwin
//go:build !windows
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small comment, this file could be named indexfile_unix.go for consistency with the others.

}

func (f *indexFileFromOS) Read(off, sz uint32) ([]byte, error) {
r := make([]byte, sz)
Copy link
Contributor

@jtibshirani jtibshirani Apr 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are really supporting Windows, we should use mmap here instead of direct reads. This provides the best experience, and I think it's an easier mental model for debugging/ support, to consistently use file mapping. Maybe we could re-introduce mmap-go, but just here for Windows? Or implement it directly, doesn't look too bad.


package index

func init() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part doesn't look right to me -- isn't this already handled in builder_windows.go?

By the way, I'm generally confused on why we are storing the system umask and manually applying it sometimes :)

@keegancsmith
Copy link
Member Author

@jtibshirani thanks for the review! I haven't even read this yet, this was totally vibe coding (but of course would be reviewed and improved by me before marking ready for review).

@jtibshirani
Copy link
Contributor

hehe understood! I was just too interested and wanted to get in a review this week :)

@adlternative
Copy link

Thank you for fixing this issue. However, I'm currently avoiding building zoekt for Windows in Gitea by using the go build tag "!unix" as a temporary workaround. I'll only re-enable this capability once you restore Windows support. Unfortunately, I don't have a Windows environment to test zoekt either:(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants