enhancement: Track disk utilization in $HOME

[holzman]

We extended j-r-u for monitoring disk usage in $HOME for our service. Looks like at least one other person was interested in something similar: #186

[welcome]

Thanks for submitting your first pull request! You are awesome! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please make sure you followed the pull request template, as this will help us review your contribution more quickly.

You can meet the other Jovyans by joining our Discourse forum. There is also a intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[krassowski]

#186 suggests to only show something when usage is high. I would tend to agree that this is a nice feature but it might be important not to overwhelm user with information unless they opt-in (there is only so much space on the status bar).

[holzman]

Well, my take on it is that we should use that space for tracking the resource trinity (CPU, memory, disk) regardless of if their relative values. Should I change this to include an option to enable/disable this?

[holzman]

Ok, I added a configuration boolean, but this will only patch cleanly to 0.7.2. If the approach LGTY, I'll update it for the changes in 1.0.

[yuvipanda]

Unfortunately this doesn't work when $HOME is not its own volume.

If I test this on a hub that mounts NFS for $HOME directory:

>>> python -c 'import os; import psutil; print(psutil.disk_usage(os.getenv("HOME")))'
sdiskusage(total=9223372036853727232, used=2028294111232, free=9223370008559616000, percent=0.0)

This is because I think the output being reported is df -h, while what we actually need here is du -d1 -h $HOME. Unfortunately the latter takes a lot of time to run, as it has to go through each file. This makes it a fairly expensive operation.

So I agree displaying $HOME usage would be incredibly helpful - unfortunately I don't think what we have here works on most hosted systems.

[holzman]

Unfortunately this doesn't work when $HOME is not its own volume.

This is because I think the output being reported is df -h, while what we actually need here is du -d1 -h $HOME. Unfortunately the latter takes a lot of time to run, as it has to go through each file. This makes it a fairly expensive operation.

So I agree displaying $HOME usage would be incredibly helpful - unfortunately I don't think what we have here works on most hosted systems.

Hi @yuvipanda,

Good point. I agree that du is a non-starter.
Best practice for providing home areas over NFS is to make sure they're quotaed -- so what do you think of me changing the code to reporting usage/limit according to quota instead, and falling back to df if the space isn't quotaed?

I think this captures the key use case, which is to give users a heads-up if they're going to run out of room.

[yuvipanda]

@holzman does quota work on NFS? I had thought it does not. Also on everything other than XFS, it only supports it based on per-userid, rather than per-directory, right?

[perllaghu]

I can assert that psutil does work on nfs mounts [at least, on volume mounts in docker images, which are quota'd NFS directories] - reporting the quota'd capacity.

I am replicating [duplicating?] this work at https://github.com/perllaghu/jupyter-resource-usage [it looks surprisingly similar :) ]

It works fine, but I have a few details I want to clarify before I can offer it

[holzman]

Sorry to drop this - got busy and we've just been running a forked j-r-u with this patch.

As far as NFS, @yuvipanda reported different results for his hub with NFS.

How about we just add appropriate docs/warnings to the config boolean?

[holzman]

Closing this since #233 is now merged (thanks @iandesj and @perllaghu!)