Add work-in-progress for visualizing gradients tutorial (issue #3186) #3389
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3389
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 031ba22 with merge base 06f9c4b ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
github-actions
bot
added
advanced
docathon-h1-2025
|
Generally seems to be headed in the right direction in terms of tone and organization from my perspective. |
There was a problem hiding this comment.
Thanks for the working on this tutorial. Overall I'd say though that this section (prior to the actual visualizing gradients part) can be much shorter.
By the end of this tutorial, you will be able to:
Differentiate between leaf and non-leaf tensors
have a diagram from https://github.com/szagoruyko/pytorchviz, point to the leafs
Know when to use\
retain_gradvs. ``require_grad`
"use requires_grad for leaf, use retain_grad for non-leaf"
Fixes #3186
Description
Add initial draft for visualizing gradients tutorial. Link is here
This write-up starts by discussing the difference between leaf and non-leaf tensors, and the associated
requires_gradandretains_gradclass attributes.It then will go through a real-world example of visualizing gradients by using the
retains_gradin a more complicated neural network like ResNet (this part is a work-in-progress).I put the tutorial in the advanced_source directory but perhaps it would be better sorted as an intermediate tutorial or a recipe. Open to suggestions.
What's written so far is how I imagined structuring the tutorial. If you have any comments about the overall flow / material let me know. Feel free to comment on the wordage and tone as well, just know that I plan on revising tutorial as this is just the first go at it.
Checklist