Skip to content

Commit c310763

Browse files
Zha0q1apeforest
authored andcommitted
update profiler tutorial (apache#15580)
* update profiler tutorial * Update profiler.md * Update profiler.md * Update profiler.md * Re-Trigger build * Re-Trigger build * Re-Trigger build * Update profiler.md
1 parent 5e6ba7b commit c310763

File tree

1 file changed

+15
-12
lines changed

1 file changed

+15
-12
lines changed

docs/tutorials/python/profiler.md

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -195,10 +195,10 @@ print(profiler.dumps())
195195
You can also dump the information collected by the profiler into a `json` file using the `profiler.dump()` function and view it in a browser.
196196

197197
```python
198-
profiler.dump()
198+
profiler.dump(finished=False)
199199
```
200200

201-
`dump()` creates a `json` file which can be viewed using a trace consumer like `chrome://tracing` in the Chrome browser. Here is a snapshot that shows the output of the profiling we did above.
201+
`dump()` creates a `json` file which can be viewed using a trace consumer like `chrome://tracing` in the Chrome browser. Here is a snapshot that shows the output of the profiling we did above. Note that setting the `finished` parameter to `False` will prevent the profiler from finishing dumping to file. If you just use `profiler.dump()`, you will no longer be able to profile the remaining sections of your model.
202202

203203
![Tracing Screenshot](https://raw.githubusercontent.com/dmlc/web-data/master/mxnet/tutorials/python/profiler/profiler_output_chrome.png)
204204

@@ -214,11 +214,6 @@ Should the existing NDArray operators fail to meet all your model's needs, MXNet
214214
Let's try profiling custom operators with the following code example:
215215

216216
```python
217-
218-
import mxnet as mx
219-
from mxnet import nd
220-
from mxnet import profiler
221-
222217
class MyAddOne(mx.operator.CustomOp):
223218
def forward(self, is_train, req, in_data, out_data, aux):
224219
self.assign(out_data[0], req[0], in_data[0]+1)
@@ -246,15 +241,17 @@ class CustomAddOneProp(mx.operator.CustomOpProp):
246241

247242
inp = mx.nd.zeros(shape=(500, 500))
248243

249-
profiler.set_config(profile_all=True, continuous_dump = True)
244+
profiler.set_config(profile_all=True, continuous_dump=True, \
245+
aggregate_stats=True)
250246
profiler.set_state('run')
251247

252248
w = nd.Custom(inp, op_type="MyAddOne")
253249

254250
mx.nd.waitall()
255251

256252
profiler.set_state('stop')
257-
profiler.dump()
253+
print(profiler.dumps())
254+
profiler.dump(finished=False)
258255
```
259256

260257
Here, we have created a custom operator called `MyAddOne`, and within its `forward()` function, we simply add one to the input. We can visualize the dump file in `chrome://tracing/`:
@@ -267,10 +264,10 @@ Please note that: to be able to see the previously described information, you ne
267264

268265
```python
269266
# Set profile_all to True
270-
profiler.set_config(profile_all=True, aggregate_stats=True, continuous_dump = True)
267+
profiler.set_config(profile_all=True, aggregate_stats=True, continuous_dump=True)
271268
# OR, Explicitly Set profile_symbolic and profile_imperative to True
272-
profiler.set_config(profile_symbolic = True, profile_imperative = True, \
273-
aggregate_stats=True, continuous_dump = True)
269+
profiler.set_config(profile_symbolic=True, profile_imperative=True, \
270+
aggregate_stats=True, continuous_dump=True)
274271

275272
profiler.set_state('run')
276273
# Use Symbolic Mode
@@ -280,9 +277,15 @@ c = b.bind(mx.cpu(), {'a': inp})
280277
y = c.forward()
281278
mx.nd.waitall()
282279
profiler.set_state('stop')
280+
print(profiler.dumps())
283281
profiler.dump()
284282
```
285283

284+
### Some Rules to Pay Attention to
285+
1. Always use `profiler.dump(finished=False)` if you do not intend to finish dumping to file. Otherwise, calling `profiler.dump()` in the middle of your model may lead to unexpected behaviors; and if you subsequently call `profiler.set_config()`, the program will error out.
286+
287+
2. You can only dump to one file. Do not change the target file by calling `profiler.set_config(filename='new_name.json')` in the middle of your model. This will lead to incomplete dump outputs.
288+
286289
## Advanced: Using NVIDIA Profiling Tools
287290

288291
MXNet's Profiler is the recommended starting point for profiling MXNet code, but NVIDIA also provides a couple of tools for low-level profiling of CUDA code: [NVProf](https://devblogs.nvidia.com/cuda-pro-tip-nvprof-your-handy-universal-gpu-profiler/), [Visual Profiler](https://developer.nvidia.com/nvidia-visual-profiler) and [Nsight Compute](https://developer.nvidia.com/nsight-compute). You can use these tools to profile all kinds of executables, so they can be used for profiling Python scripts running MXNet. And you can use these in conjunction with the MXNet Profiler to see high-level information from MXNet alongside the low-level CUDA kernel information.

0 commit comments

Comments
 (0)