Skip to content

Add user-defined checker for server side #502

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Dec 17, 2020

Conversation

hunterlxt
Copy link
Member

@hunterlxt hunterlxt commented Dec 14, 2020

Users can provide custom checker to decide if the grpc call should be aborted in advance or be continued. For operations like check common name, a lot of redundant code can be avoided.

How to implement?

Execute the custom closure provided by the user at the execute function from src/call/server.rs. If the user declares that the grpc call has been aborted, then skip the subsequent grpc handler.

Signed-off-by: Xintao [email protected]

Signed-off-by: Xintao <[email protected]>
Signed-off-by: Xintao <[email protected]>
Signed-off-by: Xintao <[email protected]>
@hunterlxt hunterlxt changed the title Add interceptor for server side Add user-defined checker for server side Dec 15, 2020
Signed-off-by: Xintao <[email protected]>
@hunterlxt
Copy link
Member Author

Post the benchmark here (first is this PR, second is master):

rust_protobuf_async_unary_ping_pong  
==========This PR===========
I1215 17:18:11.618205813  102765 report.cc:82]               QPS: 10142.1
I1215 17:18:11.619358937  102765 report.cc:122]              QPS: 10142.1 (253.6/server core)
I1215 17:18:11.619372193  102765 report.cc:127]              Latencies (50/90/95/99/99.9%-ile): 91.7/123.5/134.5/161.3/240.2 us
I1215 17:18:11.619376904  102765 report.cc:137]              Server system time: 47.25%
I1215 17:18:11.619380564  102765 report.cc:139]              Server user time:   13.92%
I1215 17:18:11.619383896  102765 report.cc:141]              Client system time: -8.72%
I1215 17:18:11.619387240  102765 report.cc:143]              Client user time:   38.43%
I1215 17:18:11.619390774  102765 report.cc:148]              Server CPU usage: 13.08%
I1215 17:18:11.619394341  102765 report.cc:153]              Client Polls per Request: 0.00
I1215 17:18:11.619397488  102765 report.cc:155]              Server Polls per Request: 0.00
I1215 17:18:11.619401247  102765 report.cc:160]              Server Queries/CPU-sec: 16580.23
I1215 17:18:11.619404768  102765 report.cc:162]              Client Queries/CPU-sec: 34134.65
==========Master==========
I1215 17:24:23.904895795  114810 report.cc:82]               QPS: 9831.0
I1215 17:24:23.906136198  114810 report.cc:122]              QPS: 9831.0 (245.8/server core)
I1215 17:24:23.906152399  114810 report.cc:127]              Latencies (50/90/95/99/99.9%-ile): 96.6/129.5/141.5/188.2/874.9 us
I1215 17:24:23.906158311  114810 report.cc:137]              Server system time: 21.79%
I1215 17:24:23.906162716  114810 report.cc:139]              Server user time:   17.25%
I1215 17:24:23.906166820  114810 report.cc:141]              Client system time: 43.20%
I1215 17:24:23.906170846  114810 report.cc:143]              Client user time:   39.99%
I1215 17:24:23.906175181  114810 report.cc:148]              Server CPU usage: 14.88%
I1215 17:24:23.906179467  114810 report.cc:153]              Client Polls per Request: 0.00
I1215 17:24:23.906183266  114810 report.cc:155]              Server Polls per Request: 0.00
I1215 17:24:23.906187841  114810 report.cc:160]              Server Queries/CPU-sec: 25179.17
I1215 17:24:23.906192131  114810 report.cc:162]              Client Queries/CPU-sec: 11817.08

rust_protobuf_async_unary_qps_unconstrained
==========This PR===========
test 1
I1215 17:21:33.244628766  110304 report.cc:82]               QPS: 358874.7
I1215 17:21:33.246340175  110304 report.cc:122]              QPS: 358874.7 (8971.9/server core)
I1215 17:21:33.246356691  110304 report.cc:127]              Latencies (50/90/95/99/99.9%-ile): 15125.5/31634.8/37362.7/50631.8/120242.1 us
I1215 17:21:33.246362334  110304 report.cc:137]              Server system time: 327.65%
I1215 17:21:33.246366880  110304 report.cc:139]              Server user time:   1245.02%
I1215 17:21:33.246371064  110304 report.cc:141]              Client system time: 313.07%
I1215 17:21:33.246375266  110304 report.cc:143]              Client user time:   1418.86%
I1215 17:21:33.246379530  110304 report.cc:148]              Server CPU usage: 92.14%
I1215 17:21:33.246383797  110304 report.cc:153]              Client Polls per Request: 0.00
I1215 17:21:33.246387476  110304 report.cc:155]              Server Polls per Request: 0.00
I1215 17:21:33.246391926  110304 report.cc:160]              Server Queries/CPU-sec: 22806.26
I1215 17:21:33.246396124  110304 report.cc:162]              Client Queries/CPU-sec: 20721.03
test 2
I1215 17:29:37.166054863  125765 report.cc:82]               QPS: 362230.9
I1215 17:29:37.167597202  125765 report.cc:122]              QPS: 362230.9 (9055.8/server core)
I1215 17:29:37.167613433  125765 report.cc:127]              Latencies (50/90/95/99/99.9%-ile): 14751.4/31115.6/36854.0/51507.7/152245.2 us
I1215 17:29:37.167619003  125765 report.cc:137]              Server system time: 311.68%
I1215 17:29:37.167623687  125765 report.cc:139]              Server user time:   1253.79%
I1215 17:29:37.167627819  125765 report.cc:141]              Client system time: 323.53%
I1215 17:29:37.167631929  125765 report.cc:143]              Client user time:   1377.64%
I1215 17:29:37.167636283  125765 report.cc:148]              Server CPU usage: 91.89%
I1215 17:29:37.167640540  125765 report.cc:153]              Client Polls per Request: 0.00
I1215 17:29:37.167644304  125765 report.cc:155]              Server Polls per Request: 0.00
I1215 17:29:37.167648892  125765 report.cc:160]              Server Queries/CPU-sec: 23152.24
I1215 17:29:37.167653144  125765 report.cc:162]              Client Queries/CPU-sec: 21293.02
==========Master==========
test 1
I1215 17:25:42.032429849  117193 report.cc:82]               QPS: 372686.6
I1215 17:25:42.034157581  117193 report.cc:122]              QPS: 372686.6 (9317.2/server core)
I1215 17:25:42.034173201  117193 report.cc:127]              Latencies (50/90/95/99/99.9%-ile): 14713.2/30466.6/35666.2/46606.1/84983.4 us
I1215 17:25:42.034178724  117193 report.cc:137]              Server system time: 339.40%
I1215 17:25:42.034183273  117193 report.cc:139]              Server user time:   1252.90%
I1215 17:25:42.034187402  117193 report.cc:141]              Client system time: 332.76%
I1215 17:25:42.034191547  117193 report.cc:143]              Client user time:   1418.92%
I1215 17:25:42.034195797  117193 report.cc:148]              Server CPU usage: 92.54%
I1215 17:25:42.034200009  117193 report.cc:153]              Client Polls per Request: 0.00
I1215 17:25:42.034203867  117193 report.cc:155]              Server Polls per Request: 0.00
I1215 17:25:42.034208396  117193 report.cc:160]              Server Queries/CPU-sec: 23422.26
I1215 17:25:42.034212615  117193 report.cc:162]              Client Queries/CPU-sec: 21275.95
test 2
I1215 17:26:45.717547988  119224 report.cc:82]               QPS: 348712.0
I1215 17:26:45.719688674  119224 report.cc:122]              QPS: 348712.0 (8717.8/server core)
I1215 17:26:45.719706265  119224 report.cc:127]              Latencies (50/90/95/99/99.9%-ile): 15538.3/32525.0/38522.9/54028.6/98141.4 us
I1215 17:26:45.719711910  119224 report.cc:137]              Server system time: 311.73%
I1215 17:26:45.719716463  119224 report.cc:139]              Server user time:   1162.84%
I1215 17:26:45.719729997  119224 report.cc:141]              Client system time: 320.58%
I1215 17:26:45.719734891  119224 report.cc:143]              Client user time:   1399.97%
I1215 17:26:45.719739290  119224 report.cc:148]              Server CPU usage: 92.94%
I1215 17:26:45.719743446  119224 report.cc:153]              Client Polls per Request: 0.00
I1215 17:26:45.719747186  119224 report.cc:155]              Server Polls per Request: 0.00
I1215 17:26:45.719751664  119224 report.cc:160]              Server Queries/CPU-sec: 23652.89
I1215 17:26:45.719755846  119224 report.cc:162]              Client Queries/CPU-sec: 20267.52

skyzh
skyzh previously approved these changes Dec 15, 2020
Copy link
Member

@skyzh skyzh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Previously I thought we could use static dispatch (e.g. f: impl Fn(xxx) -> xxx), but it seems not approachable.

@hunterlxt
Copy link
Member Author

hunterlxt commented Dec 15, 2020

This test add some logic to test add_checker.

rust_protobuf_async_unary_qps_unconstrained
==========direct===========
test 1
I1215 17:47:10.035174170  160773 report.cc:82]               QPS: 349207.6
I1215 17:47:10.036404026  160773 report.cc:122]              QPS: 349207.6 (8730.2/server core)
I1215 17:47:10.036419828  160773 report.cc:127]              Latencies (50/90/95/99/99.9%-ile): 15344.1/32501.2/38570.5/55781.5/124581.4 us
I1215 17:47:10.036425791  160773 report.cc:137]              Server system time: 294.41%
I1215 17:47:10.036430528  160773 report.cc:139]              Server user time:   1227.77%
I1215 17:47:10.036434678  160773 report.cc:141]              Client system time: 284.09%
I1215 17:47:10.036438758  160773 report.cc:143]              Client user time:   1366.20%
I1215 17:47:10.036443089  160773 report.cc:148]              Server CPU usage: 92.61%
I1215 17:47:10.036447636  160773 report.cc:153]              Client Polls per Request: 0.00
I1215 17:47:10.036451449  160773 report.cc:155]              Server Polls per Request: 0.00
I1215 17:47:10.036455999  160773 report.cc:160]              Server Queries/CPU-sec: 22924.62
I1215 17:47:10.036460201  160773 report.cc:162]              Client Queries/CPU-sec: 21160.45
test 2
I1215 17:48:16.068267522  162875 report.cc:82]               QPS: 364934.3
I1215 17:48:16.069491695  162875 report.cc:122]              QPS: 364934.3 (9123.4/server core)
I1215 17:48:16.069508067  162875 report.cc:127]              Latencies (50/90/95/99/99.9%-ile): 14265.7/30523.3/36288.4/54138.6/248986.8 us
I1215 17:48:16.069513687  162875 report.cc:137]              Server system time: 334.97%
I1215 17:48:16.069518391  162875 report.cc:139]              Server user time:   1270.32%
I1215 17:48:16.069522828  162875 report.cc:141]              Client system time: 325.90%
I1215 17:48:16.069527243  162875 report.cc:143]              Client user time:   1381.27%
I1215 17:48:16.069531688  162875 report.cc:148]              Server CPU usage: 91.44%
I1215 17:48:16.069536258  162875 report.cc:153]              Client Polls per Request: 0.00
I1215 17:48:16.069540207  162875 report.cc:155]              Server Polls per Request: 0.00
I1215 17:48:16.069544976  162875 report.cc:160]              Server Queries/CPU-sec: 22720.63
I1215 17:48:16.069558983  162875 report.cc:162]              Client Queries/CPU-sec: 21376.59
==========add_checker==========
test 1
I1215 17:54:07.843513272  171438 report.cc:82]               QPS: 358224.1
I1215 17:54:07.844719267  171438 report.cc:122]              QPS: 358224.1 (8955.6/server core)
I1215 17:54:07.844736554  171438 report.cc:127]              Latencies (50/90/95/99/99.9%-ile): 15026.9/31433.2/37209.2/51882.1/153227.1 us
I1215 17:54:07.844742560  171438 report.cc:137]              Server system time: 327.68%
I1215 17:54:07.844747430  171438 report.cc:139]              Server user time:   1303.24%
I1215 17:54:07.844751760  171438 report.cc:141]              Client system time: 325.78%
I1215 17:54:07.844755941  171438 report.cc:143]              Client user time:   1391.79%
I1215 17:54:07.844760669  171438 report.cc:148]              Server CPU usage: 92.21%
I1215 17:54:07.844765109  171438 report.cc:153]              Client Polls per Request: 0.00
I1215 17:54:07.844768950  171438 report.cc:155]              Server Polls per Request: 0.00
I1215 17:54:07.844773499  171438 report.cc:160]              Server Queries/CPU-sec: 21946.40
I1215 17:54:07.844777745  171438 report.cc:162]              Client Queries/CPU-sec: 20856.45
test2
I1215 17:55:06.026081504  174155 report.cc:82]               QPS: 344677.3
I1215 17:55:06.027349050  174155 report.cc:122]              QPS: 344677.3 (8616.9/server core)
I1215 17:55:06.027365879  174155 report.cc:127]              Latencies (50/90/95/99/99.9%-ile): 15117.4/32490.4/38971.1/60911.3/231242.6 us
I1215 17:55:06.027371898  174155 report.cc:137]              Server system time: 299.75%
I1215 17:55:06.027378742  174155 report.cc:139]              Server user time:   1247.28%
I1215 17:55:06.027385438  174155 report.cc:141]              Client system time: 324.03%
I1215 17:55:06.027392179  174155 report.cc:143]              Client user time:   1337.43%
I1215 17:55:06.027399116  174155 report.cc:148]              Server CPU usage: 92.22%
I1215 17:55:06.027406312  174155 report.cc:153]              Client Polls per Request: 0.00
I1215 17:55:06.027412927  174155 report.cc:155]              Server Polls per Request: 0.00
I1215 17:55:06.027420547  174155 report.cc:160]              Server Queries/CPU-sec: 22268.15
I1215 17:55:06.027427724  174155 report.cc:162]              Client Queries/CPU-sec: 20745.42

Signed-off-by: Xintao <[email protected]>
@yiwu-arbug
Copy link

@BusyJay

@gregwebs
Copy link

It seems this library is missing a middleware implementation. This patch is a very limited form of middleware where an abort can be called. The way to make it more of a full middleware is to pass the checker the payload and to allow it to modify the payload (sometimes this is achieved by passing a new payload on to a handler continuation function).

@gregwebs
Copy link

This seems like a good enough solution until we take the time to create a proper middleware interface.

let _ = client.say_hello(&req).unwrap();
let _ = client.say_hello(&req).unwrap();

flag_2.store(true, Ordering::Relaxed);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should use SeqCst order for both of the atomic operations.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address comment, BTW any ordering used here is correct.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mistake, if not use SeqSct, flag_2.store(true, Ordering::Relaxed); maybe be reordered behind assert.

yiwu-arbug
yiwu-arbug previously approved these changes Dec 16, 2020
Copy link

@yiwu-arbug yiwu-arbug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I'm not familiar with grpc-rs. @BusyJay can you take a look?

Signed-off-by: Xintao <[email protected]>
Signed-off-by: Xintao <[email protected]>
Signed-off-by: Xintao <[email protected]>
@hunterlxt hunterlxt requested a review from BusyJay December 17, 2020 09:19
@hunterlxt
Copy link
Member Author

leave some test results with trait object version FYI:

  1. QPS: 354827.0
  2. QPS: 357519.2
  3. QPS: 356797.2

Signed-off-by: Xintao <[email protected]>
@BusyJay
Copy link
Member

BusyJay commented Dec 17, 2020

I do prefer a complete interceptor implementation. But I'm also OK with current design.

@BusyJay BusyJay merged commit aef8a77 into tikv:master Dec 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants