Added documentation on RBAC and service account #14

liyinan926 · 2017-09-21T19:01:39Z

foxish · 2017-09-21T20:39:43Z

src/jekyll/running-on-kubernetes.md

+
+#### Driver
+
+The Spark driver pod uses a Kubernetes service account to access the Kubernetes API server to create and watch executor pods. The service account used by the driver pod must have the appropriate permission for the driver to be able to do its work. Specifically, at minimum, the service account must have the [`edit`](https://kubernetes.io/docs/admin/authorization/rbac/#user-facing-roles) [`Role` or `ClusterRole`](https://kubernetes.io/docs/admin/authorization/rbac/#role-and-clusterrole) granted. By default, the driver pod is automatically assigned the `default` service account in the same namespace if no service account is specified when the pod gets created. Depending on the version and setup of Kubernetes deployed, this `default` service account may or may not have the `edit` role granted under the default Kubernetes [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/) policies. Sometimes users may need to specify a custom service account that has the right role granted. Spark on Kubernetes supports specifying a custom service account to be used by the driver pod through the configuration property `spark.kubernetes.authenticate.driver.serviceAccountName=<service account name>`. For example to make the driver pod to use the `spark` service account, a user simply adds the following option to the `spark-submit` command:


Largely LGTM
" in the same namespace " -> in the namespace specified using --kubernetes.namespace.

Can you break this into multiple lines?

foxish · 2017-09-21T20:40:52Z

Thanks!
Can you insert linebreaks? it'll make it easier to comment on individual lines as well.

kimoonkim

The doc looks great. Thanks for writing this!

liyinan926 · 2017-09-22T21:36:49Z

@foxish do you have any more comments? If not, I will merge.

foxish · 2017-09-22T21:38:45Z

src/jekyll/running-on-kubernetes.md

+
+The Spark driver pod uses a Kubernetes service account to access the Kubernetes API server to create and watch executor pods. The service account used by the driver pod must have the appropriate permission for the driver to be able to do its work. 
+
+Specifically, at minimum, the service account must have the [`edit`](https://kubernetes.io/docs/admin/authorization/rbac/#user-facing-roles) [`Role` or `ClusterRole`](https://kubernetes.io/docs/admin/authorization/rbac/#role-and-clusterrole) granted. By default, the driver pod is automatically assigned the `default` service account in the namespace specified by `--kubernetes-namespace`, if no service account is specified when the pod gets created. 


Must have "edit" on pods and services?

Having the default "edit" role is a bit more than we need right?

Made it more specific that a role that allows creating pods and secrets is needed.

foxish · 2017-09-22T21:41:45Z

src/jekyll/running-on-kubernetes.md

+kubectl create clusterrolebinding spark-role --clusterrole=edit --serviceaccount=default:spark --namespace=default
+```
+
+Note that a `Role` can only be used to grant access to resources (like pods) within a single namespace, whereas a `ClusterRole` can be used to grant access to cluster-scoped resources (like nodes) as well as namespaced resources (like pods) across all namespaces. For Spark on Kubernetes, since the driver always creates executor pods in the same namespace, a `Role` is sufficient, although users may use a `ClusterRole` instead. For more information on RBAC authorization and how to configure Kubernetes service accounts for pods, please refer to [here](https://kubernetes.io/docs/admin/authorization/rbac/) and [here](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/).   


replace here with short description of link.

foxish · 2017-09-22T21:43:11Z

src/jekyll/running-on-kubernetes.md

+
+#### Resource Staging Server
+
+The Resource Staging Server (RSS) watches Spark driver pods to detect completed Spark applications so it knows when to safely delete resource bundles of the applications. When running as a pod in the same Kubernetes cluster as the Spark applications, by default (`spark.kubernetes.authenticate.resourceStagingServer.useServiceAccountCredentials` defaults to `true`), the RSS uses the default Kubernetes service account token located at `/var/run/secrets/kubernetes.io/serviceaccount/token` and the CA certificate located at `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`. 


Would help to make it clearer that the location is within the pod, and is the default location that k8s uses. Unclear if it's the local machine being referred to here.

foxish · 2017-09-22T21:44:13Z

LGTM after comments

foxish

Ok to merge

foxish · 2017-09-22T22:56:39Z

src/jekyll/running-on-kubernetes.md

@@ -290,6 +290,55 @@ the command may then look like the following:

 ## Advanced

+### Configuring Kubernetes Roles and Service Accounts


We should specify that this applies to clusters with RBAC. Some clusters use custom authorizers.

Added documentation on RBAC and service account

6033ad1

liyinan926 requested review from kimoonkim and foxish September 21, 2017 19:02

foxish reviewed Sep 21, 2017

View reviewed changes

Addressed comments

029ef05

liyinan926 mentioned this pull request Sep 21, 2017

Added reference YAML files for RBAC configs for driver and shuffle service apache-spark-on-k8s/spark#502

Merged

kimoonkim approved these changes Sep 22, 2017

View reviewed changes

foxish reviewed Sep 22, 2017

View reviewed changes

Addressed more comments

455aa88

liyinan926 force-pushed the rbac branch from 4731003 to 455aa88 Compare September 23, 2017 03:52

foxish merged commit 7397244 into apache-spark-on-k8s:master Sep 24, 2017


		#### Driver

		The Spark driver pod uses a Kubernetes service account to access the Kubernetes API server to create and watch executor pods. The service account used by the driver pod must have the appropriate permission for the driver to be able to do its work. Specifically, at minimum, the service account must have the [`edit`](https://kubernetes.io/docs/admin/authorization/rbac/#user-facing-roles) [`Role` or `ClusterRole`](https://kubernetes.io/docs/admin/authorization/rbac/#role-and-clusterrole) granted. By default, the driver pod is automatically assigned the `default` service account in the same namespace if no service account is specified when the pod gets created. Depending on the version and setup of Kubernetes deployed, this `default` service account may or may not have the `edit` role granted under the default Kubernetes [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/) policies. Sometimes users may need to specify a custom service account that has the right role granted. Spark on Kubernetes supports specifying a custom service account to be used by the driver pod through the configuration property `spark.kubernetes.authenticate.driver.serviceAccountName=<service account name>`. For example to make the driver pod to use the `spark` service account, a user simply adds the following option to the `spark-submit` command:


		The Spark driver pod uses a Kubernetes service account to access the Kubernetes API server to create and watch executor pods. The service account used by the driver pod must have the appropriate permission for the driver to be able to do its work.

		Specifically, at minimum, the service account must have the [`edit`](https://kubernetes.io/docs/admin/authorization/rbac/#user-facing-roles) [`Role` or `ClusterRole`](https://kubernetes.io/docs/admin/authorization/rbac/#role-and-clusterrole) granted. By default, the driver pod is automatically assigned the `default` service account in the namespace specified by `--kubernetes-namespace`, if no service account is specified when the pod gets created.


		#### Resource Staging Server

		The Resource Staging Server (RSS) watches Spark driver pods to detect completed Spark applications so it knows when to safely delete resource bundles of the applications. When running as a pod in the same Kubernetes cluster as the Spark applications, by default (`spark.kubernetes.authenticate.resourceStagingServer.useServiceAccountCredentials` defaults to `true`), the RSS uses the default Kubernetes service account token located at `/var/run/secrets/kubernetes.io/serviceaccount/token` and the CA certificate located at `/var/run/secrets/kubernetes.io/serviceaccount/ca.crt`.

		@@ -290,6 +290,55 @@ the command may then look like the following:

		## Advanced

		### Configuring Kubernetes Roles and Service Accounts

Added documentation on RBAC and service account #14

Added documentation on RBAC and service account #14

Uh oh!

Conversation

liyinan926 commented Sep 21, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

foxish commented Sep 21, 2017

Uh oh!

kimoonkim left a comment

Choose a reason for hiding this comment

Uh oh!

liyinan926 commented Sep 22, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

foxish commented Sep 22, 2017

Uh oh!

foxish left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!