Provisioning updates. #5361

kwoodson · 2017-09-11T18:14:14Z

These are provisioning updates and features that allow node bootstrapping.

List of updates:

Removed the need to specify openshift_node_bootstrap in ami creation.
Skip failures in evaluate groups if no etcd hosts are defined
More default variables for the openshift_aws role
- Termination policy for scale groups
- replace instances or replace all
- label support
- ability to run bootstrap.yml on startup
- moved the logic of the user_data into a template
- remove any openshift.fact files because ami creation and where the node ends up are different
- install bootstrap.yml into AMI
- variable to turn on/off - run the bootstrap.yml on start
- install openshift-ansible-roles into AMI for later boot use (use yedit to modify files)
- override the user_data if desired
Added the feature to create a configmap per node group
Added default variables for openshift_master to support node group configmaps
- config dir
- cloud provider
- node-config settings
  - kubeletargs
  - node labels

Instead of using a template for the node-config.yaml, we call oc adm create-node-config. We have a list of edits that occur to clean up the generation before it is specialized for each node group. Once each node group has specialized their own copy they are placed in the kube-system namespace with the name node-config- configmap.

This is going to require code changes from @smarterclayton to take the configmap and lay it down as the node-config.yaml.

kwoodson · 2017-09-11T18:15:03Z

aos-ci-test

openshift-bot · 2017-09-11T22:22:24Z

success: "aos-ci-jenkins/OS_3.6_containerized, aos-ci-jenkins/OS_3.6_containerized_e2e_tests" for f6b17c7 (logs)

openshift-bot · 2017-09-11T22:24:57Z

success: "aos-ci-jenkins/OS_3.6_NOT_containerized, aos-ci-jenkins/OS_3.6_NOT_containerized_e2e_tests" for f6b17c7 (logs)

kwoodson · 2017-09-13T17:43:33Z

aos-ci-test

openshift-bot · 2017-09-14T00:26:45Z

success: "aos-ci-jenkins/OS_3.6_NOT_containerized, aos-ci-jenkins/OS_3.6_NOT_containerized_e2e_tests" for a59a356 (logs)

openshift-bot · 2017-09-14T00:29:05Z

success: "aos-ci-jenkins/OS_3.6_containerized, aos-ci-jenkins/OS_3.6_containerized_e2e_tests" for a59a356 (logs)

kwoodson · 2017-09-21T02:52:58Z

/retest

kwoodson · 2017-09-22T18:40:53Z

/retest

kwoodson · 2017-09-22T18:43:36Z

/retest

kwoodson · 2017-10-09T18:52:30Z

@michaelgugino @sdodson, @smarterclayton
These changes have passed the latest round of provision and install tests.

smarterclayton · 2017-10-16T07:20:39Z

roles/openshift_master/defaults/main.yml

+
+openshift_master_node_config_networkconfig_mtu: 1450
+
+openshift_master_node_config_kubeletargs_cpu: 500m


These seem like node group specific settings, applied as defaults to a node-config generated per node group.

sdodson · 2017-10-17T19:35:28Z

I don't want to block this PR any longer. We definitely need to plan follow up work though.
/lgtm

smarterclayton · 2017-10-17T20:41:05Z

Yeah, this can continue in subsequent PRs. I have GCE bootstrapping and I'm close to a PR that turns it on for origin's conformance_gce suite, which will stepping stone to doing a full image build -> cluster turn up model.

kwoodson · 2017-10-17T20:47:05Z

@sdodson @smarterclayton,
I'd like to change the order of the installation. I'd like to install the control plane along withe the nodes on the masters, scale up with the nodes, then finish by installing the openshift_hosted playbooks. That way everything comes up in the right order.

smarterclayton · 2017-10-17T21:15:37Z

roles/openshift_aws/templates/user_data.j2

+  content: |
+    openshift_group_type: {{ openshift_aws_node_group_type }}
+{%   if openshift_aws_node_group_type != 'master' %}
+- path: /etc/origin/node/csr_kubeconfig


I'd prefer to call this bootstrap.kubeconfig to be consistent with its use and other kubeconfig files. Can be a followup.

Ok, sounds good.

kwoodson · 2017-10-17T21:21:58Z

/test logging

smarterclayton · 2017-10-17T21:27:40Z

roles/openshift_master/defaults/main.yml

+
+openshift_master_client_binary: "{{ openshift.common.client_binary if openshift is defined else 'oc' }}"
+
+openshift_master_config_imageconfig_format: "{{ oreg_url if oreg_url != '' else 'registry.access.redhat.com/openshift3/ose-${component}:${version}' }}"


Don't we already have this in another form?

I think the one that is used is oreg_url so I stayed with it. I'm open to change here.

openshift-ci-robot · 2017-10-17T22:27:26Z

@kwoodson: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/openshift-jenkins/logging	`47d2e20`	link	`/test logging`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

michaelgugino

I would like to see a note in the documentation or in the plays themselves that indicates the provisioning parts are still under active development.

sdodson · 2017-10-18T02:14:02Z

I would like to see a note in the documentation or in the plays themselves that indicates the provisioning parts are still under active development.

I agree, I think we'd forego docs.openshift.com documentation for now but we should make it clear in playbooks/aws/README.md that these are alpha quality.

smarterclayton · 2017-10-23T13:59:13Z

We definitely need to split openshift-cluster/config.yml into three equal playbooks that can be called independently - master, node, and hosted/post-install. Bootstrapping needs master up to get the bootstrap.kubeconfig, and ideally as kenny notes we want to:

provision the control plane physical resources (optionally provision non-bootstrapped nodes)
install the control plane
set up the bootstrapping config for use by bootstrapped nodes
provision bootstrapped node groups AND/OR configure non-bootstrapped nodes
install hosted components once sufficient nodes are available

Is that slated yet?

kwoodson · 2017-10-23T14:53:54Z

@smarterclayton,

I opened up this PR this morning on something I had been testing. #5846.

I'd like to continue this discussion in that PR.

openshift-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Sep 11, 2017

kwoodson force-pushed the fix_bootstrap_files branch from f6b17c7 to a59a356 Compare September 12, 2017 13:55

kwoodson force-pushed the fix_bootstrap_files branch 2 times, most recently from 453e5c6 to dcdc2c7 Compare September 18, 2017 21:46

openshift-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 18, 2017

kwoodson force-pushed the fix_bootstrap_files branch 2 times, most recently from f4c4291 to 3fc903b Compare September 20, 2017 21:09

openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Sep 20, 2017

kwoodson force-pushed the fix_bootstrap_files branch from 3fc903b to cb14a7a Compare September 20, 2017 22:48

kwoodson changed the title ~~Do not remove files for bootstrap if resolv or dns.~~ Provisioning updates. Sep 20, 2017

kwoodson force-pushed the fix_bootstrap_files branch 3 times, most recently from 32cd654 to ff05261 Compare September 22, 2017 18:08

kwoodson requested a review from smarterclayton September 22, 2017 18:13

kwoodson self-assigned this Sep 22, 2017

kwoodson force-pushed the fix_bootstrap_files branch from ff05261 to e1142cf Compare September 22, 2017 18:43

kwoodson force-pushed the fix_bootstrap_files branch 3 times, most recently from 2c7a78b to 0e43e4e Compare September 27, 2017 13:36

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 11, 2017

kwoodson force-pushed the fix_bootstrap_files branch from eed1e73 to 8d3ac16 Compare October 11, 2017 17:31

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 11, 2017

kwoodson force-pushed the fix_bootstrap_files branch 2 times, most recently from 6c787e1 to 73bf47f Compare October 12, 2017 19:56

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 12, 2017

kwoodson force-pushed the fix_bootstrap_files branch from 73bf47f to ab59305 Compare October 12, 2017 20:40

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 12, 2017

smarterclayton reviewed Oct 16, 2017

View reviewed changes

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 17, 2017

kwoodson force-pushed the fix_bootstrap_files branch from ab59305 to 6bf2d71 Compare October 17, 2017 13:39

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 17, 2017

Do not remove files for bootstrap if resolv or dns.

47d2e20

kwoodson force-pushed the fix_bootstrap_files branch from 6bf2d71 to 47d2e20 Compare October 17, 2017 19:34

openshift-ci-robot assigned sdodson Oct 17, 2017

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 17, 2017

smarterclayton reviewed Oct 17, 2017

View reviewed changes

michaelgugino reviewed Oct 17, 2017

View reviewed changes

sdodson merged commit 63b77fb into openshift:master Oct 18, 2017

kwoodson deleted the fix_bootstrap_files branch March 5, 2018 15:42


		openshift_master_node_config_networkconfig_mtu: 1450

		openshift_master_node_config_kubeletargs_cpu: 500m


		openshift_master_client_binary: "{{ openshift.common.client_binary if openshift is defined else 'oc' }}"

		openshift_master_config_imageconfig_format: "{{ oreg_url if oreg_url != '' else 'registry.access.redhat.com/openshift3/ose-${component}:${version}' }}"

Provisioning updates. #5361

Provisioning updates. #5361

Uh oh!

Conversation

kwoodson commented Sep 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kwoodson commented Sep 11, 2017

Uh oh!

openshift-bot commented Sep 11, 2017

Uh oh!

openshift-bot commented Sep 11, 2017

Uh oh!

kwoodson commented Sep 13, 2017

Uh oh!

openshift-bot commented Sep 14, 2017

Uh oh!

openshift-bot commented Sep 14, 2017

Uh oh!

kwoodson commented Sep 21, 2017

Uh oh!

kwoodson commented Sep 22, 2017

Uh oh!

kwoodson commented Sep 22, 2017

Uh oh!

kwoodson commented Oct 9, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

smarterclayton Oct 16, 2017

Choose a reason for hiding this comment

Uh oh!

sdodson commented Oct 17, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

smarterclayton commented Oct 17, 2017

Uh oh!

kwoodson commented Oct 17, 2017

Uh oh!

smarterclayton Oct 17, 2017

Choose a reason for hiding this comment

Uh oh!

kwoodson Oct 17, 2017

Choose a reason for hiding this comment

Uh oh!

kwoodson commented Oct 17, 2017

Uh oh!

smarterclayton Oct 17, 2017

Choose a reason for hiding this comment

Uh oh!

kwoodson Oct 17, 2017

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Oct 17, 2017

Uh oh!

michaelgugino left a comment

Choose a reason for hiding this comment

Uh oh!

sdodson commented Oct 18, 2017

Uh oh!

smarterclayton commented Oct 23, 2017

Uh oh!

kwoodson commented Oct 23, 2017

Uh oh!

Uh oh!

kwoodson commented Sep 11, 2017 •

edited

Loading

kwoodson commented Oct 9, 2017 •

edited

Loading

sdodson commented Oct 17, 2017 •

edited

Loading