Rebase kubernetes 1.8.1 #43

mfojtik · 2017-10-12T11:31:52Z

Rebase for kube 1.8.1

mfojtik · 2017-10-12T11:32:31Z

@deads2k @liggitt @sttts @soltysh PTAL (feel free to review as I go, I hope to have 100% of the list done today).

sttts · 2017-10-12T11:52:10Z

pkg/kubectl/kubectl.go

@@ -142,6 +142,30 @@ var ResourcesShortcutStatic = []ResourceShortcuts{
 		ShortForm: schema.GroupResource{Group: "extensions", Resource: "psp"},
 		LongForm:  schema.GroupResource{Group: "extensions", Resource: "podSecurityPolicies"},
 	},
+	{
+		ShortForm: schema.GroupResource{Group: "apps.openshift.io", Resource: "dc"},


with shortcuts in discovery, do we still need this?

with shortcuts in discovery, do we still need this?

@mfojtik we support one level of skew. I think we supported shortnames in kube 1.7, we just need to make sure we've wired them.

@mfojtik we support one level of skew. I think we supported shortnames in kube 1.7, we just need to make sure we've wired them.

Here openshift/origin#16823

thanks! i will drop this carry

sttts · 2017-10-12T11:59:16Z

How do UPSTREAM: <carry>: update clientset generator for openshift groups and UPSTREAM: <carry>: update client namer rules for amguity relate? Also UPSTREAM: <carry>: update group references.

deads2k · 2017-10-12T11:59:32Z

Can you post a google doc like https://docs.google.com/spreadsheets/d/1AVI6MoSxELwDQspL9F1Y10S39eLm1VpSpMGdfXRKxHU/edit

sttts · 2017-10-12T12:00:18Z

.gitignore

@@ -110,12 +110,9 @@ kubernetes.tar.gz
 # generated files in any directory
 # TODO(thockin): uncomment this when we stop committing the generated files.
 #zz_generated.*
-zz_generated.openapi.go


Is the actual openapi file missing?

@sttts @soltysh said I should add it in a commit to the end of this rebase.

mfojtik · 2017-10-12T12:01:50Z

@deads2k sure, I'm tracking this locally in text doc, but will upload that to spreadsheet

liggitt · 2017-10-12T15:32:06Z

don't forget the checklist in openshift/origin#16065

mfojtik · 2017-10-12T15:33:24Z

Spreadsheet: https://docs.google.com/a/redhat.com/spreadsheets/d/1IHQtn37AoDVjyEk8elOT047Gz-bdY7k3wKixPKm99lo/edit?usp=sharing

liggitt · 2017-10-12T15:34:39Z

we can drop UPSTREAM: <carry>: Tolerate node ExternalID changes with no cloud provider, we've unified with upstream for nodes being able to delete/create themselves

mfojtik · 2017-10-12T15:35:10Z

I plan to squash all codegen/client-gen carries into one commit. Also need to sort out the 78caf9d31d UPSTREAM: <carry>: allow controller context injection to share informers which seems to drift significantly in upstream...

liggitt · 2017-10-12T15:39:56Z

we can drop UPSTREAM: 45294: Fix protobuf generator for aliases to repeated types

liggitt · 2017-10-12T15:41:21Z

UPSTREAM: 49137: proxier is it really nil merged looking different... can we drop it?

liggitt · 2017-10-12T15:43:34Z

this merged, can we drop UPSTREAM: 48960: No warning event for DNSSearchForming?

mfojtik · 2017-10-12T15:46:53Z

@liggitt dropped and updated the spreadsheet, thanks!

liggitt · 2017-10-12T15:47:31Z

squash

UPSTREAM: <carry>: update namespace lifecycle to allow review APIs
UPSTREAM: <carry>: ignored namespace lifecycle resources

mfojtik · 2017-10-12T15:53:30Z

@liggitt squashed and also squashed the client-gen/codegen carries into one.

mfojtik · 2017-10-16T09:27:11Z

tracking integration failures (note this is result of make test-integration, I will try to re-run the failed ones)

TestCustomResourceCascadingDeletion
E1016 09:21:51.819755   14346 garbagecollector.go:259] error syncing item &garbagecollector.node{identity:garbagecollector.objectReference{OwnerReference:v1.OwnerReference{APIVersion:"mygroup.example.com/v1beta1", Kind:"foo97227a", Name:"dependentcc967", UID:"6ebe74e9-b253-11e7-82f3-0ebee34bef42", Controller:(*bool)(nil), BlockOwnerDeletion:(*bool)(nil)}, Namespace:"crd-cascading"}, dependentsLock:sync.RWMutex{w:sync.Mutex{state:0, sema:0x0}, writerSem:0x0, readerSem:0x0, readerCount:1, readerWait:0}, dependents:map[*garbagecollector.node]struct {}{}, deletingDependents:false, deletingDependentsLock:sync.RWMutex{w:sync.Mutex{state:0, sema:0x0}, writerSem:0x0, readerSem:0x0, readerCount:0, readerWait:0}, beingDeleted:false, beingDeletedLock:sync.RWMutex{w:sync.Mutex{state:0, sema:0x0}, writerSem:0x0, readerSem:0x0, readerCount:0, readerWait:0}, owners:[]v1.OwnerReference{v1.OwnerReference{APIVersion:"mygroup.example.com/v1beta1", Kind:"foo97227a", Name:"ownern2xkc", UID:"6ebe3db0-b253-11e7-82f3-0ebee34bef42", Controller:(*bool)(nil), BlockOwnerDeletion:(*bool)(nil)}}}: rpc error: code = Internal desc = transport: write unix @->localhost:33317419759551005710: write: broken pipe

EDIT: This passed when ran just this test manually. No other integration failures.

soltysh · 2017-10-16T12:37:35Z

UPSTREAM: <drop>: merge multiple registrations for the same group we are going to carry this patch as long as we have our legacy API, which is quite a few versions. Maybe we should just rename it to carry in that case?

soltysh · 2017-10-16T12:49:06Z

@deads2k these:

UPSTREAM: 00000: make AsVersionedObjects default cleanly - I can't find any changes like that in upstream where they ever proposed/rejected? I'd like to describe this commit accordingly as carry or an actual upstream PR
UPSTREAM: 00000: disambiguate operation names for legacy discovery this looks like a carry patch, is it?

deads2k · 2017-10-16T12:58:48Z

@deads2k these:

UPSTREAM: 00000: make AsVersionedObjects default cleanly - I can't find any changes like that in upstream where they ever proposed/rejected? I'd like to describe this commit accordingly as carry or an actual upstream PR
UPSTREAM: 00000: disambiguate operation names for legacy discovery this looks like a carry patch, is it?

Both have a shot at being accepted upstream, though I think AsVersionedObjects has an open pull refactoring it.

soltysh · 2017-10-16T13:58:53Z

So I'm missing following 3 commits here:

UPSTREAM: <carry>: Fix to avoid REST API calls at log level 2.
UPSTREAM: <carry>: Do not error out on pods in kube-system
UPSTREAM: <carry>: allow controller context injection to share informers

All the rest lgtm.

liggitt · 2017-10-16T14:07:48Z

I think AsVersionedObjects has an open pull refactoring it.

kubernetes#53861 moves and unexports it

mfojtik · 2017-10-16T14:12:31Z

@soltysh

4e9886f
still investigating
3c51e41

:100644 100644 b3995e5... 7ed4881... M plugin/pkg/admission/webhook/admission.go

…to work without modifying api :100644 100644 d6e45cb... a6e8623... M pkg/apis/admissionregistration/validation/validation.go :100644 100644 7ed4881... 8281825... M plugin/pkg/admission/webhook/admission.go :100644 100644 8dee1f3... c423b67... M plugin/pkg/admission/webhook/admission_test.go :100644 100644 baa1e2a... 748c700... M test/e2e/framework/framework.go

:100644 100644 fed687e... bcc87f4... M staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/unstructured/unstructured.go

:100644 100644 28a8d1f... 895fd23... M pkg/kubelet/kubelet_pods.go :100644 100644 ed9301a... f280934... M pkg/kubelet/kuberuntime/fake_kuberuntime_manager.go :100644 100644 063a89d... 28a831d... M pkg/kubelet/kuberuntime/kuberuntime_gc.go :100644 100644 389b1e5... 4a41952... M pkg/kubelet/kuberuntime/kuberuntime_gc_test.go :100644 100644 7e5d1d0... 7286bf4... M pkg/kubelet/kuberuntime/kuberuntime_manager.go

:100644 100644 eec16ea... bba5bc4... M pkg/printers/customcolumn.go :100644 100644 dc1e721... 94f915c... M pkg/printers/customcolumn_test.go

:100644 100644 895fd23... 162030f... M pkg/kubelet/kubelet_pods.go

…mapper :100644 100644 c5980bd... f9853d6... M pkg/kubectl/cmd/util/factory_test.go :100644 100644 2a7c7bc... d3d8b50... M pkg/kubectl/cmd/util/shortcut_restmapper.go :100644 100644 20d671e... 3f5bcce... M pkg/kubectl/cmd/util/shortcut_restmapper_test.go :100644 100644 9d7a8fc... 49bbc93... M pkg/kubectl/kubectl.go

…it time from 2 seconds to 5 seconds

… tolerations by namespace level empty (default, whitelist) tolerations.

…reflector_last_resource_version{name="xx"}

…rrect symlinks Symlinks relative to a working directory were being constructed to the wrong location, leading to failure to refresh client certs.

The Get() function of non-namespace lister passes a temporary object to indexer.Get() in order to fetch the actual object from the indexer. This may cause Go to allocate the temporary object on the heap instead of the stack, as it is passed into interfaces. For non-namespaced objects, Get(&Type{ObjectMeta: v1.ObjectMeta{Name: name}}) should be equivalent to GetByKey(name). This could be the root cause of excessive allocations, e.g. in tests clusterRoleLister.Get() has trigger 4 billion allocations. See openshift/origin#16954 Signed-off-by: Christian Heimes <[email protected]>

…journald log driver When using the legacy docker container runtime and when a container has terminationMessagePolicy=FallbackToLogsOnError and when docker is configured with a log driver other than json-log (such as journald), the kubelet should not try to get the container's log from the json log file (since it's not there) but should instead ask docker for the logs.

mfojtik · 2017-11-07T13:18:46Z

@gnufied without that pick the tests are passing, do we still need it? if yes it seems that it needs some rework for 1.8.x

EDIT: solved by picking the missing commits.

We should be careful while generating multiattach errors. We seem to be generating too many of them because old code had minor bug.

:100644 100644 492ca3d... 2ee9a4b... M pkg/controller/volume/attachdetach/reconciler/reconciler_test.go

:100644 100644 48b5d52... 7f1327d... M pkg/kubectl/cmd/testing/fake.go :100644 100644 05014de... 29c63bc... M pkg/kubectl/cmd/util/factory.go :100644 100644 abb10d5... 01ad1ff... M pkg/kubectl/cmd/util/factory_object_mapping.go

…t.client (drop in 1.9)

liggitt · 2017-11-20T14:42:17Z

LGTM

sttts reviewed Oct 12, 2017

View reviewed changes

mfojtik force-pushed the rebase-1.8.1 branch 2 times, most recently from 52fbeef to 123cc2a Compare October 12, 2017 14:31

mfojtik force-pushed the rebase-1.8.1 branch from 123cc2a to c3e125a Compare October 12, 2017 15:42

mfojtik force-pushed the rebase-1.8.1 branch from c3e125a to a9fea7e Compare October 12, 2017 15:45

mfojtik force-pushed the rebase-1.8.1 branch 2 times, most recently from bf7902d to 93448eb Compare October 12, 2017 15:51

mfojtik force-pushed the rebase-1.8.1 branch from 93448eb to 1fe41a2 Compare October 12, 2017 15:54

deads2k and others added 17 commits November 7, 2017 14:17

UPSTREAM: 53896: decode admission responses into a fresh object

4e23503

:100644 100644 b3995e5... 7ed4881... M plugin/pkg/admission/webhook/admission.go

UPSTREAM: 53464: output empty creationTimestamps as null

735c248

:100644 100644 fed687e... bcc87f4... M staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/unstructured/unstructured.go

UPSTREAM: 51750: output <none> for colums not found

fab08f0

:100644 100644 eec16ea... bba5bc4... M pkg/printers/customcolumn.go :100644 100644 dc1e721... 94f915c... M pkg/printers/customcolumn_test.go

UPSTREAM: 54593: Removed containers are not always waiting

4f52607

:100644 100644 895fd23... 162030f... M pkg/kubelet/kubelet_pods.go

UPSTREAM: 54763: make iptables wait flag generic; increase the max wa…

9d59240

…it time from 2 seconds to 5 seconds

UPSTREAM: 54812: Allow override of cluster level (default, whitelist)…

3102892

… tolerations by namespace level empty (default, whitelist) tolerations.

UPSTREAM: 54828: trigger endpoint update on pod deletion

2495f6a

UPSTREAM: 54921: rename metric reflector_xx_last_resource_version to …

9318a8b

…reflector_last_resource_version{name="xx"}

UPSTREAM: 55028: kubelet: dockershim: remove orphaned checkpoint files

5b8f4aa

UPSTREAM: 54979: When cert dir is relative, cert rotation builds inco…

339287f

…rrect symlinks Symlinks relative to a working directory were being constructed to the wrong location, leading to failure to refresh client certs.

UPSTREAM: 53793: User separate client for leader election in scheduler

d84ed50

UPSTREAM: 55223: Fix protobuf generator for aliases to repeated types

e0dd265

mfojtik force-pushed the rebase-1.8.1 branch from 29f4b55 to e0dd265 Compare November 7, 2017 13:18

gnufied and others added 6 commits November 7, 2017 15:07

UPSTREAM: 53401: Fix spam of multiattach errors in event logs

24f53e1

We should be careful while generating multiattach errors. We seem to be generating too many of them because old code had minor bug.

UPSTREAM: 53682: Make sure we use rwlocks not just RLock

4020059

UPSTREAM: 53831: Fix volume reconciler test flake

2734462

:100644 100644 492ca3d... 2ee9a4b... M pkg/controller/volume/attachdetach/reconciler/reconciler_test.go

regenerate listers

140fa32

UPSTREAM: <drop>: get.go doesn't handle metav1.Status returned by res…

0d5291c

…t.client (drop in 1.9)

mfojtik mentioned this pull request Nov 16, 2017

Rebase 1.8.1 openshift/origin#17115

Merged

19 tasks

mfojtik force-pushed the rebase-1.8.1 branch 2 times, most recently from 934687d to 0d5291c Compare November 20, 2017 14:40

liggitt merged this pull request into openshift:release-1.8.1 Nov 20, 2017

Rebase kubernetes 1.8.1 #43

Rebase kubernetes 1.8.1 #43

Uh oh!

Conversation

mfojtik commented Oct 12, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mfojtik commented Oct 12, 2017

Uh oh!

sttts Oct 12, 2017

Choose a reason for hiding this comment

Uh oh!

deads2k Oct 12, 2017

Choose a reason for hiding this comment

Uh oh!

deads2k Oct 12, 2017

Choose a reason for hiding this comment

Uh oh!

mfojtik Oct 12, 2017

Choose a reason for hiding this comment

Uh oh!

sttts commented Oct 12, 2017

Uh oh!

deads2k commented Oct 12, 2017

Uh oh!

sttts Oct 12, 2017

Choose a reason for hiding this comment

Uh oh!

mfojtik Oct 12, 2017

Choose a reason for hiding this comment

Uh oh!

mfojtik commented Oct 12, 2017

Uh oh!

liggitt commented Oct 12, 2017

Uh oh!

mfojtik commented Oct 12, 2017

Uh oh!

liggitt commented Oct 12, 2017

Uh oh!

mfojtik commented Oct 12, 2017

Uh oh!

liggitt commented Oct 12, 2017

Uh oh!

liggitt commented Oct 12, 2017

Uh oh!

liggitt commented Oct 12, 2017

Uh oh!

mfojtik commented Oct 12, 2017

Uh oh!

liggitt commented Oct 12, 2017

Uh oh!

mfojtik commented Oct 12, 2017

Uh oh!

mfojtik commented Oct 16, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

soltysh commented Oct 16, 2017

Uh oh!

soltysh commented Oct 16, 2017

Uh oh!

deads2k commented Oct 16, 2017

Uh oh!

soltysh commented Oct 16, 2017

Uh oh!

liggitt commented Oct 16, 2017

Uh oh!

mfojtik commented Oct 16, 2017

Uh oh!

mfojtik commented Nov 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

liggitt commented Nov 20, 2017

Uh oh!

Uh oh!

mfojtik commented Oct 12, 2017 •

edited

Loading

mfojtik commented Oct 16, 2017 •

edited

Loading

mfojtik commented Nov 7, 2017 •

edited

Loading