running openshift on localhost with auth proxy makes kubelet queries impossible

[TomasTomecek]

Setup

openshift

openshift start --loglevel=5 --listen=https://127.0.0.1:8443 --master=https://127.0.0.1:8443

httpd

apache listens on remotely accesible network interface and forwards everything to localhost:

https://<address>:8443 → https://localhost:8443
https://<address>:10250 → https://localhost:10250

We have kerberos authentication in the apache proxy, meaning everything what access' https://<address>:10250 has to have valid kerberos ticket. And that's the problem.

Issue

kubelet is attached to localhost and when openshift tries to make calls to it, it's going through proxy and not through localhost (which is somewhat obvious).

The main reason we are exposing kubelet in proxy is to get container logs.

Would it possible to configure openshift in a way so it would try to reach out to kubelet via localhost and not the public address (or hostname)?

[brenton]

Typically the master and the node are not on the same host. The Master knows how to contact the Node based on it's DNS name. You can name it anything you want so long that it resolves from each Master. I'm certain if you registered the Node with a name that resolved to 127.0.0.1 then it would reach the Node over the loopback interface.

That said, the scenario you're describing sounds like it would only make sense if there was only one Master and and one Node. If you had two Masters (A and B) how would Master A reach the Node that is colocated with Master B if it was only listening on localhost?

I'm certain I'm missing some background on the motivation for this setup so please provide more detail. What containers logs are you trying to capture?

[TomasTomecek]

You are right, my use case is very specific.

You can name it anything you want

How can I configure that? I didn't set up anything, I've just started all-in-one openshift with the command in my first post.

it would only make sense if there was only one Master and and one Node

Yes, we are planning to have only one master and one node on same machine.

What containers logs are you trying to capture?

The scenario is simple:

1. create build (POST /.../builds)
2. get build logs (GET kubelet:10250/.../containerLogs/...)

[brenton]

How can I configure that? I didn't set up anything, I've just started all-in-one openshift with the command in my first post.

Technically the only way to do this today is to start the Master and Node in separate processes but still on the same node. When you do this you have to explicitly create the node via the api. We could probably patch the code easily to ensure the all-in-one process situation accesses the Node over the loopback.

Alternatively you could see what the hostname is being set to. Check osc get nodes. If you made that resolve to 127.0.0.1 it should go over the loopback.

[brenton]

@TomasTomecek, any luck?

[TomasTomecek]

Didn't have time, tomorrow, hopefully. I think that the /etc/hosts trick could work.

[TomasTomecek]

The architecture was changed and we are not hitting this issue anymore (apache listens on 9443, proxies requests to 8443 (master is running on all interfaces), hence kubelet requests are not going through proxy anymore)