[Conformance][templates] templateservicebroker security test

[sdodson]

From https://ci.openshift.redhat.com/jenkins/job/test_pull_request_openshift_ansible_extended_conformance_install_update/945/consoleFull#198825387156cbb9a5e4b02b88ae8c2f77

May be happening on every job not certain yet.

[Conformance][templates] templateservicebroker security test

[AfterEach] [Conformance][templates] templateservicebroker security test
  /go/src/github.com/openshift/origin/test/extended/templates/templateservicebroker_security.go:89


• Failure [52.669 seconds]
[Conformance][templates] templateservicebroker security test
/go/src/github.com/openshift/origin/test/extended/templates/templateservicebroker_security.go:199
  should pass security tests [It]
  /go/src/github.com/openshift/origin/test/extended/templates/templateservicebroker_security.go:197

  Expected error:
      <*url.Error | 0xc421973500>: {
          Op: "Get",
          URL: "https://localhost:45126/brokers/template.openshift.io/v2/catalog",
          Err: {
              Op: "dial",
              Net: "tcp",
              Source: nil,
              Addr: {
                  IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
                  Port: 45126,
                  Zone: "",
              },
              Err: {
                  Syscall: "getsockopt",
                  Err: 0x6f,
              },
          },
      }
      Get https://localhost:45126/brokers/template.openshift.io/v2/catalog: dial tcp [::1]:45126: getsockopt: connection refused
  not to have occurred

  /go/src/github.com/openshift/origin/test/extended/templates/templateservicebroker_security.go:94

[smarterclayton]

@bparees @jim-minter

[bparees]

/cc @deads2k since i think this was one of the reworked tests intended to stand up its own TSB and then port-forward itself to access the TSB.

[deads2k]

Did this get fixed by something? I'm not seeing the errors in prow.

[danwinship]

@deads2k just happened in #15930. https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin_extended_templates/814/

[deads2k]

@deads2k just happened in #15930. https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin_extended_templates/814/

Ok, looks like

Expected error:
    <*url.Error | 0xc42130bd10>: {
        Op: "Get",
        URL: "https://localhost:42916/brokers/template.openshift.io/v2/catalog",
        Err: {
            Op: "dial",
            Net: "tcp",
            Source: nil,
            Addr: {
                IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1],
                Port: 42916,
                Zone: "",
            },
            Err: {
                Syscall: "getsockopt",
                Err: 0x6f,
            },
        },
    }
    Get https://localhost:42916/brokers/template.openshift.io/v2/catalog: dial tcp [::1]:42916: getsockopt: connection refused
not to have occurred

on the port forward. Guess I'll add some logging/health checking on it.

[mfojtik]

raising to P0 as almost every PR fails with this right now

[bparees]

I backed this test out of conformance last night: #15976

but we absolutely need to get to the bottom of why this is failing and get that test back in. Somehow something merged that consistently broke a conformance test (but only in the install+upgrade scenario), so that's really bad.

[jim-minter]

I think there's a good chance this is down to a race trying to use the port-forward socket before oc port-forward had actually started. If so, #15966 should fix this along with a number of other races in EnsureTSB().