How to import and use common kcl code?

[irizzant]

Hi,

I have some KCL code which I'd like to reuse, for example some schemas.

How would I reuse it in function-kcl?

[Peefy]

cc @zong-zhe

[maulik13]

KCL itself allows you to get modules from a remote repository. We use the dependency field to point to a GitHub repo which hosts the KCL modules with common schemas and utilities.

Here is an example, https://github.com/vfarcic/crossplane-sql/blob/main/kcl/compositions.k#L36

This requires setting up access to GitHub private repository which is not easy to do using functions-kcl. We ended up using DeploymentRuntimeConfig for function-kcl pod which provides Git config to access private repos using an init container.

Here is our runtime config

apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
  name: fn-kcl
spec:
  deploymentTemplate:
    spec:
      selector: {}
      template:
        spec:
          volumes:
            # Shared volume for the git configuration file
            - name: git-config
              emptyDir: {}
          initContainers:
            - name: init-gitconfig
              image: alpine:3.16
              volumeMounts:
                - name: git-config
                  mountPath: /config
              command:
                - /bin/sh
                - -c
                - |
                  # Write the git configuration file with the desired content.
                  # The file is created at /config/gitconfig so that the main container can use it.
                  mkdir /config/git
                  cat <<EOF > /config/git/config
                  [credential "https://github.com"]
                    username = infra-bot
                    helper = "!f() { test \"\$1\" = get && echo \"password=\${GITHUB_TOKEN}\"; }; f"
                  EOF
          containers:
            - name: package-runtime
              env:
                - name: GITHUB_TOKEN
                  valueFrom:
                    secretKeyRef:
                      name: gitsecret
                      key: password
              volumeMounts:
                - name: git-config
                  mountPath: /.config

[irizzant]

Thanks @maulik13 , the example you provided:
https://github.com/vfarcic/crossplane-sql/blob/main/kcl/compositions.k#L36

points to a git repo which is fine.
source support also seems to support OCI images, private repos and local sources.

When should I use source support and when dependencies instead?

[mdaops]

@irizzant

So I'd recommend just using OCI by default, there is an open issue about it always pulling latest from the oci path.

We prepackage dependencies into bundles and then share those around as oci packages. We then do

apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
  name: function-kcl
spec:
  deploymentTemplate:
    spec:
      selector: {}
      template:
        spec:
          volumes:
            - name: docker-config
              secret:
                secretName: gcr-json-key
                items:
                - key: .dockerconfigjson
                  path: config.json
          containers:
            - name: package-runtime
              volumeMounts:
                - name: docker-config
                  mountPath: /.docker
                  readOnly: true

which works well for us.

[maulik13]

Thanks @maulik13 , the example you provided: vfarcic/crossplane-sql@main/kcl/compositions.k#L36

points to a git repo which is fine. source support also seems to support OCI images, private repos and local sources.

When should I use source support and when dependencies instead?

Source does not support private repos from what I remember. OCI works, but we decided not to do this extra packaging of config to OCI images. Pointing to a Git ref is transparent and easy to debug.

[irizzant]

Thanks, maybe these examples could be put in the documentation as a reference?

[johngmyers]

I'm trying to do this, but I don't want to pull the code from remote sources. I'm trying to put it in a custom image, but can't figure out how to get it to import.

I've tried

FROM xpkg.upbound.io/crossplane-contrib/function-kcl:v0.11.2
ADD kcl /kcl
ENV KCL_MOD_PATH=/kcl

But it looks like KRM is still trying to load it from /tmp.

I want a curated set of packages baked into the image and I don't want it to be executing code downloaded directly from ghcr.io or who knows where else.

[johngmyers]

I found a solution to getting a custom function-kcl image to pull local modules is to include in the Composition something like:

      spec:
        dependencies: "local = { path = \"/kcl/local\" }"

I'll file a separate issue on a method to remove the need to put this boilerplate in every Composition.

[johngmyers]

I was able to disable downloading from the Internet by adding:

ENV KPM_REG="disallowed.invalid" OCI_REG_PLAIN_HTTP=off

though the resulting error message is a bit misleading.

[irizzant]

How do I manage transitive dependencies?

For example, I have created a repo which contains my kcl common models, something like https://github.com/kcl-lang/konfig/tree/main/models.

My kcl.mod file in the common models repo root is pretty similar to this which declares a dependency on k8s.

Now when I include my modules repo as shown here, do I need to declare the dependency on k8s again?

In other words, if A depends on B which depends on C, does A need to declare dependency on C?

By having a look at the examples it looks like it's needed