Prepare release 0.65.0 (#2960)

* Prepare release 0.65.0

* fixes for quickstart

* improve release notes

Author: avishniakov

README.md

@@ -327,7 +327,7 @@ the Apache License Version 2.0.
     <a href="https://github.com/zenml-io/zenml-projects">Projects Showcase</a>
     <br />
     <br />
-    🎉 Version 0.64.0 is out. Check out the release notes
+    🎉 Version 0.65.0 is out. Check out the release notes
     <a href="https://github.com/zenml-io/zenml/releases">here</a>.
     <br />
     🖥️ Download our VS Code Extension <a href="https://marketplace.visualstudio.com/items?itemName=ZenML.zenml-vscode">here</a>.

RELEASE_NOTES.md

@@ -1,5 +1,69 @@
 <!-- markdown-link-check-disable -->
 
+# 0.65.0
+
+## New Features and Improvements
+
+### New Quickstart Experience
+
+This example demonstrates how ZenML streamlines the transition of machine learning workflows from local environments to 
+cloud-scale operations.
+
+### Run Single Step as a ZenML Pipeline
+
+If you want to run just an individual step on your stack, you can simply call the step
+as you would with a normal Python function. ZenML will internally create a pipeline with just your step
+and run it on the active stack.
+
+### Other improvements and fixes
+
+* Updated AzureML Step Operator to work with SDKv2 and use Service Connectors
+* Added timestamps to log messages
+* Fixed issue with loading artifacts from artifact store outside of current active artifact store
+* Support of templated names for Model Version (`{date}` and `{time}` are currently supported placeholders)
+* `run_with_accelerate` step wrapper can be used as a Python Decorator on top of ZenML steps
+
+## Breaking changes
+
+* Workspace scoped POST endpoint `full-stack` was removed and merged with `stacks` POST endpoint
+
+
+## What's Changed
+* Remove broken JIRA sync workflow by @strickvl in https://github.com/zenml-io/zenml/pull/2924
+* Fix Hugging Face Spaces permissions by @strickvl in https://github.com/zenml-io/zenml/pull/2925
+* Fixes for `run_with_accelerate` by @avishniakov in https://github.com/zenml-io/zenml/pull/2935
+* Bump azure skypilot to a stable 0.6.1 by @avishniakov in https://github.com/zenml-io/zenml/pull/2933
+* Add Timestamps to Logs and Update Dashboard URL Message by @htahir1 in https://github.com/zenml-io/zenml/pull/2934
+* Adding 0.64.0 to migration tests by @bcdurak in https://github.com/zenml-io/zenml/pull/2923
+* Removed docker build docs + fixed CLI command for zenml pipeline build list by @htahir1 in https://github.com/zenml-io/zenml/pull/2938
+* Throw an error when running integration installs when uv == False but pip is not installed by @mennoliefstingh in https://github.com/zenml-io/zenml/pull/2930
+* Update AzureML step operator to SDK v2 and add service connector support by @stefannica in https://github.com/zenml-io/zenml/pull/2927
+* Improving the AzureML orchestrator docs by @bcdurak in https://github.com/zenml-io/zenml/pull/2940
+* Update mlflow docs by @htahir1 in https://github.com/zenml-io/zenml/pull/2941
+* Tell users where they can import `DockerSettings` from by @strickvl in https://github.com/zenml-io/zenml/pull/2947
+* Fail early when specifying invalid materializers by @schustmi in https://github.com/zenml-io/zenml/pull/2950
+* Add GitHub Codespaces and VS Code Remote Container support by @htahir1 in https://github.com/zenml-io/zenml/pull/2949
+* Automatically detect whether code download is necessary by @schustmi in https://github.com/zenml-io/zenml/pull/2946
+* Enable running a single step on the active stack by @schustmi in https://github.com/zenml-io/zenml/pull/2942
+* Dynamic (templated) names for model versions by @avishniakov in https://github.com/zenml-io/zenml/pull/2909
+* Adding an orchestrator URL to the AzureML orchestrator by @bcdurak in https://github.com/zenml-io/zenml/pull/2952
+* Update python version of latest docker image by @schustmi in https://github.com/zenml-io/zenml/pull/2954
+* Make `run_with_accelerate` a pythonic decorator by @avishniakov in https://github.com/zenml-io/zenml/pull/2943
+* Bugfix for artifacts coming from a different artifact store by @bcdurak in https://github.com/zenml-io/zenml/pull/2928
+* Stack Request cleanup and improvements by @bcdurak in https://github.com/zenml-io/zenml/pull/2906
+* Silence pydantic protected namespace warnings by @schustmi in https://github.com/zenml-io/zenml/pull/2955
+* Update key for finished onboarding survey by @schustmi in https://github.com/zenml-io/zenml/pull/2956
+* Extend notebook source replacement code to other objects apart from ZenML steps by @schustmi in https://github.com/zenml-io/zenml/pull/2919
+* Fix stack register CLI command by @schustmi in https://github.com/zenml-io/zenml/pull/2958
+* Lightening studio orchestrator by @safoinme in https://github.com/zenml-io/zenml/pull/2931
+* Introduce new quickstart with a focus on Stack switching by @AlexejPenner in https://github.com/zenml-io/zenml/pull/2937
+* Bugfix for the required prompts for the AzureML wizard by @bcdurak in https://github.com/zenml-io/zenml/pull/2959
+
+## New Contributors
+* @mennoliefstingh made their first contribution in https://github.com/zenml-io/zenml/pull/2930
+
+**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.64.0...0.65.0
+
 # 0.64.0
 
 ## New Features and Improvements

docs/book/how-to/build-pipelines/run-an-individual-step.md

@@ -1,6 +1,6 @@
 # Run an individual step on your stack
 
-If you want to run just an invividual step on your stack, you can simply call the step
+If you want to run just an individual step on your stack, you can simply call the step
 as you would with a normal Python function. ZenML will internally create a pipeline with just your step
 and run it on the active stack.
 

examples/quickstart/configs/training_aws.yaml

@@ -5,7 +5,7 @@ settings:
   resources:
     memory: 64GB
   docker:
-    parent_image: "zenmldocker/zenml-public-pipelines:quickstart-0.64.0-py3.11-aws"
+    parent_image: "zenmldocker/zenml-public-pipelines:quickstart-0.65.0-py3.11-aws"
     skip_build: False
 # Uncomment the following two lines to specify the accelerator for your aws orchestrator
   orchestrator.sagemaker:

examples/quickstart/configs/training_azure.yaml

@@ -1,7 +1,7 @@
 # Environment configuration
 settings:
   docker:
-    parent_image: "zenmldocker/zenml-public-pipelines:quickstart-0.64.0-py3.11-azure"
+    parent_image: "zenmldocker/zenml-public-pipelines:quickstart-0.65.0-py3.11-azure"
     skip_build: True
 # Uncomment the following two lines to specify the accelerator for your skypilot vm orchestrator
 #  orchestrator.vm_azure:

examples/quickstart/configs/training_gcp.yaml

@@ -3,7 +3,7 @@ settings:
   resources:
     memory: 64GB
   docker:
-    parent_image: "zenmldocker/zenml-public-pipelines:quickstart-0.64.0-py3.11-gcp"
+    parent_image: "zenmldocker/zenml-public-pipelines:quickstart-0.65.0-py3.11-gcp"
     skip_build: True
 # Uncomment the following two lines to specify the accelerator for your vertex orchestrator
 #  orchestrator.vertex:

examples/quickstart/quickstart.ipynb

@@ -250,6 +250,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "import materializers\n",
     "from steps import (\n",
     "    evaluate_model,\n",
     "    load_data,\n",
@@ -263,6 +264,8 @@
     "from zenml import Model, pipeline\n",
     "from zenml.client import Client\n",
     "\n",
+    "materializers\n",
+    "\n",
     "# Initialize the ZenML client to fetch objects from the ZenML Server\n",
     "client = Client()\n",
     "\n",
@@ -454,7 +457,7 @@
    "outputs": [],
    "source": [
     "# Set the cloud provider here\n",
-    "CLOUD_PROVIDER = None  # Change this to \"AWS\" or \"AZURE\" as needed\n",
+    "CLOUD_PROVIDER = None  # Change this to \"GCP\", \"AWS\" or \"AZURE\" as needed\n",
     "stack_name = None  # INSERT_STACK_NAME_HERE\n",
     "\n",
     "from zenml import pipeline\n",
@@ -466,7 +469,7 @@
     "    !pip install 'kfp>=2.6.0' gcsfs google-cloud-secret-manager 'google-cloud-container>=2.21.0' 'google-cloud-artifact-registry>=1.11.3' 'google-cloud-storage>=2.9.0' 'google-cloud-aiplatform>=1.34.0' 'google-cloud-build>=3.11.0' kubernetes\n",
     "\n",
     "    parent_image = (\n",
-    "        \"zenmldocker/zenml-public-pipelines:quickstart-0.64.0-py3.11-gcp\"\n",
+    "        \"zenmldocker/zenml-public-pipelines:quickstart-0.65.0-py3.11-gcp\"\n",
     "    )\n",
     "    skip_build = True\n",
     "\n",
@@ -478,7 +481,7 @@
     "    )\n",
     "\n",
     "    parent_image = (\n",
-    "        \"zenmldocker/zenml-public-pipelines:quickstart-0.64.0-py3.11-aws\"\n",
+    "        \"zenmldocker/zenml-public-pipelines:quickstart-0.65.0-py3.11-aws\"\n",
     "    )\n",
     "    skip_build = False  # Sagemaker does not support running straight from public docker images\n",
     "\n",
@@ -491,7 +494,7 @@
     "    !pip install urllib3 --upgrade\n",
     "\n",
     "    parent_image = (\n",
-    "        \"zenmldocker/zenml-public-pipelines:quickstart-0.64.0-py3.11-azure\"\n",
+    "        \"zenmldocker/zenml-public-pipelines:quickstart-0.65.0-py3.11-azure\"\n",
     "    )\n",
     "    skip_build = True\n",
     "\n",
@@ -526,6 +529,17 @@
     "Note: The whole process may take a bit longer the first time around, as your pipeline code needs to be built into docker containers to be run in the orchestration environment of your stack. Any consecutive run of the pipeline, even with different parameters set, will not take as long again thanks to docker caching."
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cfad9bd5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!rm -rf results\n",
+    "!rm -rf sample_data"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -592,7 +606,7 @@
     "                    \"NVIDIA_TESLA_P4\",\n",
     "                )\n",
     "            ),\n",
-    "            \"resources\": ResourceSettings(memory=\"32GB\", gpu=1),\n",
+    "            \"resources\": ResourceSettings(memory=\"32GB\", gpu_count=1),\n",
     "        }\n",
     "    )\n",
     "if CLOUD_PROVIDER == \"AWS\":\n",
@@ -613,8 +627,8 @@
     "    from zenml.integrations.azure.flavors import AzureMLOrchestratorSettings\n",
     "\n",
     "    # find out your options here: https://docs.zenml.io/stack-components/orchestrators/azureml#settings\n",
-    "    # The quickest way is porbably to configure a compute-instance in azure ml. This instance shoudl contain\n",
-    "    # a gpu. Then sepcify the name of the compute instance here.\n",
+    "    # The quickest way is probably to configure a compute-instance in azure ml. This instance should contain\n",
+    "    # a gpu. Then specify the name of the compute instance here.\n",
     "\n",
     "    compute_name = None  # Insert the name of your compute instance here\n",
     "\n",

examples/quickstart/requirements.txt

@@ -1,4 +1,4 @@
-zenml[server]>=0.64.0
+zenml[server]==0.65.0
 notebook
 pyarrow
 datasets

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "zenml"
-version = "0.64.0"
+version = "0.65.0"
 packages = [{ include = "zenml", from = "src" }]
 description = "ZenML: Write production-ready ML code."
 authors = ["ZenML GmbH <[email protected]>"]

src/zenml/VERSION

@@ -1 +1 @@
-0.64.0
+0.65.0

src/zenml/zen_server/deploy/helm/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v2
 name: zenml
-version: "0.64.0"
+version: "0.65.0"
 description: Open source MLOps framework for portable production ready ML pipelines
 keywords:
 - mlops

src/zenml/zen_server/deploy/helm/README.md

@@ -20,8 +20,8 @@ ZenML is an open-source MLOps framework designed to help you create robust, main
 To install the ZenML chart directly from Amazon ECR, use the following command:
 
 ```bash
-# example command for version 0.64.0
-helm install my-zenml oci://public.ecr.aws/zenml/zenml --version 0.64.0
+# example command for version 0.65.0
+helm install my-zenml oci://public.ecr.aws/zenml/zenml --version 0.65.0
 ```
 
 Note: Ensure you have OCI support enabled in your Helm client and that you are authenticated with Amazon ECR.

src/zenml/zen_stores/migrations/versions/0.65.0_release.py

@@ -0,0 +1,23 @@
+"""Release [0.65.0].
+
+Revision ID: 0.65.0
+Revises: bf2120261b5a
+Create Date: 2024-08-28 15:42:13.858104
+
+"""
+
+# revision identifiers, used by Alembic.
+revision = "0.65.0"
+down_revision = "bf2120261b5a"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    """Upgrade database schema and/or data, creating a new revision."""
+    pass
+
+
+def downgrade() -> None:
+    """Downgrade database schema and/or data back to the previous revision."""
+    pass