Update start workflows container script for unified storage feature to create workflows sub dir

[ruijiang-rjian]

Description

For projects with unified storage(s3), there will be a project shared dir /home/sagemaker-user/shared-files/ to replace /home/sagemaker-user/src/, this change is mainly to adapt to this new use case: if it's using git, everything is kept the same; if it's using unified storage, then we're creating /home/sagemaker-user/shared-files/workflows/ and all the sub-directories under /workflows

Type of Change

• Image update - Bug fix
• Image update - New feature
• Image update - Breaking change
• SMD image build tool update
• Documentation update

Release Information

Does this change need to be included in patch version releases? By default, any pull requests will only be added to the next SMD image minor version release once they are merged in template folder. Only critical bug fix or security update should be applied to new patch versions of existed image minor versions.

• Yes (Critical bug fix or security update)
• No (New feature or non-critical change)
• N/A (Not an image update)

If yes, please explain why:
[Explain the criticality of this change and why it should be included in patch releases]

How Has This Been Tested?

[Describe the tests you ran]

Checklist:

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works

Test Screenshots (if applicable):

tested in gamma project with the new start container script, local runner was able to start and verified workflows subfolder was created under the new "shared-files" project directory:
Tested the following scenarios:

1. bash /etc/sagemaker-ui/workflows/start-workflows-container.sh (backwards compatibility)
2. bash /etc/sagemaker-ui/workflows/start-workflows-container.sh "$is_s3_flag” (with is_s3_flag=1, Git project)
3. bash /etc/sagemaker-ui/workflows/start-workflows-container.sh "$is_s3_flag” (is_s3_flag=0, shared storage project: Project is using S3 storage, project directory set to: /home/sagemaker-user/shared-files)

sagemaker-user@default:~$ bash /etc/sagemaker-ui/workflows/start-workflows-container.sh
Hit:1 https://download.docker.com/linux/ubuntu jammy InRelease
Hit:2 https://apt.corretto.aws stable InRelease                                                                                                   
Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease                                                                                  
Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease
Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Hit:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
Reading package lists... Done
Hit:1 https://download.docker.com/linux/ubuntu jammy InRelease
Hit:2 https://apt.corretto.aws stable InRelease                                                                                
Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease                                                                         
Hit:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease
Hit:6 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
docker-ce-cli is already the newest version (5:28.2.2-1~ubuntu.22.04~jammy).
docker-compose-plugin is already the newest version (2.29.2-1~ubuntu.22.04~jammy).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
WARNING! Your password will be stored unencrypted in /home/sagemaker-user/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credential-stores

Login Succeeded
[+] Running 3/3
 ✔ Container mwaa-292-db         Healthy                                                                                                     10.7s 
 ✔ Container mwaa-292-webserver  Started                                                                                                     10.8s 
 ✔ Container mwaa-292-scheduler  Started                                                                                                     10.8s 
/opt/conda/lib/python3.12/site-packages/supervisor/options.py:13: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
workflows_healthcheck: started

Related Issues

[Link any related issues here]

Additional Notes

[Any additional information that might be helpful for reviewers]

[agupta01]

If these changes get built into a patch SMD and deployed to existing spaces (without storage), then will this break existing functionality?

[ruijiang-rjian]

If these changes get built into a patch SMD and deployed to existing spaces (without storage), then will this break existing functionality?

no it shouldn't break, I've tested it in an existing Gamma space without storage too to make sure it's backwards compatible

[vasu2856]

Can we reuse this definition from here - #668 as a param?

[vasu2856]

Can you refactor to have these common functions in the post launch script and just use these as params within the workflows container?