Include `pyenv` global python when selecting environments for the Create Environment command

[brsolomon-deloitte]

Type: Bug

Behaviour

Expected vs. Actual

When I use Python: Create Environment -> venv, it does not find any of the Python executables installed with pyenv, even though the separate Python: Select Interpreter command does find these versions.

None of #15439 seems to resolve the issue here.

Steps to reproduce:

From Python: Create Environment:

From the direct Python: Select Interpreter command:

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.11.2
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): venv
• Value of the python.languageServer setting: Default

>

{
    "workbench.colorTheme": "Default Dark+",
    "workbench.startupEditor": "none",
    "terminal.external.linuxExec": "xterm-256color",
    "[python]": {
        "editor.formatOnType": true
    },
    "python.defaultInterpreterPath": "python3",
    "python.experiments.optInto": [
    
      "pythonDiscoveryModule"
    ]
}

$ pyenv versions
* system (set by /Users/brsolomon/.pyenv/version)
  3.7.16
  3.8.16
  3.9.16
  3.10.10

User Settings

languageServer: "Pylance"

experiments
• optInto: ["pythonDiscoveryModule"]

Extension version: 2023.4.1
VS Code version: Code 1.76.2 (Universal) (ee2b180d582a7f601fa6ecfdad8d9fd269ab1884, 2023-03-14T17:54:09.061Z)
OS version: Darwin arm64 22.3.0
Modes:
Sandboxed: No

System Info

Item	Value
CPUs	Apple M1 Pro (10 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	2, 2, 3
Memory (System)	16.00GB (0.09GB free)
Process Argv	--verbose . --crash-reporter-id 1b493729-f4cd-4bd6-8bee-40a1ebc4a121
Screen Reader	no
VM	0%

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vsdfh931:30280409
vshan820:30294714
vstes263cf:30335440
vscorecescf:30445987
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30581797
vsaa593:30376534
pythonvs932:30410667
cppdebug:30492333
vsclangdf:30486550
c4g48928:30535728
dsvsc012:30540252
pynewext54:30681850
azure-dev_surveyone:30548225
pyindex848:30662994
nodejswelcome1cf:30587006
2e4cg342:30602488
pyind779:30671433
f6dab269:30613381
pythonsymbol12:30671437
pythonb192cf:30669361
azdwalk:30687957
pythonms35cf:30686773

[FarukBuldur]

With the same configurations I am not able to see .venv interpreter under select interpreter drop-down as well.

[karthiknadig]

@brsolomon-deloitte For Create env command we only show system and python.org installs of python. pyenv has its own way of creating and managing virtual environments, which is not supported yet. Creating a virtual environment with pyenv using the venv library can break experience due to how pyenv shims redirect python based on the version info inside .python-version in your project root. This is by design.

@FarukBuldur When using Create env command you will not see virtual envs listed there. This is by design for the create env command. But if you are not seeing when you run the select interpreter command then it is a bug.

[brsolomon-deloitte]

For Create env command we only show system and python.org installs of python. pyenv has its own way of creating and managing virtual environments, which is not supported yet. Creating a virtual environment with pyenv using the venv library can break experience due to how pyenv shims redirect python based on the version info inside .python-version in your project root. This is by design.

From a user perspective, I find this to be confusing and have never, in several years of using pyenv + venv, experienced this issue. (Whether with pyenv's global, shell, or local.) Can you please link to those issues? Is this mentioned in the VSCode docs as intended behavior, to exclude pyenv Pythons from discovered interpreters for Create env?

It is confusing that the Python: Select Interpreter does find pyenv Pythons, but then the sub-step for interpreters within Python: Create Environment, which otherwise seems to look like the same UI component, does not find them.

[karthiknadig]

@brsolomon-deloitte For create environment command we intentionally filter out python installs we don't fully support. We chose venv and conda as basic support to start with. We released this feature recently and our long-term intent is that we want to make it pluggable so environment providers like pyenv, poetry, pipenv, etc can contribute to this UI. This way they can control the experience in a way expected by their users. conda for example, that we have, only supports creating a local conda environment using the --prefix command.

pyenv has plugins that allow users to manage virutal environments like pyenv-virtualenv plugin. With that plugin you have to use pyenv virtualenv command to create virtual envs. There is also pyenv and conda behavior, users have to use pyenv to manage the conda environment created by pyenv.

We can definitely adjust the filter to include pyenv in that list that are specifically under the versions directory. But, we will not include the environments created using plugins or pyenv managed conda.

/cc @karrtikr

[brsolomon-deloitte]

@karthiknadig I understand your rationale here, thanks for explaining.

Here is my input as a consistent user of pyenv + stdlib venv and then I'll stop waving my hands about this :)

I have occasionally used pyenv-virtualenv, but it is not strictly necessary as the medium for virtual environment creation when using pyenv - venv will fully suffice. Rather, pyenv-virtualenv is a wrapper that adds some additional feature such as

pyenv virtualenv forwards any options to the underlying command that actually creates the virtual environment (conda, virtualenv, or python -m venv)

In my experience, I have seen truly zero difference in venv behavior when working with pyenv Pythons versus a system distribution of Python or a Homebrew install of Python. I think pyenv might have had some bumps in the road years back but it has matured to a really solid tool that makes venv behave as it should, which is agnostic to what tool is used to manage python itself.

[karthiknadig]

@brsolomon-deloitte No worries, we just started with what we knew to work with. I have marked this as a feature request to update the filter to include global pythons in pyenv.