Pythonpath doesn't work

I’m trying to develop tools for a design studio that uses “rez” for package management.
The approach I would typically take in this environment would be to create a package(s) that sets the PYTHONPATH for all the libraries. In most software, this works fine (eg Maya, 3dsMax, etc).

I tried setting PYTHONPATH and launching revit-2025 with pyrevit. From a pyrevit script, when I query sys.path and PYTHONPATH, the paths appear correctly (both in PYTHONPATH and they appear in the standard sys.path), but the imports fail with "no module named foo … " .

Adding packages with sys.path appears to work, as does adding the packages to my extension “lib” folder.

I don’t want to have to bundle all the dependencies in an extension, I have a package manager(rez) which handles adding all the necessary packages to PYTHONPATH.

Is there any way to get pyrevit to expose PYTHONPATH such that imports work?
Can anyone explain why even with the packages in sys,path without explicitly adding them in the script, that the imports fail?

Hi @radh-pmac , welcome to pyRevit community!

Just to make sure I understood correctly: does rez add multiple paths to the PYTHONPATH variable?

As of now, the cpython interpreter (I assume you’re using it, since IronPython doesn’t play well with external cpyhton environments and pypi packages in general) takes the whole content of the PYTHONPATH variable and adds it to the sys.path as a single item (if I interpret pyRevit source code correctly)

So it doesn’t really add the right paths if there are more than one.
If it isn’t what you’re experiencing, please report back with concrete examples and outputs (print(sys.path) in a script).

Thanks for the welcome!
Yes, it gathers all the dependencies I might need and adds them to PYTHONPATH.
Oddly, when I launch Revit, if I query PYTHONPATH and sys.path, the paths ARE there, but the imports fail.
That code you posted is interesting…Does this look like a pyrevit bug?
Is sysPaths the direct equivalent of sys.path? If so, then it should be a list, and the PYTHONPATH string needs decomposed by path separator and iterated over to add to the sys.paths.

Please post here the output of print(sys.path), and confirm if you are running it under CPython.
PYTHONPATH content doesn’t matter, and I suspect that you’re seeing all the paths in a single list item separated by : instead of being separate items, for the reason I explained above.

Well, usually that environment variable is used to specify a single location where all the needed packages are, so I would call it “not implementing an edge case” :wink: but yes, we’re not following the behavior of standard python here.

That’s what I was saying in my previous message :wink:

Thanks again Andrea!
My setup:-
image

usually that environment variable is used to specify a single location

According to the python documentation PYTHONPATH is indeed a list of directories, not a single site-packages folder.

" The PYTHONPATH environment variable is often used to add directories to the search path. If this environment variable is found then the contents are added to the module search path."

The initialization of the sys.path module search path — Python 3.13.1 documentation

I would call it “not implementing an edge case”

It’s pretty common in VFX studios to configure environments using PYTHONPATH. Each DCC has it’s own dependencies and each project (or department, or even user) might have it’s own requirements, so the typical approach is to construct the environment in the pythonpath before launching the application. Building bespoke site-packages folders for every permutation this might involve is non viable.
I wouldn’t describe this as an edge case by any means.

Here’s the output from my script.

Pre launch revit PYTHONPATH

C:\Users\USER\src\ORG\REPO\crevit;C:\Users\USER\src\ORG\REPO\cpy\src;c:\users\USER\packages\click\8.1.7.3\python-3.11\python;c:\users\USER\packages\pillow\10.4.0.1\python-3.11.9-sg.2.0.0.1\python;c:\users\USER\packages\pdf2image\1.17.0.4\python-3.11.9-sg.2.0.0.1\python;c:\users\USER\packages\prometheus_client\0.21.0.2\python-3.11.9-sg.2.0.0.1\python;c:\users\USER\packages\requests\2.32.3.3\python-3.11\python;c:\users\USER\packages\rez\3.1.1.2\python-3.11.9-sg.2.0.0.1\python;c:\users\USER\packages\rfc3339\6.2.0.2\python-3.11.9-sg.2.0.0.1\python;c:\users\USER\packages\sgtk\0.21.7.1\tk-core\python;c:\users\USER\packages\shotgun_api3\3.5.1.2\python-3.11.9-sg.2.0.0.1\python

After launching revit:-
os.environ['PYTHONPATH']: 
C:\Users\USER\src\ORG\REPO\crevit;C:\Users\USER\src\ORG\REPO\cpy\src;c:\users\USER\packages\click\8.1.7.3\python-3.11\python;c:\users\USER\packages\pillow\10.4.0.1\python-3.11.9-sg.2.0.0.1\python;c:\users\USER\packages\pdf2image\1.17.0.4\python-3.11.9-sg.2.0.0.1\python;c:\users\USER\packages\prometheus_client\0.21.0.2\python-3.11.9-sg.2.0.0.1\python;c:\users\USER\packages\requests\2.32.3.3\python-3.11\python;c:\users\USER\packages\rez\3.1.1.2\python-3.11.9-sg.2.0.0.1\python;c:\users\USER\packages\rfc3339\6.2.0.2\python-3.11.9-sg.2.0.0.1\python;c:\users\USER\packages\sgtk\0.21.7.1\tk-core\python;c:\users\pm\packages\shotgun_api3\3.5.1.2\python-3.11.9-sg.2.0.0.1\python


sys.version: 
3.12.3 (tags/v3.12.3:f6650f9, Apr 9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)]

sys.path: 
[
    'C:\\Users\\USER\\AppData\\Roaming\\pyRevit-Master\\bin\\cengines\\CPY3123\\python312.zip', 'C:\\Users\\USER\\AppData\\Roaming\\pyRevit-Master\\bin\\cengines\\CPY3123', 
    'C:\\Program Files\\dotnet\\shared\\Microsoft.NETCore.App\\8.0.12\\', 
    'C:\\Users\\USER\\src\\ORG\\REPO\\crevit;C:\\Users\\USER\\src\\ORG\\REPO\\cpy\\src;c:\\users\\USER\\packages\\click\\8.1.7.3\\python-3.11\\python;c:\\users\\USER\\packages\\pillow\\10.4.0.1\\python-3.11.9-sg.2.0.0.1\\python;c:\\users\\USER\\packages\\pdf2image\\1.17.0.4\\python-3.11.9-sg.2.0.0.1\\python;c:\\users\\USER\\packages\\prometheus_client\\0.21.0.2\\python-3.11.9-sg.2.0.0.1\\python;c:\\users\\USER\\packages\\requests\\2.32.3.3\\python-3.11\\python;c:\\users\\USER\\packages\\rez\\3.1.1.2\\python-3.11.9-sg.2.0.0.1\\python;c:\\users\\USER\\packages\\rfc3339\\6.2.0.2\\python-3.11.9-sg.2.0.0.1\\python;c:\\users\\USER\\packages\\sgtk\\0.21.7.1\\tk-core\\python;c:\\users\\USER\\packages\\shotgun_api3\\3.5.1.2\\python-3.11.9-sg.2.0.0.1\\python',
    'C:\\Users\\USER\\src\\ORG\\REPO\\crevit\\crevit.extension\\CG.tab\\About.panel\\test.pushbutton',
    'C:\\Users\\USER\\src\\ORG\\REPO\\crevit\\crevit.extension\\lib',
    'C:\\Users\\USER\\AppData\\Roaming\\pyRevit-Master\\pyrevitlib', 
    'C:\\Users\\USER\\AppData\\Roaming\\pyRevit-Master\\site-packages'
]

So, you’re right, the sys.path is malformed.

Further PYTHONPATH docs :

1. Command line and environment — Python 3.13.1 documentation

PYTHONPATH

Augment the default search path for module files. The format is the same as the shell’s PATH: one or more directory pathnames separated by os.pathsep (e.g. colons on Unix or semicolons on Windows). Non-existent directories are silently ignored.

In addition to normal directories, individual PYTHONPATH entries may refer to zipfiles containing pure Python modules (in either source or compiled form). Extension modules cannot be imported from zipfiles.

The default search path is installation dependent, but generally begins with prefix/lib/pythonversion (see PYTHONHOME above). It is always appended to PYTHONPATH.

An additional directory will be inserted in the search path in front of PYTHONPATH as described above under Interface options. The search path can be manipulated from within a Python program as the variable sys.path.

I’m a python developer for 10 years now, and while I don’t pretend to be “the majority”, I never needed to touch the PYTHONPATH envvar myself.
I (and almost every developers I got in touch with) use virtual environments to isolate the dependencies (starting from the python interpreter version itself) and guarantee the exact reproducibility of the development/runtime environment;
From what I understand, the python environment managers that I used/know of (venv, conda, poetry, uv, pipenv) don’t change the PYTHONPATH to set the directory of the packages; they usually add them directly in the sys.path.

It seems to me that using multiple folders in that environment variable is a recipe for disaster for a few reason:

  • you potentially could have multiple versions of the same package, leading to an unpredictability on which version would be loaded by the interpreter, leading to errors on the packages that depend on the exact version of the ignored one
  • You lose the portability and quick reproducibility of the environment.

That being said… I don’t know if the current state of pyRevit ignoring the python standards is an oversight by the original developer or due to the reason that, like me, didn’t count the multiple paths in the PYTHONPATH something useful if at all possible (I just learned this is possible thanks to you!).
If you’re willing to contribute, feel free to submit a PR to fix this!

Yea it’s very much sector specific.If you’re not needing to have granular control over dependencies at runtime I imagine it’s not something you’ll have come across.

I’ll try to put a PR in, but I’m not up to speed on cs, and I imagine the dev environment isn’t trivial?

Thanks again!

Bug reported : [Bug]: PYTHONPATH translation to sys.path fails if not single folder. · Issue #2527 · pyrevitlabs/pyRevit

I still fail to see the benefit of this, and I would very much like to learn more about it; do you by any chance have some links to point me to about this framework?

Here it is : AcademySoftwareFoundation/rez: An integrated package configuration, build and deployment system for software

I saw the repository and the documentation, but by “framework” I was referring about the use case you mentioned :sweat_smile:

Are there guides / documents about such approach out there (regardless of the specific tool used)?

I’d take a look at this video which goes in to great detail about the problems that vfx studios come across, and how Rez helps solve them.
This link starts with a decent description of the problem https://youtu.be/y5VSPeeqJbM?si=Q2lo0lOlFuS5yKzx&t=163

VFX pipeline isn’t very well documented to be honest, studios typically develop their in-house tooling which doesn’t get shared publicly.

I’d check the ASF/Rez slack channel for interesting discussions https://slack.aswf.io/

There are some other examples mentioned in this thread which do a similar job : Rez package manager alternatives : r/vfx

1 Like

I’ve created a draft PR that should fix the issue. I don’t have the bandwidth to setup a development environment currently to install the prerequisites, build and test the fix. Is that something you could help out with?

fix: Fix incorrect handling of PYTHONPATH in SetupSearchPaths() by reformstudios · Pull Request #2535 · pyrevitlabs/pyRevit

If you don’t, we don’t. :stuck_out_tongue:
Just getting the whole pyrevit beast going involves already a lot of ‘free’ time for us. As a user proposing a change, you need to take ownership of the change and go down the rabbit hole.

That being said, I appreciate the beginning of an involvement !

Thanks,
Ok, can you help me get started? The documentation appears to be lacking.

From the contributing page : Contributing to pyRevit Source

It links to : Building pyRevit Source

Which notes : “See the comments on the build script inside dev/”

Which I assume points to : pyRevit/dev/_build.py at develop · pyrevitlabs/pyRevit

Which has no notes.

Can you point me in the right direction for either instructions or a video tutorial showing how to setup a development environment.

Thanks again

yeah, need to update the docs on notion

this is up to date:
https://docs.pyrevitlabs.io/dev-guide/
graciously written down by @sanzoghenzo

I’m following the instructions but at the “pipenv install” step, I get the error :

:   Building wheel for regex (pyproject.toml): started
:   Building wheel for regex (pyproject.toml): finished with status 'error'
: Failed to build regex
: error: subprocess-exited-with-error
:
:   × Building wheel for regex (pyproject.toml) did not run successfully.
:   │ exit code: 1
:   ╰─> [13 lines of output]
:       running bdist_wheel
:       running build
:       running build_py
:       creating build\lib.win-amd64-cpython-310\regex
:       copying regex_3\__init__.py -> build\lib.win-amd64-cpython-310\regex
:       copying regex_3\regex.py -> build\lib.win-amd64-cpython-310\regex
:       copying regex_3\_regex_core.py -> build\lib.win-amd64-cpython-310\regex
:       copying regex_3\test_regex.py -> build\lib.win-amd64-cpython-310\regex
:       warning: build_py: byte-compiling is disabled, skipping.
:
:       running build_ext
:       building 'regex._regex' extension
:       error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools":
https://visualstudio.microsoft.com/visual-cpp-build-tools/
:
:
:   note: This error originates from a subprocess, and is likely not a problem with pip.
:   ERROR: Failed building wheel for regex
: ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (regex)
ERROR: Couldn't install package: [1m{}[0m
 [33mPackage installation failed...[0m```

Clicking the link, I downloaded the installer, installed the build tools, but it still raises this error. Any ideas what I need to do?

No idea.
I see in the error it points to cpython.
For deployment scripts it uses regular flavor of python. You can check that in the main.yml file in the GitHub workflows of the repo, but pretty sure it is 310

I confirm that the python version configured in the Pipfile is 3.10.

You can try to bypass the problem by replacing each occurrence of pipenv run pyrevit in the guide with python dev/pyrevit.py, and pray that no dependency is missing :sweat_smile:

We were thinking about switching the toolchain to nuke to stay in the c# realm for the compiled part, but as always it was Wishful thinking without real resouce planning (which resources? :rofl: )