Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[master] Fix urlunparse for salt file urls on python-3.12 #66948

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jfindlay
Copy link
Contributor

@jfindlay jfindlay commented Oct 7, 2024

What does this PR do?

Previously, the salt.utils.url.create() method naively assumed the compiled URL provided by urllib.parse.urlunparse() always starts with the literal string file:///, that is, the begininning of the URL string starting with the scheme and including an empty authority section. Beginning with python-3.12, however, generating a URL with urlunparse() results in the syntax file: for the portion of the URL including the scheme and authority sections.

Since urlunparse() accepts the arbitrary scheme salt, use it instead and then normalize the rendered scheme to salt:// as SaltStack expects.

Without this change, salt can't even find its own top file:

$ sudo salt-call --local -l trace state.show_top
[...]
[DEBUG   ] Could not find file 'salt://s' in saltenv 'base'
[...]
local:
    ----------

Previous Behavior

$ venv3.11/bin/python
Python 3.11.10 (main, Sep 23 2024, 08:54:27) [GCC 14.2.1 20240921] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.parse import urlunparse
>>> urlunparse(("file", "", "top.sls", "", "", ""))
'file:///top.sls'
>>> urlunparse(("salt", "", "top.sls", "", "", ""))
'salt:top.sls'
>>> 

New Behavior

$ venv3.12/bin/python
Python 3.12.6 (main, Sep 23 2024, 08:48:49) [GCC 14.2.1 20240921] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.parse import urlunparse
>>> urlunparse(("file", "", "top.sls", "", "", ""))
'file:top.sls'
>>> urlunparse(("salt", "", "top.sls", "", "", ""))
'salt:top.sls'
>>> 

Commits signed with GPG?

Yes

@jfindlay jfindlay requested a review from a team as a code owner October 7, 2024 04:38
@salt-project-bot-prod-environment salt-project-bot-prod-environment bot changed the title Fix urlunparse for salt file urls [master] Fix urlunparse for salt file urls Oct 7, 2024
dmurphy18
dmurphy18 previously approved these changes Oct 7, 2024
@jfindlay jfindlay changed the title [master] Fix urlunparse for salt file urls [master] Fix urlunparse for salt file urls on python-3.12 Oct 7, 2024
@bdrx312
Copy link
Contributor

bdrx312 commented Oct 7, 2024

Essentially duplicate of #66899

return "salt://{}".format(url[len("file:///") :])
url = salt.utils.data.decode(urlunparse(("salt", "", path, "", query, "")))
# urllib returns `salt:` but salt wants `salt://`, so normalize here
if url.startswith("salt:{path}"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing the f: if url.startswith(f"salt:{path}"):

@@ -46,8 +46,11 @@ def create(path, saltenv=None):
path = salt.utils.data.decode(path)

query = f"saltenv={saltenv}" if saltenv else ""
url = salt.utils.data.decode(urlunparse(("file", "", path, "", query, "")))
return "salt://{}".format(url[len("file:///") :])
url = salt.utils.data.decode(urlunparse(("salt", "", path, "", query, "")))
Copy link
Contributor

@bdrx312 bdrx312 Oct 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure that you can safely just replace "file" with "salt". There is special handling for a file scheme; I am not sure the extent of what is different but it could affect how the path and query are handled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, reverted to original format with file:.

Previously, the `salt.utils.url.create()` method naively assumed the
compiled URL provided by `urllib.parse.urlunparse()` always starts with
the literal string `file:///`, that is, the begininning of the URL
string starting with the `scheme` and including an empty `authority`
section.  Beginning with
[python-3.12](python/cpython@0edfc66),
however, generating a URL with `urlunparse()` results in the syntax
`file:` for the portion of the URL including the `scheme` and
`authority` sections.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants