Don't type `protocol` as `ClassVar`

[kylebarron]

Protocol is defined as a ClassVar:

filesystem_spec/fsspec/spec.py

Line 109 in ffdfed1

protocol: ClassVar[str | tuple[str, ...]] = "abstract"

This means that pylance gives me an error when I try to set it in a constructor:

Typing protocol as a ClassVar makes sense if you assume that you have separate, concrete classes for each implementation, but this is not always true. In Obstore, we have one class (FsspecStore) which dynamically supports different protocols based on the type of obstore.store used inside it.

[martindurant]

Bah, typing proves a hindrance again :|

I'm happy for you to update it to what would be most convenient.

[kylebarron]

You can just type it as str | tuple[str, ...]

[martindurant]

Please make a PR

[ap--]

Hello everyone,

I just saw this issue, and wanted to chime in.

protocol being a class attribute is very convenient for URI parsing. I can retrieve the FileSystem class via the fsspec.registry and apply parsing logic (for example _strip_protocol) without having to instantiate the actual filesystem.

There is however already one exception that I am aware of which is the generic filesystem. With exception I mean, a FileSystem class where _strip_protocol is not a classmethod. But tbf this specific example should not be used for uri parsing anyways and I don't want to derail here.

All that being said, it looks to me as if obstore could just implement the urlpath chaining interface and it would solve the problem at hand?

So instead of:

from obstore.fsspec import FsspecStore

store = FsspecStore("https")
resp = store.cat_file("https://raw.githubusercontent.com/[...]/main/README.md")
assert resp.startswith(b"# obstore") 

you could just write:

import fsspec

store, path = fsspec.url_to_fs("obstore::https://raw.githubusercontent.com/[...]/main/README.md")
resp = store.cat_file(path)
assert resp.startswith("b# obstore")

the obstore filesystem's protocol would always be obstore but its target_protocol would vary. Even the original syntax would still be supported if the first argument of the FsspecStore constructor would be target_protocol

[kylebarron]

I think you misunderstand what my problem is. I'm not suggesting to remove protocol from being a class attribute. But since that's typed as a ClassVar, it can't be set at runtime from within a class instance.

fsspec.url_to_fs("obstore::https://raw.githubusercontent.com/[...]/main/README.md")

That's a non-starter because it means users would have to change the format of their URLs to use obstore. Obstore is another implementation to access the same object storage providers, not a provider in its own right.

[martindurant]

since that's typed as a ClassVar, it can't be set at runtime

well you can - just mypy will complain unless you apply an appropriate ignore flag. In python, there is absolutely no reason that a there shouldn't be an instance attribute that falls back to a class attribute if not set.

[ap--]

I think you misunderstand what my problem is. I'm not suggesting to remove protocol from being a class attribute.

What I'm saying is: it's actually beneficial for urlpath parsing, that the suggested contract is a class attribute instead of an instance attribute.

If you implement it right the obstore user base can still use the pattern they are used to. But you could also register obstore.fsspec.FsspecStore under the "obstore" protocol to allow chaining syntax to be used to use your implementation.