Add Code.Fragment.lines/1 #14493
Add Code.Fragment.lines/1 #14493
Conversation
lib/elixir/lib/string.ex
Outdated
| defp lines(<<?\n, rest::binary>>, acc), | ||
| do: [<<acc::binary, ?\n>> | lines(rest, <<>>)] | ||
|
|
||
| defp lines(<<char, rest::binary>>, acc), |
There was a problem hiding this comment.
Shouldn't we handle /r as well? the python version does:
"asd\radf".splitlines(True)
['asd\r', 'adf']
Side note: LSP document synchronisation treats /r as legit newlines as well
There was a problem hiding this comment.
The Ruby does one not and I am skeptical about doing so because it is clearly not a newline on terminals:
$ iex
Erlang/OTP 27 [erts-15.1.2] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit]
Interactive Elixir (1.19.0-dev) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> IO.puts "foo\rbar"
bar
:ok
So treating it as a newline looks very Windows centric. :)
There was a problem hiding this comment.
Perhaps support the Unicode Line Separator? That would seem consistent with String.split/1 which uses the Unicode definition of whitespace.
There was a problem hiding this comment.
So treating it as a newline looks very Windows centric. :)
Wasn't it classic MacOS?
There was a problem hiding this comment.
Perhaps support the Unicode Line Separator? That would seem consistent with
String.split/1which uses the Unicode definition of whitespace.
python does support it
"asd
adf".splitlines(True)
['asd\u2028', 'adf']
There was a problem hiding this comment.
Python has it's own rules https://docs.python.org/3/library/stdtypes.html#str.splitlines basing on https://peps.python.org/pep-0278/ and https://peps.python.org/pep-3116/
And they quite recently added \v and \f to list of line boundaries (in 3.2)
There was a problem hiding this comment.
And then you have Erlang's interpretation of newlines, which only considers \r\n and \n as well:
~/OSS/elixir[jv-string-lines *%]$ cat example | elixir -e "IO.stream() |> Enum.each(&IO.inspect/1)"
"LINE\n"
"LINE\rLINE\n"
<<76, 73, 78, 69, 12, 76, 73, 78, 69, 11, 76, 73, 78, 69, 194, 133, 76, 73, 78,
69, 226, 128, 168, 76, 73, 78, 69, 226, 128, 169, 76, 73, 78, 69>>
So I am thinking the best is to find a new home for this function. Perhaps the Code module indeed.
There was a problem hiding this comment.
I still use Textmate often and it shows something very similar to Zed.
There was a problem hiding this comment.
That's even more conservative as it only shows \n but I assume there is a configuration somewhere for it to read it Windows style.
There was a problem hiding this comment.
Thank you for the feedback. I have renamed it to Code.lines/1, allowing us to mirror what the Elixir compiler does.
lib/elixir/lib/code.ex
Outdated
| considered, namely `\r\n` and `\n`. If you would like the retrieve | ||
| lines without their line endings, use `String.split(string, ["\r\n", "\n"])`. |
There was a problem hiding this comment.
It feels like the disclaimer about ending will narrow the use case for this quite a bit?
Perhaps we could add a trim: true option later to remove the ending, in which case it would make it more generally useful.
Which kind of use cases do we have in mind for this?
There was a problem hiding this comment.
The use case is to fetch a range from a source file but preserving its original source code. I can add trim: true but trim typically means trimming more (spaces, tabs, etc). It is also something folks can do by doing another pass on the data.
|
I am thinking about moving this to |
|
💚 💙 💜 💛 ❤️ |
Inspired by
String#linesin Ruby andstring.splitlines(True)in Python.