Progress on implementation

[jasny]

If started working on an implementation. See https://github.com/jasny/php-src/tree/generics

I'll try to post updates on the progress. If anybody else is working on this, please connect so we can collaborate.

[jasny]

@natebrunette I could open a pull request if the fork PHPGenerics/php-src was up to date. However, I prefer to be made a collaborator on that project, so I can push to the fork here.

[jasny]

Added support for generics to the lexer, for class declarations

class Foo<T>
{
    public T $one;
    public T $two;
} 

---

Also added support for object instantiation. The syntax for object instantiation wouldn't be accepted by the lexer though, giving this error;

php-src/Zend/zend_language_parser.y: error: shift/reduce conflicts

The problem is that the following syntax is already valid

call (new Date < FOO, BAR > ('now') )

I've tried to use a virtual token and set precedence as is done to solve the well known "dangling else" issue. Unfortunately this isn't applicable, because conflict is not confined to the parsed statement (or some other reason. In any case it didn't work.)

For now, the use of parenthesis is required for instantiation (and for functions when added).

$foo = (new Foo<int, DateTime>)(42);

The use of parenthesis don't necessarily need to be the final syntax. Further discussion on how to deal with this conflict is required. Ignoring it (giving priority to generics over operators) would require advanced use of YACC.

[jasny]

Added a struct for generic property declarations to Zend.c, which hold the name (like T) and (in the future) can hold a type as Machine in T is Machine.

typedef struct _zend_generic_name {
    zend_string *name;
    zend_string *type;
} zend_generic_name;

The compiler takes the list of names from the lexer and creates an array of zend_generic_name values.

Added generic_names and num_generics properties to the zend_class_entry. The compiler set these properties and the ZEND_ACC_GENERIC flag for a generic class.

At the moment, generics are ignored at instantiation (TODO).

[mindplay-dk]

The problem is that the following syntax is already valid

call (new Date < FOO, BAR > ('now') )

As I recall, someone did work on the parser earlier and actually solved this?

Something about making the parser favor Date<Foo> over Date < FOO > whenever this could be part of a generic type expression?

It's only technically a BC break, in the very unlikely use-case where someone is using the , operator in a pretty exotic way - it's not likely to break any real-world code at all. (and new Date() < FOO, BAR > ('now') is an easy fix in the unlikely event that such code exists in the wild.)

I don't recall who it was though. 😐

[zmitic]

Pardon my ignorance but I am really excited about generics and I am a big fan of psalm. The question is; will code like this:

class Foo<T>
{
    public T $one;
    public T $two;
} 

be opcached as if user wrote:

class Foo
{
    public $one;
    public $two;
} 

i.e. completely ignore generics during runtime? Wouldn't this approach be easier to implement?

Even Facebook, multi-billion dollar company, didn't solve runtime checks. I don't think it is worth even trying, especially with static-analysis tools we have now.

PS: I am in a camp that would really like an option to disable all runtime type checks via php.ini, perfect for production to gain extra speed 😄

[mindplay-dk]

i.e. completely ignore generics during runtime? Wouldn't this approach be easier to implement?

Definitely.

But it's awkwardly inconsistent with PHP, which is already fully reflected and fully run-time type-checked - we don't want to make matters worse by introducing another inconsistency to a language that's already infamous for having way too many of those.

This is actually covered by the RFC:

php-generics-rfc/generics.txt

Lines 426 to 430 in cc72197

	=== Reification ===
	Because PHP is a reflective language, type arguments are fully [[https://en.wikipedia.org/wiki/Reification_(computer_science)#Reification_and_reflective_programming_languages|reified]] - which means they actually exist (in memory) at run-time.
	This differs from generics in Hack, where type-hints are [[https://github.com/facebook/hhvm/issues/5317|not reified]] and are unavailable at run-time. Type erasure would be inconsistent with the existing PHP type system, where any type-declaration is generally available at run-time via reflection.

With regards to run-time type-checking of properties, specifically, PHP 7.4 actually has that - which is something that increases consistency of the type-system overall, so, again, we want to continue in that direction and improve the consistency of the language.

I don't think it is worth even trying, especially with static-analysis tools we have now.

One area where run-time type-checking has real value is in the context of web-development, where all your input comes from unchecked $_GET, $_POST or json_decode() and likely has to translate into your models somehow.

The main problem with omitting a feature like this is just simply user expectations. If it looks like a type-hint, users will expect a type-check, because that's how PHP works. If the language doesn't consistently deliver the expected type-checks, you have to reason about every declaration you write, every change you make during refactoring, "is this going to type-check", and so on, which can very easily lead to subtle bugs or potential vulnerabilities.

In my opinion, PHP being a web-first language, these things aren't just details with no practical value, they're core to the overall reliability of the language.

[zmitic]

Wouldn't this approach be easier to implement?

Definitely.

Then:

😄 😄 😄

If it looks like a type-hint, users will expect a type-check

Maybe I am too subjective but I don't see this as any problem at all. It can also be an extension, I wouldn't care.

I think PHP should allow advanced developers an opportunity to do new things and not live in WP-and-siblings era. This is new feature, documentation can clearly say that typecheck is not performed. I would be totally fine with it, that is why we have IDE and static analysis tools.

It might as well attract new developers from other languages. Reminder; this is not something beginners would use.

In my opinion, PHP being a web-first language, these things aren't just details with no practical value, they're core to the overall reliability of the language.

This is exactly my point; by getting extra speed, PHP could be used in other ways, not just for web. Right now, I have an application that process 92 files, each having 28 million CSV rows (NOAA data).

The task; from available ~25 values, I need to make ~10 by combining source values. To make code clean, I made it using tagged services; lots of small files implementing my interface.

The problem; parameters are checked many times, over and over again. I didn't test execution without typehints but it surely must affect it.
I am basically penalized for writing clean code. If I had put all that code in one big method, it would be nightmare to maintain but no performance loss.

By adding me an option to disable runtime checks, it would be my responsibility if something goes wrong. The same applies for generics; better to have anything than nothing.

Code example:

I hope you understand my point. If I made an error somewhere, I would easily spot it during development. But later, I really don't need all these typechecks during runtime.

---

Note:
I am aware that TypeError would not be caught but again; my responsibility.

[mindplay-dk]

this is not something beginners would use.

I've seen beginners successfully use basic generics in TypeScript, so I have to disagree.

If I did agree, I don't think this should be an excuse to make it harder for beginners if/when they do use it.

Consistency in a language benefits everyone, beginners or experts, on a daily basis.

by getting extra speed, PHP could be used in other ways

Speed is nice - but PHP isn't "fast", and you don't pick a scripting language for it's "speed".

By adding me an option to disable runtime checks, it would be my responsibility if something goes wrong

Maybe, but if so, that argument applies to the whole language - again, not an excuse to introduce inconsistencies and surprises in a language that's trying to be accessible to people of all skill levels.

Dart, for example, has that option - but across the board, for the whole language, not just for a few features because someone was impatient or couldn't be bothered to finish.

It sounds like you and I would both be very happy with something like TypeScript, where all the type-hints are strictly checked at compile-time and leave no run-time footprint at all - but it was designed that way, so when you're coding in TypeScript, you know what to expect, and that's just not how PHP works.

In my opinion, it's an extremely bad idea to introduce partial type erasure into a reflected, type-checked language - coding is hard enough with all the inconsistencies in userland, without also making the language chuck-full of "fun" surprises.

[zmitic]

I've seen beginners successfully use basic generics in TypeScript, so I have to disagree.

This makes sense; if developer makes a mistake, Typescript won't compile. So it is easy to spot errors and learn generics fast; it is exactly how I did when dealing with Angular.

But then we might never get them. My biggest hope was https://twitter.com/php_plus but repository is now private and they are very silent.

tl;dr
the idea was similar to TS; it would transpile to PHP on the fly and add annotations when needed so psalm would work. Plus few other features.

[muglug]

@jasny will there be support for instanceof Collection<Foo>?

[mindplay-dk]

@jasny just came across Dart's test-suite for generics, which is pretty excellent - the inline documentation is really useful. Also, I poked around the history in this issue tracker and came across a thread about the ambiguity problem - I added a comment here to note that the situation is probably worse now than when we started. Part of me suspects we can't introduce generics now without some sort of (however minor) breaking change to the language syntax. 😶

[jasny]

@mindplay-dk I'm going to take as many liberties to the syntax as needed to get generics working. I'm explicitly staying out of the discussion.

[nikic]

I've also started some basic prototyping work for generics at php/php-src@master...nikic:generics-2. My initial goal would be to only support "purely abstract" generics, i.e. without the ability to instantiate generic objects (which covers half of the generics use case). Not sure how much time I will invest in this.

[zmitic]

@nikic
Love the Something works commit message 😄 : php/php-src@f0f387c

Not sure how much time I will invest in this

Is there something others (non C developers) can do to help?

[muglug]

@nikic that looks amazing.

Clearly PHP doesn't need generics – development has gone just fine for the last 25 years without the features – but I think it's a banner feature, and adding it would improve outsiders' perception of PHP markedly.

Also, from personal experience a number of workplaces, thinking about problems using generic concepts has helped clarify problems, and helped me and others avoid common pitfalls – another great reason why it would be a solid addition.

[jkufner]

Clearly PHP doesn't need generics – development has gone just fine for the last 25 years without the features

I can't agree with this. The need for generics arises from the relatively new type system. Once we have classes and type hints, we hit limitations and hence the need for generics. We did not miss generics for the first 20 years because we had no type system where they could miss.

[SageDroid]

Great to see progress here! 👍

@nikic that looks amazing.

Clearly PHP doesn't need generics – development has gone just fine for the last 25 years without the features – but I think it's a banner feature, and adding it would improve outsiders' perception of PHP markedly.

Also, from personal experience a number of workplaces, thinking about problems using generic concepts has helped clarify problems, and helped me and others avoid common pitfalls – another great reason why it would be a solid addition.

I started PHP development in 2018 and before that I mainly developed with Java. For me the lack of generics was the biggest limitation and the biggest challenge. In many scenarios you could use generics to write cleaner, more reusable and better maintainable code than you do now. I would be infinitely grateful for generics in PHP. Unfortunately, I lack the knowledge in C to be able to help productively with the implementation.