porting guide 2.19 - describe lazy complex var eval impact on builtin filters/tests

[s-hertel]

This is generally covered by https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_12.html#lazy-templating, but t I think it would be helpful to give a couple examples as well.

[oraNod]

Suggested change

	* Passing nested non-scalars with embedded templates that may resolve to ``Undefined`` to Jinja2
	filter plugins such as ``default`` and ``mandatory``, and test plugins including ``defined`` and ``undefined``
	no longer evaluate the same, since nested non-scalars with embedded templates are only templated on use.
	Nested non-scalars with embedded templates that might resolve to ``Undefined`` evaluate differently when passed to Jinja2 filter plugins such as ``default`` and ``mandatory``, as well as test plugins that include ``defined`` and ``undefined``. This happens because embedded templates in nested non-scalars are templated on use only. If your playbooks require immediate evaluate of nested templates in complex variables, you might need to adjust your usage.

[oraNod]

In the first sentence, can we change are templated on use only to `are templated only when ansible-core uses them?

The detail about the test plugins seems ambiguous to me. Does ansible-core no longer evaluate the same as it does in the version I'm using now? Or is the outcome of that evaluation different somehow in 2.19?

Perhaps we extend no longer evaluate the same to no longer evaluate the same as in previous versions.

If that's the case, do we have a recommended action? Should I remove instances of defined and undefined from test plugins? Or do I need to validate my test plugins with 2.19 to observe the behaviour and make changes as appropriate?

[s-hertel]

Thanks for the reviews and wordsmithing. I agree this isn't as clear as it should be, but not sure how to fix it yet...

In the first sentence, can we change are templated on use only to `are templated only when ansible-core uses them?

I think this is true before and after 2.19. In 2.19 though, the user has more control over how ansible-core uses the variable, not that they should care necessarily.

The detail about the test plugins seems ambiguous to me.

This detail (well to clarify: both test and filter plugins) should be the point really - otherwise I'm restating https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_12.html#lazy-templating.

If test/filter plugin usage in a task used to succeed and now fails, it might not be immediately obvious how https://docs.ansible.com/ansible/devel/porting_guides/porting_guide_12.html#lazy-templating relates to the the failure, but this entry would hopefully be useful for someone looking for it. The solution is to update the usage depending on what the intention was. Either reference the correct part of the variable, or (and I can't think of a good reason for this, though I'd love to know if anyone else can think of one) eagerly evaluate the variable.

I can find/think of contrived examples¹²³ that passed before 2.19 and fail on 2.19, but I don't think it's going to be useful to give a specific recommended action, since it depends on how someone is realistically depending on recursively eager variable evaluation. It should just work like people typically expect.

The plugins didn't change and are working correctly, and no changes should be needed for 3rd party filter/test plugins either in relation to this.

No special validation of test/filter plugins should be done, just the normal steps for any major release: run the test suite, playbooks, and roles, and if they pass you're not affected.

Footnotes

1. The is undefined assertion under the second bullet point of https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_core_2.14.html#playbook ↩

2. A hypothetical example using |default: https://github.com/ansible/ansible-documentation/pull/2639/commits/6311ce1840ba5123341fa6e0cad5de24d2ededb0 ↩

3. An ancient issue using is defined and reporting lack of eager evaluation as a bug (!?!?!), but not giving the real use case: https://github.com/ansible/ansible/issues/18514 ↩

[oraNod]

Thanks @s-hertel for the pointer to the section about lazy templating (also for the detailed response). That context was helpful to understand the bits here about plugins.

I can see I was off the mark with my initial suggestion but I've revised that. Hopefully that makes a bit more sense. Cheers for explaining stuff and putting up with my suggestions!

[oraNod]

Thanks @s-hertel for the pointer to the section about lazy templating (also for the detailed response). That context was helpful to understand the bits here about plugins.

I can see I was off the mark with my initial suggestion but I've revised that. Hopefully that makes a bit more sense. Cheers for explaining stuff and putting up with my suggestions!

[oraNod]

How about adding some comments to each line to give more hints? Feel free to ignore if that's not a good suggestion though.

nested: "{{ undefined_variable }}" # This template is not evaluated until it is accessed.