summaryrefslogtreecommitdiff
path: root/lib/ansible/template
Commit message (Collapse)AuthorAgeFilesLines
* Prevent losing unsafe from lookups (#77609) (#77652)Martin Krizek2022-05-091-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch fixes a bug which under certain conditions results in data returned from lookups not being marked as unsafe. Each time Templar.do_template is invoked a new AnsibleContext is created and stored effectively at two places: 1) as an instance variable in templar_obj.cur_context 2) as a local variable called new_context in do_template method of Templar Due to custom functionality in Ansible's Context that allows for nested templating it is possible that during resolving variable's value template/do_template method is called recursively again, again creating a new context. At that point the problem manifests itself because as mentioned in 1) above the context is overwriten on the templar object which means that any subsequent calls to _lookup will use the new context to mark it as unsafe which is now different to the local new_context which is used for testing for unsafe property. The solution to the problem appears to be to restore the original context inside do_template and also to eliminate the local variable new_context to prevent problems in the future. It appears that we don't have a better way of storing the context other than as some form of global variable and so this appears to be the "best" solution possible at this point. Hopefully data tagging will be the solution here. For more examples see unit and integration tests included in this patch. Fixes #77535 (cherry picked from commit 3980eb8c09d170a861351f8aff4a1aa1a8cbb626)
* [2.11] Fix collection redirects for filter and test plugins (#77210) (#77228)Sloane Hertel2022-03-151-36/+53
| | | | | | | | | | | | | | * Fix collection redirects for filter and test plugins (#77210) * Fix collection redirects for jinja2 filters/tests * Handle recursive redirects Co-authored-by: Matt Martz <matt@sivel.net> (cherry picked from commit 8063643b4cec51a72377da5f3fa354d3ff9e737a) * The error message is only capitalized on 2.13. Make test more flexible. (cherry picked from commit 734777ef05b8c200f45336a8b9b41f9f9af597c1)
* Globals should be accessible when importing a template without the context ↵Martin Krizek2021-08-051-7/+9
| | | | | | | (#75384) (#75409) Fixes #75371 (cherry picked from commit 5a3807656860cc52c7e6b3eebc5c9397585525ba)
* Ensure Jinja2 template header overrides are used (#75306) (#75342)Martin Krizek2021-08-031-5/+11
| | | | | Fixes #75275 (cherry picked from commit 767b2f07b00be12b9366655095cf24120d35092e)
* fix unsafe preservation across newlines (#74960) (#74973)Brian Coca2021-06-111-1/+4
| | | | | | | CVE-2021-3583 ensure we always have unsafe Co-authored-by: Rick Elrod <rick@elrod.me> (cherry picked from commit 4c8c40fd3d4a58defdc80e7d22aa8d26b731353e)
* Correctly set path and fullpath for template vars (#73924)Brian Coca2021-03-261-3/+12
| | | | | | * Correctly set path and fullpath for template vars don't expect path to always be full path also added exception/tb on action fail
* Reduce complexity of Templar._lookup slightly (#73277)Matt Martz2021-03-011-48/+49
| | | | | | | * A little more complexity reduction * restore logic * Readability * Add ran check back * Add clog
* Local vars should have highest precedence in AnsibleJ2Vars (#72830)Martin Krizek2021-01-261-22/+14
| | | | | | | | | | | | | | | | | | Ability to add local variables into AnsibleJ2Vars was added in 18a9eff11f0a6e51b17405ce596bd9ff7e676320 to fix #6653. Local variables are added using ``AnsibleJ2Vars.add_locals()`` method when creating a new context - typically when including/importing a template with context. For that use case local template variables created using ``set`` should override variables from higher contexts - either from the play or any parent template, or both; Jinja behaves the same way. Also removes AnsibleJ2Vars.extras instance variable which is not used. Also adds missing test for #6653. Fixes #72262 Fixes #72615 ci_complete
* Consolidate filters/tests handling into JinjaPluginIntercept (#71463)Martin Krizek2021-01-211-71/+34
| | | | | | | | | | | | | | | | | | | | | | | | | * Consolidate filters/tests handling into JinjaPluginIntercept ci_complete * Postpone loading all ansible plugins * Do we need to create an overlay? ci_complete * Typo ci_complete * Add FIXME * conditional.py: use public Environment.parse() method * Remove remaining occurrences of shared_loader_obj being passed to Templar * __UNROLLED__ not needed with this change anymore * Incorrect rebase at some point?
* Do not pretend expression is filename in compile() Python built-in call (#73113)Oldřich Jedlička2021-01-191-1/+1
| | | | | | | | When calling compile(), the filename argument should be either a real file name or a string. According to Python docs, suggested one is '<string>'. Keep the current behaviour (encapsulate the actual expression), but enclose it into angle brackets. Signed-off-by: Oldřich Jedlička <oldium.pro@gmail.com>
* Use _wrap_native_text only for builtin STRING_TYPE_FILTERS (#71801)Martin Krizek2020-12-091-1/+2
|
* Fix parsing of values when using an empty string as key (#57132) (#72545)Yadnesh Kulkarni2020-11-201-1/+1
| | | Signed-off-by: Yadnesh Kulkarni <ykulkarn@redhat.com>
* rethink wording (#70028)Brian Coca2020-10-301-2/+2
| | | | | | | * rethink wording * removed unrequired requirement * fix tests * fixed versions Co-authored-by: Sloane Hertel <shertel@redhat.com>
* Only apply the unroll wrapper once (#72003)Martin Krizek2020-09-301-1/+3
| | | | | Co-authored-by: Matt Martz <matt@sivel.net> Fixes #71920
* Provide more information in AnsibleUndefinedVariable (#71666)Martin Krizek2020-09-221-2/+2
| | | | | * Provide more information in AnsibleUndefinedVariable Fixes #55152
* Force template module to use non-native Jinja2 (#68560)Martin Krizek2020-09-032-29/+90
| | | Fixes #46169
* Skip literal_eval for string filters results in native jinja. (#70988)Martin Krizek2020-08-112-4/+50
| | | Fixes #70831
* Misc typo fixes (#71089)Abhijeet Kasurde2020-08-051-1/+1
|
* native types: properly handle Undefined in nested data (#68432)Martin Krizek2020-08-041-18/+28
|
* Emit proper error for `x in y` when y is undefined (#70990)Martin Krizek2020-07-301-0/+4
| | | Fixes #70984
* Deprecation revisited (#69926)Felix Fontein2020-06-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Allow to specify collection_name separately for deprecation. * Use new functionality in Ansible. * Use new functionality in tests. * Update tagging/untagging functions. * Update pylint deprecated sanity test. * Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections). * Improve version validation. Re-add version_added validation. * Make sure collection names are added to return docs before schema validation. * Extra checks to avoid crashes on bad data. * Make C# module utils code work, and update/extend tests. * Add changelog fragment. * Stop extracting collection name from potentially tagged versions/dates. * Simplify C# code. * Update Windows modules docs. * Forgot semicolons.
* Add custom globals to the environment, and not per template (#69278)Matt Martz2020-06-081-10/+9
| | | | | * Add custom globals to the environment, and not per template * Add changelog fragment
* Auto unroll generators produced by jinja filters (#68014)Matt Martz2020-06-081-2/+55
| | | | | | | | | | | | | | | | | | | * Auto unroll generators produced by jinja filters * Unroll for native in finalize * Fix indentation Co-authored-by: Sam Doran <sdoran@redhat.com> * Add changelog fragment * ci_complete * Always unroll regardless of jinja2 * ci_complete Co-authored-by: Sam Doran <sdoran@redhat.com>
* various deprecation, display, warning, error fixes for collections ↵Matt Davis2020-06-051-16/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | redirection (#69822) * various deprecation, display, warning, error fixes * Update lib/ansible/utils/display.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update lib/ansible/utils/display.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update lib/ansible/utils/display.py Co-authored-by: Felix Fontein <felix@fontein.de> * cleanup, test fixes * add collection name to deprecated() calls * clean up redirect entries from uncommitted tests * fix dep warning/error header text to match previous Co-authored-by: Felix Fontein <felix@fontein.de>
* Version source tagging (automatic and manual) for version_added and ↵Felix Fontein2020-05-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | deprecation versions (#69680) * Track collection for version_added. Validate *all* version numbers in validate-modules. For tagged version numbers (i.e. version_added), consider source collection to chose validation. * Make tagging/untagging functions more flexible. * Tag all versions in doc fragments. * Tag all deprecation versions issued by code. * Make Display.deprecated() understand tagged versions. * Extend validation to enforce tagged version numbers. * Tag versions in tests. * Lint and fix test. * Mention collection name in collection loader's deprecation/removal messages. * Fix error IDs. * Handle tagged dates in Display.deprecated(). * Also require that removed_at_date and deprecated_aliases.date are tagged. * Also automatically tag/untag removed_at_date; fix sanity module removal version check. * Improve error message when invalid version number is used (like '2.14' in collections).
* collection routing (#67684)Matt Davis2020-05-261-32/+57
| | | | | | | * `meta/` directory in collections * runtime metadata for redirection/deprecation/removal of plugin loads * a compatibility layer to keep existing content working on ansible-base + collections * a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better * supported Ansible version validation on collection loads
* Properly handle unicode in safe_eval (#68576)Martin Krizek2020-05-201-2/+7
| | | | | | | | | * Properly handle unicode in safe_eval Fixes #66943 * Update lib/ansible/template/safe_eval.py Co-authored-by: Sam Doran <sdoran@redhat.com>
* native types: literal_eval all the things (#68938)Martin Krizek2020-04-171-5/+7
| | | | | | | With https://github.com/pallets/jinja/pull/1190 merged our short-circuit is no longer valid (has it ever been?) as now data like ' True ' may go through our ansible_native_concat function as opposed to going through intermediate call to Jinja2's native_concat before. Now we need to always send data through literal_eval to ensure native types are returned.
* Prevent templating unused variables for {%include%} (#68749)Martin Krizek2020-04-142-3/+39
| | | Fixes #68699
* ansible_native_concat: use to_text rather than jinja2's text_type (#68038)Martin Krizek2020-03-241-4/+4
| | | | jinja2._compat.text_type has been removed in jinja2's master branch so use ansible's to_text instead.
* Address fixme and handle filter/test errors for collections better (#68047)Matt Martz2020-03-231-4/+8
| | | | | * Address fixme and handle fitler/test errors for collections better. Fixes #66721 * Re-arrange code
* Address compat issue for collection loading on py26 (#68219)Matt Martz2020-03-231-6/+1
| | | | | | | | | | | | | | | | | | | | | * Address compat issue for collection loading on py26 * Move import_module shim to utils for compat across the codebase * Enable collection tests on py2.6 * Update changelog fragment * Simplify code using sys.moduls * Move compat to module_utils/compat/importlib * Add back errantly deleted newline * Remove hack comment Co-Authored-By: Matt Clay <matt@mystile.com> Co-authored-by: Matt Clay <matt@mystile.com>
* Templating: make sure only one variable results are cached (#67429)Felix Fontein2020-02-191-1/+1
| | | | | | | * Make sure only one variable results are cached. * Add changelog. * Add test.
* Fix case sensitivity for lookup() (#66521)Martin Krizek2020-01-271-1/+1
| | | | | | | | This brings consistency to lookup(), with_ and ansible-doc. Fixes #66464 * Add a porting guide entry
* Check vars against Mapping in DEBUG (#66355)Martin Krizek2020-01-131-2/+2
| | | | When ANSIBLE_DEBUG=1, vars are VarsWithSources now which is a Mapping. Check vars against Mapping instead of dict.
* Ensure that data within a tuple is marked as unsafe (#65918)Matt Martz2020-01-071-2/+3
| | | | | | | | | | | | | | | | | | | | | | * Use is_sequence, and Mapping throughout, add support for tuples. Fixes #65722 * Address tests * Remove unused import * Add changelog * Add docstring for clarity * Argh, linting fix * Not chasing this rabbit * wrap_var doesn't return a ref to the original item * no ref tests * Remove unused import
* Do not treat AnsibleUndefined as being unsafe (#65202)Matt Martz2019-11-251-1/+5
| | | | | | * Do not treat AnsibleUndefined as being unsafe. Fixes #65198 * fix yaml formatting
* Introduce context manager for temporary templar context changes (#60513)Matt Martz2019-10-251-0/+31
| | | | | | | | | | | | | | | | | | * Introduce context manager for temporary templar context changes. Fixes #60106 * Rename and docstring * Make set_temporary_context more generic, don't hardcode each thing you can set, apply to template action too * not None * linting fix * Ignore invalid attrs * Catch the right things, loop the right things * Use set_temporary_context in a few extra action plugins
* Wrap CLI Passwords with AnsibleUnsafeText, ensure unsafe context is not lost ↵Matt Martz2019-10-111-1/+1
| | | | | | | | | | | | during encode/decode (#63351) * Wrap .encode and .decode on AnsibleUnsafe objects * runme.sh needs to be executable * ci_complete * Update changelog with CVE
* fix collection jinja2 cache issue (#62543)Matt Davis2019-09-181-4/+2
| | | * prevents premature lookup (and potential KeyError) of Jinja filter/test function cache that's not fully populated
* add subdir support to collection loading (#60682)Matt Davis2019-08-211-9/+11
| | | | | | | | | | | | | | | * add subdir support to collection loading * collections may now load plugins from subdirs under a plugin type or roles dir, eg `ns.coll.subdir1.subdir2.myrole`->ns.coll's roles/subdir1/subdir2/myrole, `ns.coll.subdir1.mymodule`->ns.coll's plugins/modules/subdir1/mymodule.py * centralize parsing/validation in AnsibleCollectionRef class * fix issues loading Jinja2 plugins from multiple sources * resolves #59462, #59890, * sanity test fixes * string fixes * add changelog entry
* Remove UnsafeProxy (#59711)Martin Krizek2019-08-071-3/+3
| | | | | | | | | | | | | | | | * Remove UnsafeProxy Move the work from UnsafeProxy to wrap_var and add support for bytes. Where wrap_var is not needed, use AnsibleUnsafeBytes/AnsibleUnsafeText directly. Fixes #59606 * item is not always text * Address issues from reviews * ci_complete
* Add back _contains_vars method as maybe_template (#58290)Matt Martz2019-06-251-3/+22
| | | | | | | | | | * Add back _contains_vars method as maybe_template. Fixes #58282 * Remove template guard in a few places * maybe_template sounds like it might template something, rename to is_possibly_template * Add tests for is_possibly_template
* also allow None Type for safe eval (#58269)markafarrell2019-06-241-0/+1
|
* Tests as filters were deprecated, remove unused param (#57796)Martin Krizek2019-06-131-2/+2
|
* Perf improvement for Templar.is_template (#57489)Matt Martz2019-06-061-28/+40
| | | | * Faster is_template
* safe_eval fix (#57188)Brian Coca2019-06-062-3/+7
| | | | | | | | | * just dont pass locals - also fix globals - added tests * fixed tests
* Make query with errors='ignore' return a blank list (#57038)Ian Wienand2019-05-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The jinja2 query() function (or lookup with wantslist=True, which is the same thing) should always return a list. However, if you combine a query with errors='ignore' and take the error path, the current code returns a None value. This is important in a case such as - name: Conditional include file import_tasks: '{{ item }}' vars: params: files: - path/file1.yaml - path/file2.yaml loop: "{{ q('first_found', params, errors='ignore') }}" If neither file1.yaml or file2.yaml exist, this should do nothing by returning an empty list to the loop. Currently if you run the above task you'll get a rather unhelpful: Invalid data passed to 'loop', it requires a list, got this instead: . This change ensures that when a query ignores an error, it returns a empty list. The errors='ignore' case is tested in several variants with first_found. The extant (but deprecated) "skip: True" for first_found doesn't seem to be explicitly tested; a test is added here to avoid regressions before removal in 2.12. This fixes a regression you'll hit if you follow the suggestion in the deprecation message included with e17a2b502d6601be53c60d7ba1c627df419460c9 to use errors=ignore over "skip: True" for first_found. This change adds an example that points out the query/lookup difference and also fixes the error message to not mention the now deprecated "skip: True". Closes #56775
* Not native, but text (#55676)Brian Coca2019-05-241-2/+1
| | | | | | * use to_text instead of to_native * cleaned up some imports and other pyflakisms * fix missing lib messages
* Templar: encapsulate _available_variables (#55435)Martin Krizek2019-05-202-6/+18
| | | | | Ensure variables are reset between iterations
View Lorry Controller Status