How to Pull XCOM value in PostgresOperator

Question

Here I am pushing XCOM value:

task_get_username_bash = BashOperator(
                task_id='execute_bash',
                bash_command='whoami',
                xcom_push=True)

So in XCOMs it stores like {‘return_value’ : ‘$USER’} (in my case $USER = ‘airflow’).

Then I want to pull this return_value from XCOM:

task_insert_new_row = PostgresOperator(
                task_id='insert_new_row',
                trigger_rule=TriggerRule.ALL_DONE,
                sql='''INSERT INTO table_name VALUES
                (%s, %s, %s);''',
                parameters=(uuid.uuid4().int % 123456789,
                            "{{ ti.xcom_pull(task_ids='execute_bash', key='return_value') }}",
                            datetime.now()))

But PostgresOperator interpret Macros reference as str. How to pull XCOM in PostgresOperator?

Asked By: Vladyslav

||

Source

Answer 1

Problem solved:

task_insert_new_row = PostgresOperator(
                task_id='insert_new_row',
                trigger_rule=TriggerRule.ALL_DONE,
                sql='''INSERT INTO table_name VALUES
                (%s, '{{ ti.xcom_pull(task_ids='execute_bash', key='return_value') }}', %s);''',
                parameters=(uuid.uuid4().int % 123456789, datetime.now()))

Answered By: Vladyslav

Answer 2

Tell me please

Here is my code

load_data = PostgresOperator(task_id="load_data",
                            postgres_conn_id="database",
                            sql=["""INSERT INTO test (id, count, desc, updated) VALUES
                            (%s,%s, '{{ ti.xcom_pull(task_ids=["transform_data"], key="transform_data")[0][{i}]["desc"]}} ',%s);"""for i in range(0, 3)],
                            parameters=(1, 12, datetime.now()))

Here is such an error

Traceback (most recent call last): File "/opt/airflow/lib/python3.8/site-packages/airflow/models/abstractoperator.py", line 410, in _do_render_template_fields rendered_content = self.render_template( File "/opt/airflow/lib/python3.8/site-packages/airflow/models/abstractoperator.py", line 485, in render_template return [self.render_template(element, context, jinja_env, oids) for element in value] File "/opt/airflow/lib/python3.8/site-packages/airflow/models/abstractoperator.py", line 485, in <listcomp> return [self.render_template(element, context, jinja_env, oids) for element in value] File "/opt/airflow/lib/python3.8/site-packages/airflow/models/abstractoperator.py", line 470, in render_template template = jinja_env.from_string(value) File "/opt/airflow/lib/python3.8/site-packages/jinja2/environment.py", line 1105, in from_string return cls.from_code(self, self.compile(source), gs, None) File "/opt/airflow/lib/python3.8/site-packages/jinja2/environment.py", line 768, in compile self.handle_exception(source=source_hint) File "/opt/airflow/lib/python3.8/site-packages/jinja2/environment.py", line 936, in handle_exception raise rewrite_traceback_stack(source=source) File "<unknown>", line 2, in template jinja2.exceptions.TemplateSyntaxError: expected token ':', got '}'

Answered By: Olga Fokina

How to Pull XCOM value in PostgresOperator

Question:

Answers: