macros are not recognised in dbt

Question

{{ 
    config (
        pre_hook = before_begin("{{audit_tbl_insert(1,'stg_news_sentiment_analysis_incr') }}"),
        post_hook = after_commit("{{audit_tbl_update(1,'stg_news_sentiment_analysis_incr','dbt_development','news_sentiment_analysis') }}")
        )
}}

select rd.news_id ,rd.title, rd.description, ns.sentiment from live_crawler_output_rss.rss_data rd 
left join 
live_crawler_output_rss.news_sentiment ns 
on rd.news_id = ns.data_id limit 10000;

This is my model in DBT which is configured with pre and post hooks which referance a macro to insert and update the audit table.

my macro

{ % macro audit_tbl_insert (model_id_no, model_name_txt) % }

{% set run_id_value = var('run_id') %}

insert into {{audit_schema_name}}.{{audit_table_name}} (run_id, model_id, model_name, status, start_time, last_updated_at)
values 
({{run_id_value}}::bigint,{{model_id_no}}::bigint,{{model_name_txt}},'STARTED',current_timestamp,current_timestamp)

{% endmacro %}

this is the first time i’m using this macro and I see the following error.

Compilation Error in model stg_news_sentiment_analysis_incr 
(models/staging/stg_news_sentiment_analysis_incr.sql)
'audit_tbl_insert' is undefined in macro run_hooks (macros/materializations/hooks.sql) 
called by macro materialization_table_default (macros/materializations/models/table/table.sql) called by model stg_news_sentiment_analysis_incr 
(models/staging/stg_news_sentiment_analysis_incr.sql). 
This can happen when calling a macro that does not exist. 
Check for typos and/or install package dependencies with "dbt deps".

Asked By: naga satish

||

Source

Answer 1

Reading the error, it seems that your macro is being invoked; perhaps the problem is not that it isn’t being recognised. Note that stg_news_sentiment_analysis_incr is mentioned in the error log, and this appears in your first snippet as a hard-coded input to the macro executed by the book. Therefore the error appears to be occurring after we’re already inside the macro.

So what’s causing the error? On the first line it says:

Compilation Error in model stg_news_sentiment_analysis_incr

Reading your code, I don’t think that you wanted to compile the model. You seem to me to be trying to insert this string into an audit table. But you have written

pre_hook = before_begin("{{audit_tbl_insert(1,'stg_news_sentiment_analysis_incr') }}")

When the hooks compile, the following will be invoked:

{{ audit_tbl_insert(1,'stg_news_sentiment_analysis_incr') }}

And there’s your problem (if I understand your objective): 'stg_news_sentiment_analysis_incr' will be passed into the macro and interpreted as the name of a model, not as a string. See the docs – one of the examples shows a field name being passed in and used as a field name using this syntax.

In your macro, you write:

({{run_id_value}}::bigint,{{model_id_no}}::bigint,{{model_name_txt}},'STARTED',current_timestamp,current_timestamp)

… and it is here that the error is generated. Judging by the error text, it tries to compile the model, which calls a macro called materialization_table_default, which calls run_hooks, which tries to reference something called corpository_audit_tbl_insert, which doesn’t exist (at least, not at compilation time).

Answered By: Paddy Alton

Answer 2

Your macro’s definition has too much whitespace in the braces that define the jinja block:

{ % macro audit_tbl_insert (model_id_no, model_name_txt) % }

Needs to be

{% macro audit_tbl_insert (model_id_no, model_name_txt) %}

And then this should work just fine.

Answered By: tconbeer

macros are not recognised in dbt

Question:

Answers: