How can we save all the logs in a database with some extra columns?(django)

Question

I am working on a Django project in which I have to save all the logs(not just the error logs), for example when a user clicks on a tab or even a button, I should save its log. And I want to save them in my database(or better a separate database) because I need to analyze the logs and extract some information from them.

I have seen some posts in StackOverflow, but they were not very useful for me. In this post some answers have been provided, and I tried django-db-logger package in the third answer.

I have added the below code to settings.py based on the README file of this project:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
        'simple': {
            'format': '%(levelname)s %(asctime)s %(message)s'
        },
    },
    'handlers': {
        'db_log': {
            'level': 'DEBUG',
            'class': 'django_db_logger.db_log_handler.DatabaseLogHandler'
        },
    },
    'loggers': {
        'db': {
            'handlers': ['db_log'],
            'level': 'DEBUG'
        },
        'django.request': { # logging 500 errors to database
            'handlers': ['db_log'],
            'level': 'ERROR',
            'propagate': False,
        }
    }
}

I have added some logs to one of my APIs in the below code:

import logging

db_logger = logging.getLogger('db')


# _____________ user Management ______________
@api_view(['POST'])
def user(request):
    db_logger.info("In signup API")
    if request.method == "POST":
        body = json.loads(request.body.decode('utf-8'))
        body['password'] = hash_sha256(body['password'])

        duplicate = Profile.objects.filter(username=body['username'])
        if len(duplicate) != 0:
            db_logger.critical("The username is duplicate")
            return Response({"message": "نام کاربری تکراری است."}, status=status.HTTP_409_CONFLICT)

        instance = Profile.objects.create(**body)
        db_logger.info(f'User with {instance.username} created successfully')
        return Response({"username": body['username']})

After I execute the program a new table was created in my database (I am using MySQL workbench 8.0), but it contains only these columns(which have been provided in LOGGING):

But I need to save some extra information such as the username of every user that causes the log, and even the page URL. I there a way to add some parameters to logger object and then save them in the database?

If there is any other way to save this kind of information, I will be grateful for your help. I have no idea how to solve this problem.

Asked By: Aylin Naebzadeh

||

Source

Answer 1

For this purpose, there are some options in Django:

The handler is the engine that determines what happens to each message
in a logger. It describes a particular logging behavior, such as
writing a message to the screen, to a file, or to a network socket.

Maybe you can find a way here!

'handlers': {
    'console': {
        'level': 'INFO',
        'filters': ['require_debug_true'],
        'class': 'logging.StreamHandler',
        'formatter': 'simple'
    },
    'mail_admins': {
        'level': 'ERROR',
        'class': 'django.utils.log.AdminEmailHandler',
        'filters': ['special']
    }
},
'loggers': {
    'django': {
        'handlers': ['console'],
        'propagate': True,
    },
    'django.request': {
        'handlers': ['mail_admins'],
        'level': 'ERROR',
        'propagate': False,
    },
    'myproject.custom': {
        'handlers': ['console', 'mail_admins'],
        'level': 'INFO',
        'filters': ['special']
    }
}

Answered By: Shahab Rahnama

Answer 2

I have found 2 answers to my question.

1) Customizing django-db-logger package

I think in this youtube tutorial everything has been explained clearly. But the main idea is to modify the django-db-logger package after installing it in our project.

2) Write an API and call it in all the HTML files

The only problem with the first solution is that in some cases we can not record all the actions that have been done by a user, and some actions are handled on the UI side. So we can write an API, and then call it on every component and HTML page. For example, when the user is clicking from one panel to another, I could not record a log using the first approach.

Answered By: Aylin Naebzadeh

How can we save all the logs in a database with some extra columns?(django)

Question:

Answers: