GPS data storage system

Question:

I have a python script written using twisted module which is running on an ubuntu server as a service.
We have many gps devices which sends data every 10 sec. My job is to parse that data and store it in database.
Everything is working fine, but total number of concurrent connections is somehow limited to 1012 on my server. I don’t know if it is servers fault or my script.
Plz enlighten me, Is running python script as a service a good practice?or it should run by some other means and why my connections are limited to the wired number 1012.

python script:-

#!/usr/bin/python3

class Message:
    """Parse message and save it in database"""

class Echo(Protocol):
    def connectionMade(self):
        factory_obj.c += 1

    def dataReceived(self, data):
        try:
            msg = data.decode('utf-8')
        except UnicodeDecodeError:
            pass
        else:
            message_handler.handle_message(msg)
            self.transport.write(data)

    def connectionLost(self, reason):
        factory_obj.c -= 1


class EchoFactory(Factory):
    def __init__(self):
        self.c = 0

    def buildProtocol(self, addr):
        return Echo()


class Db:
    """handles database"""


factory_obj = EchoFactory()
message_handler = Message()
database_obj = Db()
a = TCP4ServerEndpoint(reactor, 4995, interface='0.0.0.0', backlog=100000)
a.listen(factory_obj)
reactor.run()

I think my script is correct, there is some problem with server configuration, although I have tried many things like playing with ulimits and all, nothing worked.
Help me on this please.

Thanks!!

Answers:

Increase max_connections to more than the number of devices you have.

Some Operating systems set ulimit to 1o24 for each process.

ulimit -n 4096

Or append these two lines to /etc/security/limits.conf

mysql hard nofile 65535 
mysql soft nofile 65535
Answered By: Rick James

Finally got an answer!! As I am running this python script as service, I just had to write LimitNOFILE=50000 under service section. Check service file below for more clarification.

[Unit]
  Description=Name of your file
  After=network.target
[Service]
  Type=simple
  User=name of the user
  ExecStart=command to run the script
  Restart=always
  LimitNOFILE=60000
[Install]
  WantedBy=multi-user.target                           

and through this we can even achieve more than 65535 connections. We just have to specify LimitNOFILE value (make sure your server can handle that)

Answered By: Priyansh Khandelwal