Efficient, DRY bi-directional lookup in both python and C++

Question

In our workflow, we have a few tables of integer ID <-> string name mappings (e.g., status/error codes, hardware IDs, etc.). We are trying to find the best way to write a bidirectional lookup library for these values in both C++ and python.

The interfaces might look like


namespace myproject{
  namespace lookup{
    typedef long ID_t;

    enum class STATUSCODE :ID_t {OK=0, ERROR=1, UNKNOWN=-1}; 
    std::string GetStatusDescription(ID_t statuscode);
    ID_t GetStatusCode(const std::string& statusdesc);

    enum class DEVICELABEL :ID_t {SuperThing_v1=0x234, LessSuperThing_v12=0x12};
    std::string GetDeviceLabel(ID_t hardwareid);
    ID_t GetHardwareID(const std::string& devicelabel); 
  }
}

#file myproject/lookup.py
class STATUSCODE(Enum):
    OK=0
    ERROR=1
    UNKNOWN=-1

def GetStatusDescription(statuscode: int) -> str:
    pass

def GetStatusCode(statusdesc: str) -> int:
    pass


class DEVICELABEL(Enum):
    SuperThing_v1=0x234 
    LessSuperThing_v12=0x12

def GetDeviceLabel(hardwareid: int) -> str:
    pass

def GetHardwareID(devicelabel: str) -> int:
    pass

Ideally, the implementation would satisfy all of the following:

The actual tables are in a single file (or one file per table, but not repeated as the enums in the above example)
Provide efficient bidirectional lookup with integer, string, and enum inputs. E.g. the python Enum class is ideal. On the C++ side I think we’re still stuck with preprocessor tricks for enum<->string mapping. The enum interface would be nice but not necessary.
The C++ interface should read the table at compile-time to avoid having to locate a data file

The lookup tables are all small and are extremely unlikely to ever grow enough that we have to worry about residing in memory. We’re really only concerned with find a good DRY way to store and read the info. I was hoping to be able to find a way to format the data table so that it could be parsed both by python and the C++ compiler (probably with the help of some preprocessor macros).
No one on our team is experienced with creating python bindings for C functions or the other way around, so we’d like to avoid that route if possible. But if that really is the best approach, suggestions for how best to implement that (swig, boost::python?) would be appreciated.

Asked By: thegreatemu

||

Source

Answer 1

There are two methods discussed already in other SO answers:

share enum between python and arduino
: write the C file from Python
Can Python load definitions from a C header file?: parse the C file to create Python values

Answered By: Ethan Furman

Answer 2

It can be done without processing, by sharing common file:
shared.py

#define my_table const struct entry my_table[]
#define statement_end() ;
#define frozenset(...) __VA_ARGS__

#if 0
def statement_end():
    return
#endif

my_table = {frozenset({"OK" , 0}), frozenset({"ERROR", 1}), frozenset({"UNKNOWN", -1})}

statement_end()

#undef my_table
#undef statement_end
#undef frozenset

When shared.py used with Python, all lines starts with # are ignored as they Python comments. When it used with C, the Python code in the #if 0 is ignored and the #define statements take effect

main.c

#include <stdio.h>

struct entry {
    char *str;
    int n;
};

#include "shared.py"

int main()
{
    /*
     * Write here great code that create bi directional lookup
     * from array of entry struct.
     * Ideas can be taken from https://stackoverflow.com/questions/6669842/how-to-best-achieve-string-to-number-mapping-in-a-c-program
     */
 
     for (const struct entry * pair = my_table; (char *)pair != (char *)my_table + sizeof(my_table); pair++)
     {
         printf("str: %s. int: %dn", pair->str, pair->n);
     }

    return 0;
}

main.py

import shared

# Write here great code that create bi directional lookup from set of frozensets, each one contain int and string
for pair in shared.my_table:
    print(pair)

Answered By: arye

Efficient, DRY bi-directional lookup in both python and C++

Question:

Answers: