Python Support
Python Support
Given that python is leading language for ML and many new interesting supports is coming from the language, therefore it was needed that we have built in support for such python code execution. However, following are the conditions that we wanted to apply.
- Python runs in single threaded manner, but we like parallel execution
- Run python code in separate process. If process creation fails then run in thread
- Read return data from python process
- Keep status of the process for reporting
- Python 2.7 and 3 support
- BangDB should compile and run with or without python - provide a switch
Currently the SVM doesn't require python, but IE may need it. Therefore, need to compile BangDB accordingly
Input file format
For training and even for prediction, user may like to send data in different formats. Therefore, we needed simple mechanism to handle this. We have separate interface "iqconvert" defined, which has following APIs
int convert(char *infile, char *outfile);
int convert(FILE *finfile, FILE *foutfile);
Currently it's implemented for csv and json to libsvm format converters. Developer should implement this for new conversion logic as appropriate