Random Forest interpretation in scikit-learn
Question:
I am using scikit-learn’s Random Forest Regressor to fit a random forest regressor on a dataset. Is it possible to interpret the output in a format where I can then implement the model fit without using scikit-learn or even Python?
The solution would need to be implemented in a microcontroller or maybe even an FPGA. I am doing analysis and learning in Python but want to implement on a uC or FPGA.
Answers:
You can check out graphviz, which uses ‘dot language’ for storing models (which is quite human-readable if you’d want to build some custom interpreter, shouldn’t be hard). There is an export_graphviz function in scikit-learn. You can load and process the model in C++ through boost library read_graphviz method or some of other custom interpreters available.
It’s unclear what you mean by this part:
Now, that I have the results, is it possible to interpret this in some format where I can then implement the fit without using sklearn or even python?
Implement the fitting process for a given dataset? tree topology? choice of parameters?
As to ‘implement… without using sklearn or python’, did you mean ‘port the bytecode or binary’ or ‘clean-code a totally new implementation’?
Assuming you meant the latter, I’d suggest GPU rather than FPGA or uC.
You could trying extracting rules from the tree ensemble model and implement the rules in hardware.
You can use TE2Rules (Tree Ensembles to Rules) to extract human understandable rules to explain a scikit tree ensemble (like GradientBoostingClassifier). It provides levers to control interpretability, fidelity and run time budget to extract useful explanations. Rules extracted by TE2Rules are guaranteed to closely approximate the tree ensemble, by considering the joint interactions of multiple trees in the ensemble.
References:
TE2Rules: You can find the code: https://github.com/linkedin/TE2Rules and documentation: https://te2rules.readthedocs.io/en/latest/ here.
Disclosure: I’m one of the core developers of TE2Rules.
I am using scikit-learn’s Random Forest Regressor to fit a random forest regressor on a dataset. Is it possible to interpret the output in a format where I can then implement the model fit without using scikit-learn or even Python?
The solution would need to be implemented in a microcontroller or maybe even an FPGA. I am doing analysis and learning in Python but want to implement on a uC or FPGA.
You can check out graphviz, which uses ‘dot language’ for storing models (which is quite human-readable if you’d want to build some custom interpreter, shouldn’t be hard). There is an export_graphviz function in scikit-learn. You can load and process the model in C++ through boost library read_graphviz method or some of other custom interpreters available.
It’s unclear what you mean by this part:
Now, that I have the results, is it possible to interpret this in some format where I can then implement the fit without using sklearn or even python?
Implement the fitting process for a given dataset? tree topology? choice of parameters?
As to ‘implement… without using sklearn or python’, did you mean ‘port the bytecode or binary’ or ‘clean-code a totally new implementation’?
Assuming you meant the latter, I’d suggest GPU rather than FPGA or uC.
You could trying extracting rules from the tree ensemble model and implement the rules in hardware.
You can use TE2Rules (Tree Ensembles to Rules) to extract human understandable rules to explain a scikit tree ensemble (like GradientBoostingClassifier). It provides levers to control interpretability, fidelity and run time budget to extract useful explanations. Rules extracted by TE2Rules are guaranteed to closely approximate the tree ensemble, by considering the joint interactions of multiple trees in the ensemble.
References:
TE2Rules: You can find the code: https://github.com/linkedin/TE2Rules and documentation: https://te2rules.readthedocs.io/en/latest/ here.
Disclosure: I’m one of the core developers of TE2Rules.