How can I test a .tflite model to prove that it behaves as the original model using the same Test Data?
Question:
I have generated a .tflite model based on a trained model, I would like to test that the tfilte model gives the same results as the original model.
Giving both the same test data and obtaining the same result.
Answers:
There is a tflite_diff_example_test in the TensorFlow code base. It generates random data and feed the same data into TensorFlow & TensorFlow lite, then compare if the difference is within a small threshold.
You can to checkout TensorFlow code from Github, and run it with bazel:
bazel run //tensorflow/contrib/lite/testing:tflite_diff_example_test
then you’ll see what arguments you need to pass.
In addition to the answer given by @miaout17, to debug / understand your tflite model (which is the spirit of the question), you can
- use
--dump_graphviz to visualize the graph as explained here https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/toco/g3doc/cmdline_examples.md#using—dump_graphviz
- use
flatc to generate a python api and then parse the model via that api
https://google.github.io/flatbuffers/flatbuffers_guide_use_python.html
- generate
json from the tflite file using flatc and print it out
- Use the tflite visualizer:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/tools/visualize.py (currently has a bug)
You may use TensorFlow Lite Python interpreter to test your tflite model.
It allows you to feed input data in python shell and read the output directly like you are just using a normal tensorflow model.
I have answered this question here.
And you can read this TensorFlow lite official guide for detailed information.
You can also use Netron to visualize your model. It allows you to load your .tflite file directly and inspect your model architecture and model weights.
I have generated a .tflite model based on a trained model, I would like to test that the tfilte model gives the same results as the original model.
Giving both the same test data and obtaining the same result.
There is a tflite_diff_example_test in the TensorFlow code base. It generates random data and feed the same data into TensorFlow & TensorFlow lite, then compare if the difference is within a small threshold.
You can to checkout TensorFlow code from Github, and run it with bazel:
bazel run //tensorflow/contrib/lite/testing:tflite_diff_example_test
then you’ll see what arguments you need to pass.
In addition to the answer given by @miaout17, to debug / understand your tflite model (which is the spirit of the question), you can
- use
--dump_graphvizto visualize the graph as explained here https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/toco/g3doc/cmdline_examples.md#using—dump_graphviz - use
flatcto generate a python api and then parse the model via that api
https://google.github.io/flatbuffers/flatbuffers_guide_use_python.html - generate
jsonfrom thetflitefile usingflatcand print it out - Use the tflite visualizer:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/tools/visualize.py (currently has a bug)
You may use TensorFlow Lite Python interpreter to test your tflite model.
It allows you to feed input data in python shell and read the output directly like you are just using a normal tensorflow model.
I have answered this question here.
And you can read this TensorFlow lite official guide for detailed information.
You can also use Netron to visualize your model. It allows you to load your .tflite file directly and inspect your model architecture and model weights.