If memory serves, you only need to support around 4-5 function calls in order to load graphs and use them in sessions. The graph itself can be built in python, and the interface looks much like a tensored variant of Janes St.'s incremental library.

I'd definitely start by getting that part working first, and then add the remaining functions if need be. But I wouldn't make that a priority. If you do succeed however, I need a better language than Python for some TF-work :)

--