Problems with LiteRT-LM v0.9.0-beta
Hello!
I fail to use your model with v0.9.0-beta. I call it from the Kotlin official bindings and I get the following error from the native runtime:
Failed to apply template: unknown function: format_function_declaration is unknown (in template:21)
I am no expert so I naively believe that the jinja template embedded in the litertml file is invalid.
Am I right?
If that's indeed the problem, unfortunatelly I don't have the possibility to override the template.
Thanks in advance for your help.
👋 dvillevalois ,i just fixxed the format_function_declaration template error you were hitting.
the stock Gemma 3n chat template calls format_function_declaration(), a custom Jinja function that exists in Google's Python tokenizer, but isn't supported by LiteRT-LM's on device template engine. So when your app tries to register tools, the template render fails.
i fine-tuned the model with a custom chat template that replaces format_function_declaration() with standard Jinja2 (tojson filter + / markers). Same function-calling behavior, fully compatible with LiteRT-LM's native engine.
tested fix in android (SDK 36, JDK 21, LiteRT-LM v0.9.0-alpha01) that loads the model, registers tools, and makes a function call. pushed the updated model.
Hey @macmacmacmac ! Thanks a lot! That works. Tested on Linux/JVM with LiteRT-LM 0.9.0-beta. Thanks again.