Skip to content

Add hybrid function calling sample - #2855

Open
VinayGuthal wants to merge 1 commit into
masterfrom
function_calling
Open

VinayGuthal wants to merge 1 commit into
masterfrom
function_calling

Conversation

@VinayGuthal

Copy link
Copy Markdown
Contributor

Description

Adds a new sample screen to test function calling tools across on-device, hybrid (PREFER_ON_DEVICE), and cloud inference modes. This sample complements the function calling support added in firebase-android-sdk #8758.

Features

  • Tool Configurations:
    • Manual (FunctionDeclaration): Defines fetchWeather(city, state, date), inspects returned FunctionCallPart entries, executes the local tool, and sends FunctionResponsePart to resolve the multi-turn exchange.
    • Automatic (AutoFunctionDeclaration): Validates automatic tool invocation handled directly by the SDK.
    • Unsupported Tool (Tool.googleSearch()): Demonstrates validation behavior and automatic cloud fallback when non-function tools are used with hybrid inference.
    • No Tools: Baseline generation.
  • Inference Mode & Method Flexibility:
    • Supports all inference modes (ONLY_ON_DEVICE, PREFER_ON_DEVICE, ONLY_IN_CLOUD).
    • Supports unary (generateContent) and streaming (generateContentStream) generation methods.
  • UI & Diagnostics:
    • Displays on-device model status and download progress.
    • Inspects function calls with argument payloads and call IDs.
    • Highlights inference source (ON_DEVICE vs IN_CLOUD).
    • Detailed Logcat and diagnostic summary card.

Summary of Changes

  • gradle/libs.versions.toml:
    • Bumped firebase-ai to 18.2.0 and firebase-ai-ondevice to 16.0.0-beta08.
  • firebase-ai/app:
    • Added FunctionCallingViewModel.kt, FunctionCallingUiState.kt, and FunctionCallingScreen.kt under com.google.firebase.quickstart.ai.
    • Registered FUNCTION_CALLING in ScreenType (Sample.kt).
    • Added "Hybrid Function Calling" sample under HYBRID and TOOLS_FC categories in FirebaseAISamples.kt.
    • Added screen dispatch in MainActivity.kt.

Testing

  • Built and verified with ./gradlew :app:assembleDebug
  • Verified code style formatting with ./gradlew ktlintCheck

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new Hybrid Function Calling feature, adding the necessary ViewModel, UI screen, state representation, and navigation routing to support testing function calling tools across on-device, hybrid, and cloud inference modes. The feedback highlights opportunities to improve robustness and consistency, specifically by using safe casting to prevent potential exceptions during JSON parsing, ensuring the streaming generation path handles automatic function calling consistently with the unary path, and avoiding redundant model downloads in the UI when the selected option has not changed.

Comment on lines +119 to +121
val city = input["city"]?.jsonPrimitive?.content
val state = input["state"]?.jsonPrimitive?.content
val date = input["date"]?.jsonPrimitive?.content

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using .jsonPrimitive on a JsonElement that is not a primitive (e.g., if the model unexpectedly returns a nested JSON object or array) will throw an IllegalArgumentException. To make the input parsing robust against unexpected model outputs, use safe casting with as? JsonPrimitive instead.

Suggested change
val city = input["city"]?.jsonPrimitive?.content
val state = input["state"]?.jsonPrimitive?.content
val date = input["date"]?.jsonPrimitive?.content
val city = (input["city"] as? JsonPrimitive)?.content
val state = (input["state"] as? JsonPrimitive)?.content
val date = (input["date"] as? JsonPrimitive)?.content

Comment on lines +285 to +302
if (toolOption == FunctionToolOption.MANUAL && detectedCalls.isNotEmpty()) {
val firstCall = detectedCalls.first()
val city = (firstCall.args["city"] as? JsonPrimitive)?.content ?: "Boston"
val state = (firstCall.args["state"] as? JsonPrimitive)?.content ?: "MA"
val date = (firstCall.args["date"] as? JsonPrimitive)?.content ?: "2024-10-17"
val weatherJson = WeatherRepository.fetchWeather(city, state, date)
executionResult = weatherJson.toString()

val followUp = model.generateContent(
content(role = "user") { text(promptText) },
content(role = "model") { part(firstCall) },
content(role = "user") {
part(FunctionResponsePart(firstCall.name, weatherJson, firstCall.id))
}
)
fullText = followUp.text.orEmpty()
source = followUp.inferenceSource
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In the STREAMING generation method, there is no handling for FunctionToolOption.AUTO to capture and set the executionResult when a function call is detected. This causes an inconsistency with the UNARY method where the execution result is captured and displayed in the UI. Adding an else if block for AUTO will resolve this.

                        if (toolOption == FunctionToolOption.MANUAL && detectedCalls.isNotEmpty()) {
                            val firstCall = detectedCalls.first()
                            val city = (firstCall.args["city"] as? JsonPrimitive)?.content ?: "Boston"
                            val state = (firstCall.args["state"] as? JsonPrimitive)?.content ?: "MA"
                            val date = (firstCall.args["date"] as? JsonPrimitive)?.content ?: "2024-10-17"
                            val weatherJson = WeatherRepository.fetchWeather(city, state, date)
                            executionResult = weatherJson.toString()

                            val followUp = model.generateContent(
                                content(role = "user") { text(promptText) },
                                content(role = "model") { part(firstCall) },
                                content(role = "user") {
                                    part(FunctionResponsePart(firstCall.name, weatherJson, firstCall.id))
                                }
                            )
                            fullText = followUp.text.orEmpty()
                            source = followUp.inferenceSource
                        } else if (toolOption == FunctionToolOption.AUTO && detectedCalls.isNotEmpty()) {
                            val firstCall = detectedCalls.first()
                            val city = (firstCall.args["city"] as? JsonPrimitive)?.content ?: "Boston"
                            val state = (firstCall.args["state"] as? JsonPrimitive)?.content ?: "MA"
                            val date = (firstCall.args["date"] as? JsonPrimitive)?.content ?: "2024-10-17"
                            executionResult = WeatherRepository.fetchWeather(city, state, date).toString()
                        }

Comment on lines +114 to +117
onClick = {
selectedModelOption = modelOpt
viewModel.checkAndDownloadModel(modelOpt.option)
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To avoid redundant network requests and job cancellations when the user taps an already selected model option, add a check to ensure checkAndDownloadModel is only called when the selection actually changes.

                    onClick = {
                        if (selectedModelOption != modelOpt) {
                            selectedModelOption = modelOpt
                            viewModel.checkAndDownloadModel(modelOpt.option)
                        }
                    },

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant