Conversation & Generation

​

ModelRunner

• Android (recommended): LeapModelDownloader.loadModel(...) / loadSimpleModel(...) — one-shot load that transparently routes through the optional Leap Model Service when installed, and adds WorkManager-backed background download staging on top.
• iOS / macOS (recommended): ModelDownloader.loadModel(...) / loadSimpleModel(...) — one-shot load that routes file transfers through URLSession. Pass sessionConfiguration: .background(withIdentifier:) for downloads that survive app suspension. (Class ships in the LeapModelDownloader SPM library product.)
• All platforms (iOS, Android, JVM, Linux native, Windows native, macOS Kotlin): LeapDownloader.loadModel(...) / loadSimpleModel(...) — the cross-platform manifest loader, with no platform-native background integration. Used directly on JVM/native and as the underlying loader inside both the iOS ModelDownloader and Android LeapModelDownloader.

public protocol ModelRunner {
  func createConversation(systemPrompt: String?) -> Conversation
  func createConversationFromHistory(history: [ChatMessage]) -> Conversation
  func unload() async
  func getPromptTokensSize(messages: [ChatMessage], addBosToken: Bool) async -> Int
  var modelId: String { get }
}

interface ModelRunner {
  val modelId: String
  fun createConversation(systemPrompt: String? = null): Conversation
  fun createConversationFromHistory(history: List<ChatMessage>): Conversation
  suspend fun unload()
  suspend fun getPromptTokensSize(messages: List<ChatMessage>, addBosToken: Boolean = true): Int
}

​

Lifecycle

• Use createConversation(systemPrompt:) for a fresh chat, or createConversationFromHistory(history:) to resume from persisted state.
• Call unload() when you’re done. On iOS this is async; on Kotlin it’s a suspend function — both release native memory.
• If the model runner is unloaded, any conversation it created becomes read-only.

​

Conversation

public protocol Conversation {
  var modelRunner: ModelRunner { get }
  var history: [ChatMessage] { get }
  var functions: [LeapFunction] { get }
  var isGenerating: Bool { get }

  func registerFunction(function: LeapFunction)
  func registerFunctions(functions: [LeapFunction])
  func appendToHistory(message: ChatMessage)
  func removeLastMessage()
  func exportToJSON() -> String

  func generateResponse(
    userTextMessage: String,
    generationOptions: GenerationOptions?
  ) -> SkieSwiftFlow<MessageResponse>

  func generateResponse(
    message: ChatMessage,
    generationOptions: GenerationOptions?
  ) -> SkieSwiftFlow<MessageResponse>
}

interface Conversation {
  val modelRunner: ModelRunner
  val history: List<ChatMessage>
  val functions: List<LeapFunction>
  val isGenerating: Boolean

  fun appendToHistory(message: ChatMessage)
  fun removeLastMessage()

  fun registerFunction(function: LeapFunction)
  fun registerFunctions(functions: List<LeapFunction>)

  fun generateResponse(
      userTextMessage: String,
      generationOptions: GenerationOptions? = null
  ): Flow<MessageResponse>

  fun generateResponse(
      message: ChatMessage,
      generationOptions: GenerationOptions? = null
  ): Flow<MessageResponse>
}

• appendToHistory(message) — record a message without triggering generation. Useful for replaying persisted state, or for inserting tool-result messages (role: .tool) after handling a function call.
• removeLastMessage() — pop the trailing message. No-op on an empty history. Useful when a generation was cancelled and you want to drop the dangling user turn.
• registerFunctions(functions) — bulk-register tool definitions; equivalent to looping over registerFunction(_:).

​

Properties

• history — a snapshot copy of the chat messages. Mutations don’t affect generation. Once the stream emits Complete, history includes the final assistant reply.
• isGenerating — true while a generation is in flight. Starting a second generation while one is running is blocked.
• functions — tool definitions the model may invoke. Registered through registerFunction(_:) / registerFunctions(_:) on both platforms.

​

Streaming generation

let user = ChatMessage(role: .user, textContent: "Hello! What can you do?")
let options = GenerationOptions()
  .with(temperature: 0.3)
  .with(minP: 0.15)
  .with(repetitionPenalty: 1.05)

Task {
  do {
    for try await response in conversation.generateResponse(
      message: user,
      generationOptions: options
    ) {
      switch onEnum(of: response) {
      case .chunk(let c):
        print(c.text, terminator: "")
      case .reasoningChunk(let r):
        print("Reasoning:", r.reasoning)
      case .functionCalls(let payload):
        handleFunctionCalls(payload.functionCalls)
      case .audioSample(let audio):
        // `audio.samples` is a `KotlinFloatArray` from Kotlin/Native — bridge to
        // `[Float]` via NSData if your renderer expects a Swift array:
        //   let nsData = LeapSDK.ArrayConversionsKt.floatArrayToNSData(array: audio.samples)
        //   let floats = nsData.withUnsafeBytes { Array($0.bindMemory(to: Float.self)) }
        audioRenderer.enqueue(audio.samples, sampleRate: Int(audio.sampleRate))
      case .complete(let completion):
        let text = completion.fullMessage.content.compactMap { part -> String? in
          if case let .text(t) = onEnum(of: part) { return t.text }
          return nil
        }.joined()
        print("\nComplete:", text)
        if let stats = completion.stats {
          print("Prompt tokens: \(stats.promptTokens), completion: \(stats.completionTokens)")
        }
      }
    }
  } catch {
    print("Generation failed: \(error)")
  }
}

class ChatViewModel(application: Application) : AndroidViewModel(application) {
    private var conversation: Conversation? = null
    private var modelRunner: ModelRunner? = null
    private var generationJob: Job? = null

    private val _generatedText = MutableStateFlow("")
    val generatedText: StateFlow<String> = _generatedText.asStateFlow()

    fun generateResponse(userInput: String) {
        generationJob = viewModelScope.launch {
            _generatedText.value = ""
            conversation?.generateResponse(userInput)
                ?.onEach { response ->
                    when (response) {
                        is MessageResponse.Chunk -> _generatedText.value += response.text
                        is MessageResponse.ReasoningChunk -> Log.d(TAG, "Reasoning: ${response.reasoning}")
                        is MessageResponse.FunctionCalls -> handleFunctionCalls(response.functionCalls)
                        is MessageResponse.AudioSample -> audioRenderer.enqueue(response.samples, response.sampleRate)
                        is MessageResponse.Complete -> Log.d(TAG, "Done. Stats: ${response.stats}")
                    }
                }
                ?.catch { e -> Log.e(TAG, "Generation failed", e) }
                ?.collect()
        }
    }

    fun stopGeneration() { generationJob?.cancel(); generationJob = null }

    override fun onCleared() {
        super.onCleared()
        generationJob?.cancel()
        runBlocking(Dispatchers.IO) { modelRunner?.unload() }
    }
}

​

Export chat history

let jsonString: String = conversation.exportToJSON()

import kotlinx.serialization.json.Json
import kotlinx.serialization.encodeToString

val jsonString = Json.encodeToString(conversation.history)

​

MessageResponse

public enum MessageResponse {
  case chunk(Chunk)                        // Chunk.text — partial assistant text
  case reasoningChunk(ReasoningChunk)      // ReasoningChunk.reasoning — thinking tokens
  case functionCalls(FunctionCalls)        // FunctionCalls.functionCalls — [LeapFunctionCall]
  case audioSample(AudioSample)            // AudioSample.samples, .sampleRate — PCM frames
  case complete(Complete)                  // Complete.fullMessage, .finishReason, .stats
}

sealed interface MessageResponse {
  data class Chunk(val text: String) : MessageResponse
  data class ReasoningChunk(val reasoning: String) : MessageResponse
  data class FunctionCalls(val functionCalls: List<LeapFunctionCall>) : MessageResponse
  data class AudioSample(val samples: FloatArray, val sampleRate: Int) : MessageResponse
  data class Complete(
    val fullMessage: ChatMessage,
    val finishReason: GenerationFinishReason,
    val stats: GenerationStats?,
  ) : MessageResponse
}

• Chunk — partial assistant text. Append to your UI buffer.
• ReasoningChunk — thinking-style tokens emitted by reasoning models (wrapped between <think> / </think> upstream). Only fires when GenerationOptions.enableThinking = true and the model supports it.
• FunctionCalls — one or more tool invocations the model wants you to execute. See Function Calling.
• AudioSample — float32 mono PCM frames from audio-capable checkpoints. The sample rate is constant for a generation; route the frames to a renderer.
• Complete — final marker. fullMessage is the assembled assistant ChatMessage (also present in conversation.history). stats is nullable (GenerationStats?); when present it holds promptTokens, completionTokens, totalTokens, tokenPerSecond (non-nullable Float), and cachedPromptTokens.

​

GenerationFinishReason

​

GenerationOptions

public class GenerationOptions {
  public var temperature: Float?
  public var topP: Float?
  public var minP: Float?
  public var repetitionPenalty: Float?
  public var topK: Int32?
  public var rngSeed: Int64?
  public var jsonSchemaConstraint: String?
  public var functionCallParser: LeapFunctionCallParser?
  public var injectSchemaIntoPrompt: Bool        // default true
  public var maxTokens: Int32?
  public var inlineThinkingTags: Bool            // default false
  public var enableThinking: Bool                // default false
  public var extras: String?

  public convenience init()                      // builder entry point

  // Builders (chainable):
  public func with(temperature: Float) -> GenerationOptions
  public func with(topP: Float) -> GenerationOptions
  public func with(minP: Float) -> GenerationOptions
  public func with(repetitionPenalty: Float) -> GenerationOptions
  public func with(topK: Int32) -> GenerationOptions
  public func with(rngSeed: Int64) -> GenerationOptions
  public func with(jsonSchema: String) -> GenerationOptions
  public func with(maxTokens: Int32) -> GenerationOptions
  public func with(injectSchemaIntoPrompt: Bool) -> GenerationOptions
  public func with(inlineThinkingTags: Bool) -> GenerationOptions
  public func with(enableThinking: Bool) -> GenerationOptions
}

let options = GenerationOptions()
    .with(temperature: 0.3)
    .with(minP: 0.15)
    .with(repetitionPenalty: 1.05)
    .with(maxTokens: 512)
    .with(jsonSchema: CityFact.jsonSchema())

data class GenerationOptions(
    var temperature: Float? = null,
    var topP: Float? = null,
    var minP: Float? = null,
    var repetitionPenalty: Float? = null,
    var topK: Int? = null,
    var rngSeed: Long? = null,
    var jsonSchemaConstraint: String? = null,
    var functionCallParser: LeapFunctionCallParser? = LFMFunctionCallParser(),
    var injectSchemaIntoPrompt: Boolean = true,
    var maxTokens: Int? = null,
    var inlineThinkingTags: Boolean = false,
    var enableThinking: Boolean = false,
    var extras: String? = null,
) {
  inline fun <reified T : Any> setResponseFormatType()

  companion object {
    fun build(buildAction: GenerationOptions.() -> Unit): GenerationOptions
  }
}

val options = GenerationOptions.build {
    temperature = 0.3f
    minP = 0.15f
    repetitionPenalty = 1.05f
    maxTokens = 512
    setResponseFormatType<CityFact>()
}

• Sampling fields (temperature, topP, minP, topK, repetitionPenalty) — standard sampling knobs. Use the values from the LEAP bundle manifest (sampling_parameters under generation_time_parameters in each model’s <Quant>.json on LiquidAI/LeapBundles); they’re tuned per checkpoint by the training team and differ from the HF model card defaults (the manifest values are the llama.cpp-engine path the SDK runs). Arbitrary “0.7” defaults from generic AI tutorials usually underperform.
• rngSeed — set for deterministic / reproducible output (testing, debugging). Default is non-deterministic.
• maxTokens — cap the response length. The model stops after this many completion tokens (prompt tokens don’t count). Defaults to “until EOS or context limit.” Useful for cost control with constrained output.
• jsonSchemaConstraint — JSON Schema string for constrained generation. Use the higher-level helpers — Swift options.with(jsonSchema: T.jsonSchema()) (or GenerationOptionsCompat.setResponseFormat(jsonSchema:)) / Kotlin setResponseFormatType<T>() — with @Generatable types. See Constrained Generation.
• injectSchemaIntoPrompt — when true (default), the schema is appended to the system message for semantic guidance in addition to the structural constraint at decode time. Set false to skip the prompt injection (matches llama-server grammar mode) — saves prompt tokens for large schemas.
• functionCallParser — picks the tokenizer expected by the model. LFMFunctionCallParser (default) for Liquid Foundation Models; HermesFunctionCallParser() for Hermes/Qwen3 formats; null to receive raw tool-call text in Chunks.
• enableThinking — turn on reasoning mode for models that support it (e.g. LFM2.5-Thinking). Reasoning tokens arrive as ReasoningChunks.
• inlineThinkingTags — when true, thinking tokens are emitted as ordinary Chunks with the literal <think>...</think> tags intact (instead of ReasoningChunk). ChatMessage.reasoningContent is still populated on the final message.
• extras — backend-specific JSON payload (internal use).

​

GenerationStats

promptTokens         Long    Prompt tokens computed (excludes tokens restored from KV cache).
completionTokens     Long    Tokens emitted during generation.
totalTokens          Long    promptTokens + completionTokens (excludes cached tokens).
tokenPerSecond       Float   Generation throughput (may be approximate on some backends).
cachedPromptTokens   Long    Prompt tokens restored from KV cache — not recomputed. 0 when the
                             cache is disabled or missed.