BF16 · Transformers / SGLang
Use Transformers for the Python examples below. The official card also lists SGLang for serving text responses.
Liquid AI · Multimodal model guide
LFM3-Omni: see, listen and respond
A compact model for text, images, video and audio input, with text and speech responses. Explore the specifications, choose a download and get started with official examples.
Independent guide. The playground is hosted by Liquid AI and may require sign-in. Benchmark numbers below are reported by the developer; this site has not run inference tests.
Information checked
LFM3-Omni-2B specifications
| Property | Specification |
|---|---|
| Total parameters | 2.17B |
| Context window | 32K tokens, shared by input and output |
| Inputs | Text, images, video and audio (16 kHz) |
| Outputs | Text, speech (24 kHz), or interleaved text and speech |
| Default video sampling | 2 fps; 4–384 frames per video |
| Language support | 16 text languages; 19 speech languages/locales, including Chinese and English |
| License | LFM Open License v1.0 |
This is a non-reasoning model. It does not generate images or video, and enable_thinking=True is unsupported. A small parameter count does not establish a minimum RAM requirement or guarantee real-time performance on every device.
What can you do with it?
- 01
Ask about an image by voice
Combine an image with a spoken question, then request a text answer or spoken response.
- 02
Understand a video with sound
Ask about visible actions and audible events together. Loading the soundtrack requires the explicit settings in the tutorial.
- 03
Transcribe and translate speech
Turn recordings into text or translate speech. Check accuracy with your own languages, accents and background noise.
- 04
Build a voice conversation
Request speech or interleaved text and audio. Applications still need recording, playback and conversation handling.
Choose your download
Use the official repositories and match the weights to your runtime. Format support alone does not mean every runtime exposes all four input modalities.
GGUF · llama.cpp
Choose quantized weights for local CPU/GPU inference. Follow Liquid AI’s llama.cpp instructions for the required model assets and supported features.
Weights are available under LFM Open License v1.0. Commercial use has conditions; read the official license before using or redistributing the model.
Run your first image question
Adapted from the official model card. These examples are provided for local use and have not been executed by this site. Use a separate Python environment; no model runs in this page.
1. Install Python dependencies
python -m pip install "transformers>=5.20.2" torch torchvision torchcodec accelerate pillow librosa soundfile
2. Save as image_qa.py
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor
checkpoint = "LiquidAI/LFM3-Omni-2B"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32
model = AutoModelForMultimodalLM.from_pretrained(
checkpoint, dtype=dtype, device_map=device, trust_remote_code=True
).eval()
processor = AutoProcessor.from_pretrained(checkpoint, trust_remote_code=True)
messages = [{"role": "user", "content": [
{"type": "image", "image": "photo.jpg"},
{"type": "text", "text": "Describe only what is visible in this image."},
]}]
inputs = processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True,
return_dict=True, return_tensors="pt",
).to(device)
with torch.no_grad():
result = model.generate(**inputs, max_new_tokens=256, generation_mode="text")
start = inputs["input_ids"].shape[1]
end = result.sequence_lengths[0]
print(processor.decode(result.sequences[0, start:end], skip_special_tokens=True))3. Run the example
python image_qa.py
Place photo.jpg beside the script, then run python image_qa.py. The first run downloads weights. CUDA is used when available; otherwise the example uses CPU float32, which can be slow and needs more memory. trust_remote_code=True loads the official repository’s custom implementation; review it before running. For installation or media-decoding errors, consult the linked runtime documentation.
Optional: include a video’s soundtrack
# Reuse model, processor and device from the image example.
messages = [{"role": "user", "content": [
{"type": "video", "video": "clip.mp4"},
{"type": "text", "text": "Describe the visible events and audible sounds."},
]}]
inputs = processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True,
return_dict=True, return_tensors="pt",
load_audio_from_video=True,
processor_kwargs={"use_audio_in_video": True},
).to(device)
with torch.no_grad():
result = model.generate(**inputs, max_new_tokens=256, generation_mode="text")
start = inputs["input_ids"].shape[1]
end = result.sequence_lengths[0]
print(processor.decode(result.sequences[0, start:end], skip_special_tokens=True))Reuse the imports and loaded model from the image example and replace the conversation and generation code with this block. Provide clip.mp4 with an audio track. Both load_audio_from_video=True and use_audio_in_video=True are needed; without them, only frames are passed. Longer media consumes more context and memory.
Official benchmarks, with context
Developer-reported results, not an independent review. The table reproduces three tasks from the model card; higher is better (↑) for all three. Checked 2026-10-30.
| Benchmark ↑ | LFM3-Omni-2B | Qwen2.5-Omni-3B | Qwen2.5-Omni-7B | Gemma 4 E2B | Gemma 4 E4B | Phi-4 multimodal |
|---|---|---|---|---|---|---|
| Daily-Omni ↑ | 70.09 | 58.81 | 61.32 | 50.63 | 54.14 | 48.87 |
| WorldSense ↑ | 47.10 | 44.99 | 45.37 | 38.11 | 43.51 | 42.37 |
| OmniBench (overall) ↑ | 46.32 | 47.11 | 49.65 | 16.81 | 20.67 | 38.97 |
LFM3-Omni leads the six listed models on Daily-Omni and WorldSense, both video-and-audio tasks. On OmniBench overall, an image-and-audio task, its 46.32 trails Qwen2.5-Omni-3B (47.11) and 7B (49.65). This supports testing it for compact audiovisual assistants, not a claim of universal superiority.
The names “3B”, “7B”, “E2B” and “E4B” are model labels, not directly comparable total parameter counts. These scores do not measure latency, memory use, or your application’s accuracy. Evaluate representative media on your target hardware before choosing a model.
Frequently asked questions
Official sources
Specifications, examples and scores can change. Use these first-party references for current details.