• demo 是跑在 playground中的。

语音助手demo

• 可以语音对话

概念

• Concepts
  Agent: A function that defines the workflow of a programmable, server-side participant. This is your application code.
  Worker: A container process responsible for managing job queuing with LiveKit server. Each worker is capable of running multiple agents simultaneously.
  Plugin: A library class that performs a specific task, like speech-to-text, from a specific provider. An agent can compose multiple plugins together to perform more complex tasks.

• agent ： 是一个函数，定义一个可编程的服务端的参与者，这个是app代码。

• worker：一个容器，负责处理管理任务，扔给livekit server 处理。每个worker都可以并发的运行多个agents。

• 插件：是一个库，可以执行某种特定任务，比如来自某个特定的人的语音到文本，

• 一个agent可以组合多个plugins 一起，处多个复杂任务。

启动worker

• 这个指令，恶意启动worker，并且等待用户连接到livekit服务：

python my_agent.py start

多才多艺的web前端： “playground”. 用于构建并测试一个agent

• Using playground for your agent UI
• To ease the process of building and testing an agent, we’ve developed a versatile web frontend called “playground”. You can use or modify this app to suit your specific requirements. It can also serve as a starting point for a completely custom agent application. 你可以使用或者修改这个app以满足你的特定的需求。
• 这个前端ui 工程可以作为一个完全定制的agent应用的起点。

Hosted playground

Source code

• 源码
  Playground docs

playground 前端 地址

https://agents-playground.livekit.io/

看起来是wss链接

需要打开音频和视频的权限

• 视频source 无法启动
• 

LiveKit Agents Playground

• 是一个设计为与agents框架一同构建的服务快速的原型化。
• 快速的切入到livekit的webrtc session 并处理或者产生音频、视频和数据流。 这个playground 包括了组件可以完全与livekit agent交互，通过音频。视频和聊天。
• The Agent Playground is designed for quickly prototyping with server side agents built with LiveKit Agents Framework. Easily tap into LiveKit WebRTC sessions and process or generate audio, video, and data streams. The playground includes components to fully interact with any LiveKit agent, through video, audio and chat.

Docs and references

• Docs for how to get started with LiveKit agents at https://docs.livekit.io/agents

• The repo containing the (server side) agent implementations (including example agents): https://github.com/livekit/agents

Try out a live version

• You can try out the agents playground at https://livekit-agent-playground.vercel.app. This will connect you to our example agent KITT.

• Setting up the playground locally

• Install dependencies

  npm install

• Copy and rename the .env.example file to .env.local and fill in the necessary environment variables.

LIVEKIT_API_KEY=<your API KEY>
LIVEKIT_API_SECRET=<Your API Secret>
NEXT_PUBLIC_LIVEKIT_URL=wss://<Your Cloud URL>

• Run the development server:

  npm run dev

• Open http://localhost:3000 with your browser to see the result.
• If you haven’t done so yet, start your agent (with the same project variables as in step 2.)
  - Connect to a room and see your agent connecting to the playground

Features

• 渲染来自于agent的视频、音频和对话
• Render video, audio and chat from your agent
• 发送音频、视频和文本到agent
• Send video, audio, or text to your agent
• 配置设置面板配置你的agent
• Configurable settings panel to work with your agent

Notes

• This playground is currently work in progress. There are known layout/responsive bugs and some features are under tested.
• The playground was tested against the kitt example in

https://github.com/livekit/agents.

• Feel free to ask questions, request features in our community slack.

Known issues

• Layout can break on smaller screens.
• Mobile device sizes not supported currently