LLM Vision | Getting Started
  • Introduction
  • Installation
  • Setup
    • Providers
      • Cloud Providers
      • Self-hosted Providers
      • Memory
      • Timeline
    • Timeline Card
    • Blueprint
    • Asking about Events
  • Usage
    • Image Analyzer
    • Video Analyzer
    • Stream Analyzer
    • Data Analyzer
    • Remember
  • Choosing the right model
Powered by GitBook
On this page
  • Prerequisites
  • The Conversation Integration
  • Creating the Script
  • Troubleshooting

Was this helpful?

  1. Setup

Asking about Events

LLM Vision can remember events and store them so you can ask about them later.

PreviousBlueprintNextUsage

Last updated 1 month ago

Was this helpful?

Prerequisites

Prerequisites: You will need to set up first.

The Conversation Integration

To be able to ask about events in the timeline, you need to install and set up one of the integrations mentioned above.

Allow the Conversation Integration access to Assist

In the conversation integration settings you will need to set 'Control Home Assistant' to 'Assist'. This allows the integration to control devices exposed to assist.

Creating the Script

  1. Create a new script with the code below.

  2. Set up an assist pipeline in Home Assistant:

    1. 'Add Assistant'

    2. Give it a name and select your conversation integration as the 'Conversation Agent'

    3. (optional) If you want to be able to talk to your Assistant, add a TTS and STT configuration

script.get_camera_events
sequence:
  - variables:
      result:
        events: >-
          {% set titles = state_attr("calendar.llm_vision_timeline","events")%}
          {% set times = state_attr("calendar.llm_vision_timeline","starts") %}
          {% set summaries = state_attr("calendar.llm_vision_timeline","summaries") %}
          ```csv
          time, title, summary
          {%for i in range(titles|count)%}
          {%if times[i] > now() - timedelta(days=1)%}
          {{times[i]}}, '{{titles[i]}}', '{{summaries[i]}}'
          {%endif%}
          {%endfor%}
          ```
  - stop: stop
    response_variable: result
alias: Get Camera Events
description: Returns all events captured by cameras today.
icon: mdi:timeline
fields: {}

You can now ask Assist about events that happened today.

Troubleshooting

If Assist cannot answer questions about events, you may need to include some additional prompting in the system prompt. Instruct it to 'use the get_camera_events tool when asked about events'.

The easiest way to automatically add events to the timeline, is to use the . In your own automations, set remember to true. If you want to add custom events, use the .

This guide has been tested with , but should work with and as well.

Expose the script to assist. Follow to learn how to expose entities to assist.

Go to

official blueprint
remember action
OpenAI Conversation
Google Generative AI
Anthropic Conversation
this guide
Settings > Voice Assistants
LLM Vision Timeline
Asking Assist about remembered events