It supports two workflows:

Streamlit UI (app.py) for interactive runs and analysis

Headless runner (headless_runner.py) for scheduled and automated jobs

The goal is to run the same prompt across multiple LLMs and compare output in a normalized format with long-term storage and history.

Tech stack

• Python 3.10+
• Streamlit
• Bright Data Scraping APIs
• Supabase

Prerequisites

Python 3.10+ recommended

A Bright Data account with Web Scraper API access

A Supabase account and project

Setup

1) Create and activate a virtual environment

Linux / macOS:

python -m venv .venv
source .venv/bin/activate

Windows (PowerShell):

python -m venv .venv
..venvScriptsActivate.ps1

2) Install dependencies

pip install -r requirements.txt

3) Configure environment variables

Create a .env file in the project root:

BRIGHTDATA_API_TOKEN=your_brightdata_api_token_here

SUPABASE_URL=your_supabase_project_url
SUPABASE_API_TOKEN=your_supabase_api_key

These credentials are required for database persistence and scheduling.

4) Creating the database

Create the llm_runs table within Supabase.

create table public.llm_runs (
 id bigint generated by default as identity primary key,
 created_at_ts bigint not null, -- unix seconds
 model_name text not null,
 prompt text not null,
 country text null,
 target_phrase text null,
 mentioned boolean not null default false,
 payload jsonb not null
);

create index if not exists llm_runs_created_at_ts_idx
 on public.llm_runs (created_at_ts);

create index if not exists llm_runs_model_idx
 on public.llm_runs (model_name);

create index if not exists llm_runs_target_idx
 on public.llm_runs (target_phrase);

Create the prompts table.

create table public.prompts (
 id bigint generated by default as identity primary key,
 created_at_ts bigint not null,
 prompt text not null,
 is_active boolean not null default true
);

create index if not exists prompts_created_at_ts_idx
 on public.prompts (created_at_ts desc);

create index if not exists prompts_active_idx
 on public.prompts (is_active);

Create the schedules table.

create table public.schedules (
 id bigint generated by default as identity primary key,

 name text not null,
 is_enabled boolean not null default true,

 next_run_ts bigint not null,
 last_run_ts bigint null,

 models jsonb not null default '[]'::jsonb,

 country text null,
 target_phrase text null,
 only_active_prompts boolean not null default true,

 locked_until_ts bigint null,
 lock_owner text null,

 repeat_every_seconds bigint not null default 86400
);

create index if not exists schedules_due_idx
 on public.schedules (is_enabled, next_run_ts);

create index if not exists schedules_lock_idx
 on public.schedules (locked_until_ts);

Starting the application

Start the headless runner

Open a terminal and run:

python headless_runner.py

This process handles scheduled jobs and database persistence.

Launch the Streamlit UI

Open a second terminal and run:

streamlit run app.py

The UI is used for interactive runs, reporting, and schedule management.

Notes

• Uses snapshot-based polling via Bright Data

• Results are normalized by Bright Data

• All runs are persisted in Supabase

• Failures are isolated per model

• The UI and headless runner are designed to run in parallel