Contact person: Business Client

Phone:Show

Email:Show

Location: Brisbane, Australia

Budget: Recommended by industry experts

Time to start: As soon as possible

Project description:
"Describe what you need

I’m building a system that can sense emotional signals in a live conversation — from audio, video and speech — and return a synchronized emotional stream for a weekly podcast.

I need one engineer who can build a real-time multimodal pipeline from scratch. The role is hands-on: prototype fast, ship weekly improvements, and make it work end-to-end.

This is inference only, not model training.

The System (High-Level)

The pipeline will:

Capture 2 video feeds from cameras

extract facial/body emotional signals

timestamp frames

Capture audio input from a dual mic receiver

run emotion model

track tone/tension/stress cues

timestamp stream

Run Whisper (or similar) in real time

speech-to-text

confidence scores

timestamped text segments

Synchronize all streams

align video/audio/text

output structured JSON

Send JSON to a conversational AI (ChatGPT or other LLM)

via local API or device connection

in real time or near real time

Display emotional timeline

simple UI (web or local)

clean visual for podcast use

The goal is not a polished product — it’s a working emotional layer for a conversation.

What You’ll Build

real-time capture layer

video emotion inference

audio emotion inference

Whisper transcription loop

timestamp + sync logic

JSON output schema

lightweight visualization

You don’t need to design the UI — just make something clear and usable." (client-provided description)