Colin Joy Lobo

Available for work

Colin Joy
Lobo.

AI Engineer

I build the agentic systems behind generative film: multi-agent pipelines on Azure AI Foundry that turn a script into finished frames, the eval harnesses that catch their mistakes, and the inference that runs the whole thing. It's in production right now on streaming series and upcoming features.

View work Get in touch
6
Production agents
4
Eval levels
50+
Artists supported
26M+
Series views
Frame from Mahabharat: Ek Dharmayudh
Frame from Mahabharat: Ek Dharmayudh
Frame from Mahabharat: Ek Dharmayudh
Frame from Mahabharat: Ek Dharmayudh
01 Mahabharat Ek Dharmayudh · JioHotstar
Frame from Khana Dil Se
Frame from Khana Dil Se
Frame from Khana Dil Se
Frame from Khana Dil Se
02 Khana Dil Se Hansal Mehta · YouTube
Fig.0 Real broadcast frames from the two AI series I work on.
Services

What I do.

Agentic pipelines

Multi-agent orchestration with automated quality checks, carrying a script through to renderable shots.

Generative video

Bulk AI video from stills to a finished cut, with multi-character lip-sync and consistent character models.

3D & VFX

Concept art to explorable 3D worlds, plus segmentation, matting and clean-plate for VFX teams.

AI infrastructure

Serverless GPU serving on RunPod and Azure: Docker, REST APIs, autoscaling and model-swapping under memory limits.

Projects

Productions, tools & systems.

A In production
Watch trailer ↗
01OTT production

Mahabharat: Ek Dharmayudh — JioHotstar

Built the multi-agent shot pipeline and quality checks for India's first AI-generated premium series. It has since crossed 26.5M views.

JioHotstarHistoryverseLipSyncCarve
A generated frame from the Khana Dil Se cooking show
Watch on YouTube ↗
02Food show

Khana Dil Se — Hansal Mehta

Generated the video for Hansal Mehta's AI cooking show on YouTube, turning each episode's script into finished shots.

Hansal MehtaYouTubeBerniniSeedance 2.0
Watch teaser ↗
03Film VFX

Chiranjeevi Hanuman — The Eternal

Storyboards and an image-to-3D pipeline for an AI-generated theatrical feature, releasing in 2026.

Image-to-3DStoryboard agentCueBlender
A cosmic concept frame from the Krishna feature
04Film

Krishna — feature film

Storyboard generation and style-transfer for an upcoming AI feature, keeping one consistent look across the film.

HistoryverseCueStyle transfergpt-image-2
Instagram ↗
05Celebrity tech

Mishty — Rhea Chakraborty's AI avatar

A photoreal, interactive AI avatar of Rhea Chakraborty, built on Galleri5's studio pipeline with motion capture, voice and lip-sync.

MishtyG5 AI StudioWanimateLipSync
B Tools & systems
01Multi-agent

Six-agent creative pipeline

Six AI agents take a script to finished shot panels, with a four-level automated review that catches a bad shot before it moves on.

Azure AI FoundryMCPRAGLLM-as-judgeSonnet 4.6
ScriptWriterShotBreakdownAssetVisualBreakdownBGPlateVideoComposer
4-level LLM-as-judge gates every hand-off ↑
Six agents in sequence; an eval harness gates each stage before the next runs.
02VFX tooling

Carve — segmentation, matting & clean-plate

A browser tool for VFX teams: segment objects, matte hair and fur, and remove anything from a shot. Built on SAM and ViTMatte.

FastAPISAM 2/3/3.1ViTMatteProPainterrepo ↗
Carve UI — auto multi-ID segmentation Auto multi-ID
Open repo ↗
03Model training

Clean-license matte model

Retrained the matting model on a clean-licensed dataset so it can ship to clients. Sharper on soft hair and fur than the original.

PyTorchViTMatte-SBiRefNetCOCOP3M-10kCorridorKey
Matte comparison — input, ground truth, research baseline and my v2.1 on a flyaway-hair portrait and a cat, with SAD scores
Held-out frames: input, ground truth, research baseline, my v2.1. Lower SAD is better (9.86 vs 14.61); note the flyaway hair and the fur along the cat's back.
View ↗
043D worlds

Vantage — panorama to 3D world

Turns two photos into an explorable 3D world (mesh, panorama, Gaussian splat), then re-renders any angle as a high-res still.

Node.jsfal.aiWorld Labs MarbleGaussian splatsgpt-image-2
An explorable generated world, fly-through.
05Desktop / hardware

MultiCam Control

A desktop app that controls a fleet of GoPro cameras at once: discovery, live preview, synced recording, and one-click download of every card.

ElectronReactFastAPIBLE · COHN
GoPro Control Center — camera fleet Camera fleet
06Storyboard app

Cue — storyboard generation app

Turns a screenplay into a shot-by-shot storyboard and keeps every character on-model from reference images. Exports to PDF, runs without a GPU.

Gemini 2.5gpt-image-2shot breakdowncharacter refsCPU-only
S2 · E2 · EXT. FOREST — THE MOONLIT LAKE — NIGHT

"Beyond the treeline, the forest opens onto a moonlit lake fed by a crashing waterfall. Through the silver mist, a woman stands drenched beneath the falls, singing — and Bhim, mace in hand, draws back into the shadows."

Approved Bhim character reference sheet
@bhim
Approved Hidimba character reference sheet
@hidimba
Cue breaks the scene into shots — and every panel below holds these two on-model →
Scene storyboard — 33 shots, all generatedB&W · 16:9
Storyboard SHOT 30 — moonlit lake and waterfall, Bhim at the shoreSHOT 30XWS
@bhim
Storyboard SHOT 50 — close-up of Hidimba beneath the fallsSHOT 50CU
@hidimba
Storyboard SHOT 60 — Bhim, wary, behind a treeSHOT 60MS
@bhim
Storyboard SHOT 90 — Hidimba, closerSHOT 90CU
@hidimba
Storyboard SHOT 130 — close on BhimSHOT 130CU
@bhim
Storyboard SHOT 150 — Bhim and Hidimba together by the lakeSHOT 150MS
@bhim @hidimba
Storyboard SHOT 170 — Hidimba, moonlitSHOT 170CU
@hidimba
Storyboard SHOT 300 — Bhim and Hidimba by the fallsSHOT 300MS
@bhim @hidimba

Real panels from one scene Cue storyboarded end to end: all 33 shots from the screenplay. Working from just the two reference sheets above, it holds Bhim and Hidimba on-model across every panel, whatever the framing.

Skills

Technical expertise.

Agentic AI

  • Multi-agent orchestration
  • Azure AI Foundry
  • MCP / tool calling
  • RAG · vector stores
  • LLM-as-judge evals
  • Typed agent handoff

Generative media

  • Seedance 2.0
  • Bernini
  • Flux · SDXL
  • Wan · Qwen
  • LoRA training
  • gpt-image-2 · ComfyUI

Computer vision / VFX

  • SAM 2 / 3 / 3.1
  • ViTMatte · BiRefNet
  • Segmentation
  • Soft-alpha matting
  • ProPainter clean-plate
  • EXR / ACES colour

Inference & infra

  • FastAPI serving
  • Model-swap under memory
  • systemd services
  • Cloudflare tunnels
  • Async job queues
  • Redis · PostgreSQL

3D & pipelines

  • World Labs Marble
  • Image-to-3D
  • Gaussian splats
  • Blender Python API
  • Retopo & texture bake
  • Scene generation

Backend & languages

  • Python
  • FastAPI · Node.js
  • React · Electron
  • Pipeline engineering
  • REST API design
  • Automation

Contact

Let's work together.

Available for hard, applied AI roles in generative media and production infrastructure. Fastest reply by email.

✉ Send email in · LinkedIn

@colinjoylobo Résumé (PDF)