# GenAI-DeepDetect: Final Implementation PRD (v2) ## Persistent Agent Memory Directive Obsidian is the preferred cross-session context store. At the start of every agent session, use Obsidian MCP to read the GenAI-DeepDetect context notes. At the end of every agent session, use Obsidian MCP to update module status, blockers, and session log. If Obsidian MCP is not connected, do not invent a successful MCP write. Use the local fallback mirror at `Obsidian/GenAI-DeepDetect/`, clearly record that MCP was unavailable, and sync those notes through Obsidian MCP as soon as the server is connected. **Deadline: Tonight, 12:00 AM** **Deploy to: HuggingFace Spaces (Gradio) — ZeroGPU tier** **Hardware: A10G (40GB VRAM), on-demand via `@spaces.GPU`** **LLM: NVIDIA NIM free API (Llama-3.1-8B-Instruct)** **Everything else: HuggingFace pretrained models** **Only training needed: Module 3 (SSTGNN) on L40S (~5 hrs, ~$6)** **Context Store: Notion (for cross-agent context handoff)** hugging face agent : curl https://huggingface.co/spaces/akagtag/deepdetection/agents.md --- ## ZeroGPU: What Changes ZeroGPU allocates an A10G only during a `@spaces.GPU`-decorated function call. GPU is **not** available at startup. This means: - All models load on **CPU** at module init (startup) - `@spaces.GPU` is applied to the `analyze()` function in `app.py` - Inside that context, `.to("cuda")` works, CUDA is live - After the function returns, GPU is released — no persistent GPU state - **You can drop the fallback module entirely** — A10G has 40GB, all real models fit Space `README.md` header must declare `hardware: zero-gpu` (see below). > **No fallback module needed.** With 40GB VRAM, M1+M2+M3+CLIP all load > comfortably. Keep `m3_fallback.py` as a file but never import it in `app.py`. --- ## Notion: Cross-Agent Context Store > Obsidian MCP is not in the currently connected servers. Notion is connected > and serves the same purpose. All context, decisions, and state are written to > and read from a Notion database at the start of each agent session. ### One-time Notion Setup Create a Notion database called **GenAI-DeepDetect Context** with these properties: - `Title` (title field) - `Module` (select: M1, M2, M3, M5-fusion, M5-llm, infra, global) - `Status` (select: pending, in-progress, done, blocked) - `Notes` (text) - `LastUpdated` (date) ### Agent Handoff Protocol At the **start** of every Claude Code session (or agent switch), load context: ```bash # Prompt to use at the start of any agent session: "Read the GenAI-DeepDetect Context Notion database and summarize current status per module before we begin." ``` At the **end** of every session, write context back: ```bash # Prompt at end of session: "Update the GenAI-DeepDetect Context Notion database with what we completed today, what's blocked, and what the next agent should pick up first." ``` This replaces ad-hoc status tracking and makes every agent session stateful. --- ## Space README.md (Required for ZeroGPU) ```yaml --- title: GenAI-DeepDetect emoji: 🔍 colorFrom: red colorTo: gray sdk: gradio sdk_version: '4.44.0' app_file: app.py pinned: true hardware: zero-gpu license: mit --- ``` Without `hardware: zero-gpu`, `@spaces.GPU` will silently fall back to CPU. You must be on HF Pro and have ZeroGPU access enabled on your account. --- ## What You Are Building A Gradio app on HuggingFace Spaces (ZeroGPU) that takes a video, runs 4 detection modules on an A10G, fuses scores, calls NVIDIA NIM for a natural-language explanation, and returns: 1. **FakeScore** (0-1, higher = more likely fake) 2. **Per-module scores** (lip-sync, fingerprint, graph-GNN) 3. **Generator attribution** (which AI tool made this) 4. **Natural-language explanation** (from Llama via NVIDIA NIM) --- ## Module Source Map | Module | What | Source | Weights | Training? | | --------- | ----------------------------- | --------------------------------------- | ------------------------------------------- | ------------- | | M1 | Lip-sync detection | `github.com/AaronComo/LipFD` | Official `ckpt.pth` from their Google Drive | NO | | M2 | Deepfake binary + attribution | `yermandy/deepfake-detection` on HF | Auto-downloads via transformers | NO | | M3 | Graph spatio-temporal GNN | arXiv:2508.05526 (implement yourself) | Train on L40S, push to HF Hub | YES (~5 hrs) | | M5-fusion | Score aggregation | 3-input attention MLP | Train on CPU in 5 minutes | YES (trivial) | | M5-llm | Explanation generation | NVIDIA NIM `meta/llama-3.1-8b-instruct` | API call, no weights needed | NO | --- ## File Structure ``` GenAI-DeepDetect/ ├── README.md # HF Space model card (with hardware: zero-gpu) ├── app.py # Gradio UI entry point ├── requirements.txt ├── packages.txt # system deps: ffmpeg, libsndfile1 ├── .env.example # NVIDIA_API_KEY placeholder │ ├── modules/ │ ├── __init__.py │ ├── m1_lipsync.py # LipFD pretrained wrapper │ ├── m2_fingerprint.py # CLIP deepfake detector wrapper │ ├── m3_sstgnn.py # SSTGNN inference (your trained model) │ ├── m3_fallback.py # ViT fallback — kept but never imported in prod │ ├── sstgnn_model.py # SSTGNN architecture definition │ ├── m5_fusion.py # Attention MLP │ └── m5_explain.py # NVIDIA NIM Llama API caller │ ├── utils/ │ ├── video.py # Frame/audio extraction with ffmpeg │ └── graph.py # Spatial-patch graph builder for M3 │ ├── weights/ │ └── fusion_mlp.pt # Tiny MLP (~12KB), committed to repo │ ├── test_assets/ │ ├── real_sample.mp4 │ └── fake_sample.mp4 │ └── lipfd/ # Copied model files from LipFD repo └── model.py ``` --- ## requirements.txt ``` spaces>=0.28.0 torch>=2.1.0 torchvision>=0.16.0 torchaudio>=2.1.0 torch-geometric>=2.4.0 transformers>=4.36.0 gradio>=4.44.0 opencv-python-headless>=4.8.0 librosa>=0.10.0 numpy>=1.24.0 Pillow>=10.0.0 openai>=1.0.0 huggingface-hub>=0.19.0 soundfile>=0.12.0 ``` `spaces` is the HuggingFace library that provides the `@spaces.GPU` decorator. ## packages.txt ``` ffmpeg libsndfile1-dev ``` --- ## ZeroGPU Module Pattern All modules follow this exact pattern: ```python # CORRECT: load on CPU at init, use GPU inside @spaces.GPU class SomeModule: def __init__(self, cache_dir="/data/model_cache"): # Always CPU at startup — GPU not allocated yet self.device = "cpu" self.model = load_model().to("cpu") def to_gpu(self): """Called inside @spaces.GPU context.""" self.device = "cuda" self.model = self.model.to("cuda") def to_cpu(self): """Optional: called after inference to free GPU memory.""" self.device = "cpu" self.model = self.model.to("cpu") ``` The `analyze()` function in `app.py` calls `to_gpu()` on each module at the start of the GPU context and optionally `to_cpu()` at the end (not strictly needed since the GPU is released anyway when the decorated function returns). --- ## Module 1: Lip-Sync (LipFD Pretrained) ### modules/m1_lipsync.py ```python import torch import cv2 import librosa import numpy as np from huggingface_hub import hf_hub_download class LipSyncModule: """ LipFD pretrained lip-sync deepfake detector. Source: github.com/AaronComo/LipFD (NeurIPS 2024) Output: score in [0,1], higher = more likely fake """ def __init__(self, cache_dir="/data/model_cache"): self.device = "cpu" self.cache_dir = cache_dir self._load_model() def _load_model(self): ckpt_path = hf_hub_download( repo_id="AkshatAgarwal/LipFD-checkpoint", filename="ckpt.pth", cache_dir=self.cache_dir ) from lipfd.model import LipFDNet self.model = LipFDNet() state_dict = torch.load(ckpt_path, map_location="cpu") self.model.load_state_dict(state_dict) self.model.eval() def to_gpu(self): self.device = "cuda" self.model = self.model.to("cuda") def to_cpu(self): self.device = "cpu" self.model = self.model.to("cpu") @torch.no_grad() def score(self, video_path: str) -> dict: frames, audio, fps = self._preprocess(video_path) if frames is None or audio is None: return {"s1": 0.5, "segments": [], "note": "no_face_or_audio"} frames_t = torch.tensor(frames, dtype=torch.float32).to(self.device) audio_t = torch.tensor(audio, dtype=torch.float32).to(self.device) logits = self.model(frames_t, audio_t) score = torch.sigmoid(logits).mean().item() return {"s1": score, "segments": self._get_segments(logits, fps)} def _preprocess(self, video_path: str): cap = cv2.VideoCapture(video_path) fps = cap.get(cv2.CAP_PROP_FPS) frames = [] while cap.isOpened(): ret, frame = cap.read() if not ret: break lip_crop = self._extract_lip_region(frame) if lip_crop is not None: lip_crop = cv2.resize(lip_crop, (96, 96)) frames.append(lip_crop) cap.release() if len(frames) < 5: return None, None, fps audio, sr = librosa.load(video_path, sr=16000) mel = librosa.feature.melspectrogram(y=audio, sr=sr) frames_arr = np.array(frames).transpose(0, 3, 1, 2) / 255.0 return frames_arr, mel, fps def _extract_lip_region(self, frame): face_cascade = cv2.CascadeClassifier( cv2.data.haarcascades + "haarcascade_frontalface_default.xml" ) gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) faces = face_cascade.detectMultiScale(gray, 1.3, 5) if len(faces) == 0: return None x, y, w, h = faces[0] lip_y = y + int(h * 0.65) lip_h = int(h * 0.35) lip_x = x + int(w * 0.2) lip_w = int(w * 0.6) return frame[lip_y:lip_y+lip_h, lip_x:lip_x+lip_w] def _get_segments(self, logits, fps): scores = torch.sigmoid(logits).cpu().numpy() return [ {"time": round(i / fps, 2), "score": round(float(s), 3)} for i, s in enumerate(scores) if s > 0.6 ] ``` --- ## Module 2: Style Fingerprinting (CLIP Pretrained) ### modules/m2_fingerprint.py ```python import torch import cv2 import numpy as np from transformers import ( AutoModelForImageClassification, AutoProcessor, CLIPModel, CLIPTokenizer, CLIPProcessor ) from PIL import Image GENERATORS = [ "Sora", "Runway Gen-2", "Wav2Lip", "Stable Diffusion v1.5", "SDXL", "Midjourney v6", "DALL-E 3", "Unknown/OOD" ] class FingerprintModule: def __init__(self, cache_dir="/data/model_cache"): self.device = "cpu" self.model = AutoModelForImageClassification.from_pretrained( "yermandy/deepfake-detection", cache_dir=cache_dir ) self.processor = AutoProcessor.from_pretrained( "yermandy/deepfake-detection", cache_dir=cache_dir ) self.model.eval() self.clip = CLIPModel.from_pretrained( "openai/clip-vit-large-patch14", cache_dir=cache_dir ) self.clip_tok = CLIPTokenizer.from_pretrained( "openai/clip-vit-large-patch14", cache_dir=cache_dir ) self.clip_proc = CLIPProcessor.from_pretrained( "openai/clip-vit-large-patch14", cache_dir=cache_dir ) self.clip.eval() self._precompute_generator_embeddings() def to_gpu(self): self.device = "cuda" self.model = self.model.to("cuda") self.clip = self.clip.to("cuda") self.gen_embeds = self.gen_embeds.to("cuda") def to_cpu(self): self.device = "cpu" self.model = self.model.to("cpu") self.clip = self.clip.to("cpu") self.gen_embeds = self.gen_embeds.to("cpu") def _precompute_generator_embeddings(self): prompts = [f"An image generated by {g} AI model" for g in GENERATORS] tokens = self.clip_tok(prompts, padding=True, return_tensors="pt") with torch.no_grad(): self.gen_embeds = self.clip.get_text_features(**tokens) self.gen_embeds = self.gen_embeds / self.gen_embeds.norm(dim=-1, keepdim=True) @torch.no_grad() def score(self, video_path: str) -> dict: frames = self._extract_frames(video_path, n=16) if not frames: return {"s2": 0.5, "attribution": {}, "top_generator": "Unknown"} fake_scores = [] for frame in frames: inputs = self.processor(images=frame, return_tensors="pt") inputs = {k: v.to(self.device) for k, v in inputs.items()} logits = self.model(**inputs).logits prob = torch.softmax(logits, dim=-1) fake_prob = prob[0][1].item() if prob.shape[-1] > 1 else prob[0][0].item() fake_scores.append(fake_prob) s2 = sum(fake_scores) / len(fake_scores) attribution = self._attribute(frames) if s2 > 0.5 else {} top_gen = max(attribution, key=attribution.get) if attribution else "Unknown" return {"s2": s2, "attribution": attribution, "top_generator": top_gen} def _attribute(self, frames: list) -> dict: img_embeds = [] for frame in frames[:8]: inputs = self.clip_proc(images=frame, return_tensors="pt") inputs = {k: v.to(self.device) for k, v in inputs.items()} embed = self.clip.get_image_features(**inputs) embed = embed / embed.norm(dim=-1, keepdim=True) img_embeds.append(embed) avg_embed = torch.cat(img_embeds).mean(dim=0, keepdim=True) sims = (avg_embed @ self.gen_embeds.T).squeeze() probs = torch.softmax(sims * 10, dim=-1) return {GENERATORS[i]: round(probs[i].item(), 4) for i in range(len(GENERATORS))} def _extract_frames(self, video_path: str, n: int = 16) -> list: cap = cv2.VideoCapture(video_path) total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) indices = np.linspace(0, max(total-1, 0), n, dtype=int) if total > 0 else [] frames = [] for idx in indices: cap.set(cv2.CAP_PROP_POS_FRAMES, idx) ret, frame = cap.read() if ret: frames.append(Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))) cap.release() return frames ``` --- ## Module 3: SSTGNN ### modules/sstgnn_model.py _(unchanged from v1 — architecture is the same)_ ```python import torch import torch.nn as nn from torch_geometric.nn import global_mean_pool from torch_geometric.utils import degree class SpectralFilterLayer(nn.Module): def __init__(self, in_ch, out_ch, K=3): super().__init__() self.coeffs = nn.ParameterList([ nn.Parameter(torch.randn(in_ch, out_ch) * 0.01) for _ in range(K) ]) self.K = K def forward(self, x, edge_index): out = x @ self.coeffs[0] x_k = x for k in range(1, self.K): row, col = edge_index deg = degree(col, x.size(0), dtype=x.dtype).clamp(min=1) norm = deg.pow(-0.5) aggr = torch.zeros_like(x) aggr.index_add_(0, col, norm[col].unsqueeze(-1) * x_k[row] * norm[row].unsqueeze(-1)) x_k = aggr out = out + x_k @ self.coeffs[k] return torch.relu(out) class TemporalDiffModule(nn.Module): def __init__(self, T, out_dim=32): super().__init__() self.proj = nn.Linear(T, out_dim) def forward(self, x_seq): fft = torch.fft.fft(x_seq, dim=1).abs() fft_pooled = fft.mean(dim=-1) return self.proj(fft_pooled) class SSTGNN(nn.Module): def __init__(self, patch_feat_dim=8, hidden_dim=128, num_frames=32, num_spectral_layers=3, spectral_K=3, fft_dim=32): super().__init__() self.input_proj = nn.Linear(patch_feat_dim + fft_dim, hidden_dim) self.spectral_layers = nn.ModuleList([ SpectralFilterLayer(hidden_dim, hidden_dim, K=spectral_K) for _ in range(num_spectral_layers) ]) self.temporal = TemporalDiffModule(T=num_frames, out_dim=fft_dim) self.classifier = nn.Sequential( nn.Linear(hidden_dim, 64), nn.ReLU(), nn.Dropout(0.3), nn.Linear(64, 1) ) def forward(self, data): fft_feat = self.temporal(data.x_temporal) x = torch.cat([data.x, fft_feat], dim=-1) x = self.input_proj(x) for layer in self.spectral_layers: x = layer(x, data.edge_index) + x x = global_mean_pool(x, data.batch) return self.classifier(x).squeeze(-1) ``` ### modules/m3_sstgnn.py ```python import torch from huggingface_hub import hf_hub_download from modules.sstgnn_model import SSTGNN from utils.graph import video_to_graph from torch_geometric.data import Batch class SSTGNNModule: def __init__(self, cache_dir="/data/model_cache"): self.device = "cpu" ckpt_path = hf_hub_download( repo_id="AkshatAgarwal/SSTGNN-deepfake", filename="sstgnn_best.pt", cache_dir=cache_dir ) self.model = SSTGNN(patch_feat_dim=8, hidden_dim=128, num_frames=32) self.model.load_state_dict(torch.load(ckpt_path, map_location="cpu")) self.model.eval() def to_gpu(self): self.device = "cuda" self.model = self.model.to("cuda") def to_cpu(self): self.device = "cpu" self.model = self.model.to("cpu") @torch.no_grad() def score(self, video_path: str) -> dict: graph = video_to_graph(video_path, patch_size=16, max_frames=32) batch = Batch.from_data_list([graph.to(self.device)]) logits = self.model(batch) s3 = torch.sigmoid(logits).item() vram = torch.cuda.max_memory_allocated() // (1024*1024) if torch.cuda.is_available() else 0 return {"s3": s3, "vram_mb": vram} ``` --- ## Module 5: Fusion + Explain _(unchanged from v1 — these run on CPU regardless)_ ### modules/m5_fusion.py ```python import torch, torch.nn as nn, os class FusionMLP(nn.Module): def __init__(self): super().__init__() self.fc1 = nn.Linear(3, 16) self.fc2 = nn.Linear(16, 3) def forward(self, s: torch.Tensor) -> tuple: h = torch.relu(self.fc1(s)) alpha = torch.softmax(self.fc2(h), dim=-1) return (alpha * s).sum(), alpha class FusionModule: def __init__(self, weights_path="weights/fusion_mlp.pt"): self.model = FusionMLP() if os.path.exists(weights_path): self.model.load_state_dict(torch.load(weights_path, map_location="cpu")) self.model.eval() def fuse(self, s1: float, s2: float, s3: float) -> dict: s = torch.tensor([s1, s2, s3]) with torch.no_grad(): fakescore, alpha = self.model(s) return { "FakeScore": round(fakescore.item(), 4), "weights": { "lip_sync": round(alpha[0].item(), 3), "fingerprint": round(alpha[1].item(), 3), "graph_gnn": round(alpha[2].item(), 3), } } ``` ### modules/m5_explain.py ```python import os from openai import OpenAI class ExplainModule: """NVIDIA NIM: meta/llama-3.1-8b-instruct. ~40 req/min free.""" def __init__(self): self.client = OpenAI( api_key=os.environ.get("NVIDIA_API_KEY", ""), base_url="https://integrate.api.nvidia.com/v1" ) self.model = "meta/llama-3.1-8b-instruct" def explain(self, fakescore, s1, s2, s3, weights, attribution, segments, top_generator) -> str: verdict = "FAKE" if fakescore > 0.5 else "REAL" confidence = ( "high" if abs(fakescore-0.5) > 0.3 else "moderate" if abs(fakescore-0.5) > 0.15 else "low" ) seg_text = "" if segments: seg_text = "Flagged timestamps: " + ", ".join( [f"{s['time']}s (score={s['score']})" for s in segments[:5]] ) attr_text = "" if attribution: top3 = sorted(attribution.items(), key=lambda x: -x[1])[:3] attr_text = "Top generators: " + ", ".join( [f"{n}: {p*100:.1f}%" for n, p in top3] ) prompt = f"""You are a forensic AI analyst. Analyze these deepfake detection results. Be specific about evidence. Results: - Verdict: {verdict} (FakeScore: {fakescore:.3f}, confidence: {confidence}) - Lip-Sync (M1): {s1:.3f} (weight: {weights.get('lip_sync', 'N/A')}) - Fingerprint (M2): {s2:.3f} (weight: {weights.get('fingerprint', 'N/A')}) - Graph-GNN (M3): {s3:.3f} (weight: {weights.get('graph_gnn', 'N/A')}) {seg_text} {attr_text} - Most likely generator: {top_generator} Write 3-5 sentences. Reference specific scores and timestamps.""" try: response = self.client.chat.completions.create( model=self.model, messages=[ {"role": "system", "content": "You are a forensic deepfake analyst. Be precise."}, {"role": "user", "content": prompt} ], max_tokens=300, temperature=0.3 ) return response.choices[0].message.content.strip() except Exception as e: return self._fallback(verdict, fakescore, s1, s2, s3, top_generator, confidence) def _fallback(self, verdict, fakescore, s1, s2, s3, top_gen, conf): if verdict == "FAKE": return ( f"Video classified as {verdict} with {conf} confidence (FakeScore: {fakescore:.3f}). " f"Lip-sync scored {s1:.2f}, indicating " f"{'significant' if s1>0.7 else 'moderate' if s1>0.5 else 'minimal'} audio-visual inconsistency. " f"Style fingerprinting scored {s2:.2f}, top attribution: {top_gen}. " f"Graph analysis scored {s3:.2f}." ) return ( f"Video classified as {verdict} with {conf} confidence (FakeScore: {fakescore:.3f}). " f"All modules returned scores below detection threshold." ) ``` --- ## Main App: app.py (ZeroGPU Version) ```python import spaces # HuggingFace ZeroGPU import gradio as gr import torch, time, os from modules.m1_lipsync import LipSyncModule from modules.m2_fingerprint import FingerprintModule from modules.m3_sstgnn import SSTGNNModule # real model; no fallback in prod from modules.m5_fusion import FusionModule from modules.m5_explain import ExplainModule CACHE = "/data/model_cache" if os.path.exists("/data") else "./cache" os.makedirs(CACHE, exist_ok=True) # All models load on CPU at startup — GPU not allocated yet print("Loading modules on CPU...") m1 = LipSyncModule(cache_dir=CACHE) m2 = FingerprintModule(cache_dir=CACHE) m3 = SSTGNNModule(cache_dir=CACHE) m5_fusion = FusionModule(weights_path="weights/fusion_mlp.pt") m5_explain = ExplainModule() print("Ready. GPU will be allocated per request via ZeroGPU.") @spaces.GPU(duration=120) # request A10G for up to 120s per call def analyze(video_file): if video_file is None: return "Upload a video.", "", "", "" start = time.time() # Move models to GPU for this request m1.to_gpu() m2.to_gpu() m3.to_gpu() try: r1 = m1.score(video_file) r2 = m2.score(video_file) r3 = m3.score(video_file) finally: # GPU released after function returns anyway, but explicit is cleaner m1.to_cpu() m2.to_cpu() m3.to_cpu() # Fusion and explain run on CPU — no GPU needed fusion = m5_fusion.fuse(r1["s1"], r2["s2"], r3["s3"]) explanation = m5_explain.explain( fakescore=fusion["FakeScore"], s1=r1["s1"], s2=r2["s2"], s3=r3["s3"], weights=fusion["weights"], attribution=r2["attribution"], segments=r1.get("segments", []), top_generator=r2["top_generator"] ) elapsed = time.time() - start verdict = "FAKE" if fusion["FakeScore"] > 0.5 else "REAL" icon = "🔴" if verdict == "FAKE" else "🟢" verdict_text = f"{icon} **{verdict}** (FakeScore: {fusion['FakeScore']:.3f})" scores_text = f"""**Per-Module Scores:** - Lip-Sync (M1): {r1['s1']:.3f} [weight: {fusion['weights']['lip_sync']:.2f}] - Fingerprint (M2): {r2['s2']:.3f} [weight: {fusion['weights']['fingerprint']:.2f}] - Graph-GNN (M3): {r3['s3']:.3f} [weight: {fusion['weights']['graph_gnn']:.2f}] **Time:** {elapsed:.1f}s | **Hardware:** A10G (ZeroGPU)""" attr_text = "**Generator Attribution:**\n" if r2["attribution"]: for gen, prob in sorted(r2["attribution"].items(), key=lambda x: -x[1]): bar = "█" * int(prob * 30) attr_text += f"- {gen}: {prob*100:.1f}% {bar}\n" else: attr_text += "- N/A (classified as real)" return verdict_text, scores_text, attr_text, explanation with gr.Blocks( title="GenAI-DeepDetect", theme=gr.themes.Base(primary_hue="red", font=["DM Sans", "sans-serif"]) ) as demo: gr.Markdown( "# GenAI-DeepDetect\n" "### Multimodal Deepfake Detection and Attribution\n" "**Modules:** LipFD | CLIP Detector | SSTGNN | Llama-3.1-8B via NVIDIA NIM | " "**Hardware:** ZeroGPU (A10G)" ) with gr.Row(): with gr.Column(scale=1): vid = gr.Video(label="Upload Video", height=300) btn = gr.Button("Analyze", variant="primary", size="lg") with gr.Column(scale=2): v_out = gr.Markdown(label="Verdict") s_out = gr.Markdown(label="Scores") with gr.Row(): a_out = gr.Markdown(label="Attribution") e_out = gr.Markdown(label="Explanation") btn.click(fn=analyze, inputs=[vid], outputs=[v_out, s_out, a_out, e_out]) gr.Markdown( "---\n**Paper:** GenAI-DeepDetect | " "**Authors:** Akshat Agarwal, Dev Chopda | SRM IST" ) if __name__ == "__main__": demo.launch() ``` --- ## Environment Secrets (HF Space Settings) | Key | Value | Source | | ---------------- | ----------- | ------------------------------ | | `NVIDIA_API_KEY` | `nvapi-...` | build.nvidia.com (free signup) | | `HF_TOKEN` | `hf_...` | huggingface.co/settings/tokens | --- ## Tonight's Timeline (Updated) | Time | Task | Duration | | --------- | -------------------------------------------------------------- | -------- | | NOW | Add `hardware: zero-gpu` to README.md, set secrets in HF Space | 10 min | | +0:10 | Clone LipFD, upload checkpoint to HF Hub | 30 min | | +0:40 | Push file structure + requirements.txt (with `spaces` package) | 15 min | | +0:55 | Wire M1 + M2 + M3 with `to_gpu`/`to_cpu` pattern, test each | 45 min | | +1:40 | Wire `@spaces.GPU` in app.py, test full pipeline | 30 min | | +2:10 | Wire M5 fusion + NVIDIA NIM explainer | 30 min | | +2:40 | Fix bugs, test edge cases (no face, audio-only, short clips) | 45 min | | +3:25 | README.md, push final | 15 min | | +3:40 | Collect scores, train MLP, push fusion weights | 15 min | | +3:55 | Write session summary to Notion context DB | 5 min | | **+4:00** | **DONE** | | --- ## Swap Guide: When SSTGNN Is Trained 1. Train on L40S, push weights: `huggingface-cli upload AkshatAgarwal/SSTGNN-deepfake sstgnn_best.pt .` 2. `m3_sstgnn.py` is already imported in `app.py` — nothing to change. 3. Commit and push. Done. --- ## Obsidian Context Store: Agent Handoff Protocol > Obsidian MCP is available on the local AI agent. Use it for all cross-session > context. At session start: read. At session end: write. ### Folder Setup (one-time) On the **first** session, the agent checks if the folder exists before creating it: ``` Check if vault folder "GenAI-DeepDetect" exists. If not, create it. Then create the following notes inside it if they don't already exist: - README.md (project overview, one-liner per module) - session-log.md (append-only log of every session) - module-status.md (current state of each module, overwrite each session) - blockers.md (open issues / questions, cleared when resolved) ``` ### Session Start (every session) ``` Read these files from the GenAI-DeepDetect Obsidian folder: - module-status.md - blockers.md - session-log.md (last 3 entries only) Summarize current state and tell me what to work on first. ``` ### Session End (every session) Append to `session-log.md`: ```markdown ## [YYYY-MM-DD HH:MM] — [modules touched] **Completed:** - ... **Broke / Fixed:** - ... **Next session starts with:** - ... **Changed paths / model IDs:** - ... ``` Overwrite `module-status.md` with the current state of all modules: ```markdown # Module Status — [date] | Module | Status | Notes | | -------------- | ----------------- | ----- | | M1 LipSync | done / wip / todo | ... | | M2 Fingerprint | ... | ... | | M3 SSTGNN | ... | ... | | M5 Fusion | ... | ... | | M5 Explain | ... | ... | | Infra/Space | ... | ... | ``` Update `blockers.md` — remove resolved items, add new ones: ```markdown # Open Blockers — [date] - [ ] ... - [ ] ... ```