The process

How Tvak works

From a photo taken at a rural clinic to a referral decision — here's what happens.

1
Photograph the lesion
A health worker photographs the skin condition using any smartphone. Clear, close-up, well-lit photos give the best results.
2
Upload to Tvak
Photo is uploaded through the browser. On mobile, the camera opens directly. No app needed.
3
Preprocessing
Image is resized to 224×224 and normalized using ImageNet statistics — identical to training conditions.
Mean [0.485, 0.456, 0.406] · Std [0.229, 0.224, 0.225]
4
Multi-task inference
The image passes through a shared EfficientNet B0 backbone, then splits into two heads — severity (4 classes) and disease name (14 classes). Both predictions happen in one forward pass.
5
Grad-CAM heatmap
Gradient-weighted class activation mapping highlights which region of the image the model focused on. Health workers can verify the AI is looking at the lesion, not the background.
6
AI-assisted referral decision
The severity classification and disease name are passed to Llama 3 (via Groq), which generates a plain-language referral instruction the health worker can act on immediately.
Healthy → No referral  ·  Mild → Treat locally  ·  Serious → Urgent referral  ·  Other → Further assessment
Architecture

The model

EfficientNet B0 pretrained on ImageNet, fine-tuned on 29,322 skin disease images using a freeze-unfreeze strategy. Two output heads share one backbone — one for severity (4 classes), one for disease name (14 classes).

Input image · 224 × 224
↓
EfficientNet B0 backbone · 1280-dim features
↓
Severity head
4 classes
Disease head
14 classes
Training

Freeze → Unfreeze

Phase 1 · 5 epochs
Frozen backbone
Only classifier heads train. Backbone frozen to preserve pretrained features. LR: 1e-3.
Phase 2 · 15 epochs
Full fine-tuning
All layers unfreeze. Low LR (1e-5) to gently adapt without forgetting pretrained knowledge.
80.7%
Severity acc.
86.5%
Disease acc.
3,674
Test images
20
Total epochs
See it in action
Upload a skin lesion photo and get a triage result in seconds.
Analyze a lesion →