🚀 Building "Sahayak": A Multi-Persona AI Assistant (Trainer, Teacher, Doctor)
Sahayak (Sanskrit for “helper”) is an AI platform that lets users interact with hyper-realistic avatars in real time. Whether you need a personal a engineer or a doctor, Sahayak can provide personalized assistance. Here’s how to build it with code, hosting details, and cost-saving tips.
🛠️ Tech Stack
- Frontend: React.js + WebRTC (video streaming).
- Backend: Node.js
- AI: Claude 3.5 (inference), ElevenLabs (voice), HeyGen (avatars).
- Database: MongoDB Atlas (user profiles).
🖥️ Frontend Code (React.js)
1. User Interface with Persona Selection
// src/components/Home.js
import React, { useState } from 'react';
import WorkoutScreen from './WorkoutScreen';
import TeacherScreen from './TeacherScreen';
import DoctorScreen from './DoctorScreen';
const Home = () => {
const [persona, setPersona] = useState('trainer');
return (
<div className="flex flex-col items-center justify-center min-h-screen bg-gradient-to-b from-blue-50 to-blue-100">
<h1 className="text-4xl font-bold text-blue-600 mb-6">
Welcome to Sahayak
</h1>
<p className="text-lg text-gray-700 mb-8">
Choose a persona to get started:
</p>
{/* Persona Selection */}
<div className="grid grid-cols-1 md:grid-cols-3 gap-6">
<button
className={`p-6 bg-white shadow-lg rounded-2xl border-2 ${
persona === 'trainer' ? 'border-blue-500' : 'border-transparent'
} hover:shadow-xl transition-all`}
onClick={() => setPersona('trainer')}
>
<img
src="<https://via.placeholder.com/100>"
alt="Trainer"
className="mx-auto mb-4"
/>
<h3 className="text-xl font-semibold text-gray-800">AI Trainer</h3>
<p className="text-gray-500">Get fitness advice and posture analysis.</p>
</button>
<button
className={`p-6 bg-white shadow-lg rounded-2xl border-2 ${
persona === 'teacher' ? 'border-blue-500' : 'border-transparent'
} hover:shadow-xl transition-all`}
onClick={() => setPersona('teacher')}
>
<img
src="<https://via.placeholder.com/100>"
alt="Teacher"
className="mx-auto mb-4"
/>
<h3 className="text-xl font-semibold text-gray-800">AI Teacher</h3>
<p className="text-gray-500">Learn and explore new concepts easily.</p>
</button>
<button
className={`p-6 bg-white shadow-lg rounded-2xl border-2 ${
persona === 'doctor' ? 'border-blue-500' : 'border-transparent'
} hover:shadow-xl transition-all`}
onClick={() => setPersona('doctor')}
>
<img
src="<https://via.placeholder.com/100>"
alt="Doctor"
className="mx-auto mb-4"
/>
<h3 className="text-xl font-semibold text-gray-800">AI Doctor</h3>
<p className="text-gray-500">Get health tips and symptom analysis.</p>
</button>
</div>
{/* Dynamic Persona Screen */}
<div className="mt-12 w-full max-w-4xl">
{persona === 'trainer' && <WorkoutScreen />}
{persona === 'teacher' && <TeacherScreen />}
{persona === 'doctor' && <DoctorScreen />}
</div>
</div>
);
};
export default Home;
2. Real-Time Video Component
// src/components/Persona.js
import React, { useEffect, useRef, useState } from 'react';
import cv from '@techstark/opencv-js';
const WorkoutScreen = () => {
const videoRef = useRef(null);
const canvasRef = useRef(null);
const [isCameraReady, setIsCameraReady] = useState(false);
// Helper function to detect pose
const detectPose = (frame) => {
// Placeholder: Implement pose detection logic here
const posePoints = [{ x: 100, y: 200 }, { x: 150, y: 250 }]; // Example output
return posePoints;
};
// Helper function to detect emotion
const detectEmotion = (frame) => {
// Placeholder: Implement emotion detection logic here
const emotions = ['Happy', 'Neutral', 'Sad'];
return emotions[Math.floor(Math.random() * emotions.length)]; // Example output
};
useEffect(() => {
const initCamera = async () => {
try {
const stream = await navigator.mediaDevices.getUserMedia({
video: true,
});
videoRef.current.srcObject = stream;
videoRef.current.onloadedmetadata = () => {
videoRef.current.play();
setIsCameraReady(true);
};
const processVideo = () => {
const video = videoRef.current;
const canvas = canvasRef.current;
const ctx = canvas.getContext('2d');
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
const frame = cv.imread(canvas);
// Perform pose detection
const posePoints = detectPose(frame);
console.log('Pose points:', posePoints);
// Perform emotion detection
const emotion = detectEmotion(frame);
console.log('Emotion:', emotion);
frame.delete();
requestAnimationFrame(processVideo);
};
requestAnimationFrame(processVideo);
} catch (err) {
console.error('Error accessing camera:', err);
}
};
initCamera();
}, []);
return (
<div className="flex flex-col items-center justify-center min-h-screen bg-gray-100">
{!isCameraReady ? (
<p className="text-lg text-gray-600">Initializing camera...</p>
) : (
<>
<canvas ref={canvasRef} className="border border-gray-300 rounded-lg" />
</>
)}
<video ref={videoRef} style={{ display: 'none' }} />
</div>
);
};
export default WorkoutScreen;
🧑💻 Backend Code (Node.js)
1. Real-Time Inference Pipeline
// routes/ai.js
const express = require('express');
const claude = require('claude-api');
const elevenlabs = require('elevenlabs-client');
const mongoose = require('mongoose');
const fetch = require('node-fetch');
const User = require('../models/User');
const router = express.Router();
// Trainer Endpoint: Analyze posture and suggest workouts
router.post('/analyze', async (req, res) => {
try {
const { keypoints, userId } = req.body;
const claudeResponse = await claude.complete(
`User's posture: ${keypoints}\\nSuggest a 15-minute workout routine.`
);
// Generate avatar video with HeyGen
const heygenResponse = await fetch('<https://api.heygen.com/generate>', {
method: 'POST',
headers: { 'Authorization': 'Bearer YOUR_KEY' },
body: JSON.stringify({
text: claudeResponse,
voice: 'elevenlabs-voice-id'
})
});
// Save the suggested plan to the user's activity log
await User.findByIdAndUpdate(userId, {
$push: {
activityLog: {
type: 'Workout Plan',
details: claudeResponse,
date: new Date(),
},
},
});
res.json({ plan: claudeResponse, avatarUrl: heygenResponse.url });
} catch (err) {
res.status(500).send({ error: 'Failed to analyze posture', details: err });
}
});
// Emotion Analysis Endpoint
router.post('/emotion', async (req, res) => {
try {
const { image, userId } = req.body;
// Simulate emotion analysis (replace with a real API/model)
const emotions = ['Happy', 'Sad', 'Neutral', 'Angry'];
const detectedEmotion = emotions[Math.floor(Math.random() * emotions.length)];
// Log emotion analysis result
await User.findByIdAndUpdate(userId, {
$push: {
activityLog: {
type: 'Emotion Analysis',
details: `Detected emotion: ${detectedEmotion}`,
date: new Date(),
},
},
});
res.json({ detectedEmotion });
} catch (err) {
res.status(500).send({ error: 'Failed to analyze emotion', details: err });
}
});
// Teacher Endpoint: Generate a study plan
router.post('/study-plan', async (req, res) => {
try {
const { topic, duration, userId } = req.body;
const claudeResponse = await claude.complete(
`Create a study plan for the topic "${topic}" for ${duration} minutes.`
);
// Save the study plan to the user's progress
await User.findByIdAndUpdate(userId, {
$push: {
activityLog: {
type: 'Study Plan',
details: claudeResponse,
date: new Date(),
},
},
});
res.json({ studyPlan: claudeResponse });
} catch (err) {
res.status(500).send({ error: 'Failed to generate study plan', details: err });
}
});
// Doctor Endpoint: Symptom analysis and recommendations
router.post('/symptoms', async (req, res) => {
try {
const { symptoms, userId } = req.body;
const claudeResponse = await claude.complete(
`Based on the symptoms: ${symptoms}, provide recommendations.`
);
// Log the doctor's advice to the user
await User.findByIdAndUpdate(userId, {
$push: {
activityLog: {
type: 'Doctor Advice',
details: claudeResponse,
date: new Date(),
},
},
});
res.json({ advice: claudeResponse });
} catch (err) {
res.status(500).send({ error: 'Failed to analyze symptoms', details: err });
}
});
module.exports = router;
2. Database Integration (MongoDB)
const mongoose = require('mongoose');
const activityLogSchema = new mongoose.Schema({
type: { type: String, required: true }, // e.g., "Workout Plan", "Emotion Analysis"
details: { type: String, required: true },
date: { type: Date, default: Date.now },
});
const userSchema = new mongoose.Schema({
name: { type: String, required: true },
email: { type: String, required: true, unique: true },
personaPreferences: {
trainer: { type: Boolean, default: false },
teacher: { type: Boolean, default: false },
doctor: { type: Boolean, default: false },
},
goals: [
{
type: { type: String, required: true }, // e.g., "Fitness", "Learning"
description: { type: String, required: true },
progress: { type: Number, default: 0 }, // Progress percentage
},
],
activityLog: [activityLogSchema],
});
module.exports = mongoose.model('User', userSchema);
🏠 Hosting Setup
- Frontend: Deploy to Vercel (free tier).
- Backend: Use Heroku (free dyno + 1000/month for hobby tier).
- Database: MongoDB Atlas (free tier).
💡 Cost-Saving Tips
- AI APIs:
- Use ElevenLabs’ community plan (0–2k/month) for voice cloning.
- chatgpt/ antropic API’s for real-time search.
🚧 Why I Paused Development
Building Sahayak cost 15k/month in API credits and cloud services. I had to pause full-time work after 4 months but plan to resume with community funding.
Connect with me to collaborate or fund Sahayak’s future!
#AIAssistant #Sahayak #BudgetAI #OpenSourceAI