Projects
1. Comprehend.it
Skills- Django, Python, Whisper, Llama-2, LangChain, LLM Prompting, Video Processing, OCR
An education platform build on Django to transcribe videos using Whisper AI and generate quizzes/notes/summaries from the transcript by leveraging Llama-2 and GPT-3.5 for prompt-engineering and integrated using LangChain to the application.
2. LangPrep
Skills- Django, Python, AWS - Amazon Polly, Amazon Textract, S3
Django application for pronunciation learning. Hosted on AWS, it uses Amazon Textract for OCR for text extraction and Amazon Polly for text to speech conversion. All documents are stored in S3 bucket.
3. Language-Agnostic Sentence Embedding Generator
Skills- PyTorch, GANs, Python, BERT, TensorBoard, Evaluation
Trained a GAN network to create a language agnostic embedding generator. Trained multi-lingual BERT based models as embedding generator and dense classifiers as the discriminator to identify source language of a text.
4. Unsupervised Monocular Depth Estimation from Video
Skills- PyTorch, Python, Vision Transformer, Deep Learning, Adversarial Training, Training Models
Enhanced depth estimation for monocular videos using self-attention and vision transformer, outperforming the baseline on KITTI dataset. Implemented self-attention layers and ViT as image encoder with adversarial loss.
5. Open-Domain Long-Form Question Answering
Skills- PyTorch, Python, Retrieval Augmented Generation (RAG), Fine-Tuning
Fine-tuned T5-base model on ELI5 data for compiling an answer from multiple sources to queries from a user. Additionally, implemented Sparse-Retrieval Index (BM25) and Dense-Retrieval Index (FAISS) to fetch relevant documents to provide as context to the T-5 model.
6. Harmony
Skills- Android Programming, Java, Python, Django, UML, Design
Developed two Android mobile applications and a Django application using Java and python for patients, caretakers, and doctors respectively to manage Severe Mental Illnesses in collaboration with AIIMS, Delhi, the premier hospital in India.