GitHub Challenge: AI-Powered Text Summarizer & Questions Generator

Denis Gavrilin — Sat, 18 Jan 2025 20:15:24 +0000

This is a submission for the GitHub Copilot Challenge : New Beginnings

What I Built

I developed a text summarization application capable of processing text files (.pdf, .txt) and YouTube video transcripts to generate summaries with self-check questions. My motivation for building this tool was to streamline learning: often, you want to extract the essential information from a book or lecture while eliminating irrelevant details, or convert video content into digestible text. This application provides quick, dry summaries with questions that can help you verify your understanding of the material.

Usage

You can run the application using the command line. The application accepts either a file path or a YouTube video URL to fetch transcripts from.

Options

Required

--file_path: Path to the input file (.txt or .pdf) or
--youtube_url: YouTube video URL to fetch transcripts from

Optional

--language: The language for output summarization
--tokens: Number of tokens for the summary, how big you want your summary, better use reasonably big amount
--chunk_size: Size of chunks your text will be split into, for each chunk there will be a summary
--num_questions: Number of questions to generate for each summary
--summary_path: Folder path to save the summary file

Demo

Currently, I didn't create a web-based demo due to API token limitations; instead, the application is available as a CLI tool. Users can supply their own OpenAI API tokens to try it out. Here are a couple of examples:

From the text available here, the tool generated this summary with questions: sample_text_summary.md.
For the YouTube video 1Y82U450zcI, it produced this summary with questions: youtube_summary_1Y82U450zcI.md.

Repository

The project's source code is publicly available on GitHub: smrz repository.

Copilot Experience

Although I'm not deeply experienced with python, I was able to develop this application with GitHub Copilot's assistance. Copilot helped generate code, handle errors and edge cases, and structure functionality. While organizing the code and creating tests were challenging, Copilot proved to be a valuable tool since I don't know python. Without Copilot, building this project in python would likely have taken much longer.

GitHub Models

For prototyping LLM capabilities within my app, I leveraged GitHub Models, specifically the OpenAI GPT-4 model (referred to as gpt-4o) via the OpenAI API. This integration enhanced the summarization and question generation features, enabling more intelligent and responsive outputs. Prior attempts to use local large language models were slower and produced less satisfactory results, so using GitHub's model was a significant improvement.

Conclusion

Building this application in one day with about 80% assistance from Copilot was fun. There are countless ways to improve the app and enhance code organization, but working within these constraints helped me focus on the core functionality. This project not only achieved its goals but also provided valuable learning experiences in leveraging AI tools for development.

Forem: Denis Gavrilin