Have you ever had a teacher who simply provided you with the solutions to your homework? Yes,that was satisfying at the time, but did you really gain any knowledge? Most likely not.
That was the main obstacle we had to overcome in order to create an AI helper for RosettaCloud's interactive Labs. Our platform teaches shell programming and DevOps, and we wanted to develop an AI assistant that would do more than just solve problems for students—it should help them learn.
In this post, I'll describe how we used LangChain, Amazon Bedrock's Nova LLM, and RAG (Retrieval-Augmented Generation) system to create a hint-first instructional chatbot that have a knowledge base and a memory, Instead of depriving pupils of educational chances, our method guarantees that the assistant leads them toward discoveries.
The "Don't Just Tell Me" Philosophy
When learning technical skills like DevOps, Shell Scripting or Kubernetes, students need to develop troubleshooting abilities and pattern recognition. If an AI simply answers questions directly, it short-circuits the learning process.
Here is how our interactive-lab will look like:
For now lets focus on AI our AI assistant:
Here's our educational philosophy baked into our AI assistant:
First interaction: Provide a hint that guides the student toward the solution
Follow-up questions: Provide more direct assistance if the student is still stuck
Context-aware memory: Remember the conversation to provide increasingly targeted help
Nova's strength in context understanding and its ability to follow complex instructions made it perfect for this approach. Let's dive into how we built it.
Architecture Overview: The RosettaCloud Chatbot Flow
Our system follows a 7-step flow:
1- User asks a question in the Angular frontend
2- Question is sent via WebSockets to API Gateway
3- The AI Chatbot Lambda processes the request
4- Two parallel operations:
A- Fetch chat history from DynamoDB
B- Search for relevant documents in our LanceDB vector database
5- Amazon Bedrock's Nova processes everything
6- Response is streamed back to the client
7- Frontend renders the response with helpful references
This architecture gives us real-time interactions with context awareness, making the student experience feel like working with a patient tutor.
The Core Components
Angular Client side app
WebSocket Communication for Real-Time Responses
Students expect real-time help. We use WebSockets to create a streaming experience:
typescript
// Angular service for WebSocket communication
@Injectable({
providedIn: 'root'
})
export class ChatbotService {
private socket: WebSocket;
private messageSubject = new Subject<ChatMessage>();
constructor() {
this.connectWebSocket();
}
private connectWebSocket() {
// Connect to our API Gateway WebSocket endpoint
this.socket = new WebSocket(environment.chatbotWebsocketUrl);
this.socket.onmessage = (event) => {
const data = JSON.parse(event.data);
this.messageSubject.next(data);
};
}
sendQuestion(question: string) {
const payload = {
prompt: question,
session_id: this.getSessionId(),
bedrock_model_id: 'amazon.nova-lite-v1:0',
model_kwargs: {
temperature: 0.7,
top_p: 0.9,
max_tokens_to_sample: 512
}
};
this.socket.send(JSON.stringify(payload));
}
getMessages(): Observable<ChatMessage> {
return this.messageSubject.asObservable();
}
}
The Lambda Backend: AI Chatbot with RAG
The heart of our system is the AI Chatbot Lambda which handles:
Processing the user's question
Retrieving relevant context from our knowledge base
Building a conversational RAG workflow
Streaming responses back to the client
Here's a simplified version of our ai_chatbot.py Lambda function:
def handle_message(event, context):
connection_id = event.get('requestContext', {}).get('connectionId', '')
api_endpoint = get_api_endpoint_from_event(event)
body = json.loads(event.get("body", "{}"))
prompt = body["prompt"]
bedrock_model_id = body["bedrock_model_id"]
model_kwargs = body["model_kwargs"]
session_id = body["session_id"]
try:
# Initialize our streaming wrapper for Bedrock
streamer = BedrockStreamer(connection_id, session_id, api_endpoint)
# Create the RAG chain with LangChain
conversation = streamer.create_rag_chain(
LANCEDB_S3_URI,
KNOWLEDGE_BASE_ID,
bedrock_model_id,
model_kwargs
)
# Stream the response
for response in streamer.stream_response(conversation, prompt):
streamer.params["Data"] = response
streamer.api_client.post_to_connection(**streamer.params)
return {"statusCode": 200, "body": json.dumps("Success")}
except Exception as e:
print(f"Error: {str(e)}")
# Error handling...
The BedrockStreamer class is where the magic happens:
Our chatbot system's foundation is the BedrockStreamer class, which manages the intricate interaction between human inquiries and AI answers. It establishes connections to Amazon Bedrock for AI processing and API Gateway for WebSocket communication. Fundamentally, this class builds the RAG (Retrieval Augmented Generation) pipeline by setting up the conversational components of LangChain, initializing vector retrievers from LanceDB, defining the "hint-first" instructional prompt method, and controlling the streaming response system. It manages important activities including using heartbeat messages to sustain WebSocket connections, adding instructional metadata to answers, and using DynamoDB to ensure session persistence. Instead of functioning as a unified teaching aid, the many elements would remain disjointed fragments in the absence of this overarching orchestration layer.
class BedrockStreamer:
def __init__(self, connectionId, session_id, api_endpoint):
self.api_client = boto3.client("apigatewaymanagementapi",
endpoint_url=api_endpoint)
self.bedrock_client = boto3.client(service_name='bedrock-runtime',
region_name=BEDROCK_REGION)
self.session_id = session_id
self.doc_sources = []
self.params = {
"Data": "",
"ConnectionId": connectionId
}
self.heartbeat_active = False
def set_prompt(self, response_style="balanced"):
# The key to our hint-first approach is in the system prompt
system_prompt = (
"You are an assistant specializing in DevOps. "
f"Use a {response_style} style - be direct, clear and efficient. "
"Use the following pieces of retrieved context to answer "
"But do not include the context in your answer. "
"You should not answer the question directly, but rather hint the user "
"to the answer. if the user asked again, then you can answer. "
"You should not answer any question not related to DevOps or SWE. "
"If the user asks about something else, say that you are not able to "
"answer that. If the user asks about DevOps, answer it. "
"the question about DevOps. If you don't know the answer, "
"say that you don't know. When showing code examples, "
"format them properly with markdown syntax."
"\n\n"
"{context}"
)
# Conversation prompt for chat history context
qa_prompt = ChatPromptTemplate.from_messages([
("system", system_prompt),
MessagesPlaceholder(variable_name="chat_history"),
("human", "{input}"),
])
return qa_prompt
Nova LLM: The Brain That Gets Educational Context
Amazon Bedrock's Nova model is particularly good at understanding the nuance between "give a hint" and "provide the answer." We configure it to provide educational guidance:
def create_rag_chain(self, lancedb_uri, knowledge_base_id, bedrock_model_id, model_kwargs):
# Get our educational prompt template
qa_prompt = self.set_prompt("educational")
# Initialize vector retriever for our shell script knowledge base
retriever = self.init_retriever(lancedb_uri, knowledge_base_id)
# Initialize Bedrock LLM with Nova
llm = BedrockChat(
model_id=bedrock_model_id,
model_kwargs=model_kwargs,
streaming=True,
client=self.bedrock_client
)
# Create chain components using LangChain
history_aware_retriever = create_history_aware_retriever(llm, retriever)
question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)
rag_chain = create_retrieval_chain(history_aware_retriever, question_answer_chain)
# Make it conversational with DynamoDB chat history
conversational_rag_chain = RunnableWithMessageHistory(
rag_chain,
lambda session_id: DynamoDBChatMessageHistory(
table_name=DYNAMO_TABLE,
session_id=self.session_id
),
input_messages_key="input",
history_messages_key="chat_history",
)
return conversational_rag_chain
Stream response back via web sockets:
Angular show it to user:
Knowledge Base Indexing: Document Indexer Lambda
For our assistant to be effective, it needs access to high-quality learning materials. We built a document indexing system that processes shell scripts and extracts educational metadata.
Our chatbot system's knowledge base is created and maintained via the crucial background operation known as Document Indexing Flow.
The Document Indexer Lambda is automatically activated by an EventBridge trigger when shell scripts are uploaded to S3. The Lambda meticulously examines each script to collect educational metadata such as the question text, difficulty level, correct answers, and verification logic. The improved script representations are then converted into vector embeddings that capture semantic meaning using this Lambda using Amazon Bedrock's Titan Embeddings model. The contextual understanding of the chatbot is powered by a searchable knowledge base created by storing these vectors in LanceDB. Without human involvement, this automated pipeline guarantees that our chatbot always has access to up-to-date, correctly indexed instructional information, enabling the system to grow naturally as more learning resources are added.
def process_shell_script(text, filename):
# Extract educational metadata from shell script comments
lab_info = {
'exercise_name': '',
'difficulty': '',
'learning_objectives': [],
'question_number': '',
'question': '',
'question_type': '',
'possible_answers': [],
'correct_answer': '',
'question_flag': '',
'check_flag': ''
}
# Parse the shell script to extract question info
# from comments and code blocks
# ...
# Create an enhanced representation that includes
# educational context
enhanced_text = f"""
SCRIPT NAME: {filename}
ORIGINAL CONTENT:
{text}
SUMMARY OF COMMENTS:
{chr(10).join(comments)}
MAIN COMMAND BLOCKS:
{chr(10) + chr(10).join([f"Block {i+1}:{chr(10)}{block}" for i, block in enumerate(command_blocks)])}
LAB QUESTION INFORMATION:
{formatted_lab_metadata}
"""
return enhanced_text, lab_info
The Hint-First Approach in Action
Let's see how this actually works in practice. Here's a sample interaction:
Lessons Learned from Building with Nova
After implementing this system, we've learned several valuable lessons:
Nova does a remarkable job of adhering to subtle instructions and maintaining the "hint first" strategy. Streaming is crucial. Compared to waiting for full answers, the WebSocket method feels considerably more interactive.
We went through numerous iterations to find the ideal balance between being too helpful and being helpful. Careful prompt engineering matters. Context is king: The assistant is genuinely helpful when it combines conversation history and document retrieval.
Finding a balance between user delight and instructional value was the most challenging aspect. Students would become irritated if the helper was very restricted. They wouldn't learn if they were too helpful. We were able to achieve that balance because of Nova's adaptability.
The Significance of This for Education
There are two main types of traditional approaches to educational chatbots:
Basic Q&A bots that offer straightforward responses without any direction
Systems that are too tight and irritate kids
With Nova, our hint-first strategy establishes a compromise—a system that:
- Promotes critical thinking
- Offers scaffolded assistance
- Increases directness gradually in accordance with student needs
- Keeps in mind the context of the conversation to offer tailored assistance
This reflects the real job of effective teachers. They point students in the direction of answers rather than solving their difficulties.
Conclusion: AI in Education's Future
Developing educational AI differs greatly from developing a general-purpose helper. A well-designed RAG system and Amazon Nova's adaptability enabled us to develop something that genuinely improves learning rather than hinders it.
As we keep improving our system, we're investigating:
- More advanced methods for knowledge retrieval
- Progress monitoring to automatically modify hint levels
- Integration with practical laboratory settings for instantaneous feedback
AI in education is not about substituting answer machines for instructors. The goal is to build assistants that, like the best human teachers, know when to guide, when to instruct, and when to hint.
Have you developed AI systems for education? Which strategies have you found work best for striking a balance between learning support and assistance? Leave a comment with your ideas!
Top comments (0)