Forem: Latitude

17 Projects for Developers to Build AI Features 100x Faster 👩‍💻🔥

Anmol Baranwal — Thu, 17 Oct 2024 15:17:40 +0000

Prompt engineering and AI are booming, with almost every startup team integrating AI to make things easier for their users.

Today, we are covering 17 projects to maximize your productivity for developers building with AI.

You will find tools related to prompt engineering, code editors, agents and many more exciting things.

This list will surprise you.

1. Latitude LLM - prompt engineering platform to build and refine prompts with AI.

Latitude is the open source prompt engineering platform to build, evaluate, and refine your prompts with AI. You can create and iterate prompts in the platform by using their SDKs or the API.

The best part is that every time a prompt runs, it automatically logs the entire context, the output and other metadata relevant for evaluations and debugging.

This is how the dashboard looks.

✅ There is support for advanced features like parameters, snippets, logic and more.

✅ You get version control for prompts, collaborative prompt manager and even evaluations in batch or real-time.

A basic user flow can be:

-→ Create a new project.

-→ Write your first prompt using the editor.

-→ Test your prompt using the playground with different inputs and see the mode's responses.

-→ Before deploying, you can upload a dataset and run a batch evaluation to assess your prompt’s performance across various scenarios. Watch this video to see how evaluations can analyze the results of your prompts.

-→ You can deploy your prompt as an endpoint for easy integration with your applications.

-→ Use the Logs section to review your prompt’s performance over time.

-→ Refine your prompt and invite team members to your Latitude workspace to collaborate.

Watch this quick demo to learn more.

You can read the docs and the concepts involved like prompts, logs and evaluation that are involved.

You can use this quickstart guide by using the cloud version or self-hosting it.

They have 536 stars on GitHub and are growing very fast.

Star Latitude LLM ⭐️

2. LiveKit Agents - build real-time multimodal AI apps.

LiveKit Agents is an end-to-end framework that enables developers to build intelligent, multimodal voice assistants (AI agents) capable of engaging users through voice, video and data channels.

Let me explain in simple words.

The Agents framework allows you to build AI-driven server programs that can see, hear and speak in real time. Your agent connects with end-user devices through a LiveKit session. During that session, your agent can process text, audio, images or video streaming from a user's device and have an AI model generate any combination of those same modalities as output and stream them back to the user.

✅ They support a lot of SDKs including Swift, Android, Flutter, Rust, Unity, Node, Go, PHP, React and more.

You can get started with pip.

pip install livekit-agents

They also have a lot of plugins that make it easy to process streaming input or generate output. For instance, there are plugins for converting text-to-speech or running inference with popular LLMs. One example of a plugin is:

pip install livekit-plugins-openai

They also provide open source React components and examples for building with LiveKit if you're using React.

You can read the docs and see the list of all plugins that are available. If you want to try, then you can do it at cloud.livekit.io.

If you're looking for some sample apps with code, check these:

⚡ A basic voice agent using a pipeline of STT, LLM, and TTS

⚡ Super fast voice agent using Cerebras hosted Llama 3.1

This is one of the most exciting projects out of 1000+ projects that I have ever seen in open source.

They have 3.2k stars on GitHub and are growing strong.

Star LiveKit Agents ⭐️

3. Julep - build stateful AI apps.

Julep is a platform for creating AI agents that remember past interactions and can perform complex tasks.

Imagine you want to build an AI agent that can do more than just answer simple questions. It needs to handle complex tasks, remember past interactions and maybe even use other tools or APIs. That's exactly where Julep comes in.

You can use it to create multi-step tasks incorporating decision-making, loops, parallel processing and a whole lot more.

✅ It will automatically retry failed steps, resend messages, and generally keep your tasks running smoothly.

✅ You can use Julep's document store to build a system for retrieving and using your own data.

mental model of julep

To get started, you can use npm or pip.

npm install @julep/sdk 

or

pip install julep

There is also a quick example that I recommend reading where a sample Agent selects a topic, generates 100 related search queries, performs the searches simultaneously, summarizes the results and shares the summary on Discord. With proper code :)

Watch this quick demo and check more examples apps to understand more.

You can read the detailed docs which has a Python quickstart guide, Nodejs quickstart guide, tutorials and how-to guides.

You might say it's similar to Langchain but they both have slightly different concepts. For instance, LangChain is great for creating sequences of prompts and managing interactions with AI models. It has a large ecosystem with lots of pre-built integrations, which makes it easier to run things quickly.

Julep on the other hand is more about building persistent AI agents that can remember things. It shines when you need complex tasks that involve multiple steps, decision-making, and integration with various tools or APIs directly within the agent's process. Read more on the detailed comparison.

Julep has 1.3k stars on GitHub and growing strong.

Star Julep ⭐️

4. Open WebUI - most loved AI Interface (Supports Ollama, OpenAI API...), runs offline.

Open WebUI is an awesome user-friendly self-hosted chat user interface designed to operate entirely offline.

This can help you build features at a rate you can never imagine.

You can use pip to quickly install it. Check the complete installation guide.


# install Open WebUI

pip install open-webui

# run Open WebUI

open-webui serve

Let's see some of the awesome features.

✅ You can customize the OpenAI API URL to link with LMStudio, GroqCloud, Mistral, OpenRouter and more.

✅ You can use it in your preferred language with our internationalization (i18n) support.

✅ There is an option of hands-free voice and video call features which gives a little more flexibility.

✅ Their official website has clear info on a bunch of models, prompts, tools and functions by the community.

✅ You can load documents directly into the chat or add files to your document library and access them using the # command before a query.

✅ You can perform web searches using providers like SearXNG, Google PSE, Brave Search, serpstack, serper, Serply, DuckDuckGo, TavilySearch and SearchApi to inject the results directly into your chat experience.

Also recommend watching this walkthrough to learn more.

You can read the docs which includes a getting started guide, FAQs (recommend reading) and tutorials.

It is built using Svelte, Python and TypeScript.

They have 41.6k stars on GitHub which says a lot about the popularity.

Star Open WebUI ⭐️

5. Quivr - RAG framework for building GenAI second brains.

Quivr, your second brain, utilizes the power of GenerativeAI to be your personal assistant. You can think of it as Obsidian but turbocharged with AI powers.

It is a platform that helps you create AI assistants, referred to as Brain. These assistants are designed with specialized cases like some can connect to specific data sources, allowing users to interact directly with the data.

While others serve as specialized tools for particular use cases, powered by Rag technology. These tools process specific inputs to generate practical outputs, such as summaries, translations and more.

Watch a quick demo of Quivr!

Some of the amazing features are:

✅ You can choose the type of Brain you want to use, based on the data source you wish to interact with.

✅ They also provide a powerful feature to share your brain with others. This can be done by sharing with individuals via their emails and assigning them specific rights.

✅ Quivr works offline, so you can access your data anytime, anywhere.

✅ You can access and continue your past conversations with your brains.

✅ But the best one that I loved is that you can install a Slack bot. Refer to this demo to see what you can do. Very cool!

Anyway, read about all the awesome stuff that you can do with Quivr.

You can read the installation guide and 60 seconds installation video. Refer to the docs for any other information.

They have also provided guides on how to deploy Quivr with Vercel, Porter, AWS and Digital Ocean.

It has 36.3k+ Stars on GitHub with 300+ releases.

Star Quivr ⭐️

6. Dify - innovation engine for GenAI apps.

Dify is an open-source platform for building AI applications.

Its intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.

They combine Backend-as-a-Service and LLMOps to improve the development of generative AI solutions. You can use the cloud or self-host it (refer to docs).

You can even build and test powerful AI workflows on a visual canvas.

Let's see some of the awesome features:

✅ Dify provides 50+ built-in tools for AI agents, such as Google Search, DALL·E, Stable Diffusion and WolframAlpha.

✅ You can monitor and analyze application logs and performance over time.

✅ You can use the RAG pipeline to extract text from PDFs, PPTs and other common document formats.

✅ A lot of integration options are available from dozens of inference providers and self-hosted solutions, covering GPT, Mistral, Llama3, and any OpenAI API-compatible models. A full list of supported model providers can be found here.

✅ You can create AI Agents with just a few clicks, letting them independently use enterprise-defined tools and data to solve complex tasks.

You can read the docs.

Two of the impressive use cases that I loved:

⚡ Building a Notion AI Assistant

⚡ Create a MidJourney Prompt Bot with Dify

Dify has 47.7k stars on GitHub and has a lot of contributors.

Star Dify ⭐️

7. Micro Agent - AI agent that writes (actually useful) code for you.

AI-assisted coding tools like GitHub Copilot and ChatGPT don't produce very reliable code and they often don't work correctly right out of the box, you find bugs, edge cases, or even references to non-existent APIs.

This can lead to a frustrating loop of trying the generated code, finding issues, going back to the AI for fixes and repeating.

The time spent debugging can negate the total time saved using AI tools in the first place.

Micro Agent uses AI to mitigate the problems of unreliable code generation.

Give it a prompt and it'll generate a test and then iterate on code until all test cases pass.

You can install it using this command.

npm install -g @builder.io/micro-agent

# Next, set your OpenAI API key when prompted or manually using this.
micro-agent config set OPENAI_KEY=<your token>

# Then you can run to start a new coding task
micro-agent

Micro Agent will prompt you to describe the function you want, generate tests, and start writing code in your preferred language to make the tests pass. Once all the tests are green, you'll have a fully functional, test-backed function ready to use.

Let's explore some of the most mind blowing use cases:

⚡ 30-second demo of Micro Agent generating tests and code for a TypeScript function that groups anagrams together from an array of strings.

⚡ Using Micro Agent to generate a simple HTML to AST parser (it was achieved on two iterations).

⚡ Unit test matching.

⚡ Visual matching (experimental).

⚡ Integration with Figma.

Micro Agent can also integrate with Visual Copilot to connect directly with Figma to ensure the highest fidelity possible design to code!

Visual Copilot connects directly to Figma to assist with pixel-perfect conversion, exact design token mapping, and precise usage of your components in the generated output.

Then, Micro Agent can take the output of Visual Copilot and make final adjustments to the code to ensure it passes TSC, lint, tests, and fully matches your design including final tweaks. Amazing right :)

You can read the docs and the official blog where the team discussed everything about the micro agent.

It's open source with 2.8k stars on GitHub.

Star Micro Agent ⭐️

8. Cline - autonomous coding agent right in your IDE.

The concept seems very similar to Cursor where Cline is an autonomous coding agent capable of creating/editing files, executing commands, and more with your permission every step of the way.

It's a VSCode extension and you can find it in the marketplace. It has 84k+ installs.

Cline works on Claude 3.5 Sonnet's agentic coding capabilities.

✅ Cline supports API providers like OpenRouter, Anthropic, OpenAI, Google Gemini, AWS Bedrock, Azure, and GCP Vertex. You can also configure any OpenAI compatible API or use a local model through Ollama.

✅ You can add context using four different commands.

@url: Paste in a URL for the extension to fetch and convert to markdown, useful when you want to give Cline the latest docs
@problems: Add workspace errors and warnings ('Problems' panel) for Cline to fix.

✅ It uses a headless browser to inspect any website, like localhost, allowing it to capture screenshots and console logs. This gives him the autonomy to fix visual bugs and runtime issues without you needing to handhold and copy-pasting error logs yourself.

✅ You can even run commands in the terminal to do awesome stuff.

You can read the docs.

Cline has 7k stars on GitHub.

Star Cline ⭐️

9. GPT Crawler - create your own custom GPT from a URL.

With GPT Crawler, you can crawl any site to generate knowledge files to make your own custom GPT from one or multiple URLs.

The objective is to make the docs site interactive, people can more simply find the answers they are looking for using a chat interface.

Watch this quick demo!

You will have to configure the crawler and then simply run it. After the crawl is complete, you will have a new output.json file, which includes the title, URL and extracted text from all the crawled pages.

You can now upload this directly to ChatGPT by creating a new GPT. Once uploaded, this GPT assistant will have all the information from those docs and be able to answer unlimited questions about them.

It's officially a assistant in ChatGPT.

You can read the docs on how to get started. You can find all the instructions on the official blog.

If you are wondering how Mitosis compiles those components, then watch this quick tutorial.

They have 18.6k stars on GitHub.

Star GPT Crawler ⭐️

10. Composio - production ready toolset for AI Agents.

Composio is the only tool needed to build complex AI automation software. It allows AI models to access third-party tools and applications to automate their interactions with them.

For instance, you can connect GitHub with the GPT model via Composio and automate reviewing PRs, resolving issues, writing test cases and more.

You can automate complex real-world workflows by using 90+ tools and integration options such as GitHub, Jira, Slack and Gmail.

You can also automate actions like sending emails, simulating clicks, placing orders and much more just by adding the OpenAPI spec of your apps to Composio.

This is how you can use this.

# install it
pip install composio-core

# Add a GitHub integration
composio add github

Here is how you can use the GitHub integration to star a repository.

from openai import OpenAI
from composio_openai import ComposioToolSet, App

openai_client = OpenAI(api_key="******OPENAIKEY******")

# Initialise the Composio Tool Set
composio_toolset = ComposioToolSet(api_key="**\\*\\***COMPOSIO_API_KEY**\\*\\***")

## Step 4
# Get GitHub tools that are pre-configured
actions = composio_toolset.get_actions(actions=[Action.GITHUB_ACTIVITY_STAR_REPO_FOR_AUTHENTICATED_USER])

## Step 5
my_task = "Star a repo ComposioHQ/composio on GitHub"

# Create a chat completion request to decide on the action
response = openai_client.chat.completions.create(
model="gpt-4-turbo",
tools=actions, # Passing actions we fetched earlier.
messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": my_task}
  ]
)

You can read the docs and examples.

Composio has 9k stars on GitHub.

Star Composio ⭐️

11. Langflow - low-code app builder for RAG and multi-agent AI apps.

Langflow is designed to make AI development easier in real-world scenarios. It's Python-based and agnostic to any model, API or database.

It is kind of a dynamic graph where each node is an executable unit. You can watch the demo.

✅ You can use dynamic inputs by using curly brackets {}.

✅ You can use the full potential of LLMs by easily fine-tuning them from spreadsheets.

✅ You can go beyond the surface and code your own components.

✅ Higher-level components naturally come from AI building blocks. Store and share your creations.

The integration options are huge so you can build almost anything. They also provide composable building blocks which are like pre-built components that can be combined in numerous ways to create AI apps.

You can read the docs which contains a quickstart guide and playground where you can directly prototype make\ing adjustments and observing different outcomes with models.

Lanflow has 31.2k stars on GitHub.

Star Langflow ⭐️

12. OpenLLM - run LLMs as OpenAI compatible API endpoint in the cloud

OpenLLM lets developers run any open-source LLMs as OpenAI-compatible API endpoints with a single command.

⚡ Support llama3, qwen2, gemma and many quantized versions full list.
⚡ OpenAI-compatible API & includes ChatGPT like UI.
⚡ Accelerated LLM decoding with state-of-the-art inference backends.
⚡ Ready for enterprise-grade cloud deployment (Kubernetes, Docker, and BentoCloud).

Get started with the following command.

pip install openllm  # or pip3 install openllm
openllm hello

OpenLLM provides a chat user interface (UI) at the /chat endpoint for an LLM server. You can visit the chat UI at http://localhost:3000/chat and start different conversations with the model.

As I said before, OpenLLM supports LLM cloud deployment via BentoML, the unified model serving framework and BentoCloud, an AI inference platform for enterprise AI teams.

If you don't know, BentoCloud provides a fully managed infrastructure optimized for LLM inference with autoscaling, model orchestration, observability, which is just a fancy way of saying it allows you to run any AI model in the cloud.

Once the deployment is complete, you can run model inference on the BentoCloud console:

You can read about the supported models and how to start the LLM server.

Explore docs as you can also chat with a model in the CLI using openllm run and specifying model version - openllm run llama3:8b.

If you love exploring walkthroughs, watch this demo by Matthew!

They have 9.9k stars on GitHub, almost hitting the mark of 10k :)

Star OpenLLM ⭐️

13. GPT Engineer - AI builds what you ask.

GPT-engineer lets you specify software in natural language, sit back, and watch as an AI writes and executes the code, and you can ask the AI to implement improvements.

It's safe to say it's an engineer who doesn't need a degree 😅

It's a commercial project for the automatic generation of web apps. It features a UI for non-technical users connected to a git-controlled codebase.

I know this feels confusing, so watch the below demo to understand how you can use GPT Engineer.

You can get started by installing the stable release using this command.

python -m pip install gpt-engineer

By default, gpt-engineer expects text input via a prompt file. It can also accept image inputs for vision-capable models. This can be useful for adding UX or architecture diagrams as additional context for GPT engineer. Read about all the awesome features.

If you want a complete walkthrough, watch this awesome demo by David!

I recommend checking out the roadmap to understand the overall vision.

They have 52.2k stars on GitHub and are on the v0.3 release.

Star GPT Engineer ⭐️

14. Void - open source alternative to cursor.

As you know, a lot of projects like PearAI and Zed came out after Cursor launched, but most of these are just forks of VSCode and don’t really improve the ecosystem that much.

If you're looking to switch, I’d recommend Cursor (not open source), Continue (similar to Cursor) and Void which is an open source alternative to Cursor.

I'm covering Void because it's backed by YCombinator so it makes them credible enough. They have not released it but you can easily get the early access to try it out.

It has very similar AI syntax features as Cursor.

✅ You can do intelligent searches With AI.

✅ You can view and edit underlying prompts.

✅ There is contextual awareness, third party integrations, Fine-Tuned generation and you can even host Ollama locally to never run out of API credits again.

As I said previously, Void is a fork of VS Code so you can transfer over all your themes, keybinds and settings with just one click.

I've covered Continue before so skipped it this time, but I would definitely prefer Cursor until Void is completely released.

Void has 7.5k stars on GitHub and is still in its early release.

Star Void ⭐️

15. Unsloth - Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory.

Unsloth makes finetuning large language models like Llama-3, Mistral, Phi-3, and Gemma 2x faster, use 70% less memory, and with no degradation in accuracy!

✅ What is finetuning?

If we want a language model to learn a new skill, a new language, some new programming language, or simply want the language model to learn how to follow and answer instructions like how ChatGPT functions, we do finetuning!

Finetuning is the process of updating the actual brains of the language model through some process called back-propagation. But, finetuning can get very slow and very resource intensive.

Unsloth can be installed locally or through another GPU service like Google Colab. Most use Unsloth through the interface Google Colab which provides a free GPU to train with.

Some of the things that stand out:

✅ Open source trains 5x faster, and the pro version claims to be 30x faster.

✅ No approximation methods are used resulting in a 0% loss in accuracy.

✅ No change of hardware, Works on Linux and Windows via WSL.

You can read the installation instructions and performance benchmarking tables on the website.

You can read the docs and all the uploaded models on Hugging Face directly.

They have also provided a detailed guide on How to Finetune Llama-3 and Export to Ollama.

Unsloth has 16.8k+ stars on GitHub.

Star Unsloth ⭐️

16. Khoj - Your AI second brain.

Khoj is the open source, AI copilot for search. Easily get answers without having to sift through online results or your own notes.

Khoj can understand your Word, PDF, org-mode, markdown, plaintext files, GitHub projects and even Notion pages.

It's available as a Desktop app, Emacs package, Obsidian plugin, Web app and Whatsapp AI. Obsidian with Khoj might be the most powerful combo!

You can get started with Khoj locally in a few minutes with the following commands.

$ pip install khoj-assistant
$ khoj

Watch it in action!

Some of the exciting features:

✅ You can share your notes and documents to extend your digital brain.

✅ Your AI agents have access to the internet, allowing you to incorporate real-time information.

✅ You'll get a fast, accurate semantic search on top of your docs.

✅ Your agents can create deeply personal images and understand your speech.

For instance, saying: "Create a picture of my dream house, based on my interests". It will draw this!

Read all the features including shareable chat, online chat, file summarization, and complete details in various categories.

You can read the docs and you can try Khoj Cloud to try it quickly.

Watch the complete walkthrough on YouTube!

It has 12.8k stars on GitHub and is backed by YCombinator.

Star Khoj ⭐️

17. Prompt Tools - tools for prompt testing.

This project has a collection of open source self-hostable tools for experimenting with, testing, and evaluating LLMs, vector databases and prompts. The core idea is to enable developers to evaluate using familiar interfaces like code, notebooks and a local playground.

In just a few lines of code, you can test your prompts and parameters across different models (whether you are using OpenAI, Anthropic or LLaMA models). You can even evaluate the retrieval accuracy of vector databases.

Get started with pip.

pip install prompttools

They provide notebook examples which you can run.

You can read the docs amd check their playground.

Prompt tools have 2.7k stars on GitHub.

Star Prompt Tools ⭐️

With the right tools, building AI features is much easier and your team can deliver quality work faster.

Do you think these projects will help teams build AI features at least faster than before?

Have a great day! Till next time.

If you loved this, please follow me for more :)

Follow Latitude for more content like this.

Latitude Follow

Latitude is the open-source LLM development platform. Build, test, and deploy your LLM-based features with confidence.

17 LLM Tools That Almost Broke The Internet ⚡

Anmol Baranwal — Thu, 19 Sep 2024 13:08:41 +0000

AI is not going down anytime soon especially because of free powerful LLMs like GPT-4o and Claude.

A lot of startups are already using it to build cool products and reach massive audiences.

Today, I'm covering 17 powerful LLM tools that went viral after launch. These examples might give you some ideas for your own projects.

Let's jump in.

By the way, I’m part of Latitude and we’re building an open source prompt engineering platform. You can check our public GitHup repo.

You would be able to do a lot of cool stuff like:

⚡ Create and iterate prompt in the platform.
⚡ Run evaluations to check your prompts with LLMs.
⚡ Optimize the output of your LLM-based features automatically.

I'm very confident that you will love it after release!

Star Latitude ⭐️

1. GPT Crawler - create your own custom GPT from a URL.

With GPT Crawler, you can crawl any site to generate knowledge files to make your own custom GPT from one or multiple URLs.

The objective is to make the docs site interactive, people can more simply find the answers they are looking for using a chat interface.

Watch this quick demo!

It's officially a assistant in ChatGPT.

You can read the docs on how to get started. You can find all the instructions on the official blog.

If you are wondering how Mitosis compiles those components, then watch this quick tutorial.

They have 18.4k stars on GitHub.

Star GPT Crawler ⭐️

2. Personal AI - your personal sweet AI.

PI is an AI chatbot (created by Inflection AI) and is built to provide helpful, engaging and fun conversations. I loved the onboarding section!

I have used GPT and Gemini but Pi seems more human-like. You can have conversations about yourself and general things in life, that you would do with a friend.

You can also ask it to convey in voice. The fact I love it because of how calm the User Interface is, it feels so peaceful and you can do all sorts of stuff.

They also offer a desktop app that is free to download.

You can watch the video if you want (AI voice).

It's not open source but very powerful.

Visit Personal AI 🔥

3. Vanna - Chat with your SQL database.

Vanna is an MIT-licensed open-source Python RAG (Retrieval-Augmented Generation) framework for SQL generation.

Basically, it's a Python package that uses retrieval augmentation to help you generate accurate SQL queries for your database using LLMs.

It's perfect for developers like me who are not fond of SQL queries!

Vanna works in two easy steps - train a RAG model on your data, and then ask questions that will return SQL queries that can be set up to run on your database automatically.

You don't need to know how this whole stuff works to use it.
You just have to train a model, which stores some metadata, and then use it to ask questions.

Get started with the following command.

pip install vanna

To make things a little easier, they have built user interfaces that you can use as a starting point for your own custom interface. Find all the interfaces including Jupyter Notebook and Flask.

You can read the docs and you can try this Colab notebook in case you want to see how it works after training.

Watch this demo for a complete walkthrough!

They have 9.5k+ stars on GitHub and are built using Python.

Star Vanna ⭐️

4. AI Dungeon - text-based adventure-story game you direct while the AI brings it to life.

AI Dungeon is one of the coolest use case of LLM that I have seen so far. It's creative and has a lot of options.

There are no rules or objectives since they are only focused on adventure.

AI Dungeon is a text-based, AI generated fantasy simulation with infinite possibilities. Unlike most games where you experience worlds created by game designers, with AI Dungeon, you can direct the AI to create worlds, characters and scenarios for your character to interact with.

Watch the official trailer :)

You could lead an army fighting back an alien invasion or be a cool detective investigating the assassination of the queen.

You can discover 200+ playable scenarios at play.aidungeon.com/discover.

If you are not aware, Scenarios are basically templates that players can use to start a new Adventure.

You can filter them based on your choice like playable/readable, multiple choice or 3rd person and so many more.

They have created a complete guidebook to answer all your queries.

You can find the official app on platstore and apple store. They have 1M+ downloads which makes them very credible.

You can even improve this by using tools like Sora (upcoming launch).

They are not open source but worth checking out.

Visit AI Dungeon 🔥

5. Chat2DB - AI-driven data management platform.

Chat2DB is an AI-first data management, development, and analysis tool.

Its core is AIGC (Artificial Intelligence Generation Code), which can convert natural language into SQL, SQL into natural language, and automatically generate reports, taking efficiency to another level.

Even operations that do not understand SQL can use it to quickly query business data and generate reports.

For instance, when you are doing database development, it will help you generate SQL directly in natural language, SQL optimization suggestions, analyze SQL performance, SQL execution plan, and can also help you quickly generate SQL test data, system code and more. Lots of jargon but it's actually very powerful :)

They have excellent support for multi-data sources and can easily integrate up to 17 different database types including PostgreSQL, MySQL, MongoDB, and Redis.

You can download or try it in the browser.

Let's see some of the exciting features:

⚡ Intelligent reports.

⚡ Data Exploration.

⚡ SQL Development.

You can read the quickstart guide on the official docs.

They have 14k+ stars on GitHub and are on the release v3.2.

Star Chat2DB ⭐️

6. Mem - magic notes app.

Mem is the AI notes app that keeps you organized.

You can use their AI assistant to summarize insights or draft content based on their notes. There is also a concept of automated tagging.

It's not free (a major drawback) but you can fill out a form for huge discounts.

This is how the dashboard looks. The search feature is very handy and there are lots of useful features.

Let's see what else we can do with Mem.

⚡ You can use Mem Chat to write content by using a deep understanding of your notes.

⚡ You can use AI to create new collections.

⚡ You can pick things from your favorite apps.

⚡ Search beyond your keywords.

I have attached a tutorial to help you on how to use Mem.

It's not open source but you can find everything on their blogs.

Visit Mem 🔥

7. Character AI - chat with AI characters as if they were humans.

Character AI is a full-stack AI tool that lets users personalize their experience by interacting with AI Characters.

You can chat with those virtual characters as if they were humans.

There are many filters and personas that you can try. Their community is very huge with around 300k+ members.

You can create a character with a simple form.

And of course, add a custom persona if the existing one isn't up to your requirements.

I found a funny official video to show you behind the scenes.

You should explore it yourself to get the overall idea.

It's not open source.

Visit Character AI 🔥

8. Prompt Perfect - optimize your prompts.

It's a tool to optimize your prompt. Using LLM to optimize the input that is again going to use LLM is kind of fancy.

PromptPerfect is a prompt optimization tool designed to enhance the quality of large language models and large model results. To use PromptPerfect, you need to input a prompt and customize various settings according to their needs.

These settings include the maximum prompt length, the number of iterations to be optimized, and the quality of the output. It's a useful one if you use models like GPT-4, Claude and Midjourney.

Some of the exciting features are:

⚡ Concept of auto-tune.

⚡ Convert to precise prompts.

⚡ Improve content based on your target audience.

⚡ Also make images.

There is a whole lot of stuff that you can do. Just explore all the features.

It has a free tier plan of 10 requests per day but it's not open source.

Visit Prompt Perfect 🔥

9. Heygen - create awesome videos using AI.

With Heygen, you can create production-ready videos from text in seconds with AI-generated avatars and voices.

HeyGen does multiple things related to video generation and is a leader in generative AI for video production purposes. As such, it sports a great AI Avatar generator, AI-powered Text-to-Speech, and AI voice cloner. These tools are wrapped in a clean User Interface that lets creators get more done in less.

It works in three simple steps.

Watch the quick demo!

It is present in a lot of languages.

They also provide ChatGPT script writer that you can use to create videos faster.

They recently received $60M funding so they're growing fast.

They're not open source but are definitely worth checking out.

Visit Heygen 🔥

10. Micro Agent - AI agent that writes (actually useful) code for you.

This can lead to a frustrating loop of trying the generated code, finding issues, going back to the AI for fixes and repeating.

The time spent debugging can negate the total time saved using AI tools in the first place.

Micro Agent uses AI to mitigate the problems of unreliable code generation.

Give it a prompt and it'll generate a test and then iterate on code until all test cases pass.

You can install it using this command.

npm install -g @builder.io/micro-agent

# Next, set your OpenAI API key when prompted or manually using this.
micro-agent config set OPENAI_KEY=<your token>

# Then you can run to start a new coding task
micro-agent

Let's explore some of the most mind blowing use cases:

⚡ 30-second demo of Micro Agent generating tests and code for a TypeScript function that groups anagrams together from an array of strings.

⚡ Using Micro Agent to generate a simple HTML to AST parser (it was achieved on two iterations).

⚡ Unit test matching.

⚡ Visual matching (experimental).

⚡ Integration with Figma.

Micro Agent can also integrate with Visual Copilot to connect directly with Figma to ensure the highest fidelity possible design to code!

Visual Copilot connects directly to Figma to assist with pixel-perfect conversion, exact design token mapping, and precise usage of your components in the generated output.

You can read the docs and the official blog where the team discussed everything about the micro agent.

It's open source with 2.1k stars on GitHub.

Star Micro Agent ⭐️

11. EditGPT - improve your writing with a powerful AI editor.

EditGPT helps you proofread, edit and improve your writing with a powerful AI editor.

There is also an editGPT extension that works directly on the ChatGPT website. It lets you see the difference between your input and ChatGPT's output.

This process doesn't allow you to edit text directly and is a bit of a hassle if you want to resubmit parts you don't like. The new editor allows you to edit text directly and you can even save your prompts.

There are a lot of options available to improve your message clearly.

There is a free tier available for 10k words per month but it's limited to 1k words per request.

It's not open source but they do support 20+ languages.

Visit Edit GPT 🔥

12. Fireflies - Automate your meeting notes.

As a developer, I'm not fond of meetings but if you attend too many meetings then this is for you!

Fireflies help your team transcribe, summarize, search, and analyze voice conversations. Automate your meeting notes!

Watch this quick demo!

They provide an insane amount of useful features:

⚡ Get automatic meeting notes, action items, & transcripts.

⚡ Find anything with AI-powered search.

⚡ Collaborate with your co-workers.

⚡ Automate workflows from meetings.

⚡ A real-time knowledge base for your entire team

Plus, you can capture & transcribe meetings in multiple ways.

There are a lot of features, but the AI ones are only included in premium plans.

The good thing is that their plans are affordable and they offer a free tier plan with 800mins/storage which is more than enough.

Visit Fireflies 🔥

13. Web ChatGPT - ChatGPT with internet access.

This leads to ChatGPT prompts with relevant web search results through web browsing. This means it adds relevant web results to your prompts to ChatGPT for more accurate and up-to-date conversations.

You can find the extension on Chrome, Edge and Firefox.

Watch the quick demo!

It has more than 1.5M users for three simple features:

⚡ Get web results for your queries.

⚡ Also provides a free one-click ChatGPT prompt library with hundreds of high-quality prompts.

⚡ Get AI-powered search answers right beside your search results by using ChatGPT, Claude, Bard and Bing AI.

It's not open source but has an excellent use case.

Visit Web ChatGPT 🔥

14. Private GPT - ask questions about your documents without the internet.

PrivateGPT is a production-ready AI project that allows you to ask questions about your documents using the power of Large Language Models (LLMs), even in scenarios without an internet connection.

100% private meaning no data leaves your execution environment at any point.

The API is divided into two logical blocks:

a. High-level API, which abstracts all the complexity of a RAG (Retrieval Augmented Generation) pipeline implementation:

Ingestion of documents: internally managing document parsing, splitting, metadata extraction, embedding generation, and storage.
Chat & Completions using context from ingested documents: abstracting the retrieval of context, the prompt engineering, and the response generation.

b. Low-level API, which allows advanced users to implement their complex pipelines:

Embeddings generation: based on a piece of text.
Contextual chunks retrieval: given a query, returns the most relevant chunks of text from the ingested documents.

You can read the installation guide to get started.

You can read the docs and the detailed architecture that is involved.

PrivateGPT is now evolving towards becoming a gateway to generative AI models and primitives, including completions, document ingestion, RAG pipelines, and other low-level building blocks.

They have 53k+ stars on GitHub and evolving at a rapid pace.

Star Private GPT ⭐️

15. Perplexica - AI-powered search engine.

You would have heard about Perplexity AI somewhere along your journey, Perplexica is an open source alternative to that!

Perplexica is an AI-powered search tool or an AI-powered search engine that goes deep into the internet to find answers. It not only searches the web but also understands your questions.

It uses advanced machine learning algorithms like similarity searching and embeddings to refine results and provides clear answers with sources cited.

Using SearxNG to stay current and fully open source, Perplexica ensures you always get the most up-to-date information without compromising your privacy.

If you're wondering about the architecture, you can read about the key components.

I also recommend reading the more detailed explanation of how these components work together. They've explained it clearly by taking an example of a scenario where a user asks: How does an A.C. work?.

Some of the features are:

⚡ Some search tools might give you outdated info because they use data from crawling bots convert them into embeddings and store them in an index. Unlike them, Perplexica uses SearxNG, a metasearch engine to get the results and rerank and get the most relevant source out of it, ensuring you always get the latest information without the overhead of daily data updates.

⚡ You can make use of local LLMs such as Llama3 and Mixtral using Ollama.

⚡ It has a copilot mode to search instead of just using the context by SearxNG, it visits the top matches and tries to find relevant sources to the user's query directly from the page.

⚡ Perplexica currently has 6 focus modes:

YouTube Search Mode.
Wolfram Alpha Search Mode.
Reddit Search Mode.
Academic Search Mode.
Writing Assistant Mode.
All Mode (searches the entire web).

You can watch the complete walkthrough on YouTube!

They have 12.5k+ stars on GitHub and are on the v1.8 release.

Star Perplexica ⭐️

16. Email Triager - writing emails using AI in your style.

EmailTriager automatically learns how your email and pre-drafts are ready to send replies for you in the background. Watch the demo attached below!

It automatically trains on your past emails and drafts show up directly in Gmail if you're worried about changing your email workflow.

As I said earlier, EmailTriager is the only email assistant that actually learns to triage and write emails like you.

This is how it would show as a result of the old way vs the new way.

It's not open source and there is no free tier which is a major drawback. But $10/month is worth it in my opinion if you write a lot of emails.

Visit Email Triager 🔥

17. Udio - generate music using AI.

Udio can help you create music from simple text prompts by specifying topics, genres and other descriptors, then watch them transform into high-quality tracks.

Watch this demo and listen to some impressive music. I loved it! Plug in your earphones :)

Once you have created your masterpiece (or just a song you like), publish it on Udio for everyone to enjoy.

You can, of course, export your creations to a DAW or upload your own audio to experience Udio’s fantastic AI-editing tools.

This is how the dashboard looks.

They also provide a step-by-step tutorial. To get more credits, you just need to watch 10s music and let them know which you prefer. A win-win situation for both of us unless the song is bad.

If you want a YouTube tutorial, then I recommend watching this!

It's not open source but I'm already loving the songs :)

Visit Udio 🔥

Did you find anything good enough?

It took me a long time to research, so let me know if you have any awesome ideas or tools that you want to share.

AI is already changing the world, developers need to adapt as well.

Have a great day! Till next time.

If you loved this, please follow me for more :)

Follow Latitude for more content like this.

Latitude Follow

Latitude is the open-source LLM development platform. Build, test, and deploy your LLM-based features with confidence.

Building an AI-powered quiz application with Next.js and OpenAI🧙✨

Arindam Majumder — Wed, 21 Aug 2024 13:58:34 +0000

In this tutorial, you'll learn how to build an AI-powered quiz application that enables users to select a topic, answer questions related to that topic, and receive their score instantly upon completing the quiz.

The questions for this quiz will be dynamically generated using the OpenAI API by providing specific prompts that return the questions in a desired JSON format. While building this application, you'll also learn how to integrate OpenAI with your software applications.

What is Latitude?

Latitude AI is an open-source prompt engineering platform that allows you to easily build, test, and deploy features powered by large language models (LLMs). This platform empowers teams to create highly functional and intelligent AI applications.

You can learn more about Latitude AI by joining our waiting list. Feel free to connect with the team and discover how we solve various challenges using AI.

Join Waitlist ⭐️

Building the quiz application with Next.js

In this section, you'll learn how to build the interface for the quiz application. The application is divided into three pages: Home Page, Test Page, and Score Page.

The Homepage displays all the available topics. The Test Page renders a question and provides a list of options for users to select the correct answer. Lastly, the Score Page displays the user's score.

Create a new Next.js Typescript project by running the code snippet below:

npx create-next-app ai-quiz-app

Add a types.d.ts file to the root of the project to define the data structure for the application's quiz questions.

interface Question {
    question: string;
    options: string[];
    answer: string;
    id: number;
}

Next, create a lib folder containing a util.ts file within the Next.js app folder:

//👇🏻 topics list
export const firstTopics = [
    { id: "AI", name: "AI Questions" },
    { id: "Python", name: "Python Questions" },
    { id: "JavaScript", name: "JavaScript Questions" },
];

//👇🏻 topics list
export const secondTopics = [
    { id: "CSS", name: "CSS Questions" },
    { id: "HTML", name: "HTML Questions" },
    { id: "UI Design", name: "UI Design Questions" },
];
//👇🏻 capitalize the first letter of each word
export const capitalize = (str: string): string => {
    str = str.replace(/%20/g, " ");
    if (str.length === 0) {
        return str;
    }
    return str.charAt(0).toUpperCase() + str.slice(1) + " Questions";
};

The firstTopics and the secondTopics array contains the list of topics available within the application and the capitalize function accepts a string as its parameter and capitalize the first letter of the sentence.

The Home Page

Copy the code snippet below into the app/page.tsx file:

"use client";
import { firstTopics, secondTopics } from "./lib/util";
import { useRouter } from "next/navigation";

export default function Home() {
    const router = useRouter();

    const handleConfirmClick = (id: string) => {
        const result = confirm(`Are you sure you want to take the ${id} test?`);
        if (result) {
            router.push(`/test/${id}`);
        } else {
            alert(`You have cancelled the ${id} test`);
        }
    };

    return (
        <main className='w-full min-h-screen flex flex-col items-center justify-center'>
            <h2 className='text-4xl font-bold text-blue-600'>Take Tests</h2>
            <p className='text-lg text-gray-500 mb-5'>
                Select a topic, take tests and get your results instantly
            </p>
            <div className='px-4'>
                <section className='w-full flex items-center space-x-5 mb-4'>
                    {firstTopics.map((topic) => (
                        <button
                            key={topic.id}
                            className={`bg-blue-500 text-white px-5 py-3 text-xl rounded-md`}
                            onClick={() => handleConfirmClick(topic.id)}
                        >
                            {topic.name}
                        </button>
                    ))}
                </section>

                <section className='w-full flex items-center space-x-5'>
                    {secondTopics.map((topic) => (
                        <button
                            key={topic.id}
                            className={`bg-blue-500 text-white px-5 py-3 text-xl rounded-md`}
                            onClick={() => handleConfirmClick(topic.id)}
                        >
                            {topic.name}
                        </button>
                    ))}
                </section>
            </div>
        </main>
    );
}

The Home page displays all available topics and directs the user to the test page when they click on a topic link.

The Test Page

Create the test page by adding a page.tsx file within a test/[id] directory. Copy the code snippet below into the test/[id]/page.tsx file:

"use client";
import { useParams } from "next/navigation";
import { useCallback, useEffect, useState } from "react";
import { useRouter } from "next/navigation";
import { capitalize } from "@/app/lib/util";

export default function Test() {
    //👇🏻 array of questions
    const [questions, setQuestions] = useState<Question[]>([]);
    //👇🏻 loading state
    const [loading, setLoading] = useState<boolean>(true);
    //👇🏻 total user's score
    const [userScore, setUserScore] = useState<number>(0);
    //👇🏻 tracks each question in the array
    const [count, setCount] = useState<number>(0);
    //👇🏻 holds the quiz topic
    const { id } = useParams<{ id: string }>();
    const router = useRouter();

    const handleSelectAnswer = (selectedAnswer: string) => {
        //👇🏻 Update the score
        setUserScore((prev) =>
            selectedAnswer === questions[count].answer ? prev + 1 : prev
        );

        //👇🏻 Check if it's the last question
        if (count < questions.length - 1) {
            //👇🏻 Move to the next question
            setCount((prev) => prev + 1);
        } else {
            //👇🏻  If it's the last question, navigate to the score page after the score has updated
            setTimeout(() => {
                router.push(
                    "/score?score=" +
                        (selectedAnswer === questions[count].answer
                            ? userScore + 1
                            : userScore)
                );
            }, 0); // 👈🏼 Ensure the score is updated before navigating
        }
    };

    if (loading) {
        return <h3 className='font-semibold text-2xl mb-3'>Loading...</h3>;
    }

    return (
        <main className='w-full min-h-screen p-6 flex flex-col items-center justify-center'>
            <h2 className='font-bold text-3xl mb-4 text-blue-500'>
                {capitalize(id)}
            </h2>
            <h3 className='font-semibold text-2xl mb-3'>
                Question: {count + 1} of {questions.length}
            </h3>

            <h3 className='text-xl mb-4'>{questions[count]?.question}</h3>

            <div className='flex flex-col lg:w-1/3 mb-6'>
                {questions[count]?.options.map((option, index) => (
                    <button
                        className='p-4 bg-[#EEEEEE]  
                rounded-xl mb-6 min-w-[200px] hover:bg-[#EF5A6F] hover:text-white text-lg'
                        key={index}
                        onClick={() => handleSelectAnswer(option)}
                    >
                        {option}
                    </button>
                ))}
            </div>
        </main>
    );
}

From the code snippet above:

The questions state holds all the questions for the selected topic, while the count state is used to navigate through the array of questions, allowing users to answer each one.
The userScore state stores the user's total score after completing the test.
The user's total score is then passed as a parameter to the score page.

The Score Page

Create a score folder containing a page.tsx file within the Next.js app folder and copy the code snippet into the file:

"use client";
import Link from "next/link";
import { useSearchParams } from "next/navigation";

export default function Score() {
    const searchParams = useSearchParams();
    const score = searchParams.get("score");

    if (!score) {
        return (
            <main className='p-4 min-h-screen w-full flex flex-col items-center justify-center'>
                <h2 className='text-2xl font-semibold'>Score</h2>
                <Link href='/' className='bg-blue-500 p-4 text-blue-50 rounded '>
                    Go Home
                </Link>
            </main>
        );
    }

    return (
        <main className='p-4 min-h-screen w-full flex flex-col items-center justify-center'>
            <h2 className='text-2xl font-semibold'>Score</h2>

            <p className='text-lg text-center mb-4'>
                You got {score} out of 10 questions correct.
            </p>

            <h1 className='font-extrabold text-5xl text-blue-500 mb-3'>
                {Number(score) * 10}%
            </h1>

            <Link href='/' className='bg-blue-500 p-4 text-blue-50 rounded '>
                Go Home
            </Link>
        </main>
    );
}

From the code snippet above, the Score page accepts the user's total score and displays the result in percentage.

How to integrate OpenAI in your Next.js application

OpenAI allows us to integrate various large language models (LLMs), such as GPT-3 and GPT-4, into our applications to build intelligent features. These models can perform a wide range of natural language processing tasks, including text generation, translation, summarization, and more. In this section, you'll learn how to generate quiz questions in your desired format using OpenAI.

Before we proceed, visit the OpenAI Developers' Platform and create a new secret key.

Create a .env.local file and copy the your newly created secret key into the file.

OPENAI_API_KEY=<your_API_key>

Install the OpenAI JavaScript SDK by running the following command in your terminal:

npm install openai

Next, let's create an API endpoint to retrieve AI-generated questions from OpenAI based on the topic selected by the user.

Add an api folder with a route.ts file inside the Next.js app directory.

cd app
mkdir api && cd api
touch route.ts

Copy the code snippet below into the api/route.ts file. It accepts a POST request that contains the selected topic from the client.

import { NextRequest, NextResponse } from "next/server";

export async function POST(req: NextRequest) {
    const { topic } = await req.json();

    console.log({ topic }); 👉🏻 // topic is JavaScript, UI Design, etc

    return NextResponse.json({ message: "Fetch complete" }, { status: 200 });
}

Within the test page, add a useEffect hook that sends a POST request to the API endpoint and returns the questions array:

const fetchQuestions = useCallback(async () => {
    const request = await fetch(`/api`, {
        method: "POST",
        headers: {
            "Content-Type": "application/json",
        },
        body: JSON.stringify({ topic: id }),
    });
    const data = await request.json();
    setQuestions(data.questions);
    setLoading(false);
}, [id]);

useEffect(() => {
    fetchQuestions();
}, [fetchQuestions]);

Add sample.json file within a libs folder and copy the following code snippet into it:

{
    "questions": [
        {
            "id": 1,
            "question": "What is the capital of France?",
            "options": ["Paris", "London", "Berlin", "Madrid"],
            "answer": "Paris"
        },
        {
            "id" : 2,
            "question": "What is the capital of Germany?",
            "options": ["Paris", "London", "Berlin", "Madrid"],
            "answer": "Berlin"
        }
    ]
}

The sample.json file defines the structure of the questions expected from OpenAI.

Finally, update the API endpoint to generate and return a list of questions in JSON format using an OpenAI LLM.

import { NextRequest, NextResponse } from "next/server";
import sampleQuestions from "@/app/lib/sample.json"
import OpenAI from "openai";

const openai = new OpenAI({
    apiKey: process.env.OPENAI_API_KEY,
});

export async function POST(req: NextRequest) {
    //👇🏻 User's selected topic
    const { topic } = await req.json();

    //👇🏻 AI prompt
        const prompt = `Generate 10 distinct questions on ${topic} and ensure they are in JSON format containing an id, topic which is ${topic}, a question attribute containing the question, an options array of 3 options, and an answer property. Please ensure that the options array is shuffled to ensure that the answer does not retain a single position.
    - Please don't make the answers too obvious and lengthy.
    - Ensure the questions are unique and not repetitive.
    - The questions should not be too simple but intermediate level.
    - Return only the JSON object containing the questions.
    You can use this as a sample: ${JSON.stringify(sampleQuestions)}
    `;

    //👇🏻 Generates the questions
    const completion = await openai.chat.completions.create({
        model: "gpt-3.5-turbo",
        messages: [
            {
                role: "user",
                content: prompt,
            },
        ],
    });

  //👇🏻 Questions result
    const aiQuestions = completion.choices[0].message.content;
    const questions = JSON.parse(aiQuestions!);

    if (questions.questions.length < 10) {
        return NextResponse.json(
            { message: "Error generating questions", questions },
            { status: 400 }
        );
    }
    //👇🏻 Returns the list of questions
    return NextResponse.json(
        { message: "Fetch complete", questions: questions.questions },
        { status: 200 }
    );
}

The code snippet above creates a precisely formatted prompt that generates the required questions using OpenAI's GPT-3 model. The generated questions are subsequently returned.

Congratulations! You’ve completed the project for this tutorial.

Next Steps

So far, you've learnt how to build an AI-generated quiz application. You can improve the application by authenticating users and saving their scores in a database.

With effective prompts, you can leverage AI to create intelligent software applications. Latitude is refining this process to unlock the full potential of AI through prompt engineering.

Want to be among the first to experience the next evolution of AI-powered applications? Join our waiting list to be part of the journey.

Thank you for reading!

7 Steps To Become A Prompt Engineer

Lucas Lima do Nascimento — Wed, 14 Aug 2024 09:40:42 +0000

Introduction

Hello there, dear reader! If you’re looking for an article where you’ll actually find a DOABLE guide for getting somewhere with prompt engineering, you’ve come to the right place!

My name is Lucas and I’ve been a software engineer for over 5 years now.

My journey into the world of software and AI began during my university studies at UFU (Universidade Federal de Uberlândia), where I pursued a Bachelor's Degree in Computer Engineering and actually took part in a Robotics Team called Roboforge — you should probably know that robotics and AI are kind of close fields too — where we actually got first place while competing in one of Latin America’s biggest competition: LARC (which stands for Latin America Robotics Competition, not that creative, I have to say).

Literally me, programming at the competition

Some really awesome robots that were playing soccer there (fully autonomous)

And in this guide, I’ll give my best shot at taking you step-by-step through what you should learn to get a position as a prompt engineer!

Step 1: Understanding the Basics of LLMs

Before diving into prompt engineering in particular, it's important to have a solid foundation in AI. You know it already — first, you have to learn to walk before you learn to run.

So, explore a little bit and understand how LLMs work. You don’t have to know everything, though, but, try to answer at least 3 questions:

What the hell is an LLM?
What types of LLM exist out there?
How do I build an LLM? (You don’t have to build one! Just learn the basics on how it’s done)

Answering these 3 questions above, you’ll probably find yourself with a bunch of new terms added to your vocabulary and, even more than that, with a bunch of new knowledge added to your toolkit.

It also can be motivating to actually see the requirements of a few prompt engineering positions and Latitude has created a platform for you to see those positions easily. You can check that out here:

⭐️ Prompt Engineering Jobs by Latitude ⭐️

Step 2: Learn by Doing: Craft a few prompts

Crafting prompts is an iterative process that involves trial and error. Start by experimenting with simple prompts for personal projects and use cases while gradually moving on to more complex ones. This hands-on approach — which we usually call project-based learning, a really interesting way to learn, which usually leads to ~23% increase in student achievement — will help you understand how AI models interpret and respond to different inputs

No ideas on how to start prompting? No worries, I’ve got you covered! Let’s check a few examples:

Hop on your favorite LLM (and if you don’t have one, feel free to check our comparison guide) and start with a few basic prompts:

Long article text here .............
....................................

Summarize the above article for me.

Or, another useful example:

Interesting topic title you found somewhere

Translate the above topic title to French.

Both of those examples can show how simple and powerful prompt engineering can really be! Feel free to try those out or any others that might suit you.

Step 3: Learn Some Programming Basics

Programming is a critical skill for any AI professional. To become a proficient prompt engineer, you should learn at least the basics of one programming language such as Python or R, that are really relevant to the field.

Besides that, there are numerous benefits that coding skills will give you: from implementing and testing your prompts programmatically to manipulating data effectively. There are numerous online resources and courses available to help you get started with programming (e.g. Python Roadmap — an awesome roadmap to guide you through your journey, or Python Project Ideas for you to learn by doing).

Step 4: Applying prompt engineering to real-world examples

Awesome, so, now, you already know how to create basic prompts and how to code a little. A really good next step would probably be to start checking some real-world scenarios and how they are being handled while retaining information about the users too!

Let’s check one example here as well:

Start by imaging a scenario where the company you’re working for has an e-commerce application.
They’re implementing a new search mechanism with AI in order that users can search with natural language
The AI here should return a JSON with the characteristics the user asked for

If you’re running an e-commerce for clothing, you’ll probably want to search for relevant information to that sector — like the age, in years. For example:

# If the user says:
# Birthday gift for my 18 months old daughter

# Maybe a good return would be:
{
  "gender": "female",
  "occasion": "birthday",
  "age_years": 1.5
}

Now, let’s run this same scenario working for an e-commerce for sports clothing. Now, searching for the same elements might not be adequate.

# If the user says:
# Birthday gift for my 10 years old daughter, she loves skateboarding

# Maybe a better return would be:
{
  "gender": "female",
  "occasion": "birthday",
  "sport_suggestion": "skate",
  "age_years": 10
}

While creating software, the user is always the most important role in the play — from e-commerces to emails, to study companions and code assistants. He is the one who actually pays you, so, always remember to keep that in mind while studying other real-world scenarios.

Step 5: Learn where things go wrong

If you got here, you probably faced a few problems here and there while playing with prompting. From math to sources, to hallucination, there are literally tons of different pitfalls you can end up into if you’re not carefully creating your prompt.

This step here is actually learned by failing, and then, learning (based on how inaccurate the result was) what you can do differently to improve your prompt.

Want source citations and your LLM is generating people and articles on the fly? Try RAG (Retrieval Augmented Generation — that you can learn more about here — and LLMs with access to the internet! They will surely be more precise.

Some really bizarre math is showing up mid-calculation? Honestly, LLMs kind of suck at math (which I, personally, find ironically funny). Maybe you can try out some different LLMs — with more focus on the math side — or a different approach to solving the same problem, running things step-by-step or skipping calculations with LLMs and using general programming to solve that instead.

Step 6: Get Some Fine-Tuning Skills and Improve Reliability

Ok. At this point, you probably know your way around prompting for most of the scenarios you’ll find out there. Unfortunately, as you’ll eventually realize, LLMs tend to be extremely chaotic when not fine-tuned properly, creating varying degrees of “creativity” in your responses.

This could not be a problem, but, generally, it is. Whenever you’re working with responses, we generally want the AI to have a limited range of variance and bias so that we can constrain our answers between the confinement of accepted answers.

If you’re in this step, I recommend you take a look at how LLMs are actually configured, check out temperature and Top P sampling, and check for yourself how changing those values and percentages can alter the variance obtained for your prompts.

Ready to build, test, and deploy some of those prompts? A really cool platform for this is Latitude’s new product, which is currently in early access! Whenever it’s released, I’m sure you’ll find a bunch of cool stuff there that will actually help you in your journey, so check it out!

⭐️ Enter the Waitlist! ⭐️

Step 7: Showcase your work and Start Applying

Hands-on experience is invaluable in the field of prompt engineering and hey, if you’re following along, I’m sure you have tons of that, so, it’s just the time to showcase it to the world!

Don’t forget to make your work tangible to HR persons! They are the ones recruiting and looking for people, so, show both scenarios: technical for the technical people and non-technical for the non-technical people — this means basically: including a variety of prompts and the corresponding AI responses to demonstrate your expertise, translating that response to non-technical terms when needed.

Once you have a solid way to showcase things -- being that from just posts or a portfolio, start applying for prompt engineering positions and it’s just a matter of waiting until you find an awesome place to work.

Kind of lost on where to apply? Latitude’s come to the rescue again — yes, they truly built a whole platform for you to check prompt engineering jobs out, so please, take a look:
⭐️ Prompt Engineering Jobs by Latitude ⭐️

Conclusion

Hey! I’m glad that you got here!

Becoming an engineer is a rewarding journey that involves mastering fundamentals, honing your skills, understanding user needs, and crafting effective products — and this goes for any engineering, from software to prompting.

If you follow those steps, you'll be REALLY well-equipped to enter this exciting field and honestly, we'd love to hear your thoughts and experiences as you embark on this path. Feel free to share your insights in the comments below and don't forget to check out Latitude's new product to further enhance your skills!

Share your LLM-based project! 🤖 🪄

cesar.wtf — Wed, 07 Aug 2024 09:39:51 +0000

Hi there,

I'm César, CEO at Latitude.

A few weeks ago, we started building a product to help teams ship LLM-based products to production with confidence.

After talking to a lot of people building cool things with LLMs, now I’m kinda addicted to seeing new use cases.

I'd love to check out your project if you're willing to share it with the community!

You can follow this simple template. Feel free to add any images or gifs:

Project’s name
Link to the project
Short description of the project

Share it in the comments section below.

And good luck with your project!

Prompt Engineering For Developers: 11 Concepts and Examples 🎯🧙‍♂️⚡

Anmol Baranwal — Thu, 01 Aug 2024 10:00:13 +0000

Prompt engineering is one of the best places to start in this era of AI.

Understanding the core concepts will help you get the most out of generative AI models like ChatGPT, Google Bard and Claude for tasks such as debugging, code translation, and generating tests including any general task.

Today, we will cover all the core concepts and principles with very detailed examples of Prompt Engineering.

Let's jump in.

By the way, I’m part of Latitude and we’re building an open source LLM development platform. You can join the waitlist at ai.latitude.so.

You would be able to do a lot of cool stuff like:

⚡ Deploy prompts as api endpoints.

⚡ Automated evaluations using LLMs.

⚡ Collaborate on prompt engineering.

I'm very confident that you will love it after its release!

Join Waitlist ⭐

What is this new term called Prompt Engineering?

Whenever we try to communicate with ChatGPT or any other conversational AI tool to get a response, that form of text, question or information given as an input is called Prompt.

Prompt engineering is one of the most common words thrown whenever we talk about Generative AI.

Prompt engineering is basically the process of writing, and refining those inputs to optimize the responses you get from these language models.

✅ Let's imagine coffee.

Think about ordering a coffee at a café. If you simply ask for a "coffee", you'll get a standard brew. But if you specify that you want a single-origin espresso with a touch of caramel, you'll get a much more refined coffee based on your needs.

It's the same principle in prompt engineering, as detailed coffee orders lead to better coffee, specific good prompts can produce more relevant and refined AI responses. There is an entire field due to the increasing use cases because everyone is looking to improve their workflow using AI.

This will help the model to perform its tasks better, such as writing marketing emails, generating code, analyzing and synthesizing text, or any of the other hundreds, if not thousands, of current applications.

They aren't human and so they aren't intuitive. Like any machine, they're just good at garbage in, garbage out. Everything is up to us to provide a better brief.

Prompt engineers aren't only responsible for providing guidance and direction to AI language models but also for developing, testing, and refining specific prompts that have already been submitted to the AI model over time.

In 1 sentence, prompt engineering can be summed as If you're looking for better results, just try to ask better questions. After reading this, I'm sure you will get a better idea of what it actually means and how you can get better at asking those questions.

Which type of LLMs do we use?

Before we proceed, it's important to understand there are two main types of LLMs but the most commonly used is:

⚡ Instruction Tuned LLM.

Fine-tuned on instructions and good attempts at following those instructions. In general, we use these whenever we are using any generative AI models.

Reinforcement learning with human feedback is one of the common instances used under this.

-→ Example:

If we ask: Asking What is the capital of India?

a specific reply based on our instruction

Instruction-type LLMs are recommended for most of the tasks and it's helpful, honest (best of their ability) and harmless.

I will be referring to this type when explaining the concepts or further discussions.

1. A good prompt is very specific and follows a particular structure.

Normally, it's okay to just greet and ask a question to a model, for instance saying this: Nice to meet you, can you tell me the console message I would get after typing console.log("Hello World").

But it would normally give bad results and would assume things on its own, which is what we don't want.

It's important to have a good structure for a prompt if you need very accurate results such as in programming tasks.

Let's see how to structure any prompt clearly:

⚡ Intro: Set up the scenario for which you're referring to. It helps to give the AI a distinct role to think of themselves in.

-→ For example, I want you to act as an interviewer. I will be the candidate and you will....

⚡ Context: Any good reference is always up to the work.

-→ For example, you will ask me the interview questions for the frontend developer junior position.

⚡ Instructions: Giving proper instructions is important after the context.

-→ For example, Do not write all the conservation at once. I want you to only do the interview with me. Ask me the questions and wait for my answers. Do not write explanations.

⚡ Ending: You can describe properly what to do.

-→ For instance, Ask me the questions one by one like an interviewer does and wait for my answers. My first sentence is 'Hi'.

So, putting all of the above together, our example prompt would look like this:

✅ I want you to act as an interviewer. I will be the candidate and you will ask me the interview questions for the frontend junior developer position. I want you to only reply as the interviewer. Do not write all the conservation at once. I want you to only do the interview with me. Ask me the questions and wait for my answers. Do not write explanations. Ask me the questions one by one like an interviewer does and wait for my answers. My first sentence is Hi

-→ Another good example of a prompt can be:

✅ I want you to act as a javascript console. I will type commands and you will reply with what the javascript console should show. I want you to only reply with the terminal output inside one unique code block, and nothing else. do not write explanations. do not type commands unless I instruct you to do so. when I need to tell you something in english, I will do so by putting text inside curly brackets {like this}. My first command is console.log("Hello World");.

this works fine

You should try to avoid any small talk and use strict prompts to get better results!

Now, it’s important to structure your prompts, but you don’t need to follow any specific structure (even the above one). You can test based on the initial results you get and improve it further. Sometimes, even less is more!

2. ✅ Active voice ❌ Passive voice.

Active voice is when the subject of a sentence performs the action expressed by the verb. (Subject + Verb + Object)

While Passive voice is when the subject of a sentence is acted upon by the verb. (Object + Verb + Subject)

-→ Example:

Active Voice: The chef cooked the meal.

Passive Voice: The meal was cooked by the chef.

If you're wondering why this is even important? Active voice gives a sense of action and more clarity to the AI model. It would lead to a better response.

Plus, using rhetorical questions within prompts can stimulate the AI model's "thinking process" and guide it toward generating more meaningful outputs.

3. Give the model time to think.

This is one of the most critical concepts that you need to understand if you're planning to get better at prompt engineering.

Let's suppose if asked to multiply 21 by 34, you might not know it immediately, but can still work it out with time. Unless you're a calculator :)

Similarly, models make more reasoning errors when trying to answer right away, rather than taking time to work out an answer. Asking for a chain of thought before an answer can help the model reason its way toward correct answers more reliably.

One of the tactics can be Instruct the model to work out its own solution before rushing to a conclusion.

Sometimes we get better results when we explicitly instruct the model to do the same.

Suppose, we want a model to evaluate a solution to a math problem. The most obvious way to approach this is to simply ask the model if the solution is correct or not.

Problem Statement:

I'm planning to build a solar power installation and need help calculating the financials.
- Land costs $100 per square foot.
- Solar panels cost $250 per square foot.
- The maintenance contract costs a flat $100,000 per year, plus an additional $10 per square foot.
Calculate the total cost for the first year of operation based on the number of square feet.

Student's Solution:

Let x represent the size of the installation in square feet.
1. Land cost: 100x
2. Solar panel cost: 250x
3. Maintenance cost: 100,000 + 10x
Total cost: 100x + 250x + 100,000 + 10x = 360x + 100,000

SYSTEM:

Evaluate whether the student’s solution is correct.

USER:

Determine if the student’s solution to the problem is accurate.

ASSISTANT:

First, let’s break down the solution:
1. Land Cost: 100x
2. Solar Panel Cost: 250x
3. Maintenance Cost: 100,000 + 10x

Combining these:
- Total cost = Land cost + Solar panel cost + Maintenance cost
- Total cost = 100x + 250x + 100,000 + 10x
- Total cost = 360x + 100,000

The student's solution is indeed correct.

Working out its own solution before coming to a conclusion is one of the most underrated techniques that is very powerful and simple!

Other tactics can be to:

🎯 Use a sequence of queries to hide the model's reasoning process.

🎯 Ask the model if it missed anything on previous passes.

Do you know, that there is also a concept of automatic chain of thought? You should read about it in the prompting guide including the stages involved.

4. Avoid prompt injections.

Prompt Injection is the process of overriding original instructions in the prompt with special user input. It often occurs when untrusted input is used as part of the prompt later on.

-→ Let's take another example:

The response generated is: Prompt injection is a technique where users manipulate the input to a language model, aiming to alter its behavior or output. This can potentially lead to unintended or malicious responses, highlighting the importance of robust input validation.

Now, you can just ask it to ignore it and do entirely something else.

It's really hard to distinguish between original developer instructions and user input instructions. But it's possible in certain situations in case of clear input, let's see one case.

Suppose, giving the same input and explicitly asking to not let it override in the next two prompts.

It was not overridden as you can see from the response snapshot below.

As instructed, the injection was possible only after 2 attempts.

There is no foolproof solution but you can adjust it based on your prompt.

5. Few-shot prompting.

Few-shot prompting is a prompt engineering technique that involves showing the AI a few examples (or shots) of the desired results. Using the examples provided, the model learns a specific behavior and gets better at carrying out similar tasks.

More like giving successful examples of completing tasks and then asking the model to perform the task.

-→ For instance, let's see an example.

Your job is to create content for our client, {{client_name}}. Here is some information about the client {{client_description}}.
Here are a few examples of content we've created in the past from briefs:
"""
Example 1:
Brief: {{brief_1}}
Content: {{content_1}}

Example 2:
Brief: {{brief_2}}
Content: {{content_2}}
"""
Here is the latest brief to create content about:
"""
Brief:{{brief_description}}
Content:

By passing along previous briefs and content generated from those briefs, the model will get an understanding of the style of the specific client that we're expecting.

I wrapped the examples in delimiters (three quotation marks) to format the prompt and help the model better understand which part of the prompt is the examples versus the instructions.

Or you can let the model know about the basic task with a few examples like:

Few shot prompting example.
The movie was good // positive
The movie was quite bad // negative
I really like the movie, but the ending was lacking // neutral
I LOVED the movie //

The model will understand and will show the output in lowercase.

While the LLMs are great, they still fall short on more complex tasks when using the zero-shot (discussed in the 7th point). In that case, Few-shot prompting can be used as a technique to enable in-context learning and improve further.

It has its fair share of limitations but works in most of the cases. You can read more about the technique including the limitations.

6. Constraints.

Constraint-based prompting involves adding constraints or conditions to your prompts, helping the language model focus on specific aspects or requirements when generating a response.

It's highly recommended for most interactions with ChatGPT. Clear constraints set boundaries or limitations on the generated response.

You can do it in different ways, such as:

⚡ Specifying that the response should be no longer than a certain word count or character limit.

-→ Let's see an example.

As you can see, it just assumed and gave up a response of 38 words when we allowed it to go up to 50 words.

this is 38 words

Let's give a better prompt by saying close to 50 words.

The response was very close to 50 words which is a better constraint.

this is 47 words

⚡ Specifying response structure.

-→ We are specifying a structure with more strict constraints.

This is the JSON response. It's important to clarify otherwise the model will assume and give text + JSON response separately.

{
  "summary": [
    "Prompt engineering involves developing prompts to get clear and useful responses from AI tools.",
    "It helps guide large language models to perform tasks based on various inputs.",
    "Generative AI tools like ChatGPT offer solutions for conversations, programming help, and automated tasks."
  ]
}

⚡ Provide explicit instructions.

-→ Defining the target audience for the response, such as aimed at software engineers or intended for advocates.

-→ Instruct ChatGPT to avoid certain types of content to make it safe and within ethical guidelines.

You can read more about constraint based prompts on the blog of Andrew Maynard with a few exercises.

7. Zero-shot prompting.

Zero-shot prompting means that the prompt used to interact with the model won't contain examples or demonstrations. The zero-shot prompt directly instructs the model to perform a task without any extra examples.

A fancy way to say "giving simple instructions as a prompt".

-→ Let's see an example.

In this prompt below, we didn't provide the model with any examples of text alongside their classifications, the LLM already understands what we mean by "sentiment".

Classify the text into neutral, negative, or positive. 
Text: I think the vacation is okay.
Sentiment:

response sentiment

Reinforcement Learning Human feedback is also a way to improve zero-shot prompts.

When zero-shot doesn't work, it's recommended to provide demonstrations or examples in the prompt which leads to few-shot prompting.

You can read the difference between Zero-Shot vs. Few-Shot vs. Fine-Tuning by labelbox. There are benchmark results at the end which was a little surprising.

🎯 One-shot prompting.

There is also a concept of One-shot prompting that involves offering a single example or reference output snippet as part of the prompt.

That is useful for generating content consistent with your writing style or for instances where you have a particular reference to follow. You can read more about it through online sources.

8. Chain-of-Thought (CoT).

Chain-of-thought (CoT) prompting encourages the model to break down complex reasoning into a series of intermediate steps, leading to a well-structured final output.

You should know that you can combine a chain of thought prompting with zero-shot prompting by asking the model to perform reasoning steps, which may often produce better output.

That is the smallest form of CoT prompting, zero-shot CoT, where you literally ask the model to think step by step. This approach yields impressive results for mathematical tasks that LLMs otherwise often solve incorrectly.

About one-shot CoT prompting - Image Source is from Wei et al. (2022)

-→ Let's see an example where you can combine it with few-shot prompting to get better results on more complex tasks that require reasoning before responding.

The odd numbers in this group add up to an even number: 4, 8, 9, 15, 12, 2, 1.
A: Adding all the odd numbers (9, 15, 1) gives 25. The answer is False.
The odd numbers in this group add up to an even number: 17,  10, 19, 4, 8, 12, 24.
A: Adding all the odd numbers (17, 19) gives 36. The answer is True.
The odd numbers in this group add up to an even number: 16,  11, 14, 4, 8, 13, 24.
A: Adding all the odd numbers (11, 13) gives 24. The answer is True.
The odd numbers in this group add up to an even number: 17,  9, 10, 12, 13, 4, 2.
A: Adding all the odd numbers (17, 9, 13) gives 39. The answer is False.
The odd numbers in this group add up to an even number: 15, 32, 5, 13, 82, 7, 1. 
A:

This is the response of a perfect result when we provided the reasoning step.

Due to improved models, even a single example might be more than enough to get the same result.

9. Reducing Hallucinations and using delimiters.

🎯 Reducing Hallucinations.

Hallucination or “making things up” is a common failure when any large language model (LLM) generates a response that is either factually incorrect, nonsensical, or disconnected from the input prompt.

It can be due to:

⚡ The model doesn't have proper context and common sense to determine those are inaccurate answers.

⚡ Trying to assume things and being extra helpful in case of being not sure about the correct response.

-→ An example of this would be an AI model designed to generate summaries of articles and end up producing a summary that includes details not present in the original article or even fabricates information entirely.

-→ The other examples can be false negatives (may fail to identify something as being a threat) or false positives(identify something as being a threat when it is not).

There are a lot of techniques that can be used to prevent these:

⚡ Retrieval Augmented Generation (RAG)
⚡ Chain-of-Verification (CoVe) prompting
⚡ Chain-of-Knowledge (CoK) prompting
⚡ ReAct prompting
⚡ Chain-of-Note (CoN) prompting
⚡ Other advanced prompt techniques

I'm not going to explore this because hallucinations aren't really an internal factor to get better at prompt engineering.

While frequent human review of LLM responses and trial-and-error prompt engineering can help you detect and address hallucinations in your application, this approach is extremely time-consuming and difficult to scale as your application grows.

🎯 Using delimiters.

Delimiters like triple quotation marks, XML tags, section titles, etc. can help to identify some of the sections of text to treat differently.

Some of the delimiters can be:

Triple quotes: """
Triple backticks/dashes: ---
Angle brackets: < >
XML Tags: <tag> </tag>

Summarize the text delimited by triple quotes in 3 bullet points.

"""insert text here"""

The example response would be.

You can read more about delimiters with practical examples by real python guide.

10. Refine your prompts since there is no "perfect prompt".

It's very rare that you will end up with a good result on the first try of any prompt. To bridge the gap between desired response and the present output, the prompt needs to be refined as much as possible.

A simple way can be:

⚡ Try something.
⚡ Analyze where the result does not give what you want.
⚡ Clarify instructions, and give more time to think.
⚡ Refine prompts with a batch of examples.

-→ Let's see an example.

Giving this as an input response.

✅ Summarize the text enclosed in brackets into approximately three bullet points and provide that summary in a JSON format.

"""Prompt engineering is the practice of developing prompts that produce clear and useful responses from AI tools. AI prompting can help direct a large language model to execute tasks based on different inputs.
The goal is to teach AI models to provide the best output to a query. If you're keeping up with the latest news in technology, you may already be familiar with the term generative AI or the platform known as ChatGPT—a publicly-available AI tool used for conversations, tips, programming assistance, and even automated solutions. """

It assumed things and gave both JSON & text which is wrong since we only needed the JSON output.

We need to refine the prompt further by specifying that we need a JSON rather than a text description.

✅ Summarize the text delimited by brackets close to 3 bullet points and give the final response as a json rather than a text description.

"""Prompt engineering is the practice of developing prompts that produce clear and useful responses from AI tools. AI prompting can help direct a large language model to execute tasks based on different inputs. The goal is to teach AI models to provide the best output to a query. If you're keeping up with the latest news in technology, you may already be familiar with the term generative AI or the platform known as ChatGPT—a publicly-available AI tool used for conversations, tips, programming assistance, and even automated solutions. """

This time, we got a better response without any unwanted results.

That's the process of iterative development including error analysis!!

🎯 Test changes systematically.

You should know this concept if you're going to further refine your prompt.

Improving performance is easier in cases where you can measure it. In some of these cases, a modification to a prompt will achieve better results on a few isolated examples but could lead to worse overall results on other sets of examples.

To make sure that a change is net positive in performance, it may be necessary to define a comprehensive test suite, also known as an eval.

Examples of good evals can be:

-→ Representative of real-world usage (or at least diverse).
-→ Contains many test cases to evaluate performance changes.
-→ Easy to automate or repeat.

Something like you're giving two prompts to compare their results on which can be better.

Computers can automate evals with objective criteria (such as questions with single correct answers) as well as any fuzzy criteria.

I know it's hard to understand if you're unaware of the concept, you can just refer to online resources or you can sign up for the below waitlist.

Latitude is building an open source LLM development platform. You can join the waitlist at ai.latitude.so.

You would be able to do a lot of cool stuff like automate evaluations using LLMs and collaborate on prompt engineering.

Join Waitlist ⭐

11. Ask the model to adopt the persona.

Have you ever heard the phrase, "fake it till you make it". I know it's controversial but the subject of numerous psychological studies suggests that it improves overall efficiency even though it seems wrong 😅

Much like the psychological concept, instructing an LLM to assume a specific role or persona can drastically increase the performance.

This technique, known as role-playing, allows the LLM to generate more accurate, contextually relevant and persona-consistent responses.

For instance, you can use a prompt like this: (reference)

I want you to act as a linux terminal. I will type commands and you will reply with what the terminal should show. I want you to only reply with the terminal output inside one unique code block, and nothing else. do not write explanations. do not type commands unless I instruct you to do so. When I need to tell you something in English, I will do so by putting text inside curly brackets {like this}. My first command is pwd.

As you can see, it gives a simple powerful response based on the input prompt.

Role-playing prompts can involve assigning specific requirements beyond assigning a role. Like, two journalists share the same profession, but there is a higher chance they might differ in traits such as personality and expertise.

I recommend reading about Role-Playing in Large Language Models like ChatGPT to understand it better.

Conclusion

The only limitation (and a huge one) is that no matter how good your prompt engineering skills are, you are still limited by how good the model itself is. The quality and overall practicality of the model's training data is very crucial.

Plus, it's tougher than ever because we can generate audio, video, or many different types of formats using AI tools so it's really getting complex these days.

One of the best articles that I recommend reading about Prompt Engineering is from Google Cloud and circleci.

Merve Noyan created an exceptional ChatGPT Prompt Generator App, allowing users to generate prompts tailored to their desired persona.

You can find a list of huge tested prompts on awesome ChatGPT prompts. Some of the examples that I have used are from this repo.

It's an art that takes a lot of trial and error, but it's worth it!
Continue to experiment and test until you get the best results.

I hope you enjoyed this, let me know if you have any questions.

Have a great day! Till next time.

You can check my work at anmolbaranwal.com. Thank you for reading! 🥰

Follow Latitude for more content like this.

Latitude Follow

Latitude is the open-source LLM development platform. Build, test, and deploy your LLM-based features with confidence.

The Ultimate LLM Leaderboard: Ranking the Best Language Models

Lucas Lima do Nascimento — Wed, 24 Jul 2024 12:30:00 +0000

Introduction

You’re there, casually scrolling your timeline as a subtle vibration from your phone catches your attention. Another AI LLM is reaching the market and is the new best toy in town that everyone should check.

The first time that happened, you were probably actually intrigued and maybe even went to check it out, but, by the fourth and even fifth time, checking new LLMs and AIs started to become a little tiring.

Want to check which are the best LLMs for each case (API, personal use/prompting, etc), ranked, without having to spend hours searching and comparing each one? This article is for you, then.

Criteria for Ranking

The ranking of language models in this leaderboard is based on a list of criteria made by me, designed to evaluate their overall performance and applicability. These criteria include:

Overview: A small resume about who is the creator of the LLM (and maybe even some polemics you should learn about)
Strengths: Where and in which cases the LLM excels at
Cost: The model's capacity to handle increasing amounts of data and user interactions along with how much would that cost you. Generally, this uses tokens for measurement. Not sure what a token is and how it’s calculated? You can consider it equal to ~4 characters of text for common English.
Ease of Integration/Community Support: How easily the model can be integrated into existing systems and workflows and the availability of resources, documentation, and community engagement to assist us with implementation and usage.

LLM Setup - The quick and easy route

HOLD FAST! Before we continue, if you’d prefer to learn by doing and using those powerful LLMs we have something exciting for you!

Join our waitlist for early access to Latitude’s new platform. It's designed to help LLM developers streamline their work and bring ideas to life with confidence, helping a ton in the measurement process I mentioned above.

We're honestly truly excited about this platform and can't wait to see what you create with it!

Don't miss out! Click the link below to join our waitlist 🚀

⭐️ Enter the Waitlist! ⭐️

Top Language Models

Now, let’s jump right into the LLMs that are most famous today. Of course, I wouldn’t be able to actually review every single one (there are literally thousands of models out there) but, here we’ll check: GPT, Gemini, LLaMA, Claude, and Copilot.

GPT

Overview: GPT (Generative Pre-trained Transformer) is one of the most well-known language models developed by OpenAI. It has a ton of models: 3.5-turbo, 4, 4o (the most recent), and more. In terms of polemics, it’s involved in kind a few of them (e.g. ChatGPT's voice closely resembles Scarlett Johnasson's, says lab analysis : NPR), so, you can also take that into consideration.

Strengths: Exceptional accuracy and versatility in generating human-like text. It has some great API documentation and integration also, while still maintaining a good interface for human interaction.

Prices: Varies depending on usage and subscription plans, the better trained the model, the more expensive it becomes, ranging from $0.5/1M tokens to $5/1M tokens on the most recent model. You can check them all here: Pricing | OpenAI

Looking for image models? It also has, with prices ranging from $0.016/image to $0.040/image. If you’re interested in using it for prompting and normal use instead of integration, it has both a free plan and a Plus plan with different models, image generation, and a price of $20/month.

Gemini

Overview: Gemini is a robust language model known for its efficiency and scalability. It’s made by Google and recently had some really awesome announcements at I/O (a conference made by Google to showcase new stuff). It’s also mentioned in a few polemics (e.g Google pauses Gemini's AI image generation after diversity controversies) that you can take into consideration.

Strengths: Amazing context window (this basically means it can use more data as the font of a single response), great language proficiency too!

Prices: For APIs, pricing based on usage too, having a free tier with low requests number and $0.35 / million tokens, later on. For personal use, Gemini Code Assist, for example, costs $19 per user, per month.

LLaMA

Overview: LLaMA (Large Language Model Meta AI) is designed to be less resource-intensive than other models, it has a different license scheme, which means that this resembles “something open source” and as the other ones, also has a few polemics in its name (e.g. Meta Stops Disclosing the Data It Uses to Train AI Models Like Llama 2 - Business Insider).

Strengths: They try to follow guidelines for responsible AI creation and innovation. It also has different models specific for different scenarios like Code LLaMA, for code generation, for example. It’s great for research and English-focused applications.

Prices: Basically free while using Meta AI or downloading and running it locally/self-host. You can also check out services for using as API, that for example can generate API keys for using LLaMA.

Claude

Overview: Claude is a versatile language model that excels in understanding and generating natural language. It’s a really interesting alternative to GPT, honestly — and, for the first time, I couldn’t find some polemics including it.

Strengths: Seems to perform better than GPT mostly regarding speed and prices, but tries to reach the same strengths, like conversational AI, etc.

Prices: For personal use, it starts out free and grows up to $20 for pro or $30 for teams. For APIs, costs $3/M tokens for input and $15 for output on their most intelligent model to date.

Copilot

Overview: Microsoft Copilot (here, I’ll evaluate both Copilot for Microsoft 365 and GitHub Copilot) and, although this one is not necessarily an LLM, it’s been publicly known that Microsoft is developing an in-house AI LLM — that is probably going to be connected to these products in the future, and, since Windows is used by a ton of people, I’ll also leave my review of them here. And let’s not forget some polemics here too (e.g. Controversial Microsoft AI screenshot feature delayed over security concerns | Science & Tech News | Sky News)

Strengths: Aims to be the best in aiding people on the daily life, from help with emails, to Excel, to code with GitHub Copilot’s suggestions and autocomplete. The strong side here is obviously if you/your company use Microsoft’s services. Outlook, Excel, Word, Teams, and all other products can benefit from here.

Prices: You'll actually find a really cool free tier here for consumption (prompting and images , but not code). But, if you want more of that uniqueness of integrating all Microsoft's products with AI, it's a subscription-based model with various tiers. You can check them all here: GitHub Copilot · Your AI pair programmer and here. Honestly, they both seem kind of expensive with GitHub Copilot seeming a little bit less, but, in company terms, they’re probably not that expensive.

Conclusion

Awesome! So, I hope that throughout this article I provided an in-depth look at the top language models, highlighting their strengths, pricing, polemics, and use cases. With Latitude's tools and insights, developers have the resources they need to choose the best LLM for their projects and optimize their performance, so, make sure to join the waitlist!

We encourage you to explore these models further and take advantage of Latitude's platform to enhance your development process. Don't forget to share your thoughts and experiences in the comments, and join the Latitude community to stay updated on the latest advancements in AI and LLMs.

21 Open Source LLM Projects to Become 10x AI Developer

Anmol Baranwal — Wed, 17 Jul 2024 09:00:56 +0000

The time of AI is high especially because of powerful LLMs like GPT-4o and Claude.

Today, I'm covering 21 open source LLM projects that can help you to build something exciting and integrate AI into your project.

As a developer, I can confidently say that AI is not as scary as others make it sound and those who don't learn will get left behind.

Let's cover it all.

By the way, I'm part of Latitude and we're building an open source LLM development platform. You can join the waitlist at ai.latitude.so.

You would be able to do a lot of cool stuff like:

⚡ Deploy prompts as api endpoints.

⚡ Automated evaluations using LLMs.

⚡ Collaborate on prompt engineering.

I'm very confident that you will love it after its release!

Join Waitlist ⭐️

1. Vanna - Chat with your SQL database.

Vanna is an MIT-licensed open-source Python RAG (Retrieval-Augmented Generation) framework for SQL generation.

Basically, it's a Python package that uses retrieval augmentation to help you generate accurate SQL queries for your database using LLMs.

It's perfect for developers like me who are not very fond of SQL queries!

Vanna works in two easy steps - train a RAG model on your data, and then ask questions that will return SQL queries that can be set up to run on your database automatically.

You don't need to know how this whole stuff works to use it.
You just have to train a model, which stores some metadata, and then use it to ask questions.

Get started with the following command.

pip install vanna

To make things a little easier, they have built user interfaces that you can use as a starting point for your own custom interface. Find all the interfaces including Jupyter Notebook and Flask.

You can read the docs and you can try this Colab notebook in case you want to see how it works after training.

Watch this demo for a complete walkthrough!

They have 9.5k+ stars on GitHub and are built using Python.

Star Vanna ⭐️

2. Khoj - Your AI second brain.

Khoj is the open source, AI copilot for search. Easily get answers without having to sift through online results or your own notes.

For me, the concept seems exciting and It can help me in researching a lot of projects.

Khoj can understand your Word, PDF, org-mode, markdown, plaintext files, GitHub projects, and even Notion pages.

It's available as a Desktop app, Emacs package, Obsidian plugin, Web app, and Whatsapp AI. Obsidian with Khoj might be the most powerful combo!

You can get started with Khoj locally in a few minutes with the following commands.

$ pip install khoj-assistant
$ khoj

Watch it in action!

Some of the exciting features:

✅ You can share your notes and documents to extend your digital brain.

✅ Your AI agents have access to the internet, allowing you to incorporate real-time information.

✅ You'll get a fast, accurate semantic search on top of your docs.

✅ Your agents can create deeply personal images and understand your speech.

For instance, saying: "Create a picture of my dream house, based on my interests". It will draw this!

Read all the features including shareable chat, online chat, file summarization, and complete details in various categories.

You can read the docs and you can try Khoj Cloud to try it quickly.

Watch the complete walkthrough on YouTube!

It has 12k stars on GitHub and is backed by YCombinator.

Star Khoj ⭐️

3. Flowise - Drag & drop UI to build your customized LLM flow.

Flowise is an open source UI visual tool to build your customized LLM orchestration flow & AI agents.

We shouldn't compare any projects but I can confidently say this might be the most useful one among the projects listed here!

Get started with the following npm command.

npm install -g flowise
npx flowise start
OR
npx flowise start --FLOWISE_USERNAME=user --FLOWISE_PASSWORD=1234

This is how you integrate the API.

import requests

url = "/api/v1/prediction/:id"

def query(payload):
  response = requests.post(
    url,
    json = payload
  )
  return response.json()

output = query({
  question: "hello!"
)}

You can read the docs.

Cloud host is not available so you would have to self-host using these instructions.

Let's explore some of the use cases:

⚡ Let's say you have a website (could be a store, an e-commerce site, or a blog), and you want to scrap all the relative links of that website and have LLM answer any question on your website. You can follow this step-by-step tutorial on how to achieve the same.

⚡ You can also create a custom tool that will be able to call a webhook endpoint and pass in the necessary parameters into the webhook body. Follow this guide which will be using Make.com to create the webhook workflow.

There are a lot of other use cases such as building a SQL QnA or interacting with API. Explore and build cool stuff!

FlowiseAI has 27.5k stars on GitHub and has more than 14k forks so it has a good overall ratio.

Star Flowise ⭐️

4. LLAMA GPT - a self-hosted, offline, ChatGPT like chatbot (Powered by Llama 2).

LlamaGPT is a self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. It's 100% private and your data doesn't leave your device.

In the recent versions, they have also provided support for Code Llama models and Nvidia GPUs.

You can install it from the Umbrel App Store or you can also install it with Kubernetes.

You can read about the supported models on the docs.

It's very simple to use!

I know, at this point it feels confusing that there are so many ways to run LLM locally. As a developer, I think it's important to evaluate which method works for our situation!

They have 10k+ stars on GitHub and offer 2 packages.

Star LLAMA GPT ⭐️

5. LocalAI - free OpenAI alternative.

LocalAI is free, open source, and considered as an alternative to OpenAI.

LocalAI acts as a drop-in replacement REST API that’s compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing.

It allows you to run LLMs, generate images, and audio (and not only) locally or on-prem with consumer-grade hardware, supporting multiple model families. The best part is that it does not require GPU.

I never thought that there would be such an option so it's a goldmine for devs who don't want to pay much. Plus, it allows to generate of Text, Audio, Video, and Images and also has voice cloning capabilities. What more do you need?

You can watch the complete walkthrough by Semaphore CI!

There are a lot of integration options and developers have built awesome stuff such as Extension for attaching LocalAI instance to VSCode.

You can read the quickstart guide and how to run it with kubernetes. Find all the resources including how to run it on AWS, k8sgpt, and more.

They have 21k+ stars on GitHub and are on the v2.18 release.

Star LocalAI ⭐️

6. Continue - enable you to create an AI software development system.

Continue is one of the best AI code assistants I've seen in my developer journey.

You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains.

You can easily set it up. Below are some of the snapshots while I was installing it.

Step 1

run it through terminal

step2 complete

run it through terminal

step3 complete

After you've configured it, you're all set to use all the amazing concepts it provides.

They have a lot of awesome features such as:

Tab to autocomplete code suggestions.

Ask questions about your codebase.

Understand terminal errors immediately.

Kick off actions with slash commands.

Refactor functions where you are coding.

Read about all the features.

You will have to install the VSCode extension from the marketplace and then read the quickstart guide.

You can read the docs.

You can also watch this basic demo on YouTube!

They have 13k+ stars on GitHub and are built using TypeScript.

Star Continue ⭐️

7. Chat2DB - AI-driven data management platform.

Chat2DB is an AI-first data management, development, and analysis tool.

Even operations that do not understand SQL can use it to quickly query business data and generate reports.

When you do any operation, it will give you some suggestions.
For instance, when you are doing database development, it will help you generate SQL directly in natural language, give you SQL optimization suggestions, help you analyze SQL performance, analyze SQL execution plan, and can also help you quickly generate SQL test data, system code, etc. It's actually very powerful :)

They have excellent support for multi-data sources and can easily integrate up to 17 different database types including PostgreSQL, MySQL, MongoDB, and Redis.

You can download or try it in the browser.

Let's see some of the exciting features:

✅ Intelligent reports.

✅ Data Exploration.

✅ SQL Development.

You can read the quickstart guide on the official docs.

They have 14k+ stars on GitHub and are on the release v3.2.

Star Chat2DB ⭐️

8. LibreChat - Enhanced ChatGPT Clone.

LibreChat is a free, open source AI chat platform. This Web UI offers vast customization, supporting numerous AI providers, services, and integrations.

It serves all AI conversations in one place with a familiar interface, and innovative additions, for as many users as you need.

Some of the features are:

✅ Upload and analyze images seamlessly with advanced models like Claude 3, GPT-4, Gemini Vision, Llava, and Assistants.

✅ Chat with files using various powerful endpoints using OpenAI, Azure, Anthropic, and Google.

✅ Multilingual UI with support for 20+ languages.

✅ Diverse Model options including OpenAI, BingAI, Anthropic (Claude), Azure OpenAI, and Google’s premier machine learning offerings.

You can read the quickstart guide to get started.

Watch this video for the complete walkthrough!

They have 15k+ stars on GitHub and offer 4 packages.

Star LibreChat ⭐️

9. Lobe Chat - modern-design LLMs/AI chat framework.

An open-source, modern-design ChatGPT/LLMs UI/Framework.
Supports speech-synthesis, multi-modal, and extensible (function call) plugin systems. You can deploy your private OpenAI with one click.

Let's see some of the exciting features of LobeChat:

✅ Multi-Model Service Provider Support.

They have expanded our support to multiple model service providers, rather than being limited to a single one.

Find the complete list of 10+ model service providers that they support.

✅ Assistant Market.

In LobeChat's Assistant Market, creators can discover an innovative community that brings together numerous carefully designed assistants.

There are so many awesome applications there. WOW!

✅ Model Vision Recognition.

LobeChat now supports large language models with visual recognition capabilities such as OpenAI's gpt-4-vision, Google Gemini Pro vision, and Zhipu GLM-4 Vision, enabling LobeChat to have multimodal interaction capabilities.

Users can easily upload or drag and drop images into the chat box, and the assistant will be able to recognize the content of the images and engage in intelligent conversations.

✅ Text to Image Generation.

You can directly utilize the Text-to-image tool during conversations with the assistant.

By using the power of AI tools such as DALL-E 3, MidJourney, and Pollinations, assistants can now implement it properly.

✅ Local Large Language Model (LLM) Support.

With the powerful infrastructure of Ollama AI and the community's collaborative efforts, you can now engage in conversations with a local LLM (Large Language Model) in LobeChat!

By running the following Docker command, you can experience conversations with a local LLM in LobeChat.

docker run -d -p 3210:3210 -e OLLAMA_PROXY_URL=http://host.docker.internal:11434/v1 lobehub/lobe-chat

✅ Progressive Web App (PWA).

They have adopted Progressive Web App PWA technology, which is a modern web technology that elevates web applications to a near-native app experience.

✅ Custom Themes.

Read about all of the features and use cases.

You can self-host or deploy it using docker. The ecosystem of lobe chat provides 4 packages: lobehub/ui, lobehub/icons, lobehub/tts, and lobehub/lint.

They also provide plugins market where you can find lots of useful plugins that can be used to introduce new function calls and even new ways to render message results. If you want to develop your own plugin, refer to 📘 Plugin Development Guide in the wiki.

You can read the docs.

You can check the live demo. It's pretty cool!

They have 35k+ stars on GitHub with more than 500 releases.

Star Lobe Chat ⭐️

10. MindsDB - The platform for customizing AI from enterprise data.

MindsDB is the platform for customizing AI from enterprise data.

With MindsDB, you can deploy, serve, and fine-tune models in real-time, utilizing data from databases, vector stores, or applications, to build AI-powered apps - using universal tools developers already know.

With MindsDB and its nearly 200 integrations to data sources and AI/ML frameworks, any developer can use their enterprise data to customize AI for their purpose, faster and more securely.

You can read the docs and quickstart guide to get started.

They currently support a total of 3 SDKs that is using using Mongo-QL, Python, and JavaScript.

There are several applications of MindsDB such as integrating with numerous data sources and AI frameworks so you can easily bring data and AI together to create & automate custom workflows.

The other common use cases include fine-tuning models, chatbots, alert systems, content generation, natural language processing, classification, regressions, and forecasting. Read more about the use cases and each of them has an architecture diagram with a little info.

For instance, the chatbot architecture diagram with MindsDB. You can read about all the solutions provided along with their SQL Query examples.

// SQL Query Example for Chatbot
CREATE CHATBOT slack_bot USING database='slack',agent='customer_support';

Just to tell you about the overall possibilities, you can check out How to Forecast Air Temperatures with AI + IoT Sensor Data. Exciting right :)

They have 25.4k+ stars on GitHub and are on the v24.7.2.0 with more than 200 releases. By the way, this is the first time I've seen 4 parts in any release as I always followed the semantic release.

Star MindsDB ⭐️

11. AutoGPT - more exciting than ChatGPT.

At the core of AutoGPT lies its primary project, a semi-autonomous agent driven by large language models (LLMs), designed to perform any tasks for you.

The AutoGPT project consists of four main components:

The Agent – also known as just "AutoGPT"
The Benchmark – AKA agbenchmark
The Forge
The Frontend

Read on how you can set up AutoGPT using the OpenAI key.

You can see this YouTube video by Fireship on what is AutoGPT.

You can also watch this AutoGPT tutorial by Sentral Media.

You can read the docs and check out the project board on what things are under development right now.

Even if you don't know much about AI, you can try AutoGPT to understand how you can save time and build cool stuff.

They have 164k+ stars on GitHub due to such an excellent use case and automation capabilities.

Star AutoGPT ⭐️

12. reor - self organizing AI note-taking app.

One of the most exciting projects that I've seen so far, especially because it runs models locally.

Reor is an AI-powered desktop note-taking app. It automatically links related notes, answers questions on your notes, and provides semantic search.

Everything is stored locally and you can edit your notes with an Obsidian-like markdown editor. The project hypothesizes that AI tools for thought should run models locally by default.

Reor stands on the shoulders of the giants Ollama, Transformers.js & LanceDB to enable both LLMs and embedding models to run locally. Connecting to OpenAI or OpenAI-compatible APIs like Oobabooga is also supported.

I know you're wondering How can it possibly be self-organizing?

a. Every note you write is chunked and embedded into an internal vector database.
b. Related notes are connected automatically via vector similarity.
c. LLM-powered Q&A does RAG on the corpus of notes.
d. Everything can be searched semantically.

You can watch the demo here!

One way to think about Reor is as a RAG app with two generators: the LLM and the human. In Q&A mode, the LLM is fed retrieved context from the corpus to help answer a query.

Similarly, in editor mode, we can toggle the sidebar to reveal related notes retrieved from the corpus. This is quite a powerful way of augmenting your thoughts by cross-referencing ideas in a current note against related ideas from your digital collection.

You can read the docs and download from the website. Mac, Linux & Windows are all supported.

They have also provided starter guides to help you get started.

You can also watch this walkthrough!

They have 6.5k stars on GitHub and are built using TypeScript.

Star reor ⭐️

13. Leon - your open source personal assistant.

Leon is an open source personal assistant who can live on your server. You're definitely curious, right? 😅

He does stuff when you ask him to. You can even talk to him and he will revert by talking to you. Similarly, you can also text him!

If you are a developer (or not), you may want to build many things that could help in your daily life. Instead of building a dedicated project for each of those ideas, Leon can help you with his Skills structure.

If you want to, Leon can communicate with you by being offline to protect your privacy. This is a list of skills that Leon can do for now.

You should read the story behind Leon. You can also watch this demo to learn more about Leon.

This is the High-level architecture schema of Leon.

You can install Leon by using this command.

# install leon global cli
npm install --global @leon-ai/cli

# install leon
leon create birth

You can read the docs.

Appwrite is one of the sponsors which says a lot about its overall impact.

It has 15k stars on GitHub and has released some drastic changes recently so make sure to read the docs with extra caution.

Star Leon ⭐️

14. Instrukt - Integrated AI in the terminal.

Instrukt is a terminal-based AI-integrated environment. It offers a platform where users can:

Create and instruct modular AI agents.
Generate document indexes for question-answering.
Create and attach tools to any agent.

Instruct them in natural language and, for safety, run them inside secure containers (currently implemented with Docker) to perform tasks in their dedicated, sandboxed space.

It's built using Langchain, Textual, and Chroma.

Get started with the following command.

pip install instrukt[all]

There are a lot of exciting features such as:

✅ A terminal-based interface for power keyboard users to instruct AI agents without ever leaving the keyboard.

✅ Index your data and let agents retrieve it for question-answering. You can create and organize your indexes with an easy UI.

✅ Index creation will auto-detect programming languages and optimize the splitting/chunking strategy accordingly.

✅ Run agents inside secure docker containers for safety and privacy.

✅ Integrated REPL-Prompt for quick interaction with agents, and a fast feedback loop for development and testing.

✅ You can automate repetitive tasks with custom commands. It also has a built-in prompt/chat history.

You can read about all the features.

You can read the installation guide.

You can also debug and introspect agents using an in-built IPython console which is a neat little feature.

Instrukt is licensed with an AGPL license meaning that it can be used by anyone for whatever purpose.

It is safe to say that Instrukt is an AI commander for your terminal.

It is a new project so they have around 240 stars on GitHub but the use case is damn good.

Star Instrukt ⭐️

15. Quivr - RAG Framework for building GenAI Second Brains.

Quivr, your second brain, utilizes the power of GenerativeAI to be your personal assistant! You can think of it as Obsidian but turbocharged with AI powers.

Quivr is a platform that enables the creation of AI assistants, referred to as Brain. These assistants are designed with specialized capabilities like some can connect to specific data sources, allowing users to interact directly with the data.

Watch a quick demo of Quivr!

Some of the amazing features are:

✅ You can choose the type of Brain you want to use, based on the data source you wish to interact with.

✅ They also provide a powerful feature to share your brain with others. This can be done by sharing with individuals via their emails and assigning them specific rights.

✅ Quivr works offline, so you can access your data anytime, anywhere.

✅ You can access and continue your past conversations with your brains.

✅ But the best one that I loved is that you can literally install a Slack bot. Refer to this demo to see what you can do. Very cool!

Anyway, read about all the awesome stuff that you can do with Quivr.

You can read the installation guide and 60 seconds installation video. I really loved this idea!

You can read the docs.

They have also provided guides on how to deploy Quivr with Vercel, Porter, AWS, and Digital Ocean.

They could provide a better free tier plan but it's more than enough to test things on your end.

It has 30k+ Stars on GitHub with 220+ releases which means they're constantly improving.

Star Quivr ⭐️

16. Open Interpreter - natural language interface for terminal.

Open Interpreter lets LLMs run code (Python, Javascript, Shell, and more) locally. You can chat with Open Interpreter through a ChatGPT-like interface in your terminal by running $ interpreter after installing.

This provides a natural-language interface to your computer's general-purpose capabilities:

✅ Create and edit photos, videos, PDFs, etc.
✅ Control a Chrome browser to perform research Plot, clean, and analyze large datasets.

I don't know about you, but their website made me say WOW!

Quickstart using this command.

pip install open-interpreter

// After installation, simply run:
interpreter

You can read the quickstart guide.

You should read about the comparison to ChatGPT's Code Interpreter and the commands that you can use.

You can read the docs.

Open Interpreter works with both hosted and local language models. Hosted models are faster and more capable, but require payment while local models are private and free but are often less capable. Choose based on your use case!

They have 48k+ stars on GitHub and are used by 300+ developers.

Star Open Interpreter ⭐️

17. CopilotKit - 10x easier to build AI Copilots.

You will agree that it's tough to add AI features in React, that's where Copilot helps you as a framework for building custom AI Copilots.

You can build in-app AI chatbots, and in-app AI Agents with simple components provided by Copilotkit which is at least 10x easier compared to building it from scratch.

You shouldn't reinvent the wheel if there is already a very simple and fast solution!

They also provide built-in (fully-customizable) Copilot-native UX components like <CopilotKit />, <CopilotPopup />, <CopilotSidebar />, <CopilotTextarea />.

Get started with the following npm command.

npm i @copilotkit/react-core @copilotkit/react-ui

This is how you can integrate a Chatbot.

A CopilotKit must wrap all components which interact with CopilotKit. It’s recommended you also get started with CopilotSidebar (you can swap to a different UI provider later).

"use client";
import { CopilotKit } from "@copilotkit/react-core";
import { CopilotSidebar } from "@copilotkit/react-ui";
import "@copilotkit/react-ui/styles.css"; 

export default function RootLayout({children}) {
  return (
    <CopilotKit url="/path_to_copilotkit_endpoint/see_below">
      <CopilotSidebar>
        {children}
      </CopilotSidebar>
    </CopilotKit>
  );
}

You can read the docs and check the demo video.

You can integrate Vercel AI SDK, OpenAI APIs, Langchain, and other LLM providers with ease. You can follow this guide to integrate a chatbot into your application.

The basic idea is to build AI Chatbots very fast without a lot of struggle, especially with LLM-based apps.

You can watch the complete walkthrough!

CopilotKit has recently crossed 7k+ stars on GitHub with 300+ releases.

Star CopilotKit ⭐️

18. GPT Engineer - AI builds what you ask.

GPT-engineer lets you specify software in natural language, sit back, and watch as an AI writes and executes the code, and you can ask the AI to implement improvements.

It's safe to say that it's an engineer who doesn't need a degree 😅

It's a commercial project for the automatic generation of web apps. It features a UI for non-technical users connected to a git-controlled codebase.

I know this feels confusing, so watch the below demo to understand how you can use GPT Engineer.

You can get started by installing the stable release using this command.

python -m pip install gpt-engineer

If you want a complete walkthrough, watch this awesome demo by David!

I recommend checking out the roadmap to understand the overall vision.

They have 51k+ stars on GitHub and are on the v0.3 release.

Star GPT Engineer ⭐️

19. Dalai - the simplest way to run LLaMA and Alpaca locally.

Dalai lets you run LLaMA and Alpaca on your computerP and is powered by llama.cpp, llama-dl CDN, and alpaca.cpp.

Dalai runs on operating systems such as Linux, Mac, and Windows so that's a plus point!

Dalai is also an NPM package:

programmatically install
locally make requests to the model
run a dalai server (powered by socket.io)
programmatically make requests to a remote dalai server (via socket.io)

You can install the package using the below npm command.

npm install dalai

You can read the memory requirements and how to archive elsewhere rather than home directory.

They have 13k stars on GitHub and are still in the very early stages.

Star Dalai ⭐️

20. OpenLLM - run LLMs as OpenAI compatible API endpoint in the cloud

OpenLLM lets developers run any open-source LLMs as OpenAI-compatible API endpoints with a single command.

⚡ Build for fast and production usage.
⚡ Support llama3, qwen2, gemma, etc, and many quantized versions full list.
⚡ OpenAI-compatible API & includes ChatGPT like UI.
⚡ Accelerated LLM decoding with state-of-the-art inference backends.
⚡ Ready for enterprise-grade cloud deployment (Kubernetes, Docker, and BentoCloud).

Get started with the following command.

pip install openllm  # or pip3 install openllm
openllm hello

OpenLLM provides a chat user interface (UI) at the /chat endpoint for an LLM server. You can visit the chat UI at http://localhost:3000/chat and start different conversations with the model.

OpenLLM supports LLM cloud deployment via BentoML, the unified model serving framework, and BentoCloud, an AI inference platform for enterprise AI teams.

If you don't know, BentoCloud provides a fully managed infrastructure optimized for LLM inference with autoscaling, model orchestration, observability, and many more, allowing you to run any AI model in the cloud.

Once the deployment is complete, you can run model inference on the BentoCloud console:

You can read about the supported models and how to start the LLM server.

Explore docs as you can also chat with a model in the CLI using openllm run and specifying model version - openllm run llama3:8b.

For people like me who love exploring walkthroughs, watch this demo by Matthew!

They have 9k+ stars on GitHub and have 100+ releases so it's growing at a rapid pace.

Star OpenLLM ⭐️

21. Unsloth - Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory.

Unsloth makes finetuning large language models like Llama-3, Mistral, Phi-3, and Gemma 2x faster, use 70% less memory, and with no degradation in accuracy!

✅ What is finetuning?

Finetuning is the process of updating the actual brains of the language model through some process called back-propagation. But, finetuning can get very slow and very resource intensive.

Unsloth can be installed locally or through another GPU service like Google Colab. Most use Unsloth through the interface Google Colab which provides a free GPU to train with.

Some of the things that stand out:

✅ Open source trains 5x faster, and the pro version claims to be 30x faster.

✅ No approximation methods are used resulting in a 0% loss in accuracy.

✅ No change of hardware, Works on Linux and Windows via WSL.

You can read the installation instructions and performance benchmarking tables on the website.

You can read the docs and all the uploaded models on Hugging Face directly.

They have also provided a detailed guide on How to Finetune Llama-3 and Export to Ollama.

They have 12.5k+ stars on GitHub and it's an efficient solution.

Star Unsloth ⭐️

I hope you learned something new!

I believe that learning to use these powerful LLM is a choice and it's on you (as a developer) to find better productivity solutions for your use case.

Have a great day! Till next time.

If you like this kind of stuff, please follow me for more :)

Follow Latitude for more content like this.

Latitude Follow

Latitude is the open-source LLM development platform. Build, test, and deploy your LLM-based features with confidence.

Understanding APIs: 10 API Concepts and Examples

Opemipo Disu — Thu, 27 Jun 2024 12:59:58 +0000

As a developer or a person in tech, you are likely to have heard of "APIs.” Hearing this term may get you curious. Imagine you're using a service that needs to fetch data from another server. APIs make this possible by serving as a bridge between the data and the application.

This article will clarify APIs by explaining the basic concepts and practices you can apply to your API development cycle. Here is a thing you should know before getting started with APIs 😂👇:

⭐️ Would you consider giving us a star on GitHub? ⭐️

What you will learn from this article?

Basic understanding of APIs, their key concepts, and their importance in your development cycle.
Types of APIs and their fundamentals.
Best practices in API development
Tools or platforms where you can experiment with APIs.

Introduction

APIs are one of the foundational components of modern development. They enable seamless interaction between applications and data. Instead of manually providing communication between applications and data, using APIs makes this interaction accessible and efficient for developers.

Whether you are building a web application, a mobile application, or integrating a service, APIs are crucial in allowing different components in your application to connect with your data seamlessly.

What is an API?

An API (Application Programming Interface) is a framework that consists of a series of commands, functions, and protocols that enable interaction between different applications. Its primary purpose is to define methods and data structures developers could use to interact with their software components.

Think of API as a waiter in a restaurant; you tell the waiter (API) what you want, and they bring it to the kitchen (server) and then deliver your food (data) back to you. That is precisely how it works!

APIs serve as links that take requests from applications, fetch the necessary data from a server, and then return the processed data to the application.

Types of APIs

There are different types of APIs tailored for various use cases and purposes. Understanding types of APIs helps developers choose the correct API for their specific needs. Here are the most common types of APIs:

Public or Open APIs: These are made publicly available to any developer with little or no restrictions, enabling developers to access a service's data and functionalities.

A good example of a place where you can get a public API is the Some Random API platform. You will find a lot of API endpoints you can use freely in that service.
Internal or Private APIs: Their primary intent is team collaboration. They are usually not open to external developers and are used to integrate systems and services within a team or an organization. They are restricted to developers granted access to work with the API.

APIs are made private for many reasons. Some are made private to secure sensitive data, accelerate development for business reasons, or enhance internal workflows. If you're working on a large-scale application, it's best to protect your API by privatizing it.
Partner APIs: This is an example of a private API; they are not open to the public. However, they are specifically designed for external partners to use. Access is typically granted through a contractual agreement, allowing partners to integrate and access certain functionalities or data.

Now that you know the major types of APIs, it's best to understand how and where they can be applied. Different types of APIs serve various purposes and are best suited for some specific projects.

REST APIs

REST (Representational State Transfer) or RESTful APIs are rules for building and interacting with web applications. They rely on standard HTTP methods and protocols to enable communication between clients and servers.

REST APIs are designed to be simple, scalable, and stateless, making them popular for web and mobile applications. Unlike a typical API, RESTful APIs are not a protocol; instead, they leverage them for interaction.

The server responds with the requested data when the client asks for it. That's how easy it is!

Principles of REST APIs

Some architectural principles guide RESTful APIs; their unique architecture makes up everything, ensuring they remain efficient and easy to use. Here are some of REST APIs' unique principles:

Statelessness: Every request from a client to a server must contain all the necessary information to understand and process the request. The server does not store any session state about the client.

This helps to simplify the server's architecture, as it doesn't need to manage and store session information, making the application more scalable. It also helps to give accurate information.
Client-Server Architecture: In a REST API, the client and server are separate components that interact through requests and responses. The client handles the user interface and user experience, while the server manages data storage.
Uniform Interface: REST APIs follow a consistent way of getting access to resources. This includes using HTTP methods like GET, POST, PUT, and DELETE to URIs (Uniform Resource Identifier) to access and manipulate resources. This makes REST APIs easier to understand and accessible, as developers can rely on the patterns they are familiar with.
Cacheability: This is one of the principles of RESTful APIs that most developers enjoy. With REST APIs, responses from the server are labeled as cacheable or non-cacheable. Caching can reduce the number of client-server interactions and improve performance.

This increases efficiency by reducing unnecessary network calls, decreasing latency, and improving overall performance.

RESTful APIs have a great structure, commonly used in the modern-day development cycle. Its major features are its principles, which make REST APIs what they are.

Image credit: Phil Sturgeon

SOAP APIs

SOAP (Simple Object Access Protocol) API is a protocol for exchanging structured information in web applications. Unlike REST, which uses simple HTTP methods, SOAP leverages XML for its message format and follows a more complex structure.

SOAP APIs are strictly used for web applications and have built-in commands to ensure the security of messages, making them suitable for applications with tight security.

Differences between REST and SOAP

There are clear differences between REST and SOAP, as noticed in the definition above. While both are used for the web, they still differ in architecture, standards, etc. Here are some of its differences:

Protocol VS Architectural Style:
- SOAP: A protocol with standards and rules.
- REST: An architectural style that uses standard HTTP methods and protocols for interaction between web applications and data.
Message Format:
- SOAP: Uses XML for message formatting.
- REST: Uses JSON but can also use XML, HTML, or plain text for message formatting.
Complexity:
- SOAP: It's more complex due to its standards and XML messaging.
- REST: Simpler and more flexible, easier to implement.
Transport:
- SOAP: Can use various protocols (HTTP, SMTP, etc.).
- REST: Typically uses HTTPS for communication.

By understanding the differences between this protocol and architectural style, developers can choose the appropriate protocol based on the needs of their specific applications.

JSON and XML

JSON (JavaScript Object Notation) and XML (eXtensible Markup Language) are standard API communication data formats. These formats serve the same primary purpose: to encode data structures between a server and a client so that both can understand.

JSON

JSON is a lightweight data-interchange format derived from JavaScript. It is easy for humans to read and write and for machines to parse and generate.

XML

XML is a markup language that defines the structure for encoding documents in a format readable to humans and machines. XML is mainly known for its ability to handle complex data structures.

Differences between JSON and XML

Readability:
- JSON: Syntax is more readable, ideal for quick interaction, and more accessible for developers.
- XML: Comes with a more complex syntax. Best tailored for representing complex data structures and documents.
Size:
- JSON: More compact and lightweight. Results in faster data transmission and less bandwidth usage.
- XML: Larger due to extensive use of markdown tags.
Data Types:
- JSON: Supports data types such as strings, numbers, arrays, and objects.
- XML: All data is written in text, requiring parsing for specific data types.
Parsing and Performance:
- JSON: Faster to parse, especially in JavaScript environments, due to compatibility.
- XML: Slower to parse and process, requiring more resources.
Schema Support:
- JSON: JSON schema is available but not as extensive as XML schema.
- XML: XML schema is very powerful for verifying document structure and data types.

You can use whichever data formats you want for communication when working with APIs. It's best to know its differences, as there is no 'perfect' data format to use in API development. You can select any of them tailored to your needs.

When to use

There are cases where you can use JSON as your data format, as well as cases where you can use XML. Knowing when and where to use them is very important.

You can use JSON when:

You need a lightweight data format.
Working with web APIs, especially in JavaScript environments.
Simplicity and readability are essential.
You are dealing with simpler data structures and need to reduce bandwidth usage.

You can use XML when:

You want to handle complex data structures.
Validation of the data format and structure is needed.
Working with applications that require extensive metadata and descriptive data.
Data interchange needs to be highly extensible and self-descriptive.

By understanding the strengths and cases when you can use both JSON and XML, developers can decide which data format to use based on the needs of their applications.

API Endpoints

At this point in this article, you may be wondering what API endpoints are because you might have come across the term "API endpoints" a few times in this article.

An API endpoint is a URL at which a client can access an API to perform actions like retrieving, updating, or deleting data. Endpoints represent the functions and services provided by an API.

The endpoint lets API interact with the application you are working on, enabling communication and exchange of information. They are accessed with HTTP methods such as GET, POST, PUT, and DELETE, which define the type of operation that will be performed.

Example of an API Endpoint

Let's consider an example of a REST API for managing student information in a web application. The base URL for the API could be https://api.example.com. Now, let's take a look at other endpoints and responses.

Retrieve a list of students:
- Endpoint: https://api.example.com/students
- HTTP method: GET
- Purpose: To retrieve a list of all registered students in the system.
- Request:
```
GET https://api.example.com/students
```
  Here is the response you get:
```
[
  {
    "id": 1,
    "name": "Opemipo Disu",
    "email": "opemipo.disu@school.com"
  },
  {
    "id": 2,
    "name": "Ralf Endrick",
    "email": "ralf.endrick@school.com"
  }
]
```
  In this example, we used the GET method to retrieve information from the system. After that, it gives us the data we requested from the endpoint in JSON format.

Another example could be an endpoint for registering students in the system. Let's create that and see its response.

Add a new student:
- Endpoint: https://api.example.com/students
- HTTP Method: POST
- Purpose: Adding a new student to the management system.
- Request:
```
POST https://api.example.com/students
Content-Type: application/json

{
  "name": "Opemipo Disu",
  "email": "opemipo.disu@student.com"
}
```
  Response:
```
{
  "id": 1,
  "name": "Opemipo Disu",
  "email": "opemipo.disu@student.com"
}
```
  In this case, you will notice we are working with the https://api.example.com/students endpoint, basically because we want to add a new student to the system; the only way the users could be accessed is by using that endpoint because it should have information related to the student in it.

Now, let's think of deleting a specific student's information. Here's how we could go about that:

Deleting a Student's information
- Endpoint: https://api.example.com/students/{id}
- HTTP method: DELETE
- Purpose: To delete a student by their ID.
- Request:
```
  DELETE https://api.example.com/students/1
```
  Response:
```
  {
    "message": "Student deleted successfully."
  }
```
When you want to delete a student's information using an API, addressing the specific data by its ID in the API endpoint ensures that you target the correct record.

By understanding how endpoints work and seeing some examples, developers can also use APIs to interact with web applications and perform various operations.

HTTP methods

HTTP methods define the action performed on resources identified by the API endpoints. We have almost 40 registered HTTP methods, but here are the four most common ones:

GET
POST
PUT
DELETE

Now, we will go into what these methods are used for and provide an example for each of the four most commonly used HTTP methods.

GET

The GET method retrieves data from the server without making any changes to the server data.

An example of how it works was showcased earlier in the endpoint for retrieving students' information.

An example, once again:

Request:

GET https://api.example.com/students

Response:

[
  {
    "id": 1,
    "name": "Opemipo Disu",
    "email": "opemipo.disu@school.com"
  },
  {
    "id": 2,
    "name": "Ralf Endrick",
    "email": "ralf.endrick@school.com"
  }
]

As seen above, the GET method was used to retrieve the data shown in JSON format from the endpoint.

POST

The POST method sends data to the server to create a new resource. Unlike GET, which is used to retrieve data, POST submits data to the server. GET is dependent on the data sent to the server by POST.

An example of how the POST method could be used was explained earlier. The student's registration example was a precise instance where the POST method could be used.

If you missed it, please take a look at it again.

Request:

POST https://api.example.com/students
Content-Type: application/json

{
  "name": "Opemipo Disu",
  "email": "opemipo.disu@student.com"
}

We sent a request using the POST method. This was used because we wanted to add a student's information to the server.

Here's the response we get by doing that:

{
  "id": 1,
  "name": "Opemipo Disu",
  "email": "opemipo.disu@student.com"
}

In the response above, the POST method automatically helped to create and register the new student. That is just how the POST method works.

PUT

This method is used to update existing resources with new data or create a new resource if it doesn’t exist. It replaces the current information of the resource with the data provided in the request.

Let's take an example of updating a student's information using the PUT method.

Request:

PUT https://api.example.com/students/1
Content-Type: application/json

{
  "name": "Opemipo Hay",
  "email": "opemipo.hay@student.com"
}

Response:

{
  "id": 1,
  "name": "Opemipo Hay",
  "email": "opemipo.hay@student.com"
}

In this case, we had to locate the information we wanted to update using its ID. We used the PUT method and added the data we wanted to update.

DELETE

This method is used to delete existing resources. When a DELETE request is made, the server deletes the resource the URI identifies.

For this, we will take an example of deleting a student's information by its ID.

Request:

DELETE https://api.example.com/students/1

Response:

{
  "message": "Student's information deleted successfully"
}

In the request, we used the DELETE method to delete the user's information using their ID. After, we got a response saying, "Student's information deleted successfully."

HTTP Status Codes

HTTP status codes are responses returned by servers to indicate the result of the client's request. They play a vital role in API communication by displaying the outcome of the client's request to the server.

Here are some common out of many HTTP status codes:

200 (OK)

When you get this response, the request is successful, and the server returns the requested data.

An example of where you can get this response is when there's a successful GET request to retrieve data. This indicates in the network tab of your developer console that the operation was successful and that the server processed the request as expected.

400 (Not Found)

You get this response when the server cannot find the requested resource or data. This could be because data wasn't fetched correctly or because the resource doesn’t exist.

An example of where you can get this error is when you use a GET request for a user that does not exist. Let's have a quick look at that:

Request:

GET https://api.example.com/users/583

Response:

{
  "status": 404,
  "message": "Resource not found"
}

The response gave an error because there was no resource in the presumed endpoint.

500 (Internal Server Error)

When you get this response, the server encountered an unexpected condition that prevented it from fulfilling the request.

You could get this response when a server-side error occurred while processing the request.

Request:

POST https://api.example.com/students
Content-Type: application/json

{
  "name": "Opemipo",
  "email": "opemipo.disu@example.com"
}

Response:

{
  "status": 500,
  "message": "Internal server error"
}

A 500 Internal Server Error indicates a general server-side error. It shows that something went wrong on the server, not necessarily due to the client's request.

While there are a few other HTTP status codes, you can read this article to learn more about them. The ones available in this article are the most common status codes.

Authentication and Authorization

While API security is essential, authentication and authorization are critical components of API security. Authentication in API development is the process of verifying the identity of a user or an application, typically through techniques like API keys, OAuth tokens, or user credentials.

Authorization, on the other hand, determines what resources or operations the authenticated entity is allowed to access.

These processes ensure only valid users or applications can access the API and perform actions based on their permissions.

Basic Concepts of API Keys and OAuth

An API key is a unique identifier used to authenticate requests associated with a project or application.

API keys are included in the API requests to identify the calling project or application. They are typically used to track and control how the API is being used. API keys should be kept secure and not exposed in code to prevent unauthorized access.

However, they are not secure and should be used with other security measures, such as environmental variables.

On the other hand, OAuth (Open Authorization) is a token-based authentication framework that allows third-party applications to access user data without exposing user credentials.

It is widely used by platforms like Google and GitHub to grant limited access to user data. It involves a flow where the user authorizes the application, and the application receives an access token that can be used to make authorized API requests. It provides a more flexible and secure method compared to API keys.

Importance for API Security

Prevention of Unauthorized Access: Authentication ensures that only users and applications can access the API, preventing unauthorized access to sensitive data.
Rate Limiting: Authentication helps track APIs' usage, enabling the implementation of rate limits to prevent data misuse.
Monitoring: Authentication allows for detailed logging and monitoring of API usage, which can be crucial for identifying errors.

Rate Limiting and Throttling

APIs use rate limiting to keep things stable and safe. This means they limit how many requests a user or application can make in a certain amount of time. This helps prevent servers from getting overloaded.

It also ensures that all users in an application get an equal distribution of the API's resources.

To manage rate limits, applications should gradually increase the wait time between retries if a limit is reached. Monitor your API usage to stay within these limits. If you store frequently used data, you can reduce the number of requests you make.

Using page numbers and filters can help you manage large data sets more quickly, reducing the load on the API.

Testing APIs

Testing APIs is essential in the API development process, ensuring that your application communicates correctly with the server and handles data as expected. Dedicated tools allow you to make API requests, check and analyze responses, and log issues early in the development cycle.

Let's explore some of the best tools for testing APIs and provide an essential guide on effectively testing an API using these tools.

API testing tools

Postman: Postman is a tool that simplifies API development. It allows you to construct and send requests, organize APIs into collections, automate tests, and generate detailed reports.

Ideal for both manual and automated testing, Postman supports various HTTP methods, making it flexible for testing.
cURL: This command-line technique enables data transfer with URLs.

cURL is used mainly because of its accessibility and flexibility, especially for developers comfortable with the command line.
Swagger: Swagger provides a suite of tools for API documentation and testing. It allows you to visualize and interact with the API's resources without manually creating requests.

Guide on how to Test an API

Define the Endpoint and Method
- Determine the API endpoint you wish to test and the HTTP method (GET, POST, PUT, DELETE) to use.
- Example: To fetch user data, you might use:
```
  GET https://api.example.com/users
```
Set Up the Request
- Postman: Open Postman, create a new request, enter the endpoint URL, and select the HTTP method. Add necessary headers like API keys and parameters.
  
  For a GET request to retrieve users, just set the URL to https://api.example.com/users and include any required headers or parameters.
Send the Request
- Click "Send" in Postman to execute the request and observe the response.
Analyze the Response
- Status Code: Indicates the success or failure of the request (e.g., 200 OK, 404 Not Found).
- Headers: Provide metadata about the response.
- Body: Contains the data returned by the API, typically in JSON or XML format.
- Example: A successful GET request might return a status code 200 and a JSON body with user data.
Handle Errors
- If the request fails, analyze the status code and error message to diagnose the issue.
  - Example: A 404 status code indicates that the endpoint is incorrect or the resource does not exist.
- Adjust the request accordingly and retry.
Automate Testing
- Postman supports scripting to automate tests. You can write pre-request scripts to set conditions and test scripts to validate responses.
  - Example: To verify a successful response, add the following script in Postman's "Tests" tab:
```
  pm.test("Status code is 200", function () {
      pm.response.to.have.status(200);
  });
```

By utilizing tools like Postman, cURL, and Swagger, you can streamline the process of testing APIs, ensuring your application interacts with external services reliably and efficiently.

Conclusion

Understanding APIs is essential for any developer who is just starting. They are a major component of modern development, enabling seamless communication and data exchange between applications.

This article has covered the fundamental concepts of APIs, including their types, key principles of REST and SOAP APIs, data formats like JSON and XML, and the importance of API endpoints and HTTP methods.

Additionally, we explored aspects of API security through authentication and authorization, as well as the importance of rate limiting and throttling.

If you found this article helpful, please consider giving us a star on GitHub

⭐ Would you consider giving us a Star on GitHub?⭐

Your support helps us to continue improving and providing valuable content to the developer community. Thank you for reaching this far in this article 💙!

Mastering Embedded Analytics: A Getting Started Guide 📊

Opemipo Disu — Tue, 11 Jun 2024 09:12:13 +0000

TL;DR

This article will walk you through all the information you need about embedded analytics and Latitude, including how to use Latitude to visualize your data, with a detailed guide.

Overview of Embedded Analytics

At this point, you may want to know what embedded analytics is. It refers to integrating data visualization into software or a web application. Instead of viewing data as numbers or as disorganized, there are different ways to display data in an organized way and embed it in an application.

Modern technologies like charting libraries, web components, APIs, and SDKs have made data visualization seamless. An alternative to visualizing your data could be using an underlying technology, but it could be challenging to achieve compared to emerging technologies such as the ones mentioned earlier.

Modern charting tools provide users with pre-built components that make them easy to work with. While using these tools, you only need to provide raw data to work with in most cases. This makes it easy to include your data visualization in your application.

ICYMI: We published an article a couple of weeks ago on the best chart libraries to use in 2024. It's best to read that article if you are looking for a chart library to work with. I've included in the article comprehensive details about the best charting libraries to work with according to your needs.

By the way, I'm a part of Latitude's team, and it would mean a lot if you could check out our open-source framework for embedded analytics and give us a star on GitHub.

We're working to provide a better developer experience, and all support is appreciated!

Star Latitude on GitHub ⭐️

Okay, now - let's get back at it! 😉

Key Concepts of Embedded Analytics

Real-time data access: Embedded analytics often provide access to real-time data. This lets users view analytics based on the most current data available. While doing this, you must know that this is a dynamic way of rendering analytics.
Customization: Most embedded analytics tools come with various pre-built components. You can customize the analytics tailored to your needs with the help of pre-built themes or custom styling. These tools also let you include features you want in your analytics. With the help of customization, you can display your analytics with the features or style you want to embed.
Integration with databases and host applications: Embedded analytics tools let you work directly with databases and host applications as data sources. While data could be rendered dynamically, many of these tools render data differently, just as various technologies are used for database management.
Focus on User Experience: This is the primary purpose of embedded analytics; it focuses on how data can be visualized in the best way for users. Providing great functionality in analytics helps users understand the context of the analysis.

There are tons of things embedded analytics could be used for. Embedding analytics works in hands with your data, but you may be wondering what its purpose is. For instance, you see a dashboard that displays the usage stats in an application as a chart—an example of embedded analytics integrated into that application. Different tools for visualizing data have various methods of sourcing data.

Getting Started with Latitude

There are different tools available for embedding analytics into your application. One option is Latitude, an open-source framework that allows you to build and embed interactive analytics based on your database or data warehouse.

With Latitude, you only need to write composable SQL queries with custom parameters to pull data and transform it into analytics on your front end. The framework works with Svelte for the front end; with the pre-built components Latitude provides, you can include your components as attributes to display your analytics.

How Latitude Works

Latitude works with two major things: your database or data warehouse and your front end, with the help of the created database. Additionally, data can be displayed statically using CSV files. By doing this, there's no need to worry about anything apart from creating SQL queries.

Latitude isn't limited to using Svelte alone for the front end; you can also integrate Latitude into your React application. All you need to do after configuring the parameters is include them in the components for the data you have to display.

With Latitude, you're on the right path to embedding your analytics into your application.

Prerequisites for getting started with Latitude

Basic knowledge of SQL.
npm >= 8.5.5 or higher installed on your local development machine.
Vast knowledge of React or Svelte.
Basic knowledge of CSS frameworks such as Tailwind for styling.

Away from the hype, Latitude is one tool you will love to try out because of its accessibility! To start with Latitude, you must install Latitude's CLI on your local machine. The command below will install Latitude's CLI globally on your local machine.

npm install -g @latitude-data/cli

After installing the CLI, you need to create your Latitude project. You can only configure your queries and views by creating a project in Latitude. Use the command below to create a Latitude project.

latitude start

This is where your actual journey of building with Latitude begins. When you run the command, you should input the project name when requested and select the type of project, whether a prebuilt project (DuckDB + CSV demo) or an empty project from scratch.

By doing that, you have your project created locally. Afterward, cd into the project name. After doing this, you can execute the code . command to work on the project from your IDE. From here, you need to understand Latitude's project structure.

Project structure

Before working on any project, you need to understand its structure, the functionalities of each path, and the file in every path.

In Latitude, there are two significant directories: queries and views. But before going into that, here is the project structure of a Latitude project:

project-name/
├── queries/
├── views/

These are the only two paths to work with. In these paths, you will find files. The queries directory contains a configuration file, source.yml, and SQL queries. Although, when you create a project from scratch, you won't find any query files. You will have to create your query files.

Also, you can add your CSV file to the queries directory if you like. In this context, fundamental knowledge of SQL or other query languages is best. queries in Latitude can be regarded as the data source displayed in the embedded analytics.

For the views path, it controls the entire front end. You will find an HTML file in that directory that works with Svelte. In that file, you interact with your queries using parameters configured in your queries.

Now, we need to start working on queries. I will use this CSV file, Top 50 Spotify Music from 2010 to 2019, as the data source in this guide. You could download the file and keep it on your query path.

The next thing to do is set up your queries. This is where your SQL knowledge will be beneficial. Create a file in the same queries directory to capture your data source. You can use the query snippet below:

select title,
       artist,
       genre,
       music_year,
       BeatsPerMinute,
from read_csv_auto('queries/spotify.csv')
order by music_year asc

The code snippet above shows that the only fields to extract data are the title, artist, genre, music_year, and BeatsPerMinute fields from the CSV file. Toward the end of the snippet, the sixth line specifies the CSV file from which to pull data.

After doing this, you need another query to sum up your data. To do this, you can give the file a suitable name for an aggregated query file. In this case, I’ll name the query file firstcomp.sql.

select
  music_year,
  count(*) as all_titles,
from { ref('first') }
group by 1
order by 1 asc

In the query above, the music_year field from the CSV file was selected, and then the primary query file, first.sql , was referenced first. It indicates that data should be grouped differently and in ascending order.

To verify if your data was extracted correctly, you will have to run the query above using Latitude's CLI with this command:

latitude run second_filename

By doing that, this should be your result:

If you get something like this, you're on the right path. After verifying your queries and data source, the next thing to do is work on how the data should be displayed.

Now, let's navigate to the views/ directory to set the front end for the analytics we want to display.

Unlike the queries/ path, where we have to work with multiple files, we will work with index.html alone. You can add this snippet to your HTML code:

<svelte:head>
  <title>Demo project</title>
</svelte:head>
<View class="gap-8 p-8">
  <Row>
    <Text.H2 class="font-bold">Top 50 Spotify music dataset (2010 to 2019)</Text.H2>
  </Row>
  <Row>
    <LineChart
      query="firstagg"
      x="music_year"
      y="title"
    />
  </Row>
  <Row>
    <Table query="first" />
  </Row>
</View>

As you can see in the code above, it works with Svelte. Now, we aim to display analytics with the components available. The LineChart component was used because we wanted to display our data as a line chart. Afterward, we added values to the components to work with the parameters in the query files. I also added a table to work with the primary query file.

When you have that code in your index.html file, you can now run the project using this command:

latitude dev

By doing this, you'll be redirected to localhost:3000 from your terminal. Then, you should have this view on your browser displaying your data analytics.

You can embed your analytics into any website using React or HTML. If you are familiar with React, this documentation will guide you on embedding your analytics application with React.

To embed this in any application, you must ensure your Latitude application is deployed on Latitude Cloud.

To deploy your application, you need to sign up for Latitude Cloud.

latitude signup

If you have an existing Latitude Cloud account, you can log in to access it.

latitude login

After doing this, you can deploy your application by running the command below.

latitude deploy

Congratulations! You have your Latitude application deployed. 🎉

Now that we have our Latitude application deployed, the last thing to do is embed the analytics in our Latitude into another website. You can use the iframe embedding technique if you're not using React.

This could be done in three ways, depending on what you want to achieve by embedding. It could be embedded directly using parameters in the iframe's value or passing signed parameters.

To embed your analytics directly, you can put this iframe in your code:

<iframe src="https://app-url.latitude.so/" width="100%" height="400"></iframe>

In this context, you need to replace the source, https://app-url.latitude.so/, with the actual URL of your deployed application. The iframe code above helps embed your entire analytics into your web application.

That is the easiest way to embed analytics into your application, literally. You can learn about the other methods; passing parameters and passing signed parameters.

As much as there are tons of tools for visualizing data and embedding analytics into your application, Latitude makes this look very easy. It's not false hype, but the developer experience of Latitude is what makes it a great tool to use. Imagine connecting your data to your front end to integrate analytics into your application seamlessly - it's something you want to try out.

You can learn more about Latitude by going through Latitude's documentation. There are a series of guides that can walk you through everything about Latitude.

Conclusion

This article transitioned from embedded analytics to how you can get started with it easily using Latitude. If you don't understand how to use any embedded analytics tool, Latitude is one you can try out because of its seamlessness and developer experience.

Note that Latitude can be used with React or Svelte. Either way, they’re both great ways to use Latitude.

Support us 🙏 ⭐️

Latitude is open-source; you can contribute to the project as we want to make it more seamless for developers. If you love the project, you can go ahead and give it a star; we appreciate it!

Star Latitude on GitHub ⭐️

I hope you got the best out of this article. I look forward to hearing your thoughts about using Latitude for your embedded analytics in the comment section. I can't wait to have you read my next article. 👋

11 Tools To Improve Your Developer Experience by 10x 🚀

Opemipo Disu — Thu, 30 May 2024 14:40:50 +0000

There are numerous tools designed to make development seamless and efficient for developers.

As a developer, leveraging the right resources for aspects like development, design, and even communication can boost your level of productivity and streamline your workflow.

This article will cover 11 tools to take your developer experience to the next level and beyond. Also, it will cover reasons why they are essential, their benefits, and how to maximize their capabilities.

Whether you are a junior or mid-level developer, this article is for you. Let's get into this!

Overview of Developer Experience

Developer Experience (DX or DevEX) refers to the overall satisfaction and productivity developers have during their development cycle as they interact with various tools and environments. Similar to what non-technical folks know as User Experience (UX), Developer Experience specifically involves tools used by developers.

A great developer experience results in increased productivity and improved developer job satisfaction. Tools that offer a superior developer experience can substantially impact your work, allowing you to streamline tasks and produce high-quality code. These tools make coding, deployment, testing, and other development processes much more efficient.

Developer Experience is directly connected to developer productivity; working with tools that make life easier for developers boosts productivity. In essence, a great developer experience supercharges your developer productivity

Whether you're just starting your career or have some coding experience, optimizing your workflow with tools that offer a great developer experience is crucial. These tools provide a variety of features, such as:

Consistency in development environments.
Enhanced collaboration with your team.
Improved structures and code quality.
Automated workflows.

Here are a bunch of developer tools that can immensely improve your developer experience:

1. Visual Studio Code:

Visual Studio Code, or VSCode, is one of the most popular integrated development environments (IDEs). Developed by Microsoft, VSCode supports various technical tasks, including coding, debugging, and compiling. Its innovative features, such as IntelliSense, provide intelligent code completions based on imported modules, significantly enhancing coding efficiency.

VSCode is highly customizable, allowing developers to tailor their workspace to suit their needs. With a vast library of extensions, developers can add functionality to streamline their workflow, whether for coding, debugging, or deployment. These extensions have many purposes, from improving code quality and structure to automating repetitive tasks.

One of the mind-blowing features of Visual Studio Code is its seamless integration with GitHub. Without leaving the IDE, you can manage your repositories, commit changes, and sync code with remote repositories. This integration facilitates efficient version control and collaboration, enabling developers to push and pull code, manage branches, and review changes within the same environment.

Key Features of Visual Studio Code

Lightweight yet powerful IDE: Visual Studio Code offers numerous features while remaining a lightweight IDE, which contributes to its speed. Its extensibility can be considered "lightweight" as it doesn't include any extensions by default; you must install the ones you need. While coding, there's a lot you can accomplish. For instance, you can debug while performing other tasks. One of the primary reasons Visual Studio Code is considered a powerful code editor is its integration with IntelliSense. Even features like debugging and built-in Git support are potent tools that Visual Studio Code provides. You can perform many remarkable tasks with it, making it incredibly powerful.
Built-in support for multiple languages: Visual Studio Code supports hundreds of major programming languages, including JavaScript, PHP, Python, Go, etc. If the language you want to work with isn't supported by default, you'll need to navigate to the Extensions tab and search for extensions compatible with your desired language. These extensions can provide some of Visual Studio Code's built-in features, such as debugging and IntelliSense. Visual Studio Code also provides guides on how to use the different languages it supports by default.
Extensive library of extensions: In Visual Studio Code, developers have many extensions created to simplify code writing. In the environment, you'll find a tab for Extensions; this tab directly interfaces with the Visual Studio Code Marketplace. This Marketplace offers numerous extensions, allowing you to choose based on your needs.

Tips for maximizing use

Exploring and utilizing extensions for better functionality: As mentioned earlier, Visual Studio Code provides you with many extensions that different developers build. These extensions have their own purposes, but most aim to improve the workflow for developers. You can navigate to the Extensions tab in the environment and choose the one you want to work with. Here are some customizable extensions that can improve your workflow:
GitHub Copilot: AI code completion tool for code completion
Prettier: Automatic code formatting extension
Figma for VSCode: Extension for streamlining design to code workflow for Visual Studio Code.
GitLens: Supercharge Git in Visual Studio Code.
Using keyboard shortcuts to enhance productivity: Many small tasks can consume time, but using keyboard shortcuts can automate these tasks. For instance, formatting your code can take a lot of time, but it can be done quickly by pressing the Shift + Option + F keys. Visual Studio Code provides numerous default keyboard shortcuts to perform tasks and accelerate development. Note that keyboard shortcuts are customizable.

2. Tunnel

Tunnel is a collaborative tool that lets developers work together on a project by providing feedback and creating custom workflows. With Tunnel, developers can report bugs in your project. With just a few lines of code, you can start with Tunnel, making integration easy.

Integration with Tunnel is made easy with frameworks like React, Next (app and page routed), and VanillaJS. Additionally, Tunnel streamlines collecting, organizing, and resolving user issues.

Key Features of Using Tunnel

Simple integration: As mentioned earlier, Tunnel provides ease of use and accessibility for developers. It ensures that setting up and managing tunnels in local development environments is straightforward and requires minimal configuration. With just a few lines of code, you can start with Tunnel. Here's an example of how to set up Tunnel in a JavaScript environment.

<script
  src="https://tunnelapp.dev/__tunnel/script.js"
  data-project-id="PROJECT_ID"
></script>

If you're in a React environment, you can use this code snippet:

import { TunnelToolbar } from @tunnel/react;

export default function App() {
  return (
    <>
      <h1>My App</h1>
      <TunnelToolbar 
        projectId=YOUR_PROJECT_ID
        branch=BRANCH_NAME
      />
    </>
  );
}

Access control: When your project is deployed, you can invite people to collaborate. It lets you restrict who can access your project and share feedback by providing their email and role without giving them access to the entire organization.
Detailed bug capture: Bug capture in Tunnel goes beyond just capturing a bug description. It gives detailed information on network logs on the request and responses gotten when loaded, even browser data, console logs, and just a few other things. Using Tunnel might be a great way to make your project almost perfect if you like paying attention to details.

Tips for maximizing use

Monitor and log traffic: With Tunnel, you can monitor your network and console logs - using them to keep track of traffic. This can help you understand usage patterns, detect access issues, and capture errors.
Collaborative development: Tunnel allows you to share your development environment with team members for pair feedback or collaborative debugging sessions. This can be particularly useful for remote teams.
Create custom workflows: With Tunnel, you can integrate your favorite development tools, like GitHub, to link your pull requests to fit specific needs and enhance your development process, or even Slack for feedback. Custom workflows are used to improve overall efficiency in Tunnel.

3. Docker

Docker is a platform that lets you build, test, and deploy your applications using containers. It's a tool primarily used by cloud and DevOps engineers. Using the significant benefits of Docker saves you a lot of stress; it reduces the workflow of writing code and running it on production.

Containers, in this context, enable developers to package an application with its dependencies into a standardized unit for development.

With Docker, all of your applications are containerized. Therefore, it helps your application run consistently in different environments, from development to production!

Key Features of Docker

Containerization of applications: Docker isolates your applications, making them lightweight and faster than running them on a virtual machine. It ensures each container has its environment, so there's no situation where everything is running at once, which could potentially slow down your application.
Consistent environments: When you use Docker, you can see there is a structure. As mentioned, it provides consistency in different environments, such as the development, testing, and production environments. This means that your application runs the same way whether you're running an application on your local machine or in the cloud.
Resource Utilization: Containers are efficient when it comes to resource usage. They can allow massive deployments, with multiple containers operating on your local machine. This means you can run more applications on the same machine, saving time and improving your capacity to handle larger workloads.

Tips for maximizing use

Optimizing Dockerfiles: When a Dockerfile is structured to minimize the number of layers, each command creates a new layer. While building your application, ensure you have a Dockerfile in order; it's just the basic instructions for the Docker Daemon to follow when assembling a container image.
Logging and monitoring: To keep track of your applications, there are two things to do: logging and monitoring. Centralized logging tools such as Graylog, Splunk, Fluntd, etc., could be used to collect and analyze logs. Monitoring is essential to keeping track of your application's performance. If something terrible wants to happen or is about to happen, monitoring tools such as Grafana, New Relic, or even Prometheus are good tools to keep track of your application.
Reading Docker's documentation: Docker is one of those tools that provides clear and concise guides for its current and potential users. The guides on Docker's documentation are updated frequently as soon as their team makes an update. Aside from that, their blog gives detailed guides for the best practices for using Docker.

4. Jira

Jira is an agile product management tool for managing tasks and projects and tracking bugs, among many other things. Jira was developed by the Atlassian team to make collaboration easy while monitoring and managing tasks.

One of the most astonishing benefits of Jira is its integration with different tools to track your progress on tasks assigned to the tool. Using tools like Jira helps your team achieve its goals quickly; with this, managing functions for everyone in your organization is all in one place.

Jira also provides insights to check the team's progress within a certain period. If you are looking for a tool to provide accurate information on vital aspects of your team's work, Jira is for you!

Key Features of Jira

Issue and bug tracking: Jira issues are tasks, bugs, or stories. Essentially, they are items assigned to team members. Jira allows you to track these issues and their progress. For instance, when a user is assigned too many tasks, Jira can provide updates on incomplete tasks. It doesn't just ensure that you've worked on a task; it ensures that it is completed. It also helps you track bugs listed for fixes in the project's timeline so you can address them.
Customizable workflows: Jira allows you to make changes to issue workflows. You can change the issue type, edit workflow settings, and more. It's exciting to see how you can create significant transitions in your workflow.

Tips for maximizing use

Integration with third-party tools: To keep track of your team's work, Jira enables you to integrate directly with the tools you use in your daily activities as a developer. Tools like GitHub, Slack, Notion, etc., can be integrated seamlessly. Integrating with them lets you easily keep track of your work without doing it manually. For instance, if you integrate with GitHub, you no longer have to include updates manually, as it keeps track of your pull requests, issues, etc., in real time.
Utilize dashboards for project insights: As mentioned earlier, Jira provides its users with access to comprehensive project insights directly on its dashboard. This feature lets you see how well or poorly the team performs regarding assigned tasks and overall productivity. The dashboard offers detailed data about your project, including its metrics and analytics.

5. Slack

Slack is one of the most popular cloud-based communication tools for teams. Its primary purpose is to ensure accessible communication among team members. Unlike email for team communication, Slack allows you to interact with your teammates or external partners freely and accomplish much within a single platform.

Besides serving as a messaging platform, Slack enables you to meet with your team via audio or video huddles without needing an external platform. Slack is much more enjoyable than email communication; you can see who is online in your workspace and send them messages directly. This fosters better communication within teams, which can lead to increased productivity.

Key Features of Slack

Real-time communication: The primary use of Slack is communication—specifically, real-time communication. It's easy to communicate with your teammates on Slack; it feels like chatting with a friend in a professional context. Communication in Slack extends beyond messaging to include features like Slack Huddles. Huddles allow you to join team conversations without using external WebRTC tools like Google Meet or Zoom. Huddles are embedded within your Slack application. One notable aspect of Huddles is that they help retain the messages shared within them directly in Slack.
Integration with productivity apps and development tools: Slack allows you to integrate with numerous applications easily and provides real-time updates. If Slack is your team's default communication mode, you'll likely receive updates quickly. For example, if you use Slack to track GitHub issues, it will send notifications on updates immediately.

Tips for maximizing use

Create channels for specific projects or topics: Channels for organized discussions: Different Slack channels serve different purposes, similar to workspaces. Channels can be public or private, but they are created for specific purposes. Some channels may be used for support, general team communication, stand-up updates, etc. These channels make team communication very organized.
Using Canvas to share critical information: In your Slack environment, you can always find Canvas; it's fun. This allows you to keep or share information with the rest of your teammates. It lets you include valuable assets such as audio or video and add your workflows.

6. Postman

Postman is arguably the most popular API management platform. It doesn't only help with managing APIs; it also assists in building, testing, and automating APIs efficiently. Developers can even contribute to open APIs available on Postman. Its user-friendly interface for creating requests and debugging is one of the main reasons for its widespread use.

Beyond API development, Postman enables you to create custom documentation for your APIs. This documentation can be updated in real time, ensuring that all changes are reflected immediately. Additionally, Postman provides its users with automated testing, monitoring, and collaboration tools that allow multiple developers to work together seamlessly.

Key Features of Postman

Automated testing and monitoring: Postman lets developers write and execute tests automatically as a part of the API development cycle. Automating tests lets developers identify issues early before use. Developers can write tests to verify the APIs' definitiveness, performance, and reliability. It also allows for CI/CD workflows where tests are executed automatically, even when there's an update in the code. Developers can also set up monitors to run API tests. It enables Postman to send notifications to the development team at scheduled intervals. The monitoring is to keep track of the API's performance.
Collaboration features for API development: Postman offers a series of collaboration features for API development, enabling teams to work together efficiently. These features ensure that all team members are on the same page throughout the API development cycle. Postman provides personal and team workspaces, allowing members to share collections, environments, and other resources. It also offers role-based access control to manage who can make changes and view workspaces and collections. This provides a secure way to collaborate with developers on your team, especially when trying to keep sensitive information away from unauthorized users.
API development and testing: Postman is primarily used to develop APIs, providing an interface that makes this process easy. With Postman, teams can efficiently build, test, and document APIs, enhancing collaboration and productivity. Testing can be integrated with CI/CD tools like Jenkins, CircleCI, Semaphore, etc. Postman offers a friendly environment for writing tests and automating workflows.

Tips for maximizing use

Share collections with team members for collaboration. Collections in Postman are groups of requests. By granting your team access, they can edit and test the same APIs. This ensures consistency and fosters collaboration among team members. Additionally, sharing collections makes it easy for team members to manage APIs collectively.
Use collections to organize API requests. Collections allow you to group related requests to make them easy to manage. Organizing helps keep your work tidy and enables you to navigate requests easily when needed. Using collections helps to run requests simultaneously and automate workflows for efficient collaboration and testing.
Automate tests to catch issues early: Automated tests allow you to run predefined tests on your APIs regularly, ensuring they function correctly and meet some requirements. Integrating these tests into your CI/CD pipeline will enable you to automatically validate your APIs with each code change, identifying and addressing problems before they reach production.

7. Jenkins

Jenkins is a popular open-source automation server and CI/CD tool that works with your existing tools to automate stages for your development's continuous integration and delivery. It helps streamline the building, testing, and deployment of your code. By automating these tasks, Jenkins helps teams deliver high-quality software faster and with fewer errors.

Using Jenkins in your development workflow is a modern DevOps practice because it automates different parts of the software development process.

Key Features of Jenkins

Automation server for CI/CD: Jenkins is widely used to implement Continuous Integration and Continuous Delivery (CI/CD) pipelines. As mentioned earlier, it helps to facilitate the automation of different stages of the software development cycle. By doing this, you get improved code quality and an increased deployment frequency.
Plugin ecosystem: Jenkins provides its users with an incredible number of plugins to extend its functionality. Each plugin has many purposes attached to it, but they're designed to make your automation cycle seamless. The extensibility allows Jenkins to integrate seamlessly with any tool or technology you use for development.
It supports various programming languages and platforms. It also supports numerous underlying technologies and tools you use for development. It enables different teams using different underlying technologies to integrate with Jenkins, making it ideal for diverse development environments. This doesn't only apply to programming languages; it also applies to building tools. It lets you integrate build tools, such as Maven, Gradle, etc., to build processes, run tests, and deploy applications. Aside from build tools, it also lets you work with code management tools, deployment tools, and testing frameworks.

Tips for maximizing use

Install plugins to extend functionality. As mentioned earlier, Jenkins has a wide range of supported plugins. Using Jenkins alone could be limited, but plugins are great as they help increase capabilities.
Configure pipelines for automated workflows: Jenkins pipelines are robust; they help define your entire CI/CD process as code. By configuring pipelines for automated workflows, you can streamline your development process, ensure consistency, and reduce interventions.
Monitor builds and deployments for issues: Monitoring builds and deployments is important to maintain healthy pipelines in Jenkins. There are different ways monitoring could be done. It could be done by using monitoring tools such as Grafana, New Relic, Sentry, etc.; it could also be monitored by setting up deployment notifications when integrating with tools like Slack, email notifications, etc.; leveraging build and deployment history in Jenkins is also a way to monitor builds and deployment. Monitoring is essential; it helps notify you when something terrible is about to happen or is happening in your pipeline.

8. Terraform

Terraform is an infrastructure-as-code (IaC) tool developed by HashiCorp to enable developers to automate infrastructure tasks and provide infrastructure resources using a configuration language called the HashiCorp Configuration Language (HCL).

With Terraform, you can describe the desired state of your code infrastructure with provided resources such as virtual machines, networks, storage, etc. Terraform automates managing resources across different cloud providers and local environments.

It's mainly used for orchestrating and automating tasks.

Key Features of Terraform

Supports multiple cloud providers: Terraform supports many providers, allowing users to manage infrastructure across different platforms using a single configuration. This enables other teams to adopt various strategies for the cloud, leveraging the strengths of different providers while maintaining consistency in infrastructure management.
Declarative configuration language: Terraform provides a declarative configuration language called HashiCorp Configuration Language (HCL) for defining infrastructure resources and ensuring consistency across different environments. It gives a syntax that Terraform can interpret for both humans and machines. This documentation will guide you through learning HCL if you are interested.

Tips for maximizing use

Apply configurations to provision infrastructure: There are different ways to provide infrastructure. Terraform's commands, such as plan and apply to preview and execute changes, could be done as they ensure accuracy and minimize errors. Also, integrating Terraform with CI/CD tools like Jenkins or CircleCI to automate provisioning is another incredible thing to do. Also, storing state files remotely with AWS S3 or Terraform Cloud is a way to apply configurations to provide infrastructure.
Define infrastructure in configuration files: Use configuration files to ensure your infrastructure code is consistent. Storing configurations allows for easy collaboration and auditing. Configuration files improve clarity, making understanding and managing your infrastructure easier. Organized files with clear formatting are easier to maintain, just as reusing code with modules promotes efficiency and reduces duplication.

9. ESLint

ESLint is a popular open-source tool for JavaScript developers; it helps to analyze code, find problems, and improve code quality. It's integrated into IDEs and can run as a part of your CI/CD process. Apart from improving code quality, it also analyzes your code by showing its potential errors and maintaining coding standards.

ESLint also helps organize your code by improving the readability of your codebase. It dramatically helps to patch incorrect patterns to provide high-quality code.

Key Features of ESLint

Customizable rules: ESLint allows you to configure rules to maintain consistency in your code. It offers many rules but can be fine-tuned easily from the project's configuration file. Each rule has its purpose but comes with many built-in rules. You can add more rules using plugins or change the available rules.
Integrates with various IDEs: As mentioned earlier, ESLint integrates with most code editors and IDEs to provide feedback and ensure fewer errors and repetitions in the environment. It helps to add an auto-fix in your IDE when working within a JavaScript environment.
Static code analysis: ESLint helps examine your code for errors and enforce coding standards without debugging. This allows developers to identify potential issues early in development, leading to more reliable and maintainable code.

Tips for maximizing use

Use recommended rulesets: Starting with ESLint's recommended rulesets to quickly implement best practices and common coding standards is a great thing to do when using ESLint, as these rulesets cover a wide range of common issues and help ensure excellent code quality.
Integrate with build systems for automated linting: ESLint lets you integrate with build tools to run automatically within your build system or CI/CD pipeline. This ensures that code is consistently checked for errors and adherence to coding standards with every build, preventing code with potential errors from being merged into the main codebase.

10. Kubernetes

Kubernetes, or K8s, is an open-source platform for orchestrating containerized applications. It's designed to manage the lifecycle, deployment, and scaling of containers, providing a unified system for automating the management of complex distributed applications.

Its features are just valid reasons why Kubernetes is labeled an essential tool for modern DevOps practices.

Key Features of Kubernetes

Extensibility: Kubernetes offers an extensible structure, allowing users to integrate additional features and extend functionality tailored to their needs. It also supports deployment across different environments, including public, private, and hybrid clouds.
Automates deployment, scaling, and operations: Kubernetes automates containerized application deployment, ensuring a reliable and smooth deployment process. It also automates scaling based on resource usage and handling tasks such as health checks and upcoming updates.
Manages containerized applications: Kubernetes also provides management capabilities, including scheduling, orchestration, and resource management. It ensures containers are distributed properly across clusters, maintaining desired states, and managing their lifecycle.

Tips for maximizing use

Leverage Kubernetes for microservices architecture: Kubernetes is designed for managing microservices. It allows you to deploy, scale, and update individual services easily. This can improve the scalability of your applications by isolating failures and enabling efficient resource usage.
Use Helm for managing Kubernetes applications: Using Helm's package manager for Kubernetes to simplify deployment and management of applications. By using Helm charts, you can define, install, and update complex Kubernetes applications with a single command.

11. Mintlify

Mintlify is a platform designed to simplify the process of creating and maintaining technical documentation for developers. It offers templates that allow users to embed their content with minimal effort, streamlining the documentation process.

Mintlify provides features to help users create and maintain up-to-date documentation. Instead of building a new documentation site from scratch, Mintlify allows users to configure a few settings and integrate their content seamlessly.

Key Features of Mintlify

Markdown Support: Mintlify provides templates, but after choosing, you must embed your content in Markdown for simplicity. It has a guide on how to write markdowns when you use Mintlify.
Customizability: You can customize a template tailored to your needs. With Mintlify, users can create and modify their templates to align with their branding and style and ensure consistency in the documentation. You can customize the navigation structure and sidebar organization to fit your needs.
A.I. search: The AI search feature is one of the most exciting things on Mintlify. It is designed to enhance the readability and efficiency of documentation by leveraging AI to provide more accurate and relevant search results based on the contents of the documentation.

Tips for maximizing use

Optimize AI search: Use Mintlify's AI search to improve the readability of information. Ensure your content is well-structured and labeled to enable users to get better search results.
Monitoring documentation analytics: Leveraging analytics is a powerful way to maximize the use of Mintlify. By understanding how users interact with your documentation, you can make some decisions to improve its effectiveness and usability.

Conclusion

By leveraging the 11 tools in this article and maximizing their capabilities, developers can significantly increase their developer experience to the next level. Each tool offers incredible features and benefits that address different aspects of the development lifecycle.

This doesn't apply to coding alone but to other aspects like collaboration, automation, linting, etc. This article explains these tools, their features, and how to get the most out of them to make your life as a developer easier.

Congratulations if you've made it to this point of the article.

Just a quick one: we have a project; Latitude is an open-source framework for embedding analytics into your application using code. I'd appreciate it if you would give us a star on our GitHub repository.

Give Latitude a Star on GitHub 🌟

If you found this article useful, let us know in the comments. We hope to have you read our next blog post! 😃

Things I wish I knew before I started an open-source project

Opemipo Disu — Thu, 23 May 2024 12:02:20 +0000

New to the world of open source? You’ll likely make a few mistakes in your first open-source project 🤷‍♂️. In this article, I’ll share key takeaways from our experience building Latitude, covering what you need to know before launching your first open-source project.

If you’re considering taking the plunge, this article is for you!

How to properly set up an OSS project on GitHub

A well-configured project requires more than just a basic setup. It needs detailed information to give potential users and contributors a clear understanding of the project’s objective and functionality. This can be achieved by crafting a clear and concise project description, accompanied by a comprehensive README file that provides step-by-step guidance on how to use the project. This article offers invaluable insights on what to include in your README files and provides tips on writing an effective one.

Beyond that, it’s also essential to establish a contribution guideline and a code of conduct to maintain a respectful and inclusive community.

A contribution guideline provides clear guidelines for making contributions to the project. This includes instructions for submitting issues and pull requests, as well as outlining coding standards and best practices. A code of conduct, on the other hand, sets the tone for a respectful and inclusive community, covering aspects like language choices, behavioral standards, and more.

Check out Latitude for examples of all these.

Choosing the right license

When I started my first open-source project, I underestimated the importance of choosing a license that fitted my project.

Open-source licenses protect users and contributors and govern how others can use, modify, or distribute an OSS project. There are two broad categories of open-source licenses: Copyleft licenses and Permissive licenses.

Copyleft Licenses ensure any derivative work inherits the original OSS license and this can protect the original author from bad actors abusing their work. Latitude, for example, uses an LGPL copyleft license, which grants some protections against commercial competitors using Latitude to compete with us. Permissive licenses, on the other hand, are less restrictive and don’t enforce any license reuse in derivative work.

Likewise, it is very important to have all contributors sign a Contributors License Agreement, which stipulates the terms under which intellectual property has been contributed to your project. This ensures contributors cannot easily sue you for code they might have contributed to your project in the past.

To have all contributors sign the CLA, there’s a handy Gtihub Action that automatically requires new contributors to do so.

Version control and branch protection rules

Open-source collaboration often involves working with contributors whom you don’t necessarily trust. To avoid potential disasters, it’s crucial to set clear boundaries for what contributors can do. Failure to do so could lead to mistakes, such as deleting main branch code, which can be extremely difficult to resolve.

Here’s a set of sane defaults we use at Latitude:

Require a pull request with approval from code owners to merge code to main branch
Enforce approval after any code change to pull requests
Require all CI status checks to pass before merge
Only give permission to merge pull requests to code owners
Optionally allow your closest team to bypass some of these restrictions

How to maintain coding standards

In an open-source project, it’s essential to maintain coding standards. When code is written with consistency and clarity, it becomes easier to understand, maintain, and contribute to, ultimately making the project more sustainable.

One effective way to ensure coding standards are upheld is by incorporating linters like ESLint into the development workflow. Linters scrutinize code for syntax errors, formatting inconsistencies, and stylistic issues, providing instant feedback to developers on how to improve their code.

In addition to linters, code formatters like Prettier can further reinforce coding standards. These tools automatically format code according to pre-defined rules, eliminating inconsistencies in indentation, spacing, and syntax.

By combining a linter with a code formatter, open-source projects can establish a unified coding standard that is both easy to enforce and maintain. This not only improves the overall quality of the code but also creates a more welcoming environment for new contributors to join and contribute to the project.

How to manage new releases

Managing releases in an OSS project can be complicated, especially when publishing versioned packages of your software. In contrast to software-as-a-service, where you control the version users access, OSS users can install a particular package version and never update it. This increases the importance of shipping stable and bug-free software.

As a general rule, I recommend against completely automating the release of your software and instead have a manual approval step required at some point in the process. Tools like changesets, the one we use at Latitude, greatly help with this. Changesets takes care of publishing new packages, updating changelogs, and tagging releases, yet it can only do so via automatic pull requests that have the same approval requirements as any other PR at Latitude.

Some more recommendations include maintaining proper human-readable changelogs, tagging releases, and making sure you follow semantic versioning – package managers rely on it for safely keeping project dependencies up to date.

Lastly, a word on prereleases. If you publish versioned packages of your software I greatly encourage you to QA new versions in alpha release. When properly tagged, package managers will ignore these package versions, allowing you to properly and safely test new features in production-like environments. Changesets can also help you with prereleases.

Security best practices

Security practices in OSS projects are not much different from the ones you’d follow in a closed-source project. So here’s a somewhat comprehensive list we follow at Latitude:

Update dependencies regularly: Make use of automated tracking of minor dependency updates with tools like Dependabot, and try to update any major version of your dependencies every so often – at Latitude we review all our dependencies once a month or so.
Code Reviews: Implement a code review process, especially for the sensitive areas of your code that might deal with production secrets.
Access control: Implement role-based access control to restrict access to sensitive parts of your project, control who can trigger new releases, and who can push code to your main branch. GitHub provides tools for all of these.
Secure storage: Never share production secrets in OSS code, ensure only code owners and administrators have access to sensitive areas of your org like shared password managers, PaaS providers and whatnot.

These practices will only strengthen your project’s security; it’s best to keep up with them.

Conclusion

In conclusion, launching a successful open-source project requires careful consideration of several key factors, including defining the project’s purpose and goals, choosing the right license, setting up a proper project on GitHub, understanding version control, maintaining coding standards, managing new releases, and implementing security best practices.

By learning from the mistakes and experiences shared in this article, first-time open-source contributors can avoid common pitfalls and set their projects up for success.

Thanks for making it to the end of this article. I hope you found the article helpful!

Do well to help me

Latitude is an open-source framework for embedding analytics into your application using code.

Please consider giving it a Star on GitHub.

Lastly, if you have any questions about what I've shared in this article, you can drop a comment below.

I hope to have you read my next piece! 😃