PrivateGPT v0.6.0: Recipes!

At PrivateGPT, our goal is to empower developers to build private AI-native applications with ease. To achieve this goal, our strategy is to provide high-level APIs that abstract away the complexities of data pipelines, large language models (LLMs), embeddings, and more. Taking a significant step forward in this direction, version 0.6.0 introduces recipes - a powerful new concept designed to simplify the development process even further.

Introducing Recipes!

Recipes are high-level APIs that encapsulate AI-native use cases. Behind the scenes, they orchestrate complex data pipelines to deliver complete results.

With the introduction of our first recipe, summarize, we're not only enriching PrivateGPT with this valuable feature but also laying the groundwork for community-driven recipes.

Summarization Recipe

The Summarize Recipe provides a method to extract concise summaries from ingested documents or texts using PrivateGPT.

POST /v1/summarize

Use Case

The primary use case for the Summarize tool is to automate the summarization of lengthy documents, making it easier for users to grasp the essential information without reading through entire texts. This can be applied in various scenarios, such as summarizing research papers, news articles, or business reports.

Key Features

1. Ingestion-compatible: The user provides the text to be summarized. The text can be directly inputted or retrieved from ingested documents within the system.
2. Customization: The summary generation can be influenced by providing specific instructions or a prompt. These inputs guide the model on how to frame the summary, allowing for customization according to user needs. Some examples of instructions: “return a list of higlights”, “add a TL&DR at the beginning”, “return the summary in Spanish”…
3. Streaming Support: The tool supports streaming, allowing for real-time summary generation, which can be particularly useful for handling large texts or providing immediate feedback. If the text to summarize is large, the time to first token will be impacted.

Bring your own Recipe

We're excited about the possibilities for future recipes in PrivateGPT, but we'd like to invite the community to take an active role in shaping the platform's capabilities. By contributing their own recipes, users can help make PrivateGPT a truly versatile and powerful AI-native toolbox, capable of driving innovation across various industries.

If you have ideas for improving the Summarize or want to add your own recipe, feel free to contribute! You can submit your enhancements via a pull request on our GitHub repository.

Other improvements

Streamlined cold-start

We've worked to make running PrivateGPT from a fresh clone as straightforward as possible, defaulting to Ollama, auto-pulling models on first run, making the tokenizer download optional...

More models and databases support

We’ve added support for Gemini (both LLM and Embeddings) and for Milvus and Clickhouse vector databases.

Special thanks to the Milvus team for the contribution, looking forward to more collaborations!

Documentation improvements and minor bugfixes

You can read through the full list of changes in the release notes.

Breaking changes

There are some breaking changes coming in this version:

• The minimum required Python version is now 3.11.9, and Poetry must be >= 1.7.1. We recommend updating to Poetry 1.8.3.
• Default LLM model changed to LLaMA 3.1 for both Ollama and Llamacpp local setups.
• Default Embeddings model unified to nomic-embed-text for both Ollama and Llamacpp local setups. It will break your current setup if you used a different model to ingest files.

We’ve prepared a full document on how to workaround and adapt to these breaking changes here.

Public roadmap released

In addition to the recent improvements and enhancements, we have been actively collecting feedback and suggestions from the community, as well as planning a comprehensive roadmap that outlines our vision for future changes and developments. This public roadmap encompasses a wide range of updates, including new features, bug fixes, and performance optimizations. Choose the task you’d like to work on and become part of the collaborators team.

Coming next… easier deployment!

Following the community request, we are focused on improving the way PrivateGPT is deployed to different environments. In the coming weeks we’ll be releasing cointainerized versions of PrivateGPT ready to be deployed, and improvements to the docker-based building capabilities of the project. Stay tuned!

Kudos to the team and the community

Javier, AI Product Engineer at Zylon, has been the most active contributor of the version followed by the Fern team, Catherine Deskur, denisovval, Shengsheng Huang, Robert Hirsch, Quentin McGaw, Proger666, Mart, Marco Braga and Jackson. Thanks all!!

Final note for companies & organizations

If you are looking for an enterprise-ready, fully private AI workspace check out Zylon’s website or request a demo. Crafted by the team behind PrivateGPT, Zylon is a best-in-class AI collaborative workspace that can be easily deployed on-premise (data center, bare metal…) or in your private cloud (AWS, GCP, Azure…).

‍