Nvidia's Chat with RTX brings a personal AI chatbot to your PC

Chat with RTX uses retrieval-augmented generation (RAG), Nvidia TensorRT-LLM software and Nvidia RTX acceleration to bring generative AI capabilities to local, GeForce-powered Windows PCs.

Nvidia, known for its chips and graphics processing units (GPUs), is venturing into the world of generative AI.

The chip manufacturing giant on February 13 released an early version of its ‘Chat with RTX,’ which lets users personalise a chatbot with their own content on Windows PCs.

What is Chat with RTX?

Chat with RTX is a free tech demo that lets users create their own chatbot using local files as sources of knowledge.

Think of it this way: you have a bunch of documents and notes stored on your computer. By adding these local files on a PC as a dataset to an open-source large language model like Mistral or Llama 2, users can generate relevant answers to queries.

Chat with RTX uses retrieval-augmented generation (RAG), Nvidia TensorRT-LLM software and Nvidia RTX acceleration to bring generative AI capabilities to local, GeForce-powered Windows PCs.

RAG is a method that improves the precision and dependability of generative AI models by incorporating information obtained from external sources.

The tool supports various file formats, including .txt, .pdf, .doc/.docx and .xml.

Additionally, it also allows users to include information from multimedia sources, including YouTube videos and playlists. For example, users can add a YouTube link to the tool and ask specific questions about the video. The tool will then give answers based on the content of the video.

What are the requirements?

Users can download and install Chat with RTX from the Nvidia website.

To use the tool, users need a PC running on Windows 10 or 11 with a GeForce RTX 30 Series GPU or higher with at least 8GB of video random access memory (VRAM), and the latest Nvidia GPU drivers.

What sets it apart?

Since Chat with RTX runs on Windows RTX PCs and workstations, the responses are swift.

And unlike other AI chatbots like ChatGPT or Gemini, Chat with RTX allows users to handle sensitive data locally without the need for third-party sharing or an internet connection.