Overview
High-quality Training Datasets Improve LLMs for Diverse Enterprise Applications
Scale AI is proud to collaborate with NVIDIA to power the next generation of LLMs for generative AI. By leveraging Scale AI’s high-quality training datasets and NVIDIA’s NeMo SteerLM - a simple LLM alignment technique that allows dynamic steering of models during inference - developers can create applications for a variety of enterprise use cases including education, chatbots, and gaming.
The Problem
Addressing Challenges of Aligning LLMs with Task-specific Training Data
The evolution of generative AI brings the challenge of tailoring language models to enterprise needs, which requires task-specific training data. To improve model performance, NVIDIA partnered with Scale AI to collect annotated datasets covering tasks such as Rewrite, Summarization, Classification, Extraction, Closed QA and more. The SteerLM models trained with this dataset are more factually accurate and coherent, while supporting the capability to adjust for preferred verbosity and complexity at inference time.
The Solution
Scale Worked with NVIDIA to Build a First-of-kind Multi-attribute Preference Dataset
To support generative AI advancements, NVIDIA and Scale have open sourced the dataset. This dataset contains 37k samples with various response attributes (helpfulness, correctness, coherence, complexity and verbosity). The Scale AI team helped build this dataset with human experts to evaluate each response on a scale of 0-4. Scale's Generative AI Data Engine combines automation and human intelligence to rapidly generate training data tailored to specific AI goals and data needs.
Scale's Generative AI Data Engine played a key role by providing expert-generated datasets, while NVIDIA's SteerLM offers a simple yet powerful solution for dynamic model steering. This collaboration enables developers to customize language models that are capable of adapting to individual learning preferences in education, enhancing chatbots for managed services, and creating immersive, human-like interactions in gaming.
Powered by Scale’s training datasets, NVIDIA’s SteerLM unlocks new capabilities in education, gaming, and beyond. In education, SteerLM enables the customization of LLM responses, addressing the varied needs of teachers and students. Adjustable knobs for Complexity and Verbosity cater to individual learning preferences, ensuring a personalized and effective learning experience. The integration of SteerLM into NVIDIA's NeMo empowers developers to adapt LLMs for many purposes including chatbots, content generation and question answering. In gaming, SteerLM enhances non-player characters (NPCs) by allowing developers to customize personalities, resulting in more emotive and realistic dialogues and emotions.
Releasing this data enables the broader research community to develop better language models and continue innovating research. For example, enterprise software developers can leverage NVIDIA’s open source toolkits and advanced models across a variety of industry applications, without having to build LLMs from scratch.
The Result
NVIDIA and Scale Collaborate to Shape the Future of Generative AI
Our SteerLM collaboration demonstrates the immense potential of combining the latest advancements in generative AI with expert-curated datasets. As NVIDIA and Scale continue to partner on additional data initiatives, including multi-turn retrieval augmented generation (RAG), we hope to unlock even more powerful enterprise applications across industries.
Generative AI is evolving rapidly. As techniques like SteerLM become more capable, the need for high-quality training datasets will continue to grow to build models. Through our world-class Generative AI Data Engine, Scale is committed to delivering the best datasets tailored to your specific needs.
Together, NVIDIA and Scale are excited to open source high-quality datasets that can be used to align LLMs. We look forward to empowering more organizations to leverage NVIDIA’s managed services and models to transform their industries and shape the future of AI.