experiment

Header

Customers

LogoCloud

Footer

PreFooter

SectionTitle

NextImage

BoxCustomer

AnnouncementContextProvider

Screen

scale.com (Production)

lottie-react

plasmic-nav

plasmic-embed-css

plasmic-query

plasmic-basic-components

plasmic-tabs

plasmic-cms

react-chartjs-2

plasmic-strapi

react-slick

radix-ui

react-quill

Logos - Valeo.svg

50-AI-logo.svg

bcg.svg

Black.png

Logos - Global Atlantic.svg

Howard_Hughes_logo.png

4DV Analytics - png.png

Logos - Uncountable.svg

Logos - Grata.svg

stability ai.svg

Logos Antrhopic.svg

image.jpg

Logos - Brex.svg

Logos - Nuro.svg

kaleido_ai_black.svg

Luminar Logo White Horizontal Vector 4.svg

Scale Collaborates with NVIDIA to Power the Next Generation of LLMs with NeMo SteerLM

NVIDIA

NVIDIA-Endeavor-building-logo.jpg

Logos - Nvidia.svg

Scale AI is proud to collaborate with NVIDIA to power the next generation of LLMs for generative AI.&nbsp; By leveraging Scale AI&rsquo;s high-quality training datasets and NVIDIA&rsquo;s NeMo SteerLM - a simple LLM alignment technique that allows dynamic steering of models during inference - developers can create applications for a variety of enterprise use cases including education, chatbots, and gaming.

High-quality Training Datasets Improve LLMs for Diverse Enterprise Applications

The evolution of generative AI brings the challenge of tailoring language models to enterprise needs, which requires task-specific training data.&nbsp; To improve model performance, NVIDIA partnered with Scale AI to collect annotated datasets covering tasks such as Rewrite, Summarization, Classification, Extraction, Closed QA and more. The SteerLM models trained with this dataset are more factually accurate and coherent, while supporting the capability to adjust for preferred verbosity and complexity at inference time.&nbsp;

Addressing Challenges of Aligning LLMs with Task-specific Training Data

Our SteerLM collaboration demonstrates the immense potential of combining the latest advancements in generative AI with expert-curated datasets.&nbsp; As NVIDIA and Scale continue to partner on additional data initiatives, including multi-turn retrieval augmented generation (RAG), we hope to unlock even more powerful enterprise applications across industries. Generative AI is evolving rapidly.&nbsp; As techniques like SteerLM become more capable, the need for high-quality training datasets will continue to grow to build models.&nbsp; Through our world-class Generative AI Data Engine, Scale is committed to delivering the best datasets tailored to your specific needs. Together, NVIDIA and Scale are excited to open source high-quality datasets that can be used to align LLMs.&nbsp; We look forward to empowering more organizations to leverage NVIDIA&rsquo;s managed services and models to transform their industries and shape the future of AI.

NVIDIA and Scale Collaborate to Shape the Future of Generative AI

To support generative AI advancements, NVIDIA and Scale have&nbsp;<a href="https://huggingface.co/datasets/nvidia/HelpSteer">open sourced the dataset</a>. This dataset contains 37k samples with various response attributes (helpfulness, correctness, coherence, complexity and verbosity).&nbsp; The Scale AI team helped build this dataset with human experts to evaluate each response on a scale of 0-4. Scale's Generative AI Data Engine combines automation and human intelligence to rapidly generate training data tailored to specific AI goals and data needs.&nbsp;
Scale's <a href="https://scale.com/rlhf">Generative AI Data Engine</a> played a key role by providing expert-generated datasets, while NVIDIA's SteerLM offers a simple yet powerful solution for dynamic model steering. This collaboration enables developers to customize language models that are capable of adapting to individual learning preferences in education, enhancing chatbots for managed services, and creating immersive, human-like interactions in gaming. Powered by Scale&rsquo;s training datasets, NVIDIA&rsquo;s SteerLM unlocks new capabilities in education, gaming, and beyond. In education, SteerLM enables the customization of LLM responses, addressing the varied needs of teachers and students. Adjustable knobs for Complexity and Verbosity cater to individual learning preferences, ensuring a personalized and effective learning experience. The integration of SteerLM into <a href="https://techcrunch.com/2022/09/20/nvidia-launches-new-services-for-training-large-language-models/">NVIDIA's NeMo </a>empowers developers to adapt LLMs for many purposes including chatbots, content generation and question answering. In gaming, <a href="https://www.nvidia.com/en-us/geforce/news/nvidia-ace-steerlm-npc-personalities/#:~:text=Our%20newest%20advancement%20is%20the,emotive%2C%20realistic%20and%20memorable%20interactions">SteerLM enhances non-player characters</a> (NPCs) by allowing developers to customize personalities, resulting in more emotive and realistic dialogues and emotions.
Releasing this data enables the broader research community to develop better language models and continue innovating research.&nbsp; For example, enterprise software developers can leverage NVIDIA&rsquo;s open source toolkits and advanced models across a variety of industry applications, without having to build LLMs from scratch.

Scale Worked with NVIDIA to Build a First-of-kind Multi-attribute Preference Dataset

Cohere works with Scale to improve its Command model with high-quality prompt/response pairs. 

Cohere Enhances its Large Language Models with Scale.

Cohere

C-Blog-Post-4x-pink--1-.png

Cohere is a natural language processing (NLP) startup whose aim is to make NLP useful and accessible to anyone who needs it. Cohere&rsquo;s platform allows users to generate, categorize, and organize text at a scale that was previously unimaginable by developing large language models that easily integrate with existing systems.

Cohere is Building the Future of Language AI

The team at Cohere had an urgent need to acquire high-quality data to train the first iteration of their instruction-following model, Command. However, the Cohere team, &ldquo;needed to build up an annotation force quickly, and Scale provided access to a large-scale, diverse, and skilled workforce to employ for this complex data creation job,&rdquo; explained Alex Wang, Technical Lead at Cohere. The team at Cohere knew they needed a partner who could provide diverse and complex inputs (prompts) and outputs (responses) to ensure their models would perform well on challenging and unusual queries their users may have.

Cohere Needed High-Quality Data For Their Instruction-Following Model.

The data provided by Scale accounted for a sizable portion of Cohere&rsquo;s first Command dataset. By collaborating on instruction-following best practices and doubling down on complex, tightly paired data, Scale contributed to performance gains for Cohere&rsquo;s Command model. Command is one of the highest-performing LLMs across <a href="https://crfm.stanford.edu/helm/latest/?group=core_scenarios" target="_blank" rel="noopener noreferrer">HELM</a> and other standardized metrics. Cohere retrains its LLMs on a weekly basis to constantly improve the models, and that requires a regular stream of high-quality data.
&nbsp;
&ldquo;Scale has been incredibly responsive to our data needs, both at a macro level (designing novel workflows for collecting complex types of data) and a micro level (adjusting existing data collection protocols to better align the data with our needs). We give feedback to Scale weekly, and they are always receptive to our feedback, positive or negative. If we request a change, they are quick to move and have the change implemented within a couple of weeks. We feel like valued customers at Scale,&rdquo; concluded Wang.

Cohere's Command LLM Improves with High-Quality Data

Large Language Models (LLMs) are typically developed in 3 stages:
<ol>
<li>Stage 1: Training with a large corpus of raw data&nbsp;</li>
<li>Stage 2: Instruction fine-tuning using prompt/response pairs</li>
<li>Stage 3: Reinforcement learning with human feedback (RLHF)&nbsp;</li>
</ol>
Scale and Cohere initially partnered on stage 2: instruction fine-tuning using prompt/response pairs. Partnering with Scale solved both of Cohere&rsquo;s challenges. &ldquo;They have a skilled workforce capable of delivering high-quality prompt/response pairs and their engineers worked closely with us to develop the necessary tooling and infrastructure. All we had to do was communicate our needs with Scale. This greatly freed up our engineering time and resources to tackle other urgent problems while still acquiring the critical data we needed,&rdquo; said Wang. Scale was also able to cover a wide distribution of questions to provide a greater lift in model performance.

Scale's Generative AI Data Engine Delivers High-Quality Prompt/Response Pairs.

Customer Success Story: Cohere | Scale AI

Creating large volumes of training data without sacrificing quality.

As part of this mission and vision, one of TRI’s key research areas is automated driving, developing two different automated driving modes in parallel - Guardian and Chauffeur.

Guardian monitors a human’s driving task, intervening only when necessary. Chauffeur mode, on the other hand, takes all responsibility for driving, leaving occupants purely as passengers.

Toyota Research Institute’s mission is to build a new approach to mobility and pioneering the technologies that drive its future.

Led by Adrien Gaidon, TRI&rsquo;s Machine Learning team found itself with large volumes of data, but limited ability to label this data.
The need for large volumes of annotated data was obvious, but with a commitment to safety being non-negotiable, the TRI team did not want to trade off quality for quantity.
To support the research teams, David Garber and Ashmi Wadhwani were tasked with managing the data annotation pipeline, data infrastructure and machine learning infrastructure.
Very quickly, they recognized the need for a labeling provider they could rely on long term to meet the ever-changing needs of TRI&rsquo;s researchers.

Large volumes of data, but limited ability to label this data

TRI will continue to leverage Scale's latest annotations and features, such as Sensor Fusion Segmentation.

Leveraging latest annotations and features

TRI looked into various solutions for its data annotation problem, but made Scale its main provider. This close working relationship led to TRI participating in the private beta of Sensor Fusion Segmentation, a challenging annotation type in which every point in a 3D point cloud needs to be painted.

Scale’s Sensor Fusion Cuboids, Sensor Fusion Segmentation and Semantic Segmentation

Customer Success Story: Toyota | Scale AI

Developing accounts payable training data with Scale Document AI

Logos - SAP.svg

SAP is a leading software corporation headquartered in Walldorf, Germany, that develops enterprise software for the &ldquo;Management of business processes, developing solutions that facilitate effective data processing.&rdquo; Known best for its enterprise resource planning (ERP) software, SAP aims to help companies and organizations of all sizes run their businesses profitably, adapt continuously, and grow sustainably. &ldquo;We have a rich spectrum of use cases ranging from document processing, analytics, master data matching, process automation, and many more,&rdquo; said Urko Sanchez, Senior Director, Head of AI Functions, Europe. &ldquo;For this collaboration with Scale, we focused on improving our products around document processing, especially those dealing with invoices, purchase orders, and payments advices,&rdquo; he added. &ldquo;By bringing AI-powered products and services, we are helping companies to modernize and become future-proof,&rdquo; he concluded.

Scale Document AI for Accounts Payable

The team at SAP had a trove of customer documents but needed a partner to create a comprehensive dataset to enhance their accounts payable products&nbsp;while respecting the data ownership, privacy, and sensitivity of its customers. High-quality data is critical for performant models. SAP needed higher quality training data to train models for processing and extracting crucial information from purchase orders and invoices in English, German, and Spanish. &ldquo;Our customers are very heterogeneous. We have customers that provide us with thousands of documents a week, while others might require months for a fraction of the same volume. Sometimes, customers might not even know in advance what amount of data they have available,&rdquo; explained Sanchez. &ldquo;Therefore, finding a reliable and flexible partner for creating high-quality training datasets was key for us,&rdquo; he added.

Document variability made high-quality training data a challenge

ML models learn associations between input and output data. If there are errors in training or validation data, the resulting model will underperform or learn the wrong task. By leveraging Scale, SAP was able to improve their Business Document Processing (BDP) services portfolio. &ldquo;Our services are now much more accurate and can deal with new types of documents and languages,&rdquo; explained Sanchez. &ldquo;Overall, we have experienced a great boost in accuracy across the board. The whole process has also been a valuable experience to understand better the trade-off between the cost of higher quality data and the accuracy of our models,&rdquo; concluded Sanchez.

Enhanced Business Document Processing Services

With deep expertise in labeling data across a wide range of use cases, the team at Scale delivered high-quality labeled data across three languages, multiple document types, and 200+ unique fields. Adding to the complexity, fields in documents that contained personally identifiable information (PII) such as names, phone numbers, and emails need to be replaced with semantically similar but different examples to preserve data privacy. Despite these challenges, Scale delivered labeled data at near-perfect accuracy on a quick ramp to high volumes.
&ldquo;We decided to go with Scale because we needed a heavily technical partner that could deal with changing requirements dynamically, guaranteeing availability, quality, and scale,&rdquo; said Sanchez. The Document AI team did this by partnering closely with SAP to understand the technical and business requirements of SAP&rsquo;s Business Document Processing (BDP) model, then leveraging a combination of machine learning-powered pre-labeling (ML pre-labeling) and Scale&rsquo;s global labeling operations to deliver high-quality data.

Data extraction at over 95% accuracy from diverse document types under 60 seconds

Optimus Ride focuses on rider safety for robotaxis

Developing safe and reliable systems with high-quality 3D training data

Optimus Ride is a Boston-based company that specializes in developing autonomous vehicles and systems for geo-fenced environments. Optimus Ride partners with local communities around the country to develop custom mobility systems capable of transporting people and goods safely and reliably. Optimus Ride’s team has focused on Geo-fenced environments to-date including mixed-use developments, city zones, office and industrial parks, ports, airports, academic campuses, retirement communities, and more.“The team is constantly developing cutting-edge algorithms to develop safe and reliable systems,” explained Michaela Mann, Product Marketing Manager at Optimus Ride. With this hyperlocal strategy and cutting-edge technology, Optimus Ride brings together the promise of self-driving technologies with real-world considerations to enable safe, efficient, and sustainable mobility networks.

Optimus Ride partners with local communities around the country to develop custom mobility systems capable of transporting people and goods safely and reliably

The Optimus Ride team initially labeled their data in-house. &ldquo;Unfortunately, it just wasn&rsquo;t as quick as we had hoped,&rdquo; explained Nuno Almeida, Team Lead and Manager of AI, Optimus Ride. As Optimus Ride expanded into new environments and worked on developing more novel perception algorithms, they collected more data than they could effectively label in-house. &ldquo;We needed a partner who could more efficiently and quickly label all this data we&rsquo;re collecting,&rdquo; added Almeida.

More data than they could effectively label in-house

"We're focused on making sure we can bring all of the wonderful technology that exists today to our customers and our clients. We&rsquo;ve deployed across four states and our nation&rsquo;s capital so far and we work really closely with and listen to the needs of our clients to adapt with them and provide different product offerings that will allow us to continue expanding,&rdquo; explained David Robert, Head of Product at Optimus Ride. &ldquo;We're going to continue to increase our deployments, and the training data Scale AI provides will ensure that what we&rsquo;re doing actually has practical use and enduring value for people,&rdquo; he concluded.

Adapting with customers and providing product offerings that can scale

The Optimus Ride team selected Scale AI as its data annotation provider for a few reasons; &ldquo;With Scale AI, the main thing was that we can get our data labeled more quickly and at a higher quality than doing it in-house,&rdquo; noted Almeida. Second, &ldquo;We really liked all the SLAs that were provided to us and the whole Scale AI team stepped up and worked with us to meet our needs,&rdquo; said Mann. &ldquo;Scale&rsquo;s commitment to quality has been really amazing. We&rsquo;ve sent over different schemas and different attributes several times. Each time, we were able to align quickly and easily address any issues as they arose,&rdquo; noted Kushagra &ldquo;Kush&rdquo; Tiwary, Perception Engineer, at Optimus Ride. &ldquo;The hardest part of perception is all the edge cases. Having a partner that can flag these and address them swiftly has been great,&rdquo; Tiwary added.
Lastly, &ldquo;From an engineering point of view, the APIs that Scale provides allows us to create a system to automate all the interactions between our systems and Scale&rsquo;s systems,&rdquo; said Almeida. &ldquo;All the logging functions allow us to easily check for errors and help with the whole process of quality assurance,&rdquo; Almeida concluded.

Labeled data more quickly and at higher quality

Customer Success Story: Optimus Ride | Scale AI

Goodcall uses Scale Rapid's text-classification data labeling and annotations to improve their chatbots

Goodcall Boosts Call Quality with Scale Rapid

Goodcall

Goodcall provides businesses with intelligent phone agents so that their customers can always get in touch. When customers call these bots to request information like the business&rsquo;s hours, location, and products or services, these phone agents perform automatic speech recognition (ASR) to convert speech-to-text. Then, once the text is acquired, the bots employ AI analysis to interpret the person&rsquo;s request and to provide the appropriate response. The intent classification models that power Goodcall&rsquo;s chatbots need to be regularly fine-tuned with real-world production data. Any performance improvement in the model also means an improvement in the customer experience for Goodcall&rsquo;s clients. With a high volume of calls coming in every day, labeling massive amounts of data with high-quality annotations is a time-consuming process. To address this challenge, Goodcall began working with Scale Rapid to acquire high-quality text annotations. These annotations provide the data that software engineers need to evaluate and improve their models, while also freeing up time and resources at the company.

Goodcall Leverages Scale Rapid to Improve their State-of-the-art Chatbots

Before working with Scale Rapid, Goodcall had mountains of real-world production data that was left unannotated. Because they were performing data annotation in-house, they couldn&rsquo;t match the scale of available data. Every piece of unlabeled data was one less example that Goodcall could use to improve their models.
By training their models on a larger, more diverse dataset, Goodcall could unlock opportunities for improving model performance. With additional data, they could represent a wider range of customer experiences and identify and address edge cases that frequently led to model failures. For example, Goodcall could recognize utterances that their models performed poorly on, and make adjustments to compensate, improving overall accuracy. To provide the best possible results for every customer call, Goodcall needed a scalable, sustainable approach for labeling large quantities of data.

Goodcall Had Too Much Data to Annotate In-house

Since Goodcall started working with Scale Rapid, they&rsquo;ve improved their model&rsquo;s intent classification by 5%. By outsourcing the labeling process, they&rsquo;ve also freed up engineering time to tackle other problems. There have been additional unexpected benefits, too &ndash; Using Scale Rapid&rsquo;s text classifications, Goodcall has identified opportunities for improvement in their speech recognition system, by uncovering limitations in their transcript and intent classification models.&nbsp;
These text labels provide Goodcall with valuable insight into their chatbots, allowing them to quantify their models, identify issues, and assess why their models perform the way they do. Armed with this information, they&rsquo;re able to make intelligent decisions to improve their product and, by extension, the customer experience.

Goodcall Provides an Even Better Customer Experience

Scale Rapid provides simple, user-friendly tools that make it easy for companies like Goodcall to acquire the high-quality annotations they need. Goodcall can easily send a large collection of samples to Scale Rapid to be reviewed. In return, they receive text classification labels that describe the meaning of certain phrases. For example, if a text sample were to say, &ldquo;I want to know when this business closes,&rdquo; Goodcall would receive a label back from Scale indicating that this phrase is requesting business hours.
Goodcall extracts even more value from the annotations they get from Scale Rapid by checking these human-reviewed labels against the unlabeled items in their system. This way, they can have a wider variety of data that they can use to improve their models. With these annotations, Goodcall can improve the accuracy of a model&rsquo;s text-to-speech transcripts, as well as enhance a bot&rsquo;s ability to correctly understand intent.

Scale Rapid Provides Goodcall with High-quality Text Classification Annotations

Customer Success Story: Goodcall | Scale AI

Scaling ground-truth data with Scale AI. Classifying large volumes of images to develop an autonomous checkout system

Standard Cognition strives to improve everyone’s retail shopping experience by making shopping a seamless and more human interaction. Standard does this by developing an autonomous checkout platform for brick and mortar retailers that can change how people shop in the future. Their approach to autonomous checkout is unique because of their vision-first approach that identifies customers by shape and movement, rather than by facial recognition, and customers can shop in a Standard-powered store with or without their mobile app.

About Standard Cognition

&ldquo;Our system was very naive because it was still very early stage,&rdquo; said Virginia Puccio, Standard&rsquo;s Head of Data Operations. To improve the performance of their deep learning models, Standard's team required a large and varied dataset. Despite this need, &ldquo;we didn&rsquo;t have a workforce to handle the volume of annotation tasks, nor did we have the tools,&rdquo; she added. Puccio and the Standard team knew they needed a scalable data labeling solution.

Improving the performance of models

Those in the San Francisco Bay Area can experience the future of shopping at Standard&rsquo;s flagship store in downtown San Francisco. As Standard expands its operations globally, more and more shoppers will be able to experience this new paradigm for themselves.

Experience the future of retail

The Standard Team selected Scale for their expertise in working with large data sets. &ldquo;It was easy to get started with Scale AI because they had the platform and the people with the skillset to perform the annotation tasks we needed,&rdquo; noted Puccio. &ldquo;While we&rsquo;re much bigger now and could potentially handle the volume of annotation tasks, why would we want to? ... We believe we should leave data annotation to experts like Scale so we can focus on perfecting the retail shopping experience with regard to the use of AI,&rdquo; Puccio concluded.
&nbsp;
By working with Scale, the Standard team have been able to focus on other aspects of their business, such as the collection of data and offering more support services to their customers.

Perfecting the retail shopping experience

Customer Success Story: Standard Cognition | Scale AI

At Harvard Medical School, the Datta Lab accelerates their research using Scale Rapid with data labeling and annotation.

Harvard Medical School's Datta Lab Accelerates their Research Using Scale Rapid

Harvard Medical School

The Datta Lab at Harvard Medical School is studying the neural mechanisms associated with behavior in rodents. To conduct their research, they place mice in an experimental rig, and then record each mouse&rsquo;s behavior using cameras. They also measure each mouse&rsquo;s neural activity using neural implants. They can then analyze this data to examine the link between neural activity and behavior, as well as to advance their understanding of specific brain areas of mice. Actually analyzing this data, however, is a time-consuming process &ndash; To interpret the behavioral data, the researchers must first label the poses of the mouse over time. While the lab can train machine learning (ML) models to perform this automatically, a large amount of video footage needs to be manually annotated first. This annotation process detracts from the time that researchers could instead be spending on the aspects of research that require their expertise. The Datta Lab at Harvard Medical School is now working with Scale Rapid to get the annotations they need. Instead of spending weeks labeling their data, they can pour their full attention into advancing their research.

With Scale Rapid Annotating their Data, the Researchers in the Datta Lab Can Focus on the Science

wdcgTEwzgKk-HD.jpg

Analyzing a mouse&rsquo;s behavioral data requires annotators to take the raw video of a mouse and extract detailed kinematic variables that describe its body&rsquo;s position at each moment in time. In particular, researchers are often interested in the mouse&rsquo;s joint position over time, such as the location of its toes, knees, and hips.
Before researchers can train a model to detect these key points, however, they first need to build a training set. This is not a quick process &ndash; An annotator needs to sit in front of a computer for hours labeling hundreds of images. Before the Datta Lab worked with Scale Rapid, this annotation was performed entirely in lab. This consumed a huge amount of time that could otherwise have been spent on the more salient aspects of research.

The Datta Lab Needed to Speed Up their Data Annotation Process

Instead of spending weeks annotating data, the lab now sends their data to Scale Rapid and gets quality annotations as quickly as overnight. The easy access to data annotations has lowered the barrier to training new models, so the lab has been able to try out new ideas and iterate more rapidly. Now, if a researcher needs to collect new training data after updating an experiment, or if they need to track a different part of the mouse in existing data, they can easily send their footage to Scale Rapid for new annotations. This makes it possible for the lab to explore possibilities that simply weren&rsquo;t feasible before.
Additionally, the ease with which the lab can obtain high volumes of data annotations has made it easier to produce high-performing ML models. Having a large amount of data can reduce the need for model fine-tuning &ndash; The majority of edge cases are naturally handled by the model, so researchers don&rsquo;t need to spend weeks adjusting their model for these shortcomings. By using Scale Rapid, the Datta Lab can now spend more of their time advancing their research in neuroscience.

With Scale Rapid, the Datta Lab gets Better Research Outcomes Faster

Now that the Datta Lab is working with Scale Rapid, they no longer need to annotate their data in-house. Instead, they can send their raw videos of mice to Scale Rapid and quickly get quality annotations in return. These annotations provide the lab with the information they need to train their ML models for automated pose tracking.
Obtaining annotations from Scale Rapid saves the lab weeks of labeling time, allowing researchers to spend their time more effectively. Working with Scale Rapid also allows the lab to acquire a higher volume of annotations than they could ever accomplish in-house. Having a large volume of high quality annotations makes it easier than ever for the lab to train high-performing ML models, while also freeing up researcher time.

Scale Rapid Quickly Provides the Datta Lab with the Annotations They Need to Advance their Research

Customer Success Story: Harvard Medical School | Scale AI

Kaleido AI asked Scale to help them generate synthetic data to help them continue to improve their model performance on object identification and improve the IoU of their model predictions. Kaleido AI had plateaued at an IoU of 0.657 with real-world data and needed to find a way to improve its model continuously.

Scale’s Synthetic Data Helps Kaleido AI Bring Better Visual AI to Everyone

Kaleido AI

kaleido_ai_color.svg

Headquartered in Vienna, Austria, Kaleido AI&rsquo;s mission is to make complicated tech simple. By creating tools that simplify and accelerate workflows, foster creativity, and enable others to bring new ideas to life, the company provides everyone - from individuals to businesses of all sizes - access to the latest advancements in visual AI.
In 2019, Kaleido AI introduced remove.bg, the automatic image background remover, built to eliminate manual work and complex tools with high learning curves. In 2020, the team launched Unscreen, an online software that removes video backgrounds in no time. These simple but effective tools dramatically increased the speed at which users could achieve their goals. As a result, their popularity skyrocketed, which ultimately led to Canva acquiring Kaleido AI in early 2021.
Later that year, Kaleido AI launched <a href="https://www.designify.com/?utm_source=partnership&amp;utm_medium=blog&amp;utm_campaign=synthetic_case_study">Designify</a>, the AI-powered tool that creates automatic designs for individuals, car dealerships, e-commerce websites, and more.
Kaleido AI&rsquo;s incredibly effective and accurate tools use machine learning algorithms to automatically detect objects and backgrounds and return background-free photos and videos. Through easy-to-use APIs and plugins, developers and editors accelerate their workflows using Premiere Pro, After Effects, or other editing software.
Kaleido AI uses Scale Synthetic for synthetic data generation, edge case identification, and Scale&rsquo;s machine learning expertise.

Synthetic Data is Key for Improving Kaleido AI's Models

Kaleido AI takes an AI-first approach to its visual editing tools, with a talented team of ML engineers on staff. Enabling simple, automated visual editing tools that work requires excellent machine learning models and excellent machine learning models need a large volume of high-quality data.
However, Kaleido AI encountered several edge cases in a specific segmentation task where their model performed poorly. Collecting and labeling tens of thousands of real-world images with a large diversity of patterns, images, backgrounds, and textures was difficult. Open datasets did not have enough high-quality images of this particular class.
In the initial attempt to solve this issue, Kaleido AI relied on real-world data to train its segmentation models. Unfortunately, collecting and labeling a large amount of diverse real-world data was complex, resource-intensive, and costly. After this initial effort, the team determined that instead of spending the time and resources to develop their models with real-world data, they should consider using synthetic data to augment their dataset. So, they sought help from the experts in synthetic data at Scale AI.&nbsp;

Kaleido AI Needed Better Data to Improve Segmentation for Challenging Edge Cases

Today, Kaleido AI&rsquo;s model is built on five hundred real-world images and almost fifteen thousand synthetic images from Scale. Kaleido AI needed to improve its model performance and efficiency to enable visual AI for everyone. Scale Synthetic helps Kaleido AI achieve its goal by enabling continuous and efficient model performance improvement on target edge cases.
Looking forward, Kaleido AI intends to continue to increase the amount of synthetic data in its dataset, using Scale&rsquo;s Synthetically generated data to improve its models. There are many use cases that Kaleido AI is considering focusing on next, and Kaleido AI will continue to look to Scale to help them improve their model performance.
With Scale Synthetic, Kaleido AI has reduced the time and effort involved with synthetic data curation, so the team can now focus on improving its core application.

With Scale Synthetic, Kaleido AI Improves Model Performance While Focusing on Building Their Core Product

Kaleido AI asked Scale to help them generate synthetic data to help them continue to improve their model performance on object identification and improve the IoU of their model predictions. Kaleido AI had plateaued at an IoU of 0.657 with real-world data and needed to improve its model continuously.
Scale&rsquo;s ML Engineers started by analyzing Kaleido AI's sample data and model inferences in Nucleus, Scale&rsquo;s data curation platform. Scale quickly identified that the model was performing poorly in segmenting objects in images with complex patterns, the objects were shaded or transparent, or where there were shadows in the backgrounds of the scenes. Scale focused on these edge cases and generated a sample of 2,650 images of synthetic data with varied lighting, textures, and patterns. However, this first pass was not sufficient to meaningfully improve model performance.
The Scale team recognized that they needed more data specifically targeted at problem edge cases to improve model performance substantially. So the team did a deep dive into <a href="https://scale.com/nucleus" target="_blank" rel="noopener noreferrer">Nucleus</a> to curate data to further identify these problem edge cases. They also introduced the ability to visualize Scale&rsquo;s synthetic images compared to real images in 2D space. To do this, the Scale team first passed all the images to a RESNET 101 backbone and took the vector embedding of each image at various layers before passing them to t-SNE. Then, the team visualized which synthetic images were furthest in 2-dimensional space from real images and which real images formed a distinct and unaddressed cluster of images. Using this method, the Scale team identified the real images furthest from synthetic images and prioritized these edge cases.
This analysis revealed that Scale needed to include more textured/patterned objects and a wider variety of object types in the synthetic data distribution.
<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fb1568c7d117f4044baaab8b8be3c1154">
Upon further investigation, the clump of red images towards the center of the plot were patterned objects.&nbsp;
This analysis identified additional patterns and other edge cases on which to focus. We also determined that white objects on white backgrounds with dirt or speckled backgrounds or where there are two layers of background (i.e., rooms with doors) were causing poor model performance.
The Scale team now had a clear understanding of what was needed to improve model performance and set about generating a larger dataset with data targeted at addressing these edge cases. In total, Scale generated 14,583 synthetic images covering a total of 12 categories covering patterns, various objects, backgrounds, and textures. With this targeted synthetic data, Kaleido AI achieved an IoU of 0.794.
<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fc9b63e6ed407451bbe168b3add1c6f66">

Synthetic Helps Kaleido AI’s ML Team Prioritize and Generate Better Data to Improve Model Performance on Business-Critical Edge Cases

Customer Success Story: Kaleido AI | Scale AI

Voxel utilizes Scale Studio to supercharge their in-house annotation team, performing video and image annotation, and categorization.

Building In-house Labeling Operations for High-Quality Training Data

Voxel

Voxel is on a mission to leverage AI and computer vision to fundamentally change how companies manage risk and operations. To do this, Voxel enhances its customers' security cameras with real-time AI to detect hazards, risky activities, and operational inefficiencies.

"We enable safety managers to keep more people safe across large warehouses and spaces. Our technology delivers insights that can stop injuries before they happen and ultimately improve the safety culture for hazardous and dynamic environments." - Harishma Dayanidhi, VP Engineering

To develop a robust computer vision system, Voxel needs large amounts of high-quality training data that they can train their models on. A plethora of situations and edge cases need to be accounted for, such as potential hazards, risky activities, and inefficiencies in an industrial setting, that Voxel's system needs to be able to identify.

Voxel is Changing How Companies Manage Risk and Operations

Voxel's computer vision team faced two challenges: 1) how to maintain high-quality training data and 2) how to automate their labeling process for faster throughput &ndash; all while retaining their in-house annotation team.&nbsp;
Voxel had already invested the time and effort to assemble an in-house annotation team of subject-matter experts who were well-versed in handling Voxel's specific use case. Voxel saw a strategic advantage in keeping its internal labeling operations. With a team in place, Voxel began looking for a solution that could introduce greater efficiency to its labeling operations.
Until now, the team had been using an open-source solution called Computer Vision Annotation Tool (CVAT). However, the computer vision team at Voxel was ramping up the volume of annotations they needed for model training and was running into significant bottlenecks with CVAT.&nbsp;
From the operations side, Voxel could not efficiently and programmatically collect data and insights on the data labeling process, resulting in significant manual effort from the data operations team. The open-source tool couldn&rsquo;t effectively link data quality to individual annotators. Thus, if the team produced a batch of low-quality labels, they couldn't determine whether it was the training, the annotators, or something else. This environment made it difficult for Voxel to automate its data labeling process and scale its labeling operations.
On the engineering side, with CVAT, Voxel needed to custom-build data pipelines for new customer projects. Given the complexity of the data pipelines, this process took multiple engineers four weeks to build the required data infrastructure for each project.

Producing Quality Training Data while Automating the Process

After kicking off the project with Scale Studio, Voxel onboarded their 20+ subject-matter experts onto the platform. Studio gave Voxel's data operations managers visibility into their in-house labeling team with annotator metrics such as throughput, efficiency, and accuracy. Studio also made it easy to streamline a data labeling process with intuitive tools and standardized workflows. They could now forecast labeling capacity and plan to match variable labeling demands. Compared to their previously ad hoc and manual approaches, Voxel's operations managers saved 20% of time each week using Studio.
"Our data operations managers were spending 20% of time each week to manually log batches of data, assign datasets to annotators, and estimate project completion times. We couldn't standardize the workflow and accurately provide visibility to other teams. With Scale Studio, we streamlined this process, giving us the clarity we needed." - Harishma Dayanidhi, VP Engineering
Studio also helped Voxel's computer vision engineering team increase its capacity. With Scale's APIs, it was easy to integrate multiple data pipelines into their operations. There was now less manual work for engineers. With Studio, the engineering team cut their time by 50% to kick off new projects.

Efficient Operations for High-Quality Training Data and Time Savings

The confluence of these two factors led them to look for a partner with strong data annotation expertise who understands the challenges and pain points of managing one's annotation team.&nbsp;&nbsp;
Scale Studio was selected because of:
<ul>
<li>Studio's comprehensive management features with training courses, benchmark tasks, and annotator metrics (i.e., throughput, efficiency, accuracy, etc.)</li>
<li>Scale's APIs for easy integration of data pipelines and quick set up of labeling projects</li>
<li>Scale's ecosystem of integrated ML tools, such as Nucleus for dataset curation and management, and Rapid for Scale-managed dataset annotation</li>
<li>Scale's experience with processing billions of annotations, confirming Scale's platform reliability and tried-and-true infrastructure</li>
<li>Finally, Scale's credo of "earn customer love" provides the Voxel team with the responsiveness and support necessary to achieve their ambitious goals</li>
</ul>
While Studio has proven incredibly easy to use for the project management and annotator teams, Voxel has called out the Scale's customer success and engineering teams' strong technical knowledge and responsiveness in getting the most out of the platform. For example, Scale partnered with the Voxel team to ensure that all frames of complex variable-frame-rate (VFR) videos were extracted to maximize the accuracy of the annotations&ndash;and thus the accuracy of the model.&nbsp;
&nbsp;

Labeling Operations Require APIs, Admin Features, and Integrated Tools

Customer Success Story: Voxel | Scale AI

Research and development of next-generation autonomous systems that can safely operate in a range of work environments

OTTO Motors is an Ontario-based company that specializes in developing flexible and intelligent mobile robots for automated materials handling in manufacturing facilities and warehouses around the world. OTTO’s industrial-grade platform moves goods intelligently through facilities when and where they are needed, streamlining production while safely avoiding pedestrians. OTTO robots operate safely and efficiently in a wide variety of environments. “One of our biggest initiatives is to make sure our robots are reliable and accurate; ensuring they function effectively in the various environments where they are being deployed,” said James Servos, Autonomy Engineering Manager at OTTO Motors.

About OTTO Motors

To research and develop next-generation systems that can better understand, navigate, and safely operate in a range of work environments, the OTTO team needed data from a variety of diverse environments. At the time the OTTO team first began working on this problem, most publicly available datasets for autonomous mobility were specifically for autonomous road vehicles like cars and trucks. &ldquo;For what we wanted to do, there was no publicly available data,&rdquo; said Nolan Lunscher, Senior Autonomy Developer at OTTO. &ldquo;We had to start from scratch on getting datasets and we needed a partner who could supply us with a steady supply of data in a domain that was quite different from what everyone else was working on,&rdquo; he added.

Developing Datasets from Scratch

By working with Scale AI, Servos and his team are able to focus on their actual work of improving its systems rather than on data annotation. &ldquo;Someone like [Lunscher] is an incredibly high-value worker as an autonomy engineer. Working with Scale means keeping him focused on developing cutting edge algorithms and state of the art networks,&rdquo; Servos noted.
&nbsp;
The diversity of environments and areas OTTO&rsquo;s systems must operate in is vast. &ldquo;Every time we deploy to a new customer site, we find new corner cases. We&rsquo;ll continue to refine our systems and push more data through Scale&rsquo;s labeling pipeline to develop generalized networks that can function in all of these locations,&rdquo; Servos concluded.

The Future of Warehousing and Manufacturing

The OTTO team selected Scale AI as its data annotation provider for a few reasons. First, Scale AI&rsquo;s ability to annotate a range of data types from 3D sensor data to camera imagery gave the OTTO team confidence in Scale AI to handle the kinds of data they had. Second, Scale AI&rsquo;s experience in the autonomous driving space meant Scale AI could help the OTTO team better understand possible edge cases. &ldquo;The On-Demand Service also gave us a good opportunity to trial the system to get a feel for how the process works and how the tools work,&rdquo; said Lunscher. &ldquo;Once we had a feel for it, we were able to jump in with our larger project,&rdquo; he noted.
&nbsp;
&ldquo;The cost of developing all of the tools and workflows to annotate data for a company like us was just completely not feasible,&rdquo; Servos added. &ldquo;The Scale AI team was able to annotate our data at a pace far above what we would have expected while keeping the overhead for data annotation reasonable,&rdquo; he concluded.

Throughput and Volume OTTO Can Count On

Customer Success Story: OTTO Motors | Scale AI

Making drones useful

To accomplish this goal, Skydio is building the easiest-to-fly, hardest-to-crash drone featuring a state-of-the-art subject tracking and obstacle avoidance system that allows Skydio’s drones to follow its customers through the woods, over the mountains, and across the sea.

Skydio’s goal is simple: to make drones useful by making them smart

To ensure their subject tracking and obstacle avoidance system functions for a wide variety of scenarios and locations, Skydio&rsquo;s autonomy team, led by Hayk Martiros, needed a large and varied dataset to train its models with. In order to focus on actually developing its models and drones, the Skydio team knew they needed a data labeling partner they could rely on.

Finding the right partner for accurate data annotation

With the release of the Skydio 2, customers can see and experience for themselves the most advanced drone on the market.

The Skydio 2

Skydio looked into various solutions for its data annotation needs, but chose Scale AI for our product features, flexibility and team. &ldquo;The team and infrastructure Scale AI had ready to go let us focus on using our data instead of labeling it,&rdquo; noted Martiros. Scale AI&rsquo;s team made itself a valuable annotation partner by being responsive and continually working to improve its products.
&ldquo;When we presented them with an unusual labeling need, they added a feature to do it,&rdquo; said Rachel Basse. &ldquo;We appreciate how the Scale AI team proactively seeks feedback and holds themselves accountable. Their QA team is careful and quick to seek clarification, and their attention to detail has produced some truly impressive results,&rdquo; she added. They readily adapt with us when unforeseen complications arise in our data. We can implement small changes to requirements with a short conversation or roll out new projects in a few weeks, with minimal development time from us,&rdquo; Martiros concluded.

Scale AI's Polygon Annotation

Customer Success Story: Skydio | Scale AI

Annotating large volumes of videos to develop advanced control technology for workboats and other commercial vessels

Sea Machines Robotics is a Boston-based autonomous technology company that specializes in advanced control technology for workboats and other commercial vessels. Their industrial-grade systems can be easily installed aboard existing commercial vessels, or included in new-build packages to enable remote and autonomous operations of all types of commercial vessels. Based in Boston and operating globally, Sea Machines is driven by a crew of mariners, engineers, coders, and autonomy scientists. “I grew up sailing and on the water,” said Ephraim Dobbins, Vice President of Technology at Sea Machines. “Joining Sea Machines was a great opportunity to do something interesting that combined a personal hobby with my professional interest in machine learning and perception,” Dobbins continued.The Sea Machines team sees autonomy as a spectrum, from fully autonomous vessels to what they call “pilot assist” or “crew assist” systems. Full autonomy can be useful in repetitive tasks such as a commuter ferry that makes the same run multiple times a day or in hazardous situations, such as search and rescue operations or oil spill clean-ups. In all other scenarios, Sea Machines’ systems can make operating vessels safer for the crews operating them with collision avoidance systems providing more situational awareness.

About Sea Machine Robotics

Dobbins and his team are focused on improving their perception systems and deploying them in the field. &ldquo;That means understanding the level of accuracy we need, and looking at how we can get to that accuracy,&rdquo; said Dobbins. The small but mighty Sea Machines team may have been able to handle annotating a smaller volume of data, they needed a partner that could scale their data pipelines and give them the flexibility to ramp volumes up or down. &ldquo;We just didn&rsquo;t feel that building the technical expertise to annotate large volumes of data was something we wanted to do. We&rsquo;d wanted to stick with someone that was very competent at doing it than building that organization ourselves,&rdquo; Dobbins concluded.

Scaling Data Pipelines

As Sea Machines continues to expand its customer base globally, the team is continuing to collect data from more and more vessels. By partnering with Scale, the Sea Machines team has been able to generate enough data to train and test their systems. &ldquo;It&rsquo;s a virtuous cycle. I can look at the performance across multiple datasets from multiple vessels and get really good information which allows us to iterate and learn more,&rdquo; Dobbins said.
As the Sea Machines system gets deployed in more vessels, more crews of mariners will be able to experience the future of workboat and commercial vessel safety. Beyond commercial vessels, Dobbins and the Sea Machines envision a future in which their systems will be available for recreational boats.

The Future of Workboat and Commercial Vessel Safety

The Sea Machines Team selected Scale AI as it&rsquo;s data annotation provider because as the Sea Machines team diversified and increased its data sources, Scale AI could provide both the quality and the scale the team needed.
By the time Dobbins joined, he &ldquo;found everything to be running smoothly.&rdquo; &ldquo;The quality of the annotation, the ability to keep up with the quality while turning volumes up and down has been great,&rdquo; he added.

Quality and Scale Sea Machines Can Count On

Customer Success Story: Sea Machines Robotics | Scale AI

Kodiak is building better AI for trucking with Scale Synthetic.

Safely handling autonomous trucking edge cases with synthetic data.

Kodiak Robotics

Kodiak Robotics is an autonomous technology company that is building self-driving capabilities and technologies for the long-haul trucking industry. Based in Mountain View, CA, Kodiak leverages a unique sensor fusion system combined with a lightweight mapping solution to safely navigate all aspects of highway driving and deliver freight efficiently and on-time. Kodiak’s team, which includes several self-driving industry veterans, is redefining the long-haul trucking industry by “building the world’s most efficient, reliable, and respected end-to-end delivery solution.”

Kodiak Is Building Perception and Autonomy Systems that Drive the Future of Freight Transportation.

In most ideal driving scenarios, trucks don&rsquo;t encounter pedestrians on the highway. When they do, though, knowing how to detect and navigate unexpected situations is a requirement for any production-level, autonomous vehicle system. Kodiak&rsquo;s software stack learns how to identify and navigate rare scenarios by training models on examples. But it&rsquo;s often difficult to collect enough examples in the real world to reliably handle certain edge cases. For Kodiak, one of those challenging edge cases is pedestrians walking on the highway.

Large Training Datasets, yet Few Examples of Important Edge Cases.

In Nucleus, Kodiak plans to continue to use Natural Language search and Autotags to find the specific scenes in their dataset that had edge cases they needed to improve their model on. This includes&mdash;among other scenarios&mdash;scenes where construction workers are present and where a vehicle is traveling under a bridge.
For efficiency, the Kodiak team centralized all of their data, including multiple labeling projects and raw, unlabeled data, into a single dataset. This allows the team to quickly iterate on model experiments, query for specific attributes or metadata on the fly, and close the loop for a more end-to-end data and model management system.
Going forward, the team is able to review both insights and model metrics in Nucleus in order to identify scenes with poor IoU (intersection over union) and curate subsets of data where their model wasn&rsquo;t performing well, in which additional synthetic data might be helpful.

Nucleus Helps Kodiak Identify Where More Synthetic Data will Improve Accuracy.

The Kodiak team chose Scale to provide synthetic data to augment Kodiak&rsquo;s existing ground-truth training data with simulated pedestrians. Scale provides a unique human-in-the-loop synthetic data generation process to create diverse and realistic synthetic data. Trained taskers can validate the placement and poses of synthetic pedestrians to ensure the synthetic data is realistic. Scale delivers the data using the same dashboard and APIs as their existing annotation pipeline, making integration seamless.
&nbsp;

Increase Model Robustness by Training on Synthesized Rare Scenarios.

Customer Success Story: Kodiak Robotics | Scale AI

Powering automated bill pay with Scale Document AI

Brex Inc. is a financial service and technology company that is building the all-in-one finance solution for businesses to, &ldquo;launch confidently, scale fast, and realize their full potential.&rdquo; The future of FinTech will have products and services that reimagine every area of the financial system with a friendlier and faster experience that unlocks more value for customers. Brex aims to be the technology first provider of financial services for businesses. For example, with Brex Premium and features like Bill Pay, businesses can manage and pay their bills in one place simply by uploading a bill or invoice. Much of the industry, however, still relies on manual, error-prone workflows including document processing.
&ldquo;The role that AI plays here is that we want to make it so a business just needs to send us an invoice, like a PDF. And we can take that PDF, extract useful information from it, and make it one click to actually pay that bill,&rdquo; explained Michael Mattheakis, a Software Engineer on the Brex Bill Pay team.
Faster and higher-quality automation of these workflows is a critical component of Brex&rsquo;s strategy, and for the future of the industry in general.

Products

Government

Resources

Customers

Customers

Scale Collaborates with NVIDIA to Power the Next Generation of LLMs with NeMo SteerLM

Cohere Enhances its Large Language Models with Scale.

Creating large volumes of training data without sacrificing quality.

Developing accounts payable training data with Scale Document AI

Developing safe and reliable systems with high-quality 3D training data

Goodcall Boosts Call Quality with Scale Rapid

Scaling ground-truth data with Scale AI. Classifying large volumes of images to develop an autonomous checkout system

Harvard Medical School's Datta Lab Accelerates their Research Using Scale Rapid

Scale’s Synthetic Data Helps Kaleido AI Bring Better Visual AI to Everyone

Building In-house Labeling Operations for High-Quality Training Data

Research and development of next-generation autonomous systems that can safely operate in a range of work environments

Making drones useful

Annotating large volumes of videos to develop advanced control technology for workboats and other commercial vessels

Safely handling autonomous trucking edge cases with synthetic data.

Powering automated bill pay with Scale Document AI

Copymint Prevention for NFT Marketplaces

Machine intelligence for faster and safer real estate transactions

Processing mission-critical logistics documents with Scale Document AI

Velodyne uses Scale Nucleus to find edge cases in 3D data and curate the most valuable data for annotation

Delivering a Seamless Cellar Management Solution

Enhancing Log Scaling and Inventory Management with Scale Rapid

Nuro mines for rare classes in unlabeled data with Nucleus Object Autotag.

Scale Rapid Helps Orchard Robotics Deliver State-of-the-Art Precision Crop Management

With Scale Studio, Vistapath Enhances Patient Experience.

Enhancing Pick-and-Place Robots with Annotations from Scale Rapid

Pietra Enhances E-Commerce Product Imagery with Scale.

With Scale Rapid, advanced.farm Scales its Apple-Picking Operations

The future of your industry starts here.