RiskRAG: A Data-Driven Solution for Improved AI Model Risk Reporting

Stable Beluga 2 is a Llama2 70B model finetuned on an Orca style Dataset. This repository contains the model from the stabilityai/StableBeluga2 repository with the following changes: -Storing weights in bfloat16 instead of float32. This leads to 2x smaller files and a small quality loss, which is not significant compared to the loss caused by NF4 quantization used in Petals by default.

-Storing weights in small shards. Each transformer block is stored in its own shard (1.71 GB each). The input and output embeddings and adjacent layernorms are in a separate shard (1.05 GB) too. This way, Petals clients and servers don't have to download any excess data besides the layers they actually use.

-Using Safetensors instead of Pickle. This allows faster loading with smaller RAM requirements.

Training Dataset Stable Beluga 2 is trained on our internal Orca-style dataset. Training data is a synthetic dataset that was created to enhance the small model's reasoning abilities. The dataset comprises a diverse collection of tasks aimed at training AI models across various domains, focusing on cautious reasoning and alignment with ethical guidelines. It includes approximately 602,000 zero-shot queries grouped into 23 categories and 126 sub-categories, each sharing a common instruction format to promote consistency. The dataset also features 55,000 few-shot samples to encourage the model's ability to learn from context, around 160,000 math problems sourced from a variety of existing datasets, and 2,000 synthetically generated conversations between doctors and patients designed to test the model's specialized skills.