#redpajamas

[mysocial8one]

Discover Red Pajama, the new project aiming to create a leading, fully open-source AI model. With a collaboration between top research institutes and a data set of 1.2 trillion tokens, Red Pajama has the potential to revolutionize the AI industry. Learn more in this latest blog post.

[littleweststreetfanclub]

Red Christmas Pajama Set For Women At Little West Street

This christmas pajama set is made from the finest cotton, this sweet print has all the delicious goodies kids enjoy in the festive season including gingerbread house & cookies! Includes a super-soft full-sleeve notched collar top and coordinated pajamas for women. Shop now.

[tumnikkeimatome]

RedPajama-Data-v2データセットの全貌

RedPajama-Data-v2データセットは、30兆トークンという圧倒的な量を誇る、先進的な言語データリソースです。このデータセットは、5つの主要言語である英語、フランス語、スペイン語、ドイツ語、イタリア語にわたる84のCommonCrawlダンプから作成されました。その目的は、高品質な言語モデルの開発を促進するためのデータ源を提供することです。 データセットの特徴と目的 RedPajama-Data-v2は、品質管理のために40以上の事前計算されたアノテーションを含む、フィルタリングと重複除去によって精選されたデータセットです。これにより、研究者や開発者は品質に基づいてデータを選択し、重み付けすることが可能となります。CommonCrawlの中で最も完全なカバレッジを誇り、ウィキペディアの類似度や重要度スコアによってフィルタリング可能です。データの追跡を容易にするため、構造はC…

[hackernewsrobot]

RedPajama v2 Open Dataset with 30T Tokens for Training LLMs

https://together.ai/blog/redpajama-data-v2

[craigbrownphd]

RedPajama Completes First Step to Open-Source ChatGPT Alternative https://www.analyticsvidhya.com/blog/2023/04/redpajama-completes-first-step-to-open-source-chatgpt-alternative/?utm_source=dlvr.it&utm_medium=tumblr

[jamalir]

Meet RedPajama: An AI Project to Create Fully Open-Source Large Language Models Beginning with the Release of a 1.2 Trillion Token Dataset - MarkTechPost

[tastydregs]

Red Pajama Is a 1.2 Trillion Token Large Language Model

RedPajama is a project to create a set of leading, fully open-source models. Today, they announced the completion of the first step of this project: the reproduction of the LLaMA training dataset of over 1.2 trillion tokens.

AI is having its Linux moment. Stable Diffusion showed that open-source can not only rival the quality of commercial offerings like DALL-E but can also lead to incredible creativity from broad participation by communities around the world. A similar movement has now begun around large language models with the recent release of semi-open models like LLaMA, Alpaca, Vicuna, and Koala; as well as fully-open models like Pythia, OpenChatKit, Open Assistant and Dolly.

We are launching RedPajama, an effort to produce a reproducible, fully-open, leading language model. RedPajama is a collaboration between Together, Ontocord.ai, ETH DS3Lab, Stanford CRFM, Hazy Research, and MILA Québec AI Institute. RedPajama has three key components:

* Pre-training data, which needs to be both high quality and have broad coverage

* Base models, which are trained at scale on this data

* Instruction tuning data and models, which improve the base model to make it usable and safe

The starting point is LLaMA, which is the leading suite of open base models for two reasons: First, LLaMA was trained on a very large (1.2 trillion tokens) dataset that was carefully filtered for quality. Second, the 7 billion parameter LLaMA model is trained for much longer, well beyond the Chincilla-optimal point, to ensure the best quality at that model size. A 7 billion parameter model is particularly valuable for the open community as it can run on a wide variety of GPUs, including many consumer grade GPUs.

The RedPajama base dataset The full RedPajama 1.2 trillion token dataset and a smaller, more consumable random sample can be downloaded through Hugging Face. The full dataset is ~5TB unzipped on disk and ~3TB to download compressed.

RedPajama-Data-1T consists of seven data slices:

CommonCrawl: Five dumps of CommonCrawl, processed using the CCNet pipeline, and filtered via several quality filters including a linear classifier that selects for Wikipedia-like pages.

C4: Standard C4 dataset

GitHub: GitHub data, filtered by licenses and quality

arXiv: Scientific articles removing boilerplate

Books: A corpus of open books, deduplicated by content similarity

Wikipedia: A subset of Wikipedia pages, removing boilerplate

StackExchange: A subset of popular websites under StackExchange, removing boilerplate

Next: Models, instructions & OpenChatKit Having reproduced the pre-training data, the next step is to train a strong base model. As part of the INCITE program, with support from Oak Ridge Leadership Computing Facility (OLCF), we are training a full suite of models, with the first becoming available in the coming weeks.

With a strong base model in hand, we are excited to instruction tune the models. Alpaca illustrated the power of instruction tuning – with merely 50K high-quality, diverse instructions, it was able to unlock dramatically improved capabilities. Via OpenChatKit, we received hundreds of thousands of high-quality natural user instructions, which will be used to release instruction-tuned versions of the RedPajama models.

Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.

Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.

A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts.  He is open to public speaking and advising engagements.

[mitchellkriegman]

morning this time of year is more orange and comes in stripes - matches well with my red pajamas (note I don’t wear pajamas) #redpajamas #sunrise https://www.instagram.com/p/CUcv3xflQve/?utm_medium=tumblr

[89love]

thank u for making my day, aly! you’re so kind 😭💓

CEE YOURE THE SWEETEST EVER OMG 💗💗🥺🥺😭😭

[alfamarama]

Just another day of Netflix & calories at Team Alfanarama headquarters. #netflix #calories #bedroom #inbed #tv #portabletv #sixties #redpajamas #lazyday #rest #relax #telly #tv #streaming #lockdown #lockdownsessions #lockdownlife #lockdownmemes #bedroom https://www.instagram.com/p/CLjbn2Ds7jA/?igshid=igxzds2an3aj

[totybluehandmadeapparel]

My favorite love quote: "Be mine forever Love." Celebrate your valentine's day!! Staying at home with your love. 💞💗💞 My limited edition Red Romper Pajamas will be available tomorrow 100% handmade DM directly or go to my website! Bio in link ☝️☝️ . . #pickmybio☝️☝️☝️ #bemineforeverlove #romperpajamas #romper #pajamasallday #valentinedaypajamas #celebratevalentineday #redpajamas #pajamasparty #totyblueapparel #latoty❤ (at Highland Park) https://www.instagram.com/p/CKsSuCCHr4z/?igshid=16ppjqyyqaj97

[tianachu]

View this post on Instagram

A post shared by Fashion illustrator-UK🇬🇧| Ukraine🇺🇦 (@tanyachulkova)

Can't wait 💋🍾❄🎄🎁🎉 @lorealmakeup #redlips #redlipstick #lorealparis #loreallipstick #illustration #illustrator #fashionillustration #digitslillustration #digitalart #animation #beautyillustration #redpajamas #pajamas #иллюстрация #иллюстраторукраина #моднаяиллюстрация #помада #лореаль #пижама #моднаяиллюстрация

[hackernewsrobot]

SlimPajama: A 627B token cleaned and deduplicated version of RedPajama

https://www.cerebras.net/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama