Does your model run on AWS Trainium? Look it up
Hugging Face presents a Space and public dataset showing whether popular Transformers models from the Hub run unmodified on AWS Trainium through torch-neuronx. For tested models it lists the Neuron stack used, installation packages and a run script.
What the article says
Hugging Face publishes a post on model support in TorchNeuron, the native PyTorch backend for AWS Trainium (package torch-neuronx). Because Transformers runs its own modeling code on Trainium like on any other device, any Hub model could in principle run; the question is which ones actually do.
To answer it, two resources are launched: a Space called torch-neuronx model support, which reports whether popular models run unmodified on Trainium, and a public dataset, aws-neuron/torch-neuronx-model-support, which stores the results. Each test loads the model on trn2, generates 32 tokens and compares the output with a CPU run or has a small judge model check the text is fluent; a pass says nothing about throughput. Only Transformers models for text-generation and image-text-to-text that fit on 4 Trainium2 devices are tested. A run.py script is provided, and more tasks (diffusers, sentence transformers) are planned.
Read the official article AnnuIA profile
Summary written by AnnuIA in its own words from the official page of Hugging Face. The content and rights belong to the publisher: please read the original article.