> For the complete documentation index, see [llms.txt](https://ginkgo-1.gitbook.io/ginkgo/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ginkgo-1.gitbook.io/ginkgo/ginkgo-aa-0-650m.md).

# Ginkgo-AA-0-650M

Ginkgo's proprietary protein language model (Ginkgo-AA-0-650M)

## Ginkgo-AA-0-650M

### **Model Overview**

This is a 650M parameter protein language model, trained to do both masked language modeling and embedding calculation. This model was trained on a proprietary dataset consisting of a large internal Ginkgo sequence database, along with publicly available sequence from UniRef. You can see more information about this model in our technical post on the Ginkgo blog. 

### **Input**

The input uses the ESMTokenizer. This token accepts uppercase characters representing amino acids as well as the special tokens `<unk>`, `<pad>`, `<cls>`, `<mask>`, and `<eos>`. Please refer to the [ESMTokenizer docs](https://huggingface.co/docs/transformers/en/model_doc/esm#transformers.EsmTokenizer) for more information on the inputs. Additionally, we allow newlines in the input in the playground, which allows you to copy and paste a protein from a FASTA file. We strip newlines, so the input will be treated as one extended sequence. All sequences are limited to 1000 amino acids.&#x20;

### **Output**

For the embedding task, we return a mean-pooled embedding representation of the amino acid sequence.&#x20;

For the masked modeling task, we return the completed sequence with mask tokens replaced with predictions.

### Colab Examples

* [**The PETase Mask Fill Challenge**](https://colab.research.google.com/drive/1KI4vwYkOq7SssYlxyl5y0pyn2VxP1fC0?usp=sharing)**:** A simple protein engineering task to demonstrate the use of Ginkgo's protein sequence model API