Transoformer
transformers
Weights:
\(W_{Q}\)
,
\(W_{K}\)
,
\(W_{V}\)
\(W_{O}\)
\(W_{up}\)
,
\(W_{down}\)
