Folasaga
Large Language Models (LLMs) ua fa'afouina le fanua o le gagana fa'anatura (NLP) i tausaga talu ai nei. O nei faʻataʻitaʻiga ua ausia taunuʻuga faʻaonaponei i galuega eseese a le NLP, e aofia ai le faʻaliliuga gagana, aotelega o tusitusiga, ma le taliina o fesili. Ae ui i lea, o le gafatia moni o LLM o loʻo taoto i lo latou gafatia e mafaufau ma malamalama i sootaga lavelave i le va o manatu. I totonu o lenei tusiga, o le a tatou suʻesuʻeina le alualu i luma i le LLM, suʻesuʻeina o auala autu, fausaga, ma luʻitau.
Talaaga
LLMs o se ituaiga o fesoʻotaʻiga neural e aʻoaʻoina i le tele o faʻamaumauga o tusitusiga e aʻoaʻo ai mamanu ma sootaga i le va o upu ma manatu. O nei faʻataʻitaʻiga e masani lava ona aofia ai se encoder ma se decoder, lea e faʻaogaina ai e le encoder le faʻaogaina o tusitusiga ma le decoder e faʻatupuina ai le faʻasologa o tusitusiga. O le encoder e masani lava o se fausaga faʻavae transformer, lea e fetaui lelei mo faʻasologa faʻasologa e pei o tusitusiga.
Faiga Fa'aliliuga
O le fausaga o le transformer, na faʻalauiloaina i le pepa "Attention is All You Manaomia" saunia e Vaswani et al. (2017), ua avea ma tulaga masani mo LLMs. O le transformer e aofia ai se encoder ma se decoder, e tofu ma le tele o laulau. O vaega taʻitasi e aofia ai ni faʻavae e lua: o se masini e faʻalogo ai le tagata lava ia ma se fesoʻotaʻiga faʻafeiloaʻi atoa (FFNN).
sulu mai fafo
fa'aulufale mai sulu.nn pei nn
vasega TransformerLayer(nn.Module):
def __init__(self, d_model, n_heads, d_ff):
sili(TransformerLayer, self).__init__()
self.self_attn = nn.MultiHeadAttention(d_model, n_heads)
self.ffnn = nn.Linear(d_model, d_ff)
self.dropout = nn.Tuu ese(0.1)
self.layer_norm = nn.LayerNorm(d_model)
def i luma(tagata, x):
# Faiga faʻapitoa
x_attn = self.self_attn(x, x)
x_attn = self.dropout(x_attn)
x_attn = x + x_attn
# FFNN
x_ffnn = self.ffnn(x_attn)
x_ffnn = self.dropout(x_ffnn)
x_ffnn = self.layer_normal(x_ffnn + x_attn)
toe fo'i x_ffnn
Agai i luma ile LLM Reasoning
LLM ua tele le alualu i luma i le mafaufau ma le malamalama i sootaga lavelave i le va o manatu. O nisi o aga'i i luma e aofia ai:
1. Fuafuaga Fa'avae Kalafi
O mafaufauga fa'avae kalafi e aofia ai le fa'atusaina o le malamalama e pei o se kalafi, lea o lo'o fa'atusalia ai manatu ma pito o lo'o fa'atusalia sootaga i lo latou va. E mafai ona a'oa'oina LLM e fa'atatau i nei kalafi, fai fa'amatalaga e uiga i sootaga i le va o nodes.
fa'aoga kalafi::Kalafi;
fa'aoga neural_network::NeuralNetwork;
fn graph_reasoning(kalafa: &Kalafi, fesili: &str) -> String {
tu'u fa'apipi'i = neural_network.query(kalafa, fesili);
tu'u i'uga = neural_network.reason(kalafa, fa'apipi'i);
result.to_string()
}
2. A'oa'oga Tele-Galue
O le a'oa'oina o le tele o galuega e aofia ai le a'oa'oina o se fa'ata'ita'iga e tasi ile tele o galuega ile taimi e tasi. O lenei faiga ua fa'aalia e fa'aleleia atili ai le fa'atinoga o LLM i galuega eseese, e aofia ai le fa'atalanoaga ma le malamalama i so'otaga lavelave.
afifi autu
faaulufale mai (
"fmt"
"gonum.org/v1/gonum/mat"
)
ituaiga Fa'ata'ita'iga struct {
*nn.Linear
*nn.Linear
}
func (m *Fa'ata'ita'iga) I luma(x *mat.Dense) *mat.Dense {
// Pasi i luma
z : = m.Linear(x)
z = m.Linear(z)
toe fo'i z
}
3. Faiga Fa'alogo
O faiga fa'alogo e mafai ai e LLM ona taula'i atu i vaega fa'apitoa o tusitusiga fa'aoga pe a fa'atupuina le galuega. O lenei faiga ua fa'aalia e fa'aleleia atili ai le fa'atinoga o LLM i galuega eseese, e aofia ai le fa'atalanoaga ma le malamalama i so'otaga lavelave.
sulu mai fafo
fa'aulufale mai sulu.nn pei nn
vasega AttentionMechanism(nn.Module):
def __init__(self, d_model, n_heads):
sili(AttentionMechanism, self).__init__()
self.query = nn.Linear(d_model, d_model)
self.key = nn.Linear(d_model, d_model)
self.value = nn.Linear(d_model, d_model)
self.dropout = nn.Tuu ese(0.1)
self.layer_norm = nn.LayerNorm(d_model)
def i luma(tagata, x):
# Fesili, ki, ma le tau
q = self.query(x)
k = self.key(x)
v = self.value(x)
# Taumafa mamafa
mamafa = torch.matmul(q, k.T) / math.sqrt(q.shape[1])
mamafa = self.dropout(weights)
mamafa = torch.softmax(mamafa, mumu=-1)
# Veta fa'amatalaga
context = torch.matmul(mamafa, v)
toe fa'afo'i fa'amatalaga
Va'aiga
Ina ia va'ava'ai i le faatinoga o LLM i galuega eseese, e mafai ona tatou fa'aogaina se siata e fa'atusatusa ai le sa'o o fa'ata'ita'iga 'ese'ese i luga o se galuega ua tu'uina atu. Ole fa'ata'ita'iga lea ole siata ile fa'aogaina ole Chart.js:
{
"type": "pa",
"fa'amatalaga": {
"labels": ["Task 1", "Task 2", "Task 3"],
"fa'amaumauga": [
{
"label": "Fa'ata'ita'iga 1",
"fa'amaumauga": [0.8, 0.7, 0.9],
"backgroundColor": "rgba(255, 99, 132, 0.2)",
"borderColor": "rgba(255, 99, 132, 1)",
"BorderWidth": 1
},
{
"label": "Fa'ata'ita'iga 2",
"fa'amaumauga": [0.9, 0.8, 0.7],
"backgroundColor": "rgba(54, 162, 235, 0.2)",
"borderColor": "rgba(54, 162, 235, 1)",
"BorderWidth": 1
}
]
},
"filifiliga": {
"igoa": {
"fa'aaliga": moni,
"text": "Sa'o o Fa'ata'ita'iga Eseese i Galuega Eseese"
},
"fua": {
"yAxes": [
{
"tiki": {
"beginAtZero": moni
}
}
]
}
}
}
Fa'ai'uga
I totonu o lenei tusiga, ua matou suʻesuʻeina le alualu i luma i le LLM, e aofia ai mafaufauga faʻavae kalafa, aʻoaʻoga tele-task, ma faʻaogaina masini. Ua matou saunia foʻi faʻataʻitaʻiga faʻataʻitaʻiga i le Python, Rust, ma le Alu e faʻaalia ai nei manatu. Mulimuli ane, ua matou fa'aogaina se siata e va'ava'ai ai i le fa'atinoga o LLM i galuega eseese. O nei aga'i i luma e iai le avanoa e fa'afouina ai le fanua o le NLP, e mafai ai e LLM ona fa'atalanoa ma malamalama i feso'ota'iga lavelave i le va o manatu.
Fa'asinomaga
- Vaswani et al. (2017). Ua na'o le ua'i atu e te Manaomia. arXiv mua'i lolomi arXiv:1706.03762.
- Devlin et al. (2018). BERT: A'o le a'oa'oga a Deep Bidirectional Transformers mo le Malamalama i le Gagana. arXiv mua'i lolomi arXiv:1810.04805.
- Radford et al. (2019). O Fa'ata'ita'iga o Gagana o Tagata A'oa'o Tele galuega e le'i va'aia. arXiv mua'i lolomi arXiv:1906.08237.