Isingeniso
Amamodeli Olimi Olukhulu (LLMs) aguqule umkhakha wokucubungula ulimi lwemvelo (NLP) eminyakeni yamuva. Lawa mamodeli azuze imiphumela esezingeni eliphezulu emisebenzini ehlukahlukene ye-NLP, ehlanganisa ukuhumusha ulimi, ukufingqa kombhalo, kanye nokuphendula imibuzo. Kodwa-ke, amandla eqiniso e-LLMs asekhonweni lawo lokucabanga nokuqonda ubudlelwano obuyinkimbinkimbi phakathi kwemiqondo. Kulesi sihloko, sizocubungula intuthuko ekucabangeni kwe-LLM, sihlole amasu ayisisekelo, izakhiwo, nezinselelo.
Ingemuva
Ama-LLM awuhlobo lwenethiwekhi ye-neural eqeqeshwe ngenani elikhulu ledatha yombhalo ukuze ifunde amaphethini nobudlelwano phakathi kwamagama nemiqondo. Lawa mamodeli ngokuvamile ahlanganisa isifaki khodi nesikhiphi khodi, lapho isifaki khodi sicubungula umbhalo wokufakwayo kanye nesikhiphi khodi sikhiqiza umbhalo ophumayo. Isishumeki ngokuvamile siyisakhiwo esisekelwe ku-transformer, esifaneleka kahle idatha elandelanayo njengombhalo.
I-Transformer Architecture
I-architecture ye-transformer, eyethulwe ephepheni elithi "Ukunakwa Yikho Konke Okudingayo" nguVaswani et al. (2017), isibe indinganiso ye-de facto yama-LLM. I-transformer iqukethe i-encoder kanye ne-decoder, ngayinye eyakhiwe izendlalelo eziningi. Isendlalelo ngasinye siqukethe izendlalelo ezincane ezimbili: indlela yokuzinaka kanye nenethiwekhi exhunywe ngokugcwele yokuphakelayo (FFNN).
ngenisa ithoshi
ngenisa ithoshi.nn njenge-nn
class TransformerLayer(nn.Module):
def __init__(self, d_model, n_heads, d_ff):
super(TransformerLayer, self).__init__()
self.self_attn = nn.MultiHeadAttention(d_model, n_heads)
self.ffnn = nn.Linear(d_model, d_ff)
self.dropout = nn.Dropout(0.1)
self.layer_norm = nn.LayerNorm(d_model)
def phambili(self, x):
# Indlela yokuzinaka
x_attn = self.self_attn(x, x)
x_attn = self.dropout(x_attn)
x_attn = x + x_attn
#FFNN
x_ffnn = self.ffnn(x_attn)
x_ffnn = self.dropout(x_ffnn)
x_ffnn = self.layer_norm(x_ffnn + x_attn)
buyisela x_ffnn
Intuthuko ku-LLM yokubonisana
Ama-LLM enze inqubekelaphambili ebonakalayo ekucabangeni nasekuqondeni ubudlelwano obuyinkimbinkimbi phakathi kwemiqondo. Ezinye zentuthuko ezibalulekile zihlanganisa:
1. Ukubonisana Okusekelwe Kugrafu
Ukucabanga okusekelwe kugrafu kuhilela ukumelela ulwazi njengegrafu, lapho amanodi amelela khona imiqondo namaphethelo amele ubudlelwano phakathi kwawo. Ama-LLM angaqeqeshelwa ukusebenzisana nalawa magrafu, enze izibikezelo mayelana nobudlelwano phakathi kwamanodi.
sebenzisa igrafu::Igrafu;
sebenzisa i-neural_network ::NeuralNetwork;
fn graph_reasoning(igrafu: &Graph, umbuzo: &str) -> Iyunithi yezinhlamvu {
vumela ukushumeka = neural_network.query(igrafu, umbuzo);
vumela umphumela = neural_network.reason(igrafu, ukushumeka);
result.to_string()
}
2. Ukufunda Kwemisebenzi Eminingi
Ukufunda imisebenzi eminingi kubandakanya ukuqeqesha imodeli eyodwa emisebenzini eminingi ngesikhathi esisodwa. Le ndlela iboniswe ukuthuthukisa ukusebenza kwama-LLM emisebenzini eyahlukene, okuhlanganisa ukucabanga nokuqonda ubudlelwano obuyinkimbinkimbi.
iphakethe main
ngenisa (
"fmt"
"gonum.org/v1/gonum/mat"
)
thayipha isakhiwo semodeli {
*nn.Umugqa
*nn.Umugqa
}
i-func (m *Imodeli) Phambili(x *mat.Dense) *mat.Dense {
// Dlulisa phambili
z := m.Linear(x)
z = m.Liyini(z)
buyisela z
}
3. Izindlela Zokunaka
Izindlela zokunaka zivumela ama-LLM ukuthi agxile ezingxenyeni ezithile zombhalo wokufakwayo lapho ekhiqiza okukhiphayo. Le ndlela iboniswe ukuthuthukisa ukusebenza kwama-LLM emisebenzini eyahlukene, okuhlanganisa ukucabanga nokuqonda ubudlelwano obuyinkimbinkimbi.
ngenisa ithoshi
ngenisa ithoshi.nn njenge-nn
I-class AttentionMechanism(nn.Module):
def __init__(self, d_model, n_heads):
super(AttentionMechanism, self).__init__()
self.query = nn.Linear(d_model, d_model)
self.key = nn.Linear(d_model, d_model)
self.value = nn.Linear(d_model, d_model)
self.dropout = nn.Dropout(0.1)
self.layer_norm = nn.LayerNorm(d_model)
def phambili(self, x):
# Umbuzo, ukhiye, kanye nenani
q = self.query(x)
k = self.key(x)
v = i-self.value(x)
# Izisindo zokunaka
izisindo = ithoshi.matmul(q, k.T) / math.sqrt(q.shape[1])
izisindo = self.dropout(izisindo)
izisindo = torch.softmax(izisindo, dim=-1)
# Ivector yokuqukethwe
umongo = ithoshi.matmul(izisindo, v)
buyisela umongo
Ukubukwa
Ukuze ubone ngeso lengqondo ukusebenza kwama-LLM emisebenzini ehlukahlukene, singasebenzisa ishadi ukuze siqhathanise ukunemba kwamamodeli ahlukene emsebenzini othile. Nasi isibonelo seshadi elisebenzisa i-Chart.js:
{
"type": "bar",
"idatha": {
"labels": ["Umsebenzi 1", "Umsebenzi 2", "Umsebenzi 3"],
"idathasethi": [
{
"label": "Imodeli 1",
"idatha": [0.8, 0.7, 0.9],
"backgroundColor": "rgba(255, 99, 132, 0.2)",
"borderColor": "rgba(255, 99, 132, 1)",
"BorderWidth": 1
},
{
"label": "Imodeli 2",
"idatha": [0.9, 0.8, 0.7],
"backgroundColor": "rgba(54, 162, 235, 0.2)",
"borderColor": "rgba(54, 162, 235, 1)",
"BorderWidth": 1
}
]
},
"izinketho": {
"isihloko": {
"Display": iqiniso,
"text": "Ukunemba Kwamamodeli Ahlukene Emisebenzini Ehlukahlukene"
},
"izikali": {
"yAxes": [
{
"imikhaza": {
"beginAtZero": kuyiqiniso
}
}
]
}
}
}
Isiphetho
Kulesi sihloko, sihlole intuthuko ekucabangeni kwe-LLM, okuhlanganisa ukucabanga okusekelwe kugrafu, ukufunda imisebenzi eminingi, kanye nezindlela zokunaka. Siphinde sanikeza izibonelo zekhodi ku-Python, Rust, kanye ne-Go ukuze sibonise le mibono. Ekugcineni, sisebenzise ishadi ukuze sibone ngeso lengqondo ukusebenza kwama-LLM emisebenzini ehlukahlukene. Le ntuthuko inamandla okuguqula umkhakha we-NLP, ivumele ama-LLM ukuthi acabange futhi aqonde ubudlelwano obuyinkimbinkimbi phakathi kwemiqondo.
Izikhombo
*Vaswani et al. (2017). Ukunakwa Yikho Konke Okudingayo. I-arXiv preprint arXiv:1706.03762.
- UDevlin et al. (2018). I-BERT: Ukuqeqeshwa kwangaphambili kwama-Deep Bidirectional Transformers ukuze Kuqondwe Ulimi. I-arXiv preprint arXiv:1810.04805.
- URadford et al. (2019). Amamodeli Olimi Ngabafundi Bemisebenzi eminingi Engagadiwe. I-arXiv preprint arXiv:1906.08237.