Hordhac
Moodooyinka Luuqadaha Waaweyn (LLMs) ayaa isbedel ku sameeyay saaxadda habaynta luqadda dabiiciga ah (NLP) sanadihii ugu dambeeyay. Moodooyinkaani waxay ku guuleysteen natiijooyin casri ah oo ku saabsan hawlo kala duwan oo NLP ah, oo ay ku jiraan tarjumaadda luqadda, soo koobida qoraalka, iyo ka jawaabista su'aalaha. Si kastaba ha ahaatee, kartida dhabta ah ee LLMs waxay ku jirtaa awoodda ay u leeyihiin inay fahmaan oo ay fahmaan xidhiidhada adag ee ka dhexeeya fikradaha. Maqaalkan, waxaan ku daah-furi doonaa horumarka laga gaaray sababaynta LLM, sahaminta farsamooyinka hoose, naqshadaha, iyo caqabadaha.
Asalkii hore
LLMs waa nooc ka mid ah shabakada neerfaha kaas oo lagu tababaray xaddi badan oo xogta qoraalka ah si ay u bartaan qaababka iyo xidhiidhka ka dhexeeya erayada iyo fikradaha. Moodooyinkani waxay caadi ahaan ka kooban yihiin cod-bixiye iyo cod-bixiye, halkaas oo cod-bixiyehu uu ka shaqeeyo qoraalka wax-gelinta iyo furaha-fududeeyaha uu soo saaro qoraalka wax-soo-saarka. Codeeyaha inta badan waa qaab-dhismeedka ku-saleysan transformer-ka, kaas oo si gaar ah ugu habboon xogta taxanaha ah sida qoraalka.
Qaab-dhismeedka Transformer
Qaab dhismeedka beddelka, oo lagu soo bandhigay xaashida "Fiiro gaar ah waa dhammaan waxaad u baahan tahay" Vaswani et al. (2017), waxa uu noqday halbeega dhabta ah ee LLMs. Transformer-ku waxa uu ka kooban yahay cod-bixiye iyo cod-bixiye, mid walbana waxa uu ka kooban yahay lakabyo badan. Lakab kastaa wuxuu ka kooban yahay laba lakab-hoosaad: hab is-fiirsi iyo shabakad-horumarin quudineed oo si buuxda ugu xidhan (FFNN).
`` Python soo dejinta toosh soo dejinta toosh.nn sida nn
dabaqa TransformerLayer(nn.Module): def init (self, d_model, n_heads, d_ff): super (TransformerLayer, self).init() self.self_attn = nn.MultiHeadAttention(d_model, n_heads) self.ffnn = nn.Linear(d_model, d_ff) self.dropout = nn.Dropout(0.1) self.layer_norm = nn.LayerNorm(d_model)
hore u dhac (naftiisa, x):
# Habka is-fiirsiga
x_attn = self.self_attn (x, x)
x_attn = is.Dropout(x_attn)
x_attn = x + x_attn
# FFNN
x_ffnn = is.ffnn (x_attn)
x_ffnn = is.Dropout(x_ffnn)
x_ffnn = self.layer_norm(x_ffnn + x_attn)
soo noqo x_ffnn
``
Horumarka laga gaaray Sababta LLM
LLMs waxay horumar la taaban karo ka sameeyeen sababaynta iyo fahamka xidhiidhada adag ee u dhexeeya fikradaha. Qaar ka mid ah horumarinta muhiimka ah waxaa ka mid ah:
1. Sababaynta Garaafka ku salaysan
Sababaynta garaaf-ku-salaysan waxay ku lug leedahay matalaadda aqoonta garaaf ahaan, halkaas oo noodhadhku ay matalaan fikrado iyo cidhifooyin ay matalaan cilaaqaadka u dhexeeya. LLM-yada waxa loo tababari karaa inay sabab uga dhigaan garaafyadan, samaynta saadaalin ku saabsan xidhiidhada u dhexeeya noodhka.
`` miridhku isticmaal garaafka :: garaafka; isticmaal neural_network :: NeuralNetwork;
fn graph_reasoning (garaafka: & garaafka, weydiinta: &str) -> Xarig { ha ku dhejiyo = neural_network.query (garaafka, weydiinta); ha natiijado = neural_network.sabab (garaafka, dhejinta); natiijada.to_string() } ``
2. Barashada Hawsha Badan
Barashada hawlo badan waxay ku lug leedahay tababarida hal nooc oo hawlo badan isku mar ah. Habkan ayaa la muujiyay si loo wanaajiyo waxqabadka LLMs ee hawlo kala duwan, oo ay ku jiraan sababaynta iyo fahamka cilaaqaadka adag.
`` tag xirmo ugu weyn
soo dejinta ( "fmt" "gonum.org/v1/gonum/mat" )
nooca qaabka qaabka { *nn.Linear *nn.Linear }
func (m *Model) Horudhac(x *mat.Dense) *mat.Dense { // Horay u dhaaf z:= m.Linear(x) z = m.Linear(z) soo noqo z } ``
3. Hababka Feejignaanta
Hababka feejignaanta ayaa u oggolaanaya LLM-yada inay diiradda saaraan qaybo gaar ah oo ka mid ah qoraalka wax-gelinta marka ay soo saarayaan wax-soo-saarka. Habkan ayaa la muujiyay si loo wanaajiyo waxqabadka LLMs ee hawlo kala duwan, oo ay ku jiraan sababaynta iyo fahamka cilaaqaadka adag.
`` Python soo dejinta toosh soo dejinta toosh.nn sida nn
Habka Feejignaanta fasalka(nn.Module): def init (self, d_model, n_heads): super (Feejignaanta Mechanism, is).init() self.query = nn.Linear(d_model, d_model) self.key = nn.Linear(d_model, d_model) self.value = nn.Linear(d_model, d_model) self.dropout = nn.Dropout(0.1) self.layer_norm = nn.LayerNorm(d_model)
hore u dhac (naftiisa, x):
# Weydiinta, furaha, iyo qiimaha
q = is-weydiin (x)
k = is.furaha (x)
v = is.qiimaha (x)
# Miisaanka feejignaanta
miisaanka = toosh.matmul(q, k.T) / xisaab.sqrt (q.qaabka[1])
miisaanyo = is-.dhiibid(miisaan)
miisaanka = toosh.softmax(miisaan, dim=-1)
# macnaha guud
macnaha guud = toosh.matmul (miisaanka, v)
soo celinta macnaha guud
``
Muuqaalka
Si loo sawiro waxqabadka LLM-yada ee hawlo kala duwan, waxaan isticmaali karnaa jaantus si aan isku barbar dhigno saxnaanta moodooyinka kala duwan ee hawsha la bixiyay. Waa kan tusaale jaantuska isticmaalaya Chart.js:
json { "type": "bar", "xogta": { "labels": ["Hawl 1", "Hawl 2", "Hawsha 3"], "datasets": [ { "label": "Model 1", "xogta": [0.8, 0.7, 0.9], "backgroundColor": "rgba(255, 99, 132, 0.2)", "BorderColor": "rgba(255, 99, 132, 1)", "BorderWidth": 1 }, { "label": "Model 2", "xogta": [0.9, 0.8, 0.7], "backgroundColor": "rgba(54, 162, 235, 0.2)", "BorderColor": "rgba(54, 162, 235, 1)", "BorderWidth": 1 } ] }, "doorashooyinka": { "ciwaan": { "muujin": run, "text": "Saxnimada Noocyada Kala Duwan ee Hawsha Kala Duwan" }, "Miisaanka": { "yAxes": [ { "ticks": { "beginAtZero": run } } ] } } }
Gabagabo
Maqaalkan, waxaanu ku baadhnay horumarka sababaynta LLM, oo ay ku jiraan sababaynta garaaf-ku-salaysan, barashada hawlo badan, iyo hababka feejignaanta. Waxaan sidoo kale ku bixinnay tusaalooyin kood ah Python, Rust, iyo Go si aan u muujino fikradahan. Ugu dambeyntii, waxaan isticmaalnay jaantus si aan ugu sawirno waxqabadka LLM-yada ee hawlo kala duwan. Horumaradaasi waxay awood u leeyihiin inay wax ka beddelaan beerta NLP, taasoo u sahlaysa LLM-yada inay caqli galiyaan oo ay fahmaan xidhiidhada adag ee ka dhexeeya fikradaha.
Tixraacyo
- Vaswani iyo al. (2017). Feejignaan waa Dhammaan waxaad u baahan tahay. arXiv preprint arXiv:1706.03762.
- Devlin iyo al. (2018). BERT: Tababarka hore ee Beddelayaasha Labada Jiho ee Qoto dheer ee Fahamka Luuqadda. arXiv preprint arXiv:1810.04805.
- Radford iyo al. (2019). Qaababka Luuqadaha waa Bartayaal hawlo badan oo aan la ilaalinayn. arXiv preprint arXiv:1906.08237.