Okwu mmalite
Ụdị Asụsụ buru ibu (LLM) agbanweela ngalaba nhazi asụsụ eke (NLP) n'afọ ndị na-adịbeghị anya. Ụdị ndị a enwetala nsonaazụ ọgbara ọhụrụ na ọrụ NLP dị iche iche, gụnyere ntụgharị asụsụ, nchịkọta ederede, na ịza ajụjụ. Agbanyeghị, ezi ikike nke LLM dabere na ikike ha ịtụgharị uche na ịghọta mmekọrịta dị mgbagwoju anya n'etiti echiche. N'isiokwu a, anyị ga-enyocha ọganihu na echiche LLM, na-enyocha usoro, ụkpụrụ ụlọ, na ihe ịma aka.
ndabere
LLM bụ ụdị netwọkụ akwara azụrụ na oke data ederede iji mụta ụkpụrụ na mmekọrịta dị n'etiti okwu na echiche. Ụdị ndị a na-enwekarị ihe ngbanwe na ihe ngbanwe, ebe ihe ngbanwe na-ahazi ederede ntinye na ihe ngbanwe na-ewepụta ederede mmepụta. Ihe ngbanwe ahụ na-abụkarị ihe nrụpụta dabere na mgbanwe, nke dabara nke ọma maka data usoro dị ka ederede.
Transformer Architecture
The transformer architecture, ewebatara na akwụkwọ "Ntị bụ ihe niile ị chọrọ" nke Vaswani et al. (2017), abụrụla ọkọlọtọ de facto maka LLM. Ihe ngbanwe ahụ nwere ihe ngbanwe na ihe ngbanwe, nke ọ bụla nwere ọtụtụ ọkwa. Igwe oyibo ọ bụla nwere okpukpu abụọ: usoro nlebara anya onwe ya na netwọk na-ebugharị nri ejikọtara nke ọma (FFNN).
"Python mbubata ọwa mbubata torch.nn ka nn
klaasị TransformerLayer(nn.Module): def init(onwe, d_model, n_heads, d_ff): ibu (TransformerLayer, self).init() self.self_attn = nn.MultiHeadAttention(d_model, n_heads) self.ffnn = nn.Linear(d_model, d_ff) onwe.dropout = nn.Dropout(0.1) self.layer_norm = nn.LayerNorm(d_model)
gbapụ n'ihu (onwe, x):
# Usoro nlebara anya onwe onye
x_attn = self.self_attn (x, x)
x_attn = onwe.dropout(x_attn)
x_attn = x + x_attn
# FFNN
x_ffnn = onwe.ffnn (x_attn)
x_ffnn = onwe.dropout(x_ffnn)
x_ffnn = self.layer_norm(x_ffnn + x_attn)
laghachi x_ffnn
``
Ọganihu na ntụgharị uche LLM
Ndị LLM enweela ọganihu dị ukwuu na ntụgharị uche na nghọta mmekọrịta dị mgbagwoju anya n'etiti echiche. Ụfọdụ n'ime ọganihu ndị isi gụnyere:
1. Echiche dabere na eserese
Esemokwu dabere na eserese gụnyere ichikota ihe ọmụma dị ka eserese, ebe ọnụ na-anọchite anya echiche na akụkụ na-anọchite anya mmekọrịta dị n'etiti ha. Enwere ike ịzụ LLM ka ha tụgharịa uche gbasara eserese ndị a, na-ebu amụma gbasara mmekọrịta dị n'etiti ọnụ.
`` nchara jiri eserese :: Eserese; jiri neural_network :: NeuralNetwork;
fn graph_reasoning(graph: &Graf, ajụjụ: &str) -> eriri { ka embedding = neural_network.query(graph, ajụjụ); hapụ nsonaazụ = neural_network.reason (graph, ntinye); nsonaazụ.to_string() } ``
2. Multi-Task Learning
Ịmụ ọtụtụ ọrụ gụnyere ọzụzụ otu ụdị n'ọtụtụ ọrụ n'otu oge. E gosipụtara ụzọ a iji meziwanye arụmọrụ nke LLM na ọrụ dị iche iche, gụnyere ntụgharị uche na nghọta mmekọrịta dị mgbagwoju anya.
``Gaa ngwugwu isi
mbubata ( "fmt" "gonum.org/v1/gonum/mat" )
pịnye struct Model { *nn.Linear *nn.Linear }
func (m *Model) Forward(x *mat.Dense) *mat.Dense { // ngafe n'ihu z:= m.Linear(x) z = m.Linear(z) laghachi z } ``
3. Usoro nlebara anya
Usoro nlebara anya na-enye ndị LLM ohere ilekwasị anya n'akụkụ ụfọdụ nke ederede ntinye mgbe ha na-emepụta mmepụta. E gosipụtara ụzọ a iji meziwanye arụmọrụ nke LLM na ọrụ dị iche iche, gụnyere ntụgharị uche na nghọta mmekọrịta dị mgbagwoju anya.
"Python mbubata ọwa mbubata torch.nn ka nn
klaasị AttentionMechanism(nn.Module): def init (onwe, d_model, n_heads): ibu (AttentionMechanism, onwe).init() self.query = nn.Linear(d_model, d_model) self.key = nn.Linear(d_model, d_model) self.value = nn.Linear(d_model, d_model) onwe.dropout = nn.Dropout(0.1) self.layer_norm = nn.LayerNorm(d_model)
gbapụ n'ihu (onwe, x):
# Ajụjụ, igodo na uru
q = ajụjụ onwe (x)
k = igodo onwe (x)
v = uru onwe (x)
# Nlebara anya arọ
arọ = ọkụ.matmul(q, k.T) / math.sqrt (q.ọdịdị[1])
arọ = onwe.dropout(arọ)
arọ = ọkụ.softmax(arọ, dim=-1)
# vector okwu
okirikiri = ọkụ.matmul (arọ, v)
laghachi onodu
``
Nlere anya
Iji jiri anya nke uche hụ arụmọrụ nke LLM na ọrụ dị iche iche, anyị nwere ike iji eserese atụnyere izi ezi nke ụdị dị iche iche na ọrụ enyere. Nke a bụ ọmụmaatụ nke eserese na-eji Chart.js:
Json { "ụdị": "ogwe", "data": { "labels": ["Task 1", "Task 2", "Ọrụ 3"], "datasets": [ { "label": "Model 1", "data": [0.8, 0.7, 0.9], "backgroundColor": "rgba(255, 99, 132, 0.2)", "BorderColor": "rgba(255, 99, 132, 1)", "BorderWiidth": 1 }, { "label": "Model 2", "data": [0.9, 0.8, 0.7], "backgroundColor": "rgba(54, 162, 235, 0.2)", "borderColor": "rgba(54, 162, 235, 1)", "BorderWiidth": 1 } ] }, "Nhọrọ": { " aha": { "ngosi": eziokwu, "text": "Izizi ụdịdị dị iche iche na ọrụ dị iche iche" }, "ọnụego": { "yAxes": [ { "akara aka": { "beginAtZero": eziokwu } } ] } } } ``
Mmechi
N'ime edemede a, anyị enyochala ọganihu dị na ntụgharị uche LLM, gụnyere ntụgharị uche dabere na eserese, mmụta ọtụtụ ọrụ, na usoro nlebara anya. Anyị ewepụtakwala ihe atụ koodu na Python, Rust, na Go iji gosi echiche ndị a. N'ikpeazụ, anyị ejirila chaatị mee ihe iji jiri anya nke uche hụ arụmọrụ LLM na ọrụ dị iche iche. Ọganihu ndị a nwere ike ịgbanwe mpaghara NLP, na-eme ka LLM nwee ike ịtụgharị uche na ịghọta mmekọrịta dị mgbagwoju anya n'etiti echiche.
Ntụaka
- Vaswani et al. (2017). Nlebara anya bụ ihe niile ị chọrọ. arXiv preprint arXiv:1706.03762.
- Devlin et al. (2018). BERT: Ọzụzụ izizi nke ndị ntụgharị nke miri emi maka nghọta asụsụ. arXiv preprint arXiv: 1810.04805.
- Radford et al. (2019). Ụdị asụsụ bụ ndị mmụta Multitask anaghị elekọta ya. arXiv preprint arXiv:1906.08237.