Nhanganyaya
Makuru Mutauro Mamodheru (LLMs) akashandura ndima yemutauro wechisikigo kugadzirisa (NLP) mumakore achangopfuura. Aya mamodheru akawana mhedzisiro yemazuva ano mumabasa akasiyana-siyana eNLP, kusanganisira kududzira mutauro, kupfupisa zvinyorwa, uye kupindura mibvunzo. Nekudaro, kugona kwechokwadi kweLLMs kuri mukukwanisa kwavo kufunga uye kunzwisisa hukama hwakaoma pakati pepfungwa. Muchinyorwa chino, isu tichaongorora kufambira mberi mukufunga kweLLM, tichiongorora ari pasi pemaitiro, zvivakwa, uye matambudziko.
Background
LLMs imhando yeneural network inodzidziswa pahuwandu hwakawanda hwe data data kudzidza mapatani uye hukama pakati pemazwi uye pfungwa. Mamodheru aya anowanzo sanganisira encoder uye decoder, uko encoder inogadzira zvinyorwa zvemukati uye decoder inogadzira zvinyorwa zvinobuda. Iyo encoder inowanzo shanduri-yakavakirwa architecture, iyo inonyanya kuenderana neyakatevedzana data senge zvinyorwa.
Transformer Architecture
The transformer architecture, yakaunzwa mupepa "Attention is All You need" naVaswani et al. (2017), yave iyo de facto chiyero cheLLMs. Iyo transformer ine encoder uye decoder, imwe neimwe inoumbwa neakawanda akaturikidzana. Nheyo yega yega ine maviri madiki-matanho: yekuzvitarisa-yega uye yakanyatso batanidza feed-forward network (FFNN).
import torch
import torch.nn as nn
kirasi TransformerLayer(nn.Module):
def __init__(self, d_model, n_heads, d_ff):
super(TransformerLayer, self).__init__()
self.self_attn = nn.MultiHeadAttention(d_model, n_heads)
self.ffnn = nn.Linear(d_model, d_ff)
self.dropout = nn.Dropout(0.1)
self.layer_norm = nn.LayerNorm(d_model)
def kumberi( pachedu, x):
# Kuzvitarisisa maitiro
x_attn = self.self_attn(x, x)
x_attn = self.dropout(x_attn)
x_attn = x + x_attn
#FFNN
x_ffnn = self.ffnn(x_attn)
x_ffnn = self.dropout(x_ffnn)
x_ffnn = self.layer_norm(x_ffnn + x_attn)
dzoka x_ffnn
Kufambira mberi muLLM Kukurukurirana
MaLLM akaita kufambira mberi kwakakosha mukufunga uye kunzwisisa hukama hwakaoma pakati pepfungwa. Zvimwe zvezvikuru zvebudiriro zvinosanganisira:
1. Graph-Based Reasoning
Kufunga kwakavakirwa pagirafu kunosanganisira kumiririra ruzivo segirafu, apo nodhi dzinomiririra pfungwa uye mipendero inomiririra hukama pakati padzo. MaLLM anogona kudzidziswa kufunga nezve magirafu aya, achiita fungidziro nezvehukama pakati pemanodhi.
shandisa girafu::Girafu;
shandisa neural_network ::NeuralNetwork;
fn graph_reasoning(girafu: &Girafu, mubvunzo: &str) -> Tambo {
rega embedding = neural_network.query(graph, query);
regai mhedzisiro = neural_network.reason(graph, embedding);
result.to_string()
}
2. Multi-Task Learning
Multi-task kudzidza kunosanganisira kudzidzisa modhi imwe chete pamabasa akawanda panguva imwe chete. Iyi nzira yakaratidzwa kuvandudza kushanda kweLLM pamabasa akasiyana-siyana, kusanganisira kufunga uye kunzwisisa hukama hwakaoma.
package main
import (
"fmt"
"gonum.org/v1/gonum/mat"
)
mhando Model chimiro {
*nn.Linear
*nn.Linear
}
func (m *Model) Pamberi(x *mat.Dense) *mat.Dense {
// Forward pass
z := m.Mutsetse(x)
z = m.Mutsetse(z)
dzoka z
}
3. Attention Mechanisms
Maitiro ekutarisisa anobvumira maLLM kuti atarise pane zvakati zvidimbu zvemashoko ekupinza kana achigadzira zvinobuda. Iyi nzira yakaratidzwa kuvandudza kushanda kweLLM pamabasa akasiyana-siyana, kusanganisira kufunga uye kunzwisisa hukama hwakaoma.
import torch
import torch.nn as nn
kirasi AttentionMechanism(nn.Module):
def __init__(self, d_model, n_heads):
super(AttentionMechanism, self).__init__()
self.query = nn.Linear(d_model, d_model)
self.key = nn.Linear(d_model, d_model)
self.value = nn.Linear(d_model, d_model)
self.dropout = nn.Dropout(0.1)
self.layer_norm = nn.LayerNorm(d_model)
def kumberi( pachedu, x):
# Mubvunzo, kiyi, uye kukosha
q = self.query(x)
k = self.key(x)
v = self.value(x)
# Attention huremu
uremu = torch.matmul(q, k.T) / math.sqrt(q.shape[1])
uremu = self.dropout(zviremu)
uremu = torch.softmax(zviremera, dim=-1)
# Context vector
context = torch.matmul(weights, v)
return context
Kuona
Kuona mashandiro eLLM pamabasa akasiyana, tinogona kushandisa chati kuenzanisa kurongeka kwemamodheru akasiyana pabasa rakapihwa. Heino muenzaniso wechati uchishandisa Chart.js:
{
"type": "bha",
"data": {
"labels": ["Basa 1", "Basa 2", "Basa rechi3"],
"datasets": [
{
"label": "Muenzaniso 1",
"data": [0.8, 0.7, 0.9],
"backgroundColor": "rgba(255, 99, 132, 0.2)",
"borderColor": "rgba(255, 99, 132, 1)",
"borderWidth": 1
},
{
"label": "Muenzaniso 2",
"data": [0.9, 0.8, 0.7],
"backgroundColor": "rgba(54, 162, 235, 0.2)",
"borderColor": "rgba(54, 162, 235, 1)",
"borderWidth": 1
}
]
},
"sarudzo": {
"zita": {
"kuratidza": chokwadi,
"text": "Kururama kweMamodheru Akasiyana Pamabasa Akasiyana-siyana"
},
"zvikero": {
"yAxes": [
{
"zvikwekwe": {
"beginAtZero": chokwadi
}
}
]
}
}
}
Mhedziso
Muchinyorwa chino, takaongorora kufambira mberi mukufunga kweLLM, kusanganisira girafu-yakavakirwa kufunga, akawanda-basa kudzidza, uye maitiro ekutarisa. Isu takapawo mienzaniso yekodhi muPython, Rust, uye Go kuenzanisira aya pfungwa. Chekupedzisira, takashandisa chati kuona maitirwo eLLM pamabasa akasiyana siyana. Kufambira mberi uku kune mukana wekushandura ndima yeNLP, zvichiita kuti maLLM afunge uye anzwisise hukama hwakaoma pakati pepfungwa.
References
- Vaswani et al. (2017). Attention ndiyo Yese yaunoda. arXiv preprint arXiv:1706.03762.
- Devlin et al. (2018). BERT: Pre-kudzidziswa kweYakadzika Bidirectional Transformers yeMutauro Kunzwisisa. arXiv preprint arXiv:1810.04805.
- Radford et al. (2019). Mutauro Mamodheru haana Kutariswa Multitask Vadzidzi. arXiv preprint arXiv:1906.08237.