Kenyelletso
Mehlala e Meholo ea Lipuo (LLMs) e fetotse tšebetso ea puo ea tlhaho (NLP) lilemong tsa morao tjena. Mehlala ena e fihletse liphetho tsa maemo a holimo mesebetsing e fapaneng ea NLP, ho kenyeletsoa phetolelo ea puo, kakaretso ea mongolo, le likarabo tsa lipotso. Leha ho le joalo, bokhoni ba 'nete ba LLM bo ho bokhoni ba bona ba ho beha mabaka le ho utloisisa likamano tse rarahaneng lipakeng tsa mehopolo. Sengoliloeng sena, re tla hlahloba tsoelo-pele ea LLM ea ho beha mabaka, ho lekola mekhoa, meaho le liphephetso.
Background
Li-LLM ke mofuta oa marang-rang a neural a koetliselitsoeng ho latela lintlha tse ngata tsa mongolo ho ithuta mekhoa le likamano lipakeng tsa mantsoe le mehopolo. Hangata mefuta ena e na le encoder le decoder, moo encoder e sebetsanang le mongolo o kentsoeng le decoder e hlahisa mongolo o hlahisoang. Hangata encoder ke meralo e thehiloeng ho li-transformer, e loketseng haholo data e latellanang joalo ka mongolo.
Moralo oa Transformer
Mehaho ea li-transformer, e hlahisitsoeng pampiring "Tlhokomeliso ke Sohle Seo U se Hlokang" ka Vaswani et al. (2017), e fetohile tekanyetso ea de facto bakeng sa LLMs. Transformer e na le encoder le decoder, e 'ngoe le e' ngoe e entsoe ka likarolo tse ngata. Lera le leng le le leng le na le likaroloana tse peli: mokhoa oa ho itlhokomela le marang-rang a hokahaneng ka botlalo (FFNN).
kenya lebone
import torch.nn joalo ka nn
sehlopha sa TransformerLayer(nn.Module):
def __init__(self, d_model, n_heads, d_ff):
super(TransformerLayer, self).__init__()
self.self_attn = nn.MultiHeadAttention(d_model, n_heads)
self.ffnn = nn.Linear(d_model, d_ff)
self.dropout = nn.Dropout(0.1)
self.layer_norm = nn.LayerNorm(d_model)
def forward(self, x):
# Mokhoa oa ho itlhokomela
x_attn = self.self_attn(x, x)
x_attn = self.dropout(x_attn)
x_attn = x + x_attn
# FFNN
x_ffnn = self.ffnn(x_attn)
x_ffnn = self.dropout(x_ffnn)
x_ffnn = self.layer_norm(x_ffnn + x_attn)
khutlela x_ffnn
Tsoelopele ho LLM Ho Bea Mabaka
Li-LLM li entse tsoelo-pele e kholo ho beha mabaka le ho utloisisa likamano tse rarahaneng lipakeng tsa mehopolo. Tse ling tsa likhatelopele tsa bohlokoa li kenyelletsa:
1. Ho Bea Mabaka ka Kerafo
Menahano e thehiloeng ho kerafo e kenyelletsa ho emela tsebo joalo ka kerafo, moo li-node li emelang mehopolo le methati e emelang likamano lipakeng tsa tsona. Li-LLM li ka koetlisetsoa ho beha mabaka ka li-graph tsena, li bolela esale pele mabapi le likamano lipakeng tsa li-node.
sebelisa kerafo::Kerafo;
sebelisa neural_network :: NeuralNetwork;
fn graph_reasoning(graph: &graph, query: &str) -> String {
tlohella embedding = neural_network.query(graph, query);
tlohella sephetho = neural_network.reason(graph, embedding);
result.to_string()
}
2. Ho Ithuta ka Mesebetsi e Mengata
Ho ithuta ka mesebetsi e mengata ho kenyelletsa ho koetlisa mohlala o le mong mesebetsing e mengata ka nako e le 'ngoe. Mokhoa ona o bontšitsoe ho ntlafatsa ts'ebetso ea LLM mesebetsing e fapaneng, ho kenyeletsoa ho beha mabaka le ho utloisisa likamano tse rarahaneng.
sephutheloana se seholo
kenya (
"fmt"
"gonum.org/v1/gonum/mat"
)
mofuta oa sebopeho sa mohlala {
*nn.Mohala
*nn.Mohala
}
func (m *Model) Forward(x *mat.Dense) *mat.Dense {
// Ho fetela pele
z := m. Linear(x)
z = m.Mohala(z)
khutlela z
}
3. Mekhoa ea ho ela hloko
Mekhoa ea ho ela hloko e lumella li-LLM ho tsepamisa maikutlo likarolong tse itseng tsa mongolo ha li hlahisa tlhahiso. Mokhoa ona o bontšitsoe ho ntlafatsa ts'ebetso ea LLM mesebetsing e fapaneng, ho kenyeletsoa ho beha mabaka le ho utloisisa likamano tse rarahaneng.
kenya lebone
import torch.nn joalo ka nn
Sehlopha sa TlhokomeloMechanism(nn.Module):
def __init__(self, d_model, n_heads):
super(AttentionMechanism, self).__init__()
self.query = nn.Linear(d_model, d_model)
self.key = nn.Linear(d_model, d_model)
self.value = nn.Linear(d_model, d_model)
self.dropout = nn.Dropout(0.1)
self.layer_norm = nn.LayerNorm(d_model)
def forward(self, x):
# Potso, senotlolo, le boleng
q = self.query(x)
k = self.key(x)
v = self.value(x)
# Litekanyo tsa tlhokomeliso
boima = torch.matmul(q, k.T) / math.sqrt(q.shape[1])
boima = self.dropout(boima)
boima = torch.softmax(boima, dim=-1)
# Vector ea litaba
moelelo = toche.matmul(boima, v)
kgutlisetsa moelelo
Ho bona pono
Ho bona ka mahlo a kelello tšebetso ea LLM mesebetsing e fapaneng, re ka sebelisa chate ho bapisa ho nepahala ha mehlala e fapaneng mosebetsing o fanoeng. Mohlala ke ona oa chate e sebelisang Chart.js:
{
"type": "bar",
"data": {
"labels": ["Mosebetsi oa 1", "Mosebetsi oa 2", "Mosebetsi oa 3"],
"lintlha tsa boitsebiso": [
{
"label": "Mohlala oa 1",
"data": [0.8, 0.7, 0.9],
"backgroundColor": "rgba(255, 99, 132, 0.2)",
"borderColor": "rgba(255, 99, 132, 1)",
"borderWidth": 1
},
{
"label": "Mohlala oa 2",
"data": [0.9, 0.8, 0.7],
"backgroundColor": "rgba(54, 162, 235, 0.2)",
"borderColor": "rgba(54, 162, 235, 1)",
"borderWidth": 1
}
]
},
"likgetho": {
"sehlooho": {
"bontša": 'nete,
"text": "Ho Nepaha ha Meetso e Fapaneng ka Mesebetsi e Fapaneng"
},
"makala": {
"yAxes": [
{
"li-ticks": {
"beginAtZero": 'nete
}
}
]
}
}
}
Qetello
Sengoliloeng sena, re hlahlobile tsoelo-pele ea ho beha mabaka ka LLM, ho kenyelletsa ho beha mabaka ho thehiloeng ho kerafo, ho ithuta ka mesebetsi e mengata, le mekhoa ea tlhokomelo. Re boetse re fane ka mehlala ea khoutu ho Python, Rust, le Go ho hlalosa mehopolo ena. Qetellong, re sebelisitse chate ho bona ka mahlo a kelello tšebetso ea LLM mesebetsing e fapaneng. Tsoelo-pele ena e na le monyetla oa ho fetola tšimo ea NLP, e nolofalletsa LLM ho beha mabaka le ho utloisisa likamano tse rarahaneng pakeng tsa likhopolo.
Litšupiso
- Vaswani et al. (2017). Tlhokomelo ke Sohle Seo U se Hlokang. arXiv preprint arXiv:1706.03762.
- Devlin et al. (2018). BERT: Koetliso ea pele ea li-Deep Bidirectional Transformers bakeng sa Kutloisiso ea Puo. arXiv preprint arXiv:1810.04805.
- Radford le ba bang. (2019). Mehlala ea Puo ke Baithuti ba sa Laoloeng Mesebetsi e Mengata. arXiv preprint arXiv:1906.08237.