Intshayelelo
IiModeli zoLwimi olukhulu (LLMs) ziwuguqule kakhulu ummandla wokusetyenziswa kolwimi lwendalo (NLP) kwiminyaka yakutshanje. Le mifuziselo ifezekise iziphumo ezikumgangatho ophezulu kwimisebenzi eyahlukeneyo ye-NLP, kubandakanywa ukuguqulelwa kolwimi, isishwankathelo sombhalo, kunye nokuphendula imibuzo. Nangona kunjalo, amandla okwenyani e-LLMs alele kubuchule babo bokuqiqa kunye nokuqonda ubudlelwane obuntsonkothileyo phakathi kweengqikelelo. Kweli nqaku, siza kuphonononga inkqubela phambili kwi-LLM yokuqiqa, sijonge iindlela ezisisiseko, uyilo, kunye nemingeni.
Imvelaphi
Ii-LLM luhlobo lothungelwano lwe-neural oluqeqeshelwe ubuninzi bedatha yokubhaliweyo ukufunda iipateni kunye nobudlelwane phakathi kwamagama kunye neengqiqo. Le mifuziselo ibandakanya i-encoder kunye ne-decoder, apho i-encoder iqhubekisa igalelo lombhalo kunye ne-decoder ivelisa umbhalo wemveliso. I-encoder idla ngokuba yi-architecture esekelwe kwi-transformer, efanelekile ngokukodwa kwiidatha ezilandelelanayo ezifana nokubhaliweyo.
Uyilo lweTransformer
I-architecture ye-transformer, eyaziswa kwiphepha elithi "Ingqalelo yiyo yonke into oyifunayo" nguVaswani et al. (2017), ibe ngumgangatho we-de facto we-LLMs. I-transformer iqulethe i-encoder kunye ne-decoder, nganye yenziwe ngamaleko amaninzi. Umaleko ngamnye uqulathe amanqwanqwa angaphantsi amabini: isixhobo sokuzihoya kunye nenethwekhi eqhagamshelwe ngokupheleleyo ye-feed-forward network (FFNN).
ngenisa itotshi
ngenisa itotshi.nn njenge nn
udidi loGuqulo lweLayer(nn.Modyuli):
def __init__(self, d_model, n_heads, d_ff):
super(TransformerLayer, self).__init__()
self.self_attn = nn.MultiHeadAttention(d_model, n_heads)
self.ffnn = nn.Linear(d_model, d_ff)
self.dropout = nn.Dropout(0.1)
self.layer_norm = nn.LayerNorm(d_model)
def phambili(self, x):
# Isixhobo sokuzihoya
x_attn = self.self_attn(x, x)
x_attn = self.dropout(x_attn)
x_attn = x + x_attn
# FFNN
x_ffnn = self.ffnn(x_attn)
x_ffnn = self.dropout(x_ffnn)
x_ffnn = self.layer_norm(x_ffnn + x_attn)
buyisela x_ffnn
Ukuqhubela phambili kwi-LLM Ukuqiqa
Ii-LLM zenze inkqubela ebonakalayo ekuqiqeni nasekuqondeni ubudlelwane obuntsonkothileyo phakathi kweekhonsepthi. Ezinye zezona nkqubela ziphambili ziquka:
1. Ukuqiqa okuSekwe kwigrafu
Ukuqiqa okusekwe kwigrafu kubandakanya ukumela ulwazi njengegrafu, apho iindawo ezimele iikhonsepthi kunye nemiphetho zimele ubudlelwane phakathi kwazo. Ii-LLMs zinokuqeqeshelwa ukuqiqa ngezi grafu, zenza uqikelelo malunga nobudlelwane phakathi kweendawo zokuhlala.
sebenzisa igrafu::Igrafu;
sebenzisa i-neural_network::NeuralNetwork;
fn igrafu_ukuqiqa(igraph: &Graph, umbuzo: &str) -> Umtya {
vumela uzinziso = neural_network.query(igrafu, umbuzo);
vumela isiphumo = neural_network.reason(igrafu, ukufakela);
result.to_string()
}
2. UkuFunda kwemiSebenzi emininzi
Ukufunda imisebenzi emininzi kubandakanya ukuqeqesha imodeli enye kwimisebenzi emininzi ngaxeshanye. Le ndlela ibonakaliswe ukuphucula ukusebenza kwee-LLM kwimisebenzi eyahlukeneyo, kubandakanywa ukuqiqa nokuqonda ubudlelwane obuntsonkothileyo.
iphakheji engundoqo
ngenisa ngaphandle (
"fmt"
"gonum.org/v1/gonum/mat"
)
uhlobo lwesakhiwo semodeli {
*nn.Umgca
*nn.Umgca
}
func (m *Imodeli) Phambili(x *mat.Dense) *mat.Dense {
// Ukudlula phambili
z := m.Umgca(x)
z = m.Umgca(z)
buya z
}
3. Iindlela zokuthathela ingqalelo
Iindlela zokuthathela ingqalelo zivumela ii-LLMs ukuba zigxininise kwiindawo ezithile zombhalo wongeniso xa zivelisa imveliso. Le ndlela ibonakaliswe ukuphucula ukusebenza kwee-LLM kwimisebenzi eyahlukeneyo, kubandakanywa ukuqiqa nokuqonda ubudlelwane obuntsonkothileyo.
ngenisa itotshi
ngenisa itotshi.nn njenge nn
iklasi yokuNika iMechanism(nn.Modyuli):
def __init__(self, d_model, n_heads):
super(AttentionMechanism, self).__init__()
self.query = nn.Linear(d_model, d_model)
self.key = nn.Linear(d_model, d_model)
self.value = nn.Linear(d_model, d_model)
self.dropout = nn.Dropout(0.1)
self.layer_norm = nn.LayerNorm(d_model)
def phambili(self, x):
# Umbuzo, isitshixo, kunye nexabiso
q = self.query(x)
k = self.key(x)
v = self.value(x)
# Ingqalelo yobunzima
ubunzima = itotshi.matmul(q, k.T) / math.sqrt(q.shape[1])
ubunzima = self.dropout(ubunzima)
ubunzima = torch.softmax(ubunzima, dim=-1)
# Vector yomxholo
umxholo = itotshi.matmul(ubunzima, v)
buyisela umxholo
Umboniso
Ukujonga ukusebenza kwee-LLM kwimisebenzi eyahlukeneyo, sinokusebenzisa itshathi ukuthelekisa ukuchaneka kweemodeli ezahlukeneyo kumsebenzi onikiweyo. Nanku umzekelo wetshati esebenzisa iChart.js:
{
"type": "bar",
"idatha": {
"labels": ["Umsebenzi 1", "Umsebenzi 2", "Umsebenzi 3"],
"iiseti zedatha": [
{
"label": "Imodeli 1",
"idatha": [0.8, 0.7, 0.9],
"backgroundColor": "rgba(255, 99, 132, 0.2)",
"borderColor": "rgba(255, 99, 132, 1)",
"BorderWidth": 1
},
{
"label": "Imodeli 2",
"idatha": [0.9, 0.8, 0.7],
"backgroundColor": "rgba(54, 162, 235, 0.2)",
"borderColor": "rgba(54, 162, 235, 1)",
"BorderWidth": 1
}
]
},
"iinketho": {
"isihloko": {
"bonisa": yinyani,
"text": "Ukuchaneka kweeModeli ezahlukeneyo kwiMisebenzi eyahlukeneyo"
},
"izikali": {
"yAxes": [
{
"amanqaku": {
"beginAtZero": yinyani
}
}
]
}
}
}
Isiphelo
Kweli nqaku, siye saphonononga inkqubela phambili kwi-LLM yokuqiqa, kubandakanywa ukuqiqa okusekwe kwigrafu, ukufunda ngemisebenzi emininzi, kunye neendlela zokuthathela ingqalelo. Siye sanikezela ngemizekelo yekhowudi kwiPython, Rust, kunye neGo ukucacisa ezi ngqikelelo. Okokugqibela, sisebenzise itshati ukujonga ukusebenza kweeLLM kwimisebenzi eyahlukeneyo. Ezi nkqubela phambili zinamandla okuguqula icandelo le-NLP, zenza ukuba ii-LLM ziqiqe kwaye ziqonde ubudlelwane obuntsonkothileyo phakathi kweengqikelelo.
IiReferensi
- Vaswani et al. (2017). Ingqalelo kuko konke okufunayo. arXiv preprint arXiv:1706.03762.
- UDevlin et al. (2018). I-BERT: Uqeqesho lwangaphambili lwe-Deep Bidirectional Transformers ukuze kuqondwe uLwimi. arXiv preprint arXiv:1810.04805.
- URadford et al. (2019). Imifuziselo yoLwimi ngaBafundi beMisebenzi emininzi eNgagadwayo. arXiv preprint arXiv:1906.08237.