Ci gaba a Tunani na LLM: Zurfin Zurfin Fasaha

Gabatarwa

Manyan Samfuran Harshe (LLMs) sun kawo sauyi a fannin sarrafa harshe na halitta (NLP) a cikin 'yan shekarun nan. Waɗannan samfuran sun sami sakamako na zamani a cikin ayyuka daban-daban na NLP, gami da fassarar harshe, taƙaitaccen rubutu, da amsa tambaya. Koyaya, haƙiƙanin yuwuwar LLMs ya ta'allaka ne ga ikonsu na tunani da fahimtar hadaddun alaƙa tsakanin ra'ayoyi. A cikin wannan labarin, za mu shiga cikin ci gaba a cikin tunanin LLM, bincika dabarun da ke ƙasa, gine-gine, da ƙalubale.

##Baya

LLMs wani nau'in cibiyar sadarwa ne na jijiyoyi wanda aka horar da akan ɗimbin bayanan rubutu don koyan tsari da alaƙa tsakanin kalmomi da ra'ayoyi. Waɗannan samfuran yawanci sun ƙunshi encoder da dikodi, inda mai rikodin ke aiwatar da rubutun shigar da mai ƙaddamarwa ke haifar da rubutun fitarwa. Rubutun yawanci tsarin gine-gine ne na tushen wuta, wanda ya dace musamman don bayanan jeri kamar rubutu.

Taswira Architecture

Tsarin gine-gine, wanda aka gabatar a cikin takarda "Hankali shine Duk Abinda kuke Bukata" na Vaswani et al. (2017), ya zama ma'auni na gaskiya don LLMs. Transformer ya ƙunshi encoder da decoder, kowanne ya ƙunshi yadudduka da yawa. Kowane Layer ya ƙunshi ƙananan yadudduka biyu: na'ura mai kulawa da kai da cikakkiyar hanyar sadarwar ciyarwa ta gaba (FFNN).

`` Python shigo da tocila shigo da tocila.nn kamar nn

Class TransformerLayer(nn.Module): def init(kai, d_model, n_heads, d_ff): super (TransformerLayer, kai).init() self.self_attn = nn.MultiHeadAttention(d_model, n_heads) self.ffnn = nn.Linear(d_model, d_ff) kai.Dropout = nn.Dropout(0.1) self.layer_norm = nn.LayerNorm(d_model)

def gaba (kai, x):
    # Tsarin kulawa da kai
    x_attn = kai.self_attn (x, x)
    x_attn = kai. dropout(x_attn)
    x_attn = x + x_attn

    # FFNN
    x_ffnn = kai.ffnn (x_attn)
    x_ffnn = kai. dropout (x_ffnn)
    x_ffnn = self.layer_norm(x_ffnn + x_attn)

    dawo x_ffn

## Ci gaba a Tunanin LLM

LLMs sun sami babban ci gaba a cikin tunani da fahimtar hadaddun alaƙa tsakanin ra'ayoyi. Wasu daga cikin mahimman ci gaban sun haɗa da:

### 1. Tunani Mai Girma

Tunanin tushen zane ya ƙunshi wakiltar ilimi azaman jadawali, inda nodes ke wakiltar ra'ayoyi da gefuna suna wakiltar alaƙa tsakanin su. Ana iya horar da LLMs don yin tunani game da waɗannan jadawali, yin tsinkaya game da alaƙa tsakanin nodes.

``tsatsa
amfani da jadawali :: Graph;
amfani da neural_network :: NeuralNetwork;

fn graph_reasoning (zane: &Graph, tambaya: &str) -> igiya {
    bari sakawa = neural_network.query (zane, tambaya);
    bari sakamako = neural_network.dalili (jadawalin, sakawa);
    sakamako.to_string()
}

2. Koyon Ayyuka da yawa

Koyon ayyuka da yawa ya ƙunshi horar da ƙira ɗaya akan ayyuka da yawa a lokaci guda. An nuna wannan hanyar don inganta ayyukan LLMs akan ayyuka daban-daban, gami da tunani da fahimtar alaƙa masu rikitarwa.

``tafi babban kunshin

shigo da ( "fmt" "gonum.org/v1/gonum/mat" )

rubuta Tsarin Model { *nn.Linear *nn.Linear }

func (m *Model) Gaba(x *mat.Dense) *mat.Dense { // Wuce gaba z:= m.Linear(x) z = m.Linear(z) dawo z }

### 3. Hankalin Hanyoyi

Hanyoyin kulawa suna ba LLMs damar mayar da hankali kan takamaiman sassa na rubutun shigarwa lokacin samar da fitarwa. An nuna wannan hanyar don inganta ayyukan LLMs akan ayyuka daban-daban, gami da tunani da fahimtar hadaddun alaƙa.

`` Python
shigo da tocila
shigo da tocila.nn kamar nn

Aji HankaliMechanism(nn.Module):
    def __init__(kai, d_model, n_heads):
        super (AttentionMechanism, kai).__init__()
        self.query = nn.Linear(d_model, d_model)
        self.key = nn.Linear(d_model, d_model)
        darajar kai = nn.Linear(d_model, d_model)
        kai.Dropout = nn.Dropout(0.1)
        self.layer_norm = nn.LayerNorm(d_model)

    def gaba (kai, x):
        # Tambaya, maɓalli, da ƙima
        q = tambayar kai (x)
        k = kai.key(x)
        v = darajar kai (x)

        # Hankalin nauyi
        nauyi = torch.matmul(q, k.T) / math.sqrt (q.siffa[1])
        nauyi = kai. dropout(nauyi)
        nauyi = torch.softmax (nauyin nauyi, dim=-1)

        # Fassarar yanayi
        mahallin = torch.matmul (masu nauyi, v)

        mayar da mahallin

Kallon gani

Don ganin aikin LLMs akan ayyuka daban-daban, zamu iya amfani da ginshiƙi don kwatanta daidaiton ƙira daban-daban akan wani aikin da aka bayar. Ga misalin ginshiƙi ta amfani da Chart.js:

Json { "type": "bar", "data": { "labels": ["Task 1", "Task 2", "Task 3"], "datasets": [ { "label": "Model 1", "bayanai": [0.8, 0.7, 0.9], "backgroundColor": "rgba(255, 99, 132, 0.2)", "BorderColor": "rgba(255, 99, 132, 1)", "BorderWidth": 1 }, { "label": "Model 2", "bayanai": [0.9, 0.8, 0.7], "backgroundColor": "rgba(54, 162, 235, 0.2)", "BorderColor": "rgba(54, 162, 235, 1)", "BorderWidth": 1 } ] }, "zaɓi": { "take": { "nuna": gaskiya, "text": "Tsarin Samfura Daban-daban akan Aiyuka Daban-daban" }, "ma'auni": { "yAxes": [ { "zama": { "beginAtZero": gaskiya } } ] } } }

## Kammalawa

A cikin wannan labarin, mun bincika ci gaba a cikin tunanin LLM, gami da tunanin tushen jadawali, koyan ayyuka da yawa, da hanyoyin kulawa. Mun kuma bayar da misalai na lamba a Python, Tsatsa, da Go don kwatanta waɗannan ra'ayoyin. A ƙarshe, mun yi amfani da ginshiƙi don hango ayyukan LLM akan ayyuka daban-daban. Waɗannan ci gaban suna da yuwuwar kawo sauyi a fagen NLP, yana ba LLMs damar yin tunani da fahimtar hadaddun alaƙa tsakanin ra'ayoyi.

## Magana

* Vaswani et al. (2017). Hankali shine Duk Abinda kuke Bukata. arXiv preprint arXiv: 1706.03762.
* Devlin et al. (2018). BERT: Gabatarwar Horowar Masu Canjin Bidiyo Mai zurfi don Fahimtar Harshe. arXiv preprint arXiv: 1810.04805.
* Radford et al. (2019). Samfuran Harshe Masu Koyan ayyuka da yawa marasa kulawa. arXiv preprint arXiv: 1906.08237.