Iterambere muri LLM Impamvu: Tekinike Yimbitse

Intangiriro

Ururimi runini (LLMs) rwahinduye urwego rwo gutunganya ururimi karemano (NLP) mumyaka yashize. Izi ngero zageze ku buhanga bugezweho mu bikorwa bitandukanye bya NLP, harimo guhindura ururimi, incamake y’inyandiko, no gusubiza ibibazo. Nyamara, ubushobozi nyabwo bwa LLMs buri mubushobozi bwabo bwo gutekereza no gusobanukirwa isano igoye hagati yibitekerezo. Muri iki kiganiro, tuzacukumbura iterambere mu bitekerezo bya LLM, dushakishe ubuhanga bwibanze, ubwubatsi, nibibazo.

Amavu n'amavuko

LLMs ni ubwoko bwurusobe rwimyitozo yatojwe kumubare munini wamakuru wanditse kugirango yige imiterere nubusabane hagati yamagambo nibitekerezo. Izi moderi mubisanzwe zigizwe na kodegisi na decoder, aho encoder itunganya inyandiko yinjiza kandi decoder ikabyara inyandiko isohoka. Ubusanzwe kodegisi nububiko bushingiye kuri transformateur, ikaba ikwiranye cyane namakuru akurikirana nkumwandiko.

Guhindura imyubakire

Ubwubatsi bwa transformateur, bwatangijwe mu mpapuro "Icyitonderwa nicyo ukeneye" na Vaswani n'abandi. (2017), yahindutse de-facto ya LLMs. Transformator igizwe na encoder na decoder, buri kimwe kigizwe nibice byinshi. Buri cyiciro kigizwe nibice bibiri: uburyo bwo kwiyitaho hamwe numuyoboro wuzuye uhuza-imbere (FFNN).

`` python itara ryinjira gutumiza itara.nn nka nn

icyiciro TransformerLayer (nn.Module): def init __ (wenyine, d_model, n_imitwe, d_ff): super (TransformerLayer, wenyine) . init __ () wenyine.wowe_attn = nn. wenyine.ffnn = nn.Umurongo (d_model, d_ff) wenyine.dropout = nn.Umusozo (0.1) wenyine.umukinyi_norm = nn.UmurongoNorm (d_model)

def imbere (wenyine, x):
    # Uburyo bwo kwiyitaho
    x_attn = wenyine.wowe_attn (x, x)
    x_attn = kwigira.dropout (x_attn)
    x_attn = x + x_attn

    # FFNN
    x_ffnn = wenyine.ffnn (x_attn)
    x_ffnn = kwikorera.dropout (x_ffnn)
    x_ffnn = wenyine.umukinyi_idasanzwe (x_ffnn + x_attn)

    garuka x_ffnn

`` `

Iterambere muri LLM Gutekereza

LLMs yateye imbere cyane mubitekerezo no gusobanukirwa isano igoye hagati yibitekerezo. Bimwe mubyingenzi byateye imbere harimo:

1. Igishushanyo-gishingiye kubitekerezo

Igishushanyo gishingiye ku gishushanyo gikubiyemo guhagararira ubumenyi nkigishushanyo, aho imitwe igereranya imyumvire nimpande zerekana isano hagati yabo. LLMs irashobora gutozwa gutekereza kuriyi shusho, itanga ubuhanuzi kubyerekeye isano iri hagati yumutwe.

`` ingese koresha igishushanyo :: Igishushanyo; koresha neural_network :: NeuralNetwork;

fn igishushanyo mbonera (igishushanyo: & Igishushanyo, ikibazo: & str) -> Ikirongo { reka gushira = neural_network.query (igishushanyo, ikibazo); reka ibisubizo = neural_network.impamvu (igishushanyo, gushira); ibisubizo.kugeza () } `` `

2. Kwiga Ibikorwa byinshi

Kwiga ibintu byinshi bikubiyemo guhugura icyitegererezo kimwe kumirimo myinshi icyarimwe. Ubu buryo bwerekanwe kunoza imikorere ya LLMs kumirimo itandukanye, harimo gutekereza no gusobanukirwa umubano utoroshye.

`` genda paki nyamukuru

gutumiza mu mahanga ( "fmt" "gonum.org/v1/gonum/mat" )

Ubwoko bw'icyitegererezo { * nn * nn }

imikorere (m * Icyitegererezo) Imbere (x * mat.Ubucucike) * mat.Ubucucike { // Imbere z: = m.Umurongo (x) z = m.Umurongo (z) garuka z } `` `

3. Uburyo bwo Kwitondera

Uburyo bwo kwitondera butuma LLM yibanda kubice byihariye byinyandiko yinjiza mugihe itanga ibisohoka. Ubu buryo bwerekanwe kunoza imikorere ya LLMs kumirimo itandukanye, harimo gutekereza no gusobanukirwa umubano utoroshye.

`` python itara ryinjira gutumiza itara.nn nka nn

icyiciro IcyitonderwaManism (nn.Module): def init __ (wenyine, d_model, n_imitwe): super (AttentionMechanism, self) . init __ () kwishakamo ibisubizo = nn.Umurongo (d_model, d_model) kwikorera.key = nn.Umurongo (d_model, d_model) kwigira.agaciro = nn.Umurongo (d_model, d_model) wenyine.dropout = nn.Umusozo (0.1) wenyine.umukinyi_norm = nn.UmurongoNorm (d_model)

def imbere (wenyine, x):
    # Ikibazo, urufunguzo, nagaciro
    q = kwibaza. (x)
    k = kwigira.key (x)
    v = kwigira.agaciro (x)

    # Itondere uburemere
    uburemere = itara.matmul (q, k.T) / imibare.sqrt (q.ishusho [1])
    uburemere = kwigira.ibitonyanga (uburemere)
    uburemere = itara.softmax (uburemere, dim = -1)

    # Imiterere
    imiterere = itara.matmul (uburemere, v)

    garuka imiterere

`` `

Kugaragara

Kugereranya imikorere ya LLMs kumirimo itandukanye, turashobora gukoresha imbonerahamwe yo kugereranya ukuri kwa moderi zitandukanye kumurimo runaka. Dore urugero rwimbonerahamwe ukoresheje Imbonerahamwe.js:

json { "ubwoko": "akabari", "amakuru": { "ibirango": ["Umukoro 1", "Umukoro 2", "Umukoro 3"], "imibare": [ { "label": "Icyitegererezo 1", "amakuru": [0.8, 0.7, 0.9], "backgroundColor": "rgba (255, 99, 132, 0.2)", "umupaka w'amabara": "rgba (255, 99, 132, 1)", "Ubugari bwumupaka": 1 }, { "label": "Model 2", "amakuru": [0.9, 0.8, 0.7], "backgroundColor": "rgba (54, 162, 235, 0.2)", "umupaka Ibara": "rgba (54, 162, 235, 1)", "Ubugari bwumupaka": 1 } ] }, "amahitamo": { "umutwe": { "kwerekana": ukuri, "inyandiko": "Ukuri kwa Moderi zitandukanye kubikorwa bitandukanye" }, "umunzani": { "yAxes": [ { "amatiku": { "gutangiraAtZero": ukuri } } ] } } } `

Umwanzuro

Muri iki kiganiro, twasuzumye iterambere mu bitekerezo bya LLM, harimo ibitekerezo bishingiye ku gishushanyo, kwiga imirimo myinshi, hamwe nuburyo bwo kwitondera. Twatanze kandi ingero zingero muri Python, Rust, na Go kugirango twerekane ibi bitekerezo. Hanyuma, twakoresheje imbonerahamwe kugirango twerekane imikorere ya LLMs kumirimo itandukanye. Iterambere rifite ubushobozi bwo guhindura ibintu murwego rwa NLP, bigafasha LLM gutekereza no gusobanukirwa isano igoye hagati yibitekerezo.

Reba

• Vaswani n'abandi. (2017). Icyitonderwa nicyo ukeneye cyose. arXiv itegura arXiv: 1706.03762.
• Devlin n'abandi. (2018). BERT: Mbere yo guhugura abahindura ibice byimbitse kugirango basobanukirwe ururimi. arXiv itegura arXiv: 1810.04805.
• Radford n'abandi. (2019). Ururimi Model ni Ntakurikiranwa Abiga Multitask. arXiv itegura arXiv: 1906.08237.