Soo rar qaabka LLM iyo tokenizer | B4N1

Soo rar qaabka LLM iyo tokenizer

3 months ago · b4n1 Read time: 5 min

** Fulinta Baaritaanka Dhacdada Toos ah ee Wakiilada DevOps iyo LLMs**

hordhac

Muuqaalka dhijitaalka ah ee degdega ah ee maanta, kooxaha DevOps waxay la kulmaan caqabad sii kordheysa oo kakan: aqoonsashada iyo xallinta shilalka sida ugu dhakhsaha badan. Hababka baaritaanka gacanta waxay noqon karaan waqti-qaadasho waxayna u nugul yihiin khaladaadka bini'aadamka, taasoo keenta waqti dheer oo hoos u dhaca iyo hoos u dhaca ku qanacsanaanta macaamiisha. Si arrintan wax looga qabto, waxaan sahamineynaa isgoysyada Automation-ka DevOps, AI ku shaqeysa baaritaanka shilalka, iyo doorka Models Language Models (LLMs) ee horumarinta aaladaha DevOps sida AWS DevOps Agent iyo Copado Agentia.

**Xaaladda Baaritaanka Dhacdada Hadda

Baadhitaanka shilku waa arrin muhiim ah oo ka mid ah DevOps, oo ku lug leh aqoonsiga, falanqaynta, iyo xallinta arrimaha saameeya nidaamyada software. Hababka dhaqameedku waxay ku tiirsan yihiin hababka gacanta, sida falanqaynta log, raadinta khaladaadka, iyo la xidhiidhka daneeyayaasha. Iyadoo hababkani ay waxtar leeyihiin, waxay noqon karaan kuwo gaabis ah oo u nugul khaladaadka, taasoo keenta:

  1. **Waqtiga hoos u dhaca oo kordhay ***: Hababka baaritaanka gacanta waxay daahiyaan xallinta shilka, taasoo keentay waqti-dhimis oo dheer iyo hoos u dhaca ku qanacsanaanta macaamilka.
  2. ** Qaladka Aadanaha ***: Hababka gacanta ayaa u nugul khaladka aadanaha, taas oo keeni karta ogaanshaha khaldan iyo xallinta dhacdooyinka.
  3. **Arrimaha miisaannada ***: Marka ay cabbirka iyo kakanaanta nidaamyada koraan, hababka baaritaanka gacanta ayaa sii adkaanaya in la cabbiro.

Doorka Wakiilada DevOps ee Baadhitaanka Dhacdooyinka

Wakiilada DevOps, sida AWS DevOps Agent iyo Copado Agentia, waxay door muhiim ah ka ciyaaraan si toos ah baaritaanka shilka. Wakiiladani waxay awoodaan:

  1. ** Ururi oo falanqeeyaan diiwaannada ***: Wakiilada DevOps waxay ururin karaan oo ay falanqeynayaan diiwaannada ilo kala duwan, iyagoo siinaya fikrado qiimo leh oo ku saabsan sababaha dhacdada.
  2. ** Raadinta qaladka otomaatig ah ***: Wakiiladu si toos ah ayay u habeyn karaan baadi-goobka qaladka, iyagoo yareynaya waqtiga iyo dadaalka looga baahan yahay baaritaanka gacanta.
  3. ** Kor u qaad xidhiidhka ***: Wakiiladu waxay fududayn karaan xidhiidhka ka dhexeeya kooxaha, daneeyayaasha, iyo nidaamyada, hubinta in macluumaadka dhacdada si hufan loo wadaago.

Doorka LLMs ee Baadhitaanka Dhacdada

Moodooyinka Luqadaha Waaweyn (LLMs) waxay wax ka beddeleen goobta habaynta luqadda dabiiciga ah (NLP), taasoo awood u siisay mashiinnada inay fahmaan oo ay dhaliyaan qoraalka aadanaha oo kale ah. Marka la eego macnaha guud ee baadhista shilka, LLMs waxay karaan:

  1. ** Falanqeeyaan warbixinnada dhacdada ***: LLMs waxay falanqeyn karaan warbixinnada dhacdada, iyagoo aqoonsanaya macluumaadka muhiimka ah iyo qaababka laga yaabo inay tilmaamayaan asalka dhacdada.
  2. **Samee dhacdooyinka kooban ee ***: LLM-yadu waxay dhalin karaan koobab kooban oo warbixinaha dhacdada, fududaynta isgaarsiinta iyo go'aan qaadashada.
  3. ** Bixi talooyinka xog-ogaal u ah**: LLM-yadu waxay bixin karaan talooyinka ku saabsan xalinta dhacdada, iyadoo la tixgalinayo nidaamka gaarka ah, deegaanka, iyo daneeyayaasha ku lug leh.

** Fulinta Baaritaanka Dhacdada Toos ah ee Wakiilada DevOps iyo LLMs**

Si loo hirgeliyo baadhis shil oo toos ah wakiilada DevOps iyo LLMs, waxaanu raaci doonaa hab tillaabo-tallaabo ah:

  1. ** Isku xidh wakiilada DevOps ***: Isku xidh wakiilada DevOps, sida AWS DevOps Agent iyo Copado Agentia, nidaamka maaraynta dhacdadaada.
  2. **Ururi oo falanqeeyaan diiwaannada ***: Habee wakiilada DevOps si ay u ururiyaan oo ay u falanqeeyaan diiwaannada ilo kala duwan, adoo siinaya fikrado qiimo leh oo ku saabsan sababaha dhacdada.
  3. Tababar qaabka LLM: Ku tababar model LLM xog-ururinta warbixinnada dhacdada, diiwaannada, iyo macluumaadka kale ee khuseeya.
  4. ** Isku-dubarid LLM iyo wakiilada DevOps ***: Isku-dubarid qaabka LLM iyo wakiilada DevOps, awoodna u siinaya LLM inay falanqeeyaan warbixinnada dhacdada oo ay bixiyaan talooyin ku saabsan xallinta dhacdada.
  5. **Deeg xalka ***: Geli xalka baadhista shilka ee otomaatiga ah, adoo hubinaya inuu yahay mid la cabiri karo, la isku halayn karo, oo sugan.

** Koodhka Tusaalaha: Isku-dhafka LLM iyo Wakiilada DevOps**

Waa kuwan tusaale kood kood ah oo ku jira Python, oo muujinaya sida loogu dari karo moodalka LLM wakiilka DevOps: `` Python soo dejinta soo dejinta Transformers ka soo dejisa AutoModelForSequenceClassification, AutoTokenizer Ka yimid aws_devops_ wakiilka soo dejinta

model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased") tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

Bilow wakiilka DevOps

wakiil = Wakiil ()

Qeex shaqada lagu falanqeynayo warbixinnada dhacdada iyadoo la adeegsanayo qaabka LLM

def analyze_incident_report(warbixin): # Horay u sii socodsii warbixinta dhacdada inputs = tokenizer.encode_plus (warbixin, add_special_tokens=Run, max_length=512, return_attention_mask=Run, return_tensors='pt')

# Falanqee warbixinta dhacdada adigoo isticmaalaya qaabka LLM
wax soo saarka = moodeel (wax-soo-galin ['input_ids'], attention_mask=garab ['fiirin_mask'])

# Soo saar summada fasalka la saadaaliyay
predicted_class = torch.argmax(outputs.logits)

soo celinta heerka la saadaaliyay

Qeex hawl si aad u bixiso talooyinka xalinta dhacdada

def give_talooyinka (la saadaaliyay_class): # Isticmaal summada fasalka la saadaaliyay si aad u bixiso talooyinka xalinta dhacdada haddii la saadaaliyay_class == 0: soo celi "Ka hubi nidaamka diiwaanka khaladaadka" elif saadaaliyay_class == 1: soo celi "xaqiiji isku xidhka shabakada" wax kale: soo celi "La xidhiidh kooxda horumarinta si aad gargaar u hesho"

Isku dhafka qaabka LLM wakiilka DevOps

Def dhacdo_investigation(): # Ka soo ururi oo falanqee diiwaannada wakiilka DevOps logs = wakiilka.get_logs ()

# Falanqee diiwaannada adigoo isticmaalaya qaabka LLM
predicted_class = falanqaynta_incident_report(logs)

# Soo jeedi talooyin ku saabsan xallinta dhacdada
talo soo jeedin = bixin_talo bixin(class_predicted_class)

soo celi talooyin

Geli xalinta baadhista dhacdada

Dhacdada_baaritaanka() `` ** 5-10ka ugu sarreeya amarada/Qodobka ugu Isticmaalka badan**

Waa kuwan 5-10ka amarrada ugu badan ee loo isticmaalo tignoolajiyadan:

  1. 'aws devops agent get-logs': Ka ururi diiwaannada wakiilka DevOps.
  2. transformers.AutoModelForSequenceClassification.from_pretrained(): Rar qaabka LLM.
  3. tokenizer.encode_plus(): Horay u sii socodsii warbixinta dhacdada.
  4. model(wax-soo-gelinta['input_ids'], attention_mask=wax-soo-gelinta['fiirin_mask']): Falanqee warbixinta dhacdada adigoo isticmaalaya qaabka LLM.
  5. torch.argmax(outputs.logits): Soo saar summada fasalka la saadaaliyay.
  6. bixi_talobixin(class_predicted_class): Bixi talooyinka xalinta dhacdada.
  7. agent.get_logs(): Ka ururi diiwaannada wakiilka DevOps.
  8. 'falanqayso_warbixin(warbixin)': Falanqee warbixinta dhacdada adoo isticmaalaya qaabka LLM.
  9. Baadhitaan_dhacdo()`: Geli xalinta baadhista dhacdada.
  10. `torch.save(model.state_dict(), 'model.pth'): Keydi qaabka LLM.

Muuqaalka

Halkan waxaa ah sawirida xalka baadhista shilalka tooska ah iyadoo la isticmaalayo Chart.js: json { "type": "bar", "xogta": { "labels": ["Baaritaanka Buugga", "Baaritaan Automated"], "datasets": [ { "label": "Waqtiga xalinta", "xogta": [10, 2], "backgroundColor": ["rgba(255, 99, 132, 0.2)", "rgba(54, 162, 235, 0.2)"], "BorderColor": ["rgba(255, 99, 132, 1)", "rgba(54, 162, 235, 1)"], "BorderWidth": 1 } ] }, "doorashooyinka": { "ciwaan": { "muujin": run, "text": "Waqtiga xalinta shilalka" }, "Miisaanka": { "yAxes": [ { "ticks": { "beginAtZero": run } } ] } } } Aragtidaani waxay isbarbar dhigaysaa wakhtiga lagu xalinayo shilalka iyadoo la adeegsanayo hababka baadhista gacanta iyo hababka baadhitaanka tooska ah iyadoo la isticmaalayo wakiilada DevOps iyo LLMs.

** Gabagabo**

Hirgelinta baadhista shilalka tooska ah ee wakiilada DevOps iyo LLMs waxay si weyn u wanaajin kartaa hufnaanta iyo waxtarka xalinta dhacdada. Marka la isku daro wakiilada DevOps iyo LLM-yada, ururadu waxa ay si otomaatig ah u samayn karaan falanqaynta diiwaanka, bixiyaan talooyinka macnaha guud ee xalinta shilka, oo ay yareeyaan wakhtiga lagu xalinayo shilalka. Waxaan sahminay isgoysyada Automation-ka DevOps, AI ku shaqeysa baarista shilka, iyo doorka LLMs ee horumarinta aaladaha DevOps sida AWS DevOps Agent iyo Copado Agentia. Iyadoo la raacayo habka tallaabo-tallaabo ee lagu qeexay qodobkan, ururradu waxay geyn karaan xal baaris oo toos ah, la isku halayn karo, oo sugan.


Have Questions?

Let's Talk

Contact me on LinkedIn