Budata ụdị LLM na tokenizer | B4N1

Budata ụdị LLM na tokenizer

6 months ago · b4n1 Read time: 6 min

** Na-eme nyocha ihe mberede na ndị ọrụ DevOps na LLMs ***

** Okwu mmalite**

N'ime odida obodo dijitalụ na-agba ọsọ taa, ndị otu DevOps na-eche ihe ịma aka siri ike ihu: ịchọpụta na idozi ihe mere ngwa ngwa enwere ike. Ụzọ nyocha nke aka nwere ike na-ewe oge ma na-enwekarị mmejọ mmadụ, na-ebute ogologo oge nkwụsịtụ na ibelata afọ ojuju ndị ahịa. Iji dozie okwu a, anyị na-enyocha njikọ nke DevOps automation, nyocha ihe omume AI kwadoro, yana ọrụ nke Ụdị Asụsụ Large (LLMs) n'ịkwalite ngwaọrụ DevOps dị ka AWS DevOps Agent na Copado Agentia.

** Ọnọdụ nyocha ihe mere ugbu a**

Nnyocha ihe mberede bụ akụkụ dị oke egwu nke DevOps, gụnyere njirimara, nyocha, na mkpebi nke okwu metụtara sistemụ ngwanrọ. Ụzọ ọdịnala na-adabere na usoro ntuziaka, dị ka nyocha ndekọ, nleba anya njehie, na nkwurịta okwu n'etiti ndị ihe metụtara. Ọ bụ ezie na ụzọ ndị a dị irè, ha nwere ike ịdị nwayọọ ma na-enwekarị mmejọ, na-eduga na:

  1. ** Ịbawanye oge nkwụsị ***: Ụzọ nyocha akwụkwọ ntuziaka nwere ike igbu oge mkpebi ihe merenụ, na-ebute ogologo oge nkwụsịtụ na ibelata afọ ojuju ndị ahịa.
  2. ** Njehie mmadụ ***: Usoro ntuziaka na-enwe ike ịdaba na njehie mmadụ, nke nwere ike ibute nchọpụta na-ezighi ezi na mkpebi nke ihe omume.
  3. ** Okwu mkpali ***: Ka nha na mgbagwoju anya nke sistemu na-eto, ụzọ nyocha akwụkwọ ntuziaka na-esiwanye ike n'ịbawanye.

** Ọrụ nke ndị ọrụ DevOps na nyocha ihe mberede ***

Ndị nnọchi anya DevOps, dị ka AWS DevOps Agent na Copado Agentia, na-arụ ọrụ dị mkpa n'ịmepụta nyocha ihe merenụ. Ndị nnọchi anya ndị a nwere ike:

  1. ** Chịkọta ma nyochaa ndekọ ***: Ndị ọrụ DevOps nwere ike ịnakọta na nyochaa ndekọ sitere na isi mmalite dị iche iche, na-enye nghọta bara uru banyere ihe kpatara ya.
  2. **Nsochi njehie akpaaka ***: Ndị ọrụ nwere ike megharịa nsuso njehie, belata oge na mbọ achọrọ maka nyocha akwụkwọ ntuziaka.
  3. ** welie nzikọrịta ozi ***: Ndị ọrụ nwere ike ikwado nzikọrịta ozi n'etiti otu, ndị otu na sistemu, hụ na ekesara ozi merenụ nke ọma.

** Ọrụ LLM na nyocha ihe mberede**

Ụdị Asụsụ buru ibu (LLM) agbanweela ngalaba nhazi asụsụ eke (NLP), na-eme ka igwe nwee ike ịghọta na imepụta ederede mmadụ. N'ihe gbasara nyocha ihe merenụ, LLM nwere ike:

  1. ** Nyochaa akụkọ ihe merenụ ***: LLM nwere ike nyochaa akụkọ ihe merenụ, na-achọpụta ozi isi na usoro nwere ike igosi isi ihe kpatara ihe merenụ.
  2. ** Mepụta nchịkọta ihe omume **: LLM nwere ike iwepụta nchịkọta nkenke nke akụkọ merenụ, na-eme ka nkwurịta okwu na ime mkpebi.
  3. Nye aro-amata ihe ndị gbara ya gburugburu: LLM nwere ike ịnye ntụzịaka gbasara ọnọdụ maka mkpebi ihe merenụ, na-eburu n'uche usoro, gburugburu ebe obibi, na ndị aka ya metụtara.

** Na-eme nyocha ihe mberede na ndị ọrụ DevOps na LLMs ***

Iji mejuputa nyocha ihe omume akpaaka na ndị ọrụ DevOps na LLM, anyị ga-agbaso usoro nzọụkwụ site na nzọụkwụ:

  1. ** Tinye ndị ọrụ DevOps ***: Jikọta ndị ọrụ DevOps, dị ka AWS DevOps Agent na Copado Agentia, na sistemụ njikwa ihe omume gị.
  2. ** Chịkọta ma nyochaa ndekọ ***: Hazie ndị ọrụ DevOps iji nakọta na nyochaa ndekọ sitere na isi mmalite dị iche iche, na-enye nghọta bara uru banyere ihe kpatara ya.
  3. ** Zụlite ụdị LLM ***: Zụlite ụdị LLM na nchịkọta data nke akụkọ merenụ, ndekọ ndekọ na ozi ndị ọzọ dị mkpa.
  4. ** Jikọta LLM na ndị ọrụ DevOps ***: Jikọta ụdị LLM na ndị ọrụ DevOps, na-enyere LLM aka nyochaa akụkọ ihe merenụ ma nye aro maka mkpebi ihe merenụ.
  5. ** Tinye ihe ngwọta ***: Wepụ ihe nyocha ihe omume na-akpaghị aka, na-ahụ na ọ nwere ike ịgbatị, ntụkwasị obi na nchekwa.

** Koodu Ọmụmaatụ: ijikọ LLM na ndị ọrụ DevOps ***

Nke a bụ ihe atụ koodu snippet na Python, na-egosi otu esi ejikọta ụdị LLM na onye nnọchi anya DevOps: "Python mbubata osisi site na ngbanwe na-ebubata AutoModelForSequenceClassification, AutoTokenizer Onye nnọchi anya mbubata aws_devops_agent

ụdị = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased") tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

Bido onye nnọchi anya DevOps

onye nnọchi anya = onye nnọchi anya ()

Kọwaa ọrụ iji nyochaa akụkọ merenụ site na iji ụdị LLM

def analyze_incident_report(akụkọ): # Hazie akụkọ ihe merenụ ntinye = tokenizer.encode_plus(akụkọ, add_special_tokens=Eziokwu, max_length=512, return_attention_mask=Eziokwu, return_tensors='pt')

# Jiri ụdị LLM nyochaa akụkọ ihe merenụ
ihe mmepụta = ụdị (ntinye ['input_ids'], attention_mask = ntinye ['attention_mask'])

# Wepụ akara klaasị amụma
amụma_class = ọkụ.argmax(outputs.logits)

laghachi amụma_class

Kọwaa otu ọrụ iji nye ntụnye maka mkpebi ihe merenụ

def provide_aro(nke klaasị buru amụma): # Jiri akara klaasị buru amụma iji nye ntụnye maka mkpebi ihe merenụ ọ bụrụ amụma_klas == 0: laghachi "Lelee ndekọ sistemụ maka njehie" elif amụma_klas == 1: laghachi "Chọpụta njikọ netwọkụ" ọzọ: laghachi "Kpọtụrụ otu mmepe maka enyemaka"

Jikọta ụdị LLM na onye nnọchi anya DevOps

def Event_investigation(): # Anakọta ma nyochaa ndekọ sitere na onye nnọchi anya DevOps ndekọ = agent.get_logs()

# Nyochaa ndekọ site na iji ụdị LLM
predicted_class = analyze_incident_report(dekọọ)

# Nye ntụnye maka mkpebi ihe merenụ
aro = enye_atụ aro(predicted_class)

laghachi aro

Wepụ ihe ngwọta nyocha ihe merenụ

Event_investigation() `` ** Iwu 5-10 Kachasị Eji mee ihe

Nke a bụ iwu/snippets 5-10 kacha eji eme ihe maka teknụzụ a:

  1. `aws devops agent get-logs': Na-anakọta ndekọ n'aka onye nnọchi anya DevOps.
  2. transformers.AutoModelForSequenceClassification.from_pretrained(): Buo ụdị LLM.
  3. tokenizer.encode_plus(): Hazie mkpesa merenụ.
  4. model(ntinye['input_ids'], attention_mask= ntinye['ttention_mask']): Nyochaa akụkọ ihe merenụ site na iji ụdị LLM.
  5. torch.argmax(outputs.logits): Wepụ akara klaasị e buru n'amụma.
  6. Na-enye_atụ aro(klas buru amụma): Nye ntụnye maka mkpebi ihe merenụ.
  7. agent.get_logs(): Anakọta ndekọ n'aka ndị nnọchite DevOps.
  8. 'nyochaa_incident_akụkọ(akụkọ)': Nyochaa akụkọ merenụ site na iji ụdị LLM.
  9. 'nchọpụta ihe merenụ()': Wepụ ihe ngwọta nyocha ihe merenụ.
  10. `torch.save(model.state_dict(), 'model.pth')': Chekwa ihe nlereanya LLM.

** Nlegharị anya ***

Nke a bụ nleba anya nke ngwọta nyocha ihe merenụ akpaghị aka site na iji Chart.js: Json { "ụdị": "ogwe", "data": { "labels": ["Nnyocha akwụkwọ ntuziaka", "Nnyocha akpaaka"], "datasets": [ { "label": "Oge a ga-edozi", "data": [10, 2], "backgroundColor": ["rgba(255, 99, 132, 0.2)", "rgba (54, 162, 235, 0.2)"], "BorderColor": ["rgba (255, 99, 132, 1)", "rgba (54, 162, 235, 1)"], "BorderWiidth": 1 } ] }, "Nhọrọ": { " aha": { "ngosi": eziokwu, "text": "Oge a ga-edozi nsogbu" }, "ọnụego": { "yAxes": [ { "akara aka": { "beginAtZero": eziokwu } } ] } } } `` Ọhụụ a na-atụnyere oge iji dozie ihe merenụ site na iji ụzọ nyocha akwụkwọ na usoro nyocha akpaaka site na iji ndị ọrụ DevOps na LLM.

Mkpebi

Ime nyocha ihe omume akpaaka na ndị ọrụ DevOps na LLM nwere ike melite arụmọrụ na ịdị irè nke mkpebi ihe merenụ. Site na ijikọ ndị ọrụ DevOps na LLM, ndị otu nwere ike megharịa nyocha ndekọ ndekọ, nye ntụzịaka na-amata ọnọdụ maka mkpebi ihe merenụ, ma belata oge iji dozie ihe omume. Anyị enyochala njikọ nke DevOps automation, nyocha ihe mere na-akwado AI, yana ọrụ LLM n'ịkwalite ngwa DevOps dị ka AWS DevOps Agent na Copado Agentia. Site n'ịgbaso usoro nzọụkwụ site n'usoro nke akọwapụtara n'isiokwu a, òtù dị iche iche nwere ike ibuga usoro nyocha ihe mberede nwere ike ịkawanye, ntụkwasị obi na nchekwa akpaghị aka.

Have Questions?

Let's Talk

Contact me on LinkedIn