Layisha imodeli yeLLM kunye netokenizer | B4N1

Layisha imodeli yeLLM kunye netokenizer

3 months ago · b4n1 Read time: 4 min

Ukuphunyezwa koPhando lweSihlo oluzenzekelayo ngee-Arhente ze-DevOps kunye nee-LLMs

Intshayelelo

Kwimeko yanamhlanje ekhawulezayo yedijithali, amaqela eDevOps ajongene nomceli mngeni owandayo: ukuchonga nokusombulula izehlo ngokukhawuleza. Iindlela zophando ezenziwa ngesandla zinokutya ixesha kwaye zithambekele kwiimpazamo zomntu, ezikhokelela ekunciphiseni ixesha elide kunye nokuncipha kokwaneliseka kwabathengi. Ukujongana nalo mba, sijonga ukudibana kwe-DevOps automation, uphando lwezehlo olunamandla e-AI, kunye nendima yeeModeli zoLwimi olukhulu (LLMs) ekuphuculeni izixhobo ze-DevOps ezifana ne-AWS DevOps Agent kunye ne-Copado Agentia.

Imeko yangoku yoPhando lwezehlo

Uphando lwezehlo luyinkalo ebalulekileyo ye-DevOps, ebandakanya ukuchongwa, ukuhlalutya, kunye nokusombulula imiba echaphazela iinkqubo zesofthiwe. Iindlela zemveli zixhomekeke kwiinkqubo zezandla, ezifana nohlalutyo lwelogi, ukulandelela iimpazamo, kunye nokunxibelelana nabachaphazelekayo. Ngelixa ezi ndlela zisebenzayo, zinokucotha kwaye zithambekele kwiimpazamo, ezikhokelela ku:

  1. Ukonyuka kwexesha lokungasebenzi: Iindlela zophando ezenziwa ngezandla zinokulibazisa ukusonjululwa kwesehlo, okukhokelela ekwehleni kwexesha elide kunye nokuncipha kokwaneliseka kwabathengi.
  2. Impazamo yomntu: Iinkqubo ezenziwa ngezandla zisesichengeni sokwenza iimpazamo ezenziwa ngabantu, nto leyo enokukhokelela ekuxilongeni ngendlela engachanekanga nokusonjululwa kweziganeko.
  3. Imiba yokwaleka: Njengoko ubukhulu kunye nobunzima beenkqubo bukhula, iindlela zophando olwenziwa ngesandla ziya zisiba nzima ukuzilinganisela.

Indima yee-arhente ze-DevOps kuPhando lweSihlo

Iiarhente ze-DevOps, ezifana ne-AWS DevOps Agent kunye ne-Copado Agentia, zidlala indima ebalulekileyo kuphando lweziganeko ezizenzekelayo. Ezi arhente zinoku:

  1. ** Qokelela kwaye uhlalutye iilogi **: Ii-arhente ze-DevOps zinokuqokelela kwaye zihlalutye iilogi ezivela kwimithombo eyahlukeneyo, zibonelela ngengqiqo ebalulekileyo kwizizathu zeziganeko.
  2. **Ukulandelela impazamo ngokuzenzekelayo **: Ii-arhente zinokwenza ngokuzenzekelayo ukulandelela iimpazamo, ukunciphisa ixesha kunye nomzamo ofunekayo kuphando lwezandla.
  3. Ukuphucula unxibelelwano: Ii-arhente zinokuququzelela unxibelelwano phakathi kwamaqela, abachaphazelekayo, kunye neenkqubo, ukuqinisekisa ukuba ulwazi lwezehlo lwabelwana ngokufanelekileyo.

Indima yeeLLM kuPhando lweZiganeko

IiModeli zoLwimi olukhulu (LLMs) ziye zaguqula ibala lokusetyenzwa kolwimi lwendalo (NLP), okwenza ukuba oomatshini baqonde kwaye bavelise isicatshulwa esifana nomntu. Kumxholo wophando lweziganeko, iiLLMs zinokuthi:

  1. Hlalutya iingxelo zeziganeko: IiLLM zinokuhlalutya iingxelo zesiganeko, zichonge ulwazi oluphambili kunye neepatheni ezinokubonisa oyena nobangela wesiganeko.
  2. Ukuvelisa izishwankathelo zeziganeko: Ii-LLM zinokuvelisa izishwankathelo ezimfutshane zeengxelo zezehlo, ziququzelele unxibelelwano kunye nokwenziwa kwezigqibo.
  3. Ukubonelela ngeengcebiso eziqonda imeko-bume: Ii-LLMs zinokunika iingcebiso zokuqonda imeko-bume ukuze kusonjululwe isiganeko, kuthathelwa ingqalelo inkqubo ethile, okusingqongileyo kunye nabachaphazelekayo ababandakanyekayo.

Ukuphunyezwa koPhando lweSihlo oluzenzekelayo ngee-Arhente ze-DevOps kunye nee-LLMs

Ukuphumeza uphando lweziganeko ezizenzekelayo kunye nee-arhente ze-DevOps kunye ne-LLMs, siyakulandela indlela yenyathelo ngalinye:

  1. Hlanganisa ii-arhente ze-DevOps: Hlanganisa ii-arhente ze-DevOps, ezifana ne-AWS DevOps Agent kunye ne-Copado Agentia, kunye nenkqubo yakho yokulawula iziganeko.
  2. ** Qokelela kwaye uhlalutye iilogi **: Qwalasela ii-arhente ze-DevOps ukuba ziqokelele kwaye zihlalutye iilogi ezivela kwimithombo eyahlukeneyo, ukubonelela ngengqiqo ebalulekileyo kwizizathu zeziganeko.
  3. Qeqesha imodeli yeLLM: Qeqesha imodeli yeLLM kwidathasethi yeengxelo zeziganeko, iilogi, kunye nolunye ulwazi olufanelekileyo.
  4. Hlanganisa i-LLM kunye nee-arhente ze-DevOps: Dibanisa imodeli ye-LLM kunye nee-arhente ze-DevOps, okwenza i-LLM ihlalutye iingxelo zezehlo kwaye inike iingcebiso zokusombulula isiganeko.
  5. Fakela isisombululo: Sebenzisa isisombululo sophando lwesehlo esizenzekelayo, uqinisekise ukuba sinokala, sithembekile kwaye sikhuselekile.

Ikhowudi yoMzekelo: Ukudibanisa iLLM kunye neearhente zeDevOps

Nanku umzekelo wekhowudi ekhutshiweyo kwiPython, ebonisa indlela yokudibanisa imodeli yeLLM kunye nearhente yeDevOps:

ukugawulwa kwamanye amazwe
ukusuka kubaguquli bangenisa i-AutoModelForSequenceClassification, i-AutoTokenizer
ukusuka aws_devops_arhente yokungenisa


imodeli = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased")
uphawu = AutoTokenizer.from_pretrained("distilbert-base-uncased")

# Qalisa iarhente yeDevOps
iarhente = iarhente ()

# Chaza umsebenzi wokuhlalutya iingxelo zesiganeko usebenzisa imodeli yeLLM
def analyze_incident_report(ingxelo):
    # Qwalasela ingxelo yesiganeko
    amagalelo = tokenizer.encode_plus(ingxelo, 
                                    add_special_tokens=Yinyani, 
                                    ubude_ubude=512, 
                                    return_ attention_mask=Yinyani, 
                                    return_tensors='pt')
    
    # Hlalutya ingxelo yesiganeko usebenzisa imodeli yeLLM
    iziphumo = imodeli(amagalelo['input_ids'], attention_mask=inputs['attention_mask'])
    
    # Khupha ileyibhile yeklasi eqikelelweyo
    predicted_class = torch.argmax(outputs.logits)
    
    buyisela udidi_luxelwe kwangaphambili

# Chaza umsebenzi wokubonelela ngeengcebiso zokusombulula isiganeko
def provide_suggestions(predicted_class):
    # Sebenzisa ileyibhile yeklasi eqikelelweyo ukunika iingcebiso ngesisombululo sesehlo
    ukuba kuxelwe kwangaphambili_iklasi == 0:
        buyisela "Jonga ilog yenkqubo yeempazamo"
    elif predicted_class == 1:
        buyisela "Qinisekisa uqhagamshelwano lwenethiwekhi"
    enye into:
        buyisela "Qhagamshelana neqela lophuhliso ukuze ufumane uncedo"

# Hlanganisa imodeli yeLLM kunye nearhente yeDevOps
def incident_investigation():
    # Qokelela kwaye uhlalutye iilog kwiarhente yeDevOps
    logs = agent.get_logs()
    
    # Hlalutya iinkuni usebenzisa imodeli yeLLM
    iklasi_ eqikelelweyo = analyze_incident_report(iilogi)
    
    # Nika iingcebiso zokusombulula isiganeko
    iingcebiso = provide_suggestions(predicted_class)
    
    buyisela amacebiso

# Beka isisombululo sophando lwesiganeko
isiganeko_uphando()

Ephezulu 5-10 Eyeyona Isetyenziswayo Imiyalelo/iziqwengana

Nantsi eyona miyalelo/iziqwengana eziphezulu ezi-5-10 kule teknoloji:

  1. ii-aws devops ze-agent get-logs: Qokelela iilog kwi-arhente ye-DevOps.
  2. i-AutoModelForSequenceClassification.from_pretrained(): Layisha imodeli yeLLM.
  3. tokenizer.encode_plus(): Lungiselela kwangaphambili ingxelo yesiganeko.
  4. imodeli(amagalelo['iinput_ids'], attention_mask=inputs['ingqalelo_imaski']): Hlalutya ingxelo yesiganeko usebenzisa imodeli yeLLM.
  5. torch.argmax(outputs.logts): Khupha ileyibhile yodidi oluqikelelweyo.
  6. Nikela_iingcebiso(iklasi_eqikelelweyo): Nika iingcebiso ngesisombululo sesehlo.
  7. i-agent.get_logs(): Qokelela iilog kwi-arhente ye-DevOps.
  8. hlalutya_ingxelo_yesiganeko(ingxelo): Hlalutya ingxelo yesiganeko usebenzisa imodeli yeLLM.
  9. uphando_ lwesiganeko (): Sebenzisa isisombululo sophando lwesiganeko.
  10. itotshi.gcina(imodeli.state_dict(), 'imodeli.pth'): Gcina imodeli yeLLM.

Umbono

Nanku umfanekiso-ngqondweni wesisombululo sophando lwesiganeko esizenzekelayo usebenzisa iChart.js:

{
  "type": "bar",
  "idatha": {
    "labels": ["UPhando Lwezandla", "uPhando oluzenzekelayo"],
    "iiseti zedatha": [
      {
        "label": "Ixesha lokusombulula",
        "idatha": [10, 2],
        "backgroundColor": ["rgba(255, 99, 132, 0.2)", "rgba(54, 162, 235, 0.2)"],
        "borderColor": ["rgba(255, 99, 132, 1)", "rgba(54, 162, 235, 1)"],
        "BorderWidth": 1
      }
    ]
  },
  "iinketho": {
    "isihloko": {
      "bonisa": yinyani,
      "text": "Ixesha lokuSombulula Iziganeko"
    },
    "izikali": {
      "yAxes": [
        {
          "amanqaku": {
            "beginAtZero": yinyani
          }
        }
      ]
    }
  }
}

Oku kubonwa kuthelekisa ixesha lokusombulula iziganeko kusetyenziswa iindlela zophando lwezandla ngokuchasene neendlela zophando ezizenzekelayo usebenzisa i-DevOps agents kunye neLLM.

Isiphelo

Ukuphumeza uphando lweziganeko ezizenzekelayo kunye nee-arhente ze-DevOps kunye ne-LLMs kunokuphucula kakhulu ukusebenza kunye nokusebenza kwesisombululo sesiganeko. Ngokudibanisa i-DevOps agents kunye ne-LLMs, imibutho inokwenza ngokuzenzekelayo ukuhlalutya kwelogi, inike iingcebiso zomxholo wokusombulula isiganeko, kunye nokunciphisa ixesha lokusombulula iziganeko. Siphonononge ukuhlangana kwe-DevOps automation, uphando lwezehlo oluqhutywa yi-AI, kunye nendima ye-LLMs ekuphuculeni izixhobo ze-DevOps ezifana ne-AWS DevOps Agent kunye ne-Copado Agentia. Ngokulandela indlela yesinyathelo-nge-nyathelo echazwe kweli nqaku, imibutho inokusebenzisa isisombululo sophando esizenzekelayo, esithembekileyo, nesikhuselekileyo.

Have Questions?

Let's Talk

Contact me on LinkedIn