Layisha imodeli ye-LLM ne-tokenizer | B4N1

Layisha imodeli ye-LLM ne-tokenizer

3 months ago · b4n1 Read time: 4 min

Ukuqalisa Uphenyo Lwesigameko Okuzenzakalelayo ngama-DevOps Agents nama-LLM

Isingeniso

Kusimo sedijithali esisheshayo sanamuhla, amaqembu e-DevOps abhekana nenselelo eyandayo eyinkimbinkimbi: ukuhlonza nokuxazulula izehlakalo ngokushesha okukhulu. Izindlela zophenyo ezenziwa mathupha zingadla isikhathi futhi zithambekele emaphutheni omuntu, okuholela ekunciphiseni isikhathi eside nokuncipha kokwaneliseka kwamakhasimende. Ukuze sibhekane nale nkinga, sihlola impambanondlela ye-DevOps automation, uphenyo lwesigameko olunamandla e-AI, kanye nendima yamamodeli olimi olukhulu (LLMs) ekuthuthukiseni amathuluzi e-DevOps afana ne-AWS DevOps Agent ne-Copado Agentia.

Isimo Samanje Sophenyo Lwesigameko

Uphenyo lwesigameko luyingxenye ebalulekile ye-DevOps, ehlanganisa ukuhlonza, ukuhlaziya, kanye nokuxazululwa kwezinkinga ezithinta izinhlelo zesofthiwe. Izindlela zendabuko zincike ezinqubweni ezenziwa mathupha, njengokuhlaziywa kwelogi, ukulandelela amaphutha, nokuxhumana nababambe iqhaza. Nakuba lezi zindlela zisebenza kahle, zingahamba kancane futhi zithambekele emaphutheni, okuholela ku:

  1. Isikhathi sokuphumula esengeziwe: Izindlela zophenyo ezenziwa mathupha zingabambezela ukuxazululwa kwesigameko, okuholela ekunciphiseni isikhathi eside nokuncipha kokwaneliseka kwamakhasimende.
  2. Iphutha lomuntu: Izinqubo ezenziwa mathupha zingase zibe nephutha lomuntu, okungaholela ekuxilongweni okungalungile kanye nokuxazululwa kwezigameko.
  3. Izinkinga zokukala: Njengoba ubukhulu nobunkimbinkimbi bezinhlelo bukhula, izindlela zophenyo ezenziwa mathupha ziba nzima kakhulu ukukala.

Iqhaza lama-ejenti e-DevOps ophenyweni lwesigameko

Ama-ejenti we-DevOps, afana ne-AWS DevOps Agent ne-Copado Agentia, adlala indima ebalulekile ekuphenyeni izigameko ngokuzenzakalelayo. Lawa ma-ejenti angakwazi:

  1. Qoqa futhi uhlaziye amalogi: Abenzeli be-DevOps bangaqoqa futhi bahlaziye amalogi emithonjeni ehlukahlukene, banikeze imininingwane ebalulekile ngezimbangela zesigameko.
  2. Ukulandelela amaphutha okuzenzakalelayo: Ama-ejenti angakwazi ukulandelela amaphutha ngokuzenzakalelayo, anciphise isikhathi nomzamo odingekayo ukuze kuphenywe ngesandla.
  3. Thuthukisa ukuxhumana: Ama-ejenti angakwazi ukuxhumana phakathi kwamathimba, ababambiqhaza, nezinhlelo, aqinisekise ukuthi ulwazi lwesigameko lwabiwa ngendlela efanele.

Iqhaza lama-LLM Ophenyweni Lwesigameko

Amamodeli Olimi Amakhulu (LLMs) aguqule umkhakha wokucubungula ulimi lwemvelo (NLP), okwenza imishini iqonde futhi ikhiqize umbhalo ofana nomuntu. Ngokwengqikithi yophenyo lwesigameko, ama-LLM angakwazi:

  1. Hlaziya imibiko yesigameko: Ama-LLM angahlaziya imibiko yesigameko, ahlonze ulwazi olubalulekile namaphethini angase abonise umsuka wesigameko.
  2. Khiqiza izifinyezo zezigameko: Ama-LLM angakwazi ukwenza izifinyezo ezifinyeziwe zemibiko yesigameko, enze kube lula ukuxhumana kanye nokuthathwa kwezinqumo.
  3. Nikeza iziphakamiso eziqaphela umongo: Ama-LLM anganikeza iziphakamiso eziqaphela umongo ukuze kuxazululwe isigameko, kucatshangelwa uhlelo oluthile, indawo, nababambiqhaza abahililekile.

Ukuqalisa Uphenyo Lwesigameko Okuzenzakalelayo ngama-DevOps Agents nama-LLM

Ukuze usebenzise uphenyo lwezigameko ezizenzakalelayo ngama-ejenti we-DevOps nama-LLM, sizolandela indlela yesinyathelo nesinyathelo:

  1. Hlanganisa ama-ejenti e-DevOps: Hlanganisa ama-ejenti e-DevOps, afana ne-AWS DevOps Agent ne-Copado Agentia, nohlelo lwakho lokuphatha izigameko.
  2. Qoqa futhi uhlaziye amalogi: Lungiselela abenzeli be-DevOps ukuze baqoqe futhi bahlaziye amalogi emithonjeni ehlukahlukene, banikeze imininingwane ebalulekile ngezimbangela zesigameko.
  3. Qeqesha imodeli ye-LLM: Qeqesha imodeli ye-LLM kudathasethi yemibiko yesigameko, amalogi, nolunye ulwazi olufanele.
  4. Hlanganisa i-LLM nabasebenzeli be-DevOps: Hlanganisa imodeli ye-LLM nabasebenzeli be-DevOps, uvumele i-LLM ukuthi ihlaziye imibiko yesigameko futhi inikeze iziphakamiso zokuxazulula isigameko.
  5. Khipha isisombululo: Sebenzisa isixazululo esizenzakalelayo sophenyo lwesigameko, uqinisekise ukuthi siyakaleka, sithembekile, futhi sivikelekile.

Ikhodi Yesibonelo: Ukuhlanganisa i-LLM nama-DevOps Agents

Nasi isibonelo samazwibela ekhodi ku-Python, esibonisa indlela yokuhlanganisa imodeli ye-LLM ne-ejenti ye-DevOps:

ngenisa ukugawulwa kwemithi
kusuka kubaguquli bangenisa i-AutoModelForSequenceClassification, i-AutoTokenizer
kusuka ku-aws_devops_ejenti yokungenisa


imodeli = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased")
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

# Qalisa i-ejenti ye-DevOps
i-ejenti = i-ejenti ()

# Chaza umsebenzi wokuhlaziya imibiko yesigameko usebenzisa imodeli ye-LLM
def analyze_incident_report(umbiko):
    # Yenza kusengaphambili umbiko wesigameko
    okokufaka = tokenizer.encode_plus(umbiko, 
                                    add_special_tokens=Iqiniso, 
                                    ubuningi_ubude=512, 
                                    return_attention_mask=Iqiniso, 
                                    return_tensors='pt')
    
    # Hlaziya umbiko wesigameko usebenzisa imodeli ye-LLM
    okuphumayo = imodeli(okufakiwe['input_ids'], attention_mask=inputs['attention_mask'])
    
    # Khipha ilebula yekilasi elibikezelwe
    predicted_class = torch.argmax(outputs.logits)
    
    buyisela iklasi_elibikezelwe

# Chaza umsebenzi ukuze unikeze iziphakamiso zokuxazulula isigameko
def provide_suggestions(predicted_class):
    # Sebenzisa ilebula yekilasi ebikezelwe ukunikeza iziphakamiso zokuxazulula isigameko
    if predicted_class == 0:
        buyisela "Hlola amaphutha esistimu ukuze uthole amaphutha"
    elif predicted_class == 1:
        buyisela "Qinisekisa ukuxhumeka kwenethiwekhi"
    okunye:
        buyisela "Thinta ithimba lokuthuthukisa ukuze uthole usizo"

# Hlanganisa imodeli ye-LLM ne-ejenti ye-DevOps
def incident_investigation():
    # Qoqa futhi uhlaziye izingodo ezivela kumenzeli we-DevOps
    izingodo = agent.get_logs()
    
    # Hlaziya izingodo usebenzisa imodeli ye-LLM
    predicted_class = analyze_incident_report(amalogi)
    
    # Nikeza iziphakamiso zokuxazulula isigameko
    iziphakamiso = provide_suggestions(predicted_class)
    
    buyisela ukusikisela

# Faka isixazululo sophenyo lwesigameko
uphenyo_lwesigameko()

Imiyalo/Amazwibela Aphezulu Okuyi-5-10

Nayi imiyalo/ amazwibela aphezulu angu-5-10 alobu buchwepheshe:

  1. ama-aws devops agent get-logs: Qoqa amalogi kumenzeli we-DevOps.
  2. ama-transformers.AutoModelForSequenceClassification.from_pretrained(): Layisha imodeli ye-LLM.
  3. tokenizer.encode_plus(): Cubungula kusengaphambili umbiko wesigameko.
  4. imodeli(input['input_ids'], attention_mask=inputs['attention_mask']): Hlaziya umbiko wesigameko usebenzisa imodeli ye-LLM.
  5. torch.argmax(outputs.logits): Khipha ilebula yesigaba esibikezelwe.
  6. nikeza_iziphakamiso(isigaba_esibikezelwe): Nikeza iziphakamiso zokuxazulula isigameko.
  7. i-agent.get_logs(): Qoqa amalogi kumenzeli we-DevOps.
  8. hlaziya_isigameko_umbiko(bika): Hlaziya umbiko wesigameko usebenzisa imodeli ye-LLM.
  9. Uphenyo_lwesigameko(): Sebenzisa isisombululo sophenyo lwesigameko.
  10. ithoshi.londoloza(imodeli.state_dict(), 'imodeli.pth'): Londoloza imodeli ye-LLM.

Ukubona

Nakhu okubonwayo kwesixazululo sophenyo lwesigameko esizenzakalelayo kusetshenziswa i-Chart.js:

{
  "type": "bar",
  "idatha": {
    "labels": ["Uphenyo Lwezandla", "Uphenyo Oluzenzakalelayo"],
    "idathasethi": [
      {
        "label": "Isikhathi Sokuxazulula",
        "idatha": [10, 2],
        "backgroundColor": ["rgba(255, 99, 132, 0.2)", "rgba(54, 162, 235, 0.2)"],
        "borderColor": ["rgba(255, 99, 132, 1)", "rgba(54, 162, 235, 1)"],
        "BorderWidth": 1
      }
    ]
  },
  "izinketho": {
    "isihloko": {
      "Display": iqiniso,
      "text": "Isikhathi Sokuxazulula Izigameko"
    },
    "izikali": {
      "yAxes": [
        {
          "imikhaza": {
            "beginAtZero": kuyiqiniso
          }
        }
      ]
    }
  }
}

Lokhu kuboniswa kuqhathanisa isikhathi sokuxazulula izigameko kusetshenziswa izindlela zophenyo ezenziwa ngesandla ngokumelene nezindlela zophenyo ezizenzakalelayo kusetshenziswa ama-DevOps agents kanye nama-LLM.

Isiphetho

Ukusebenzisa uphenyo lwezigameko ezizenzakalelayo ngama-ejenti we-DevOps nama-LLM kungathuthukisa kakhulu ukusebenza kahle nokusebenza kokuxazulula isigameko. Ngokuhlanganisa ama-ejenti e-DevOps nama-LLM, izinhlangano zingakwazi ukuhlaziya amalogi ngokuzenzakalelayo, zinikeze iziphakamiso eziqaphela umongo ukuze kuxazululwe isigameko, futhi zinciphise isikhathi sokuxazulula izigameko. Sihlole impambanondlela ye-DevOps automation, uphenyo lwesigameko olunamandla e-AI, kanye nendima yama-LLM ekuthuthukiseni amathuluzi e-DevOps afana ne-AWS DevOps Agent ne-Copado Agentia. Ngokulandela indlela yesinyathelo ngesinyathelo echazwe kulesi sihloko, izinhlangano zingasebenzisa isisombululo sophenyo sesigameko esinokwethenjelwa, esithembekile, nesivikelekile.

Have Questions?

Let's Talk

Contact me on LinkedIn