Load da samfurin LLM da tokenizer | B4N1

Load da samfurin LLM da tokenizer

6 months ago · b4n1 Read time: 6 min

** Aiwatar da Binciken Haɓaka Kai tsaye tare da Wakilan DevOps da LLMs**

** Gabatarwa ***

A cikin yanayin yanayin dijital mai sauri na yau, ƙungiyoyin DevOps suna fuskantar ƙalubale mai rikitarwa: ganowa da warware abubuwan da suka faru cikin sauri. Hanyoyin bincike na hannu na iya ɗaukar lokaci kuma suna fuskantar kuskuren ɗan adam, wanda ke haifar da tsawan lokaci mai tsawo da rage gamsuwar abokin ciniki. Don magance wannan batu, muna bincika mahadar DevOps automation, binciken da ya faru mai ƙarfi AI, da kuma rawar Manyan Harshe Model (LLMs) wajen haɓaka kayan aikin DevOps kamar AWS DevOps Agent da Copado Agentia.

Halin Da Yake Faruwa A Yanzu

Binciken aukuwa wani muhimmin al'amari ne na DevOps, wanda ya haɗa da ganowa, bincike, da warware batutuwan da suka shafi tsarin software. Hanyoyin al'ada sun dogara da tsarin aiki na hannu, kamar nazarin loggu, bin diddigin kuskure, da sadarwa tare da masu ruwa da tsaki. Duk da yake waɗannan hanyoyin suna da tasiri, za su iya zama a hankali kuma suna fuskantar kurakurai, suna haifar da:

  1. ** Ƙara yawan lokaci ***: Hanyoyin bincike na hannu na iya jinkirta ƙaddamar da matsala, haifar da tsawaita lokaci mai tsawo da rage gamsuwar abokin ciniki.
  2. Kuskuren ɗan adam: Hanyoyin hannu suna da sauƙi ga kuskuren ɗan adam, wanda zai iya haifar da rashin kuskuren ganewar asali da warware abubuwan da suka faru.
  3. Al'amurra masu daidaitawa: Yayin da girma da sarƙaƙƙiyar tsarin ke girma, hanyoyin bincike na hannu suna ƙara zama da wahala a ƙima.

** Matsayin DevOps Agents a cikin Bincike na Farko ***

Wakilan DevOps, kamar AWS DevOps Agent da Copado Agentia, suna taka muhimmiyar rawa wajen sarrafa binciken abin da ya faru. Waɗannan wakilai na iya:

  1. ** Tattara da bincika rajistan ayyukan ***: Ma'aikatan DevOps na iya tattarawa da bincikar rajistan ayyukan daga tushe daban-daban, suna ba da haske mai mahimmanci game da abubuwan da suka faru.
  2. ** Bibiyar kuskure ta atomatik ***: Ma'aikata na iya sarrafa sarrafa kuskuren, rage lokaci da ƙoƙarin da ake buƙata don binciken hannu.
  3. **Haɓaka sadarwa ***: Wakilai na iya sauƙaƙe sadarwa tsakanin ƙungiyoyi, masu ruwa da tsaki, da tsarin, tabbatar da cewa an raba bayanan abin da ya faru yadda ya kamata.

Gudunwar LLMs a cikin Binciken Hatsari

Manyan Samfuran Harshe (LLMs) sun kawo sauyi a fagen sarrafa harshe na halitta (NLP), suna ba injina damar fahimta da samar da rubutu irin na mutum. A cikin mahallin binciken abin da ya faru, LLMs na iya:

  1. **Bincika rahotannin abubuwan da suka faru ***: LLMs na iya nazarin rahotannin abubuwan da suka faru, gano mahimman bayanai da alamu waɗanda zasu iya nuna tushen abin da ya faru.
  2. ** Ƙirƙirar taƙaitaccen abin da ya faru ***: LLMs na iya samar da taƙaitaccen taƙaitaccen rahotannin abubuwan da suka faru, sauƙaƙe sadarwa da yanke shawara.
  3. **Bayar da shawarwarin sanin mahallin ***: LLMs na iya ba da shawarwari na sanin mahallin don warware matsalar, la'akari da takamaiman tsari, yanayi, da masu ruwa da tsaki.

** Aiwatar da Binciken Haɓaka Kai tsaye tare da Wakilan DevOps da LLMs**

Don aiwatar da binciken abin da ya faru ta atomatik tare da wakilan DevOps da LLMs, za mu bi hanyar mataki-mataki:

  1. ** Haɗa wakilan DevOps ***: Haɗa wakilan DevOps, kamar AWS DevOps Agent da Copado Agentia, tare da tsarin sarrafa abubuwan da suka faru.
  2. ** Tattara da bincika rajistan ayyukan ***: Sanya wakilai na DevOps don tattarawa da bincikar rajistan ayyukan daga tushe daban-daban, samar da mahimman bayanai game da abubuwan da suka faru.
  3. Koran samfurin LLM: Horar da samfurin LLM akan ma'auni na rahotannin abubuwan da suka faru, gundumomi, da sauran bayanan da suka dace.
  4. ** Haɗa LLM tare da wakilan DevOps ***: Haɗa samfurin LLM tare da wakilan DevOps, ba da damar LLM don nazarin rahotannin abubuwan da suka faru da kuma ba da shawarwari don warware matsalar.
  5. ** Sanya mafita ***: Sanya maganin binciken abin da ya faru ta atomatik, tabbatar da cewa yana da ƙima, abin dogaro, kuma amintacce.

** Lambar Misali: Haɗa LLM tare da DevOps Agents**

Ga misalin snippet code a Python, yana nuna yadda ake haɗa samfurin LLM tare da wakilin DevOps: `` Python shigo da shiga daga masu canji suna shigo da AutoModelForSequenceClassification, AutoTokenizer daga aws_devops_agent shigo da Agent

samfurin = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased") tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

Fara wakilin DevOps

wakili = wakili ()

Ƙayyade aiki don nazarin rahotannin abubuwan da suka faru ta amfani da samfurin LLM

def analyze_incident_report(rahoto): # Gabatar da rahoton abin da ya faru shigarwar = tokenizer.encode_plus (rahoto, add_special_tokens=Gaskiya, max_length=512, return_attention_mask=Gaskiya, return_tensors='pt')

# Yi nazarin rahoton abin da ya faru ta amfani da samfurin LLM
abubuwan da aka fitar = samfurin (shigar da ['input_ids'], attention_mask = abubuwan shigarwa ['attention_mask'])

# Cire lakabin aji da aka annabta
predicted_class = torch.argmax(outputs.logits)

dawo da annabta_class

Ƙayyade aiki don ba da shawarwari don warware abin da ya faru

def bayar da_shawarwari (annabta_class): # Yi amfani da alamar aji da aka annabta don ba da shawarwari don warware abin da ya faru idan an annabta_aji == 0: dawo "Duba rajistan ayyukan don kurakurai" elif annabta_class == 1: dawo "Tabbatar da haɗin yanar gizon" wani: dawo "A tuntuɓi ƙungiyar haɓaka don taimako"

Haɗa samfurin LLM tare da wakilin DevOps

def Event_investigation(): # Tattara da bincika rajistan ayyukan daga wakilin DevOps logs = agent.get_logs()

# Yi nazarin rajistan ayyukan ta amfani da ƙirar LLM
annabta_class = nazari_incident_report(logs)

# Ba da shawarwari don warware abin da ya faru
shawarwari = bada_shawarwari(predicted_class)

mayar da shawarwari

Sanya maganin binciken abin da ya faru

lamarin_bincike()

** Manyan Dokoki 5-10 da Akafi Amfani da su/Snippets**

Anan akwai manyan umarni 5-10 da aka fi amfani da su don wannan fasaha:

1. `aws devops agent get-logs': Tattara rajistan ayyukan daga wakilin DevOps.
2. `masu canzawa.AutoModelForSequenceClassification.daga_pretrained()`: Load da samfurin LLM.
3. `tokenizer.encode_plus()`: Gabatar da rahoton abin da ya faru.
4. `samfurin(shigowa['input_ids'], attention_mask=sakawar['ttention_mask'])`: Yi nazarin rahoton abin da ya faru ta amfani da samfurin LLM.
5. `torch.argmax(outputs.logits)`: Cire lakabin aji da aka annabta.
6. `Bayar da_shawarwari(annabta_aji)`: Ba da shawarwari don warware abin da ya faru.
7. `agent.get_logs()`: Tattara rajistan ayyukan daga wakilin DevOps.
8. `Analyze_incident_report(rahoton)`: Yi nazarin rahoton abin da ya faru ta amfani da samfurin LLM.
9. `binciken_hatsari()`: Sanya maganin binciken abin da ya faru.
10. `torch.save(model.state_dict(),'model.pth'): Ajiye samfurin LLM.

**Kallon gani**

Anan ga hangen nesa na maganin binciken abin da ya faru ta atomatik ta amfani da Chart.js:
Json
{
  "type": "bar",
  "data": {
    "labels": ["Binciken Manual", "Bincike Mai sarrafa kansa"],
    "datasets": [
      {
        "label": "Lokacin da za a warware",
        "bayanai": [10, 2],
        "backgroundColor": ["rgba (255, 99, 132, 0.2)", "rgba (54, 162, 235, 0.2)"],
        "BorderColor": ["rgba (255, 99, 132, 1)", "rgba (54, 162, 235, 1)"],
        "BorderWidth": 1
      }
    ]
  },
  "zaɓi": {
    "take": {
      "nuna": gaskiya,
      "text": "Lokacin da za a warware abubuwan da suka faru"
    },
    "ma'auni": {
      "yAxes": [
        {
          "zama": {
            "beginAtZero": gaskiya
          }
        }
      ]
    }
  }
}

Wannan hangen nesa yana kwatanta lokacin da za a warware abubuwan da suka faru ta amfani da hanyoyin bincike na hannu tare da hanyoyin bincike na atomatik ta amfani da wakilai na DevOps da LLMs.

Kammalawa

Aiwatar da binciken abin da ya faru ta atomatik tare da wakilan DevOps da LLMs na iya haɓaka inganci da ingancin ƙudurin abin da ya faru. Ta hanyar haɗa wakilan DevOps tare da LLMs, ƙungiyoyi za su iya sarrafa sarrafa bayanai na rajistan ayyukan, samar da shawarwari-sane da mahallin don warware abin da ya faru, da rage lokacin da za a warware abubuwan da suka faru. Mun bincika mahadar DevOps aiki da kai, binciken da ya faru da AI mai ƙarfi, da rawar LLMs wajen haɓaka kayan aikin DevOps kamar AWS DevOps Agent da Copado Agentia. Ta bin matakan mataki-mataki da aka zayyana a cikin wannan labarin, ƙungiyoyi za su iya tura ingantaccen, abin dogaro, da amintaccen maganin binciken abin da ya faru.


Have Questions?

Let's Talk

Contact me on LinkedIn