** Kirkirar Manyan Bututun LLM tare da DSPy da LangChain don Ingantattun DevOps ***

** Gabatarwa ***

Zuwan Manyan Samfuran Harshe (LLMs) ya kawo sauyi a fagen sarrafa Harshen Halitta (NLP) da kuma Ƙwararrun Ƙwararru (AI). Koyaya, yayin da sarƙaƙƙiya da sikelin tsarin tushen LLM ke ƙaruwa, haka buƙatar ƙaƙƙarfan ƙaƙƙarfan ƙaƙƙarfan ƙira da kayan aikin sarrafa kansa. A cikin wannan labarin, za mu shiga cikin duniyar DSPy da LangChain, ƙa'idodi biyu masu ƙarfi waɗanda ke ba da damar gina tsarin software mai ƙarfi da haɓakawa a cikin shekarun AI.

** Kalubale na Manyan Bututun LLM**

Manyan bututun LLM sun ƙunshi abubuwa da yawa, gami da sarrafa bayanai, horon ƙira, da ƙima. Waɗannan bututun na iya zama masu sarƙaƙƙiya, tare da dogaro da yawa da mu'amala tsakanin sassa daban-daban. Yayin da ma'aunin waɗannan bututun ya karu, haka kuma haɗarin kurakurai, jinkiri, da ƙulla albarkatu ke ƙaruwa.

Shigar da DSPy da LangChain

DSPy da LangChain ginshiƙai ne guda biyu waɗanda ke da nufin sauƙaƙe ƙungiyar ƙira da sarrafa manyan bututun LLM. DSPy wani tsari ne na Python wanda ke ba da hanya mai sauƙi da inganci don ginawa da sarrafa bututun bayanai masu rikitarwa, yayin da LangChain wani tsari ne wanda ke ba da damar gina tsarin AI mai sauƙi da juriya.

** DSPy: Tsarin tushen Python don bututun bayanai ***

DSPy tsari ne na tushen Python wanda ke ba da hanya mai sauƙi da inganci don ginawa da sarrafa hadaddun bututun bayanai. Tare da DSPy, masu haɓakawa na iya ƙirƙirar bututun bayanai waɗanda ke da ƙima, juriya, da sauƙin kulawa.

Ga misalin bututun bayanai mai sauƙi da aka gina tare da DSPy: `` Python shigo da dspy a matsayin dsp

data_source = dsp.Source( suna = "my_data_source", irin = "csv", hanya = "/hanya/to/data.csv" )

ayyana mai sarrafa bayanai

data_processor = dsp.Processor( suna = "my_data_processor", irin = "pandas", func=lambda x: x.dropna() )

Ƙayyade madaidaicin bayanai

data_sink = dsp.Sink( suna = "my_data_sink", irin = "csv", hanya = "/hanya/to/output.csv" )

Ƙirƙiri bututun bayanai

bututu = dsp. suna = "my_bututun", kafofin = [source_source], processor = [data_processor], sinks = [data_sink] )

Guda bututun

pipeline.run()

A cikin wannan misali, muna ayyana tushen bayanai, na'ura mai sarrafa bayanai, da ma'aunin bayanai, sannan mu ƙirƙiri bututun bayanai wanda ke haɗa waɗannan abubuwan. Ana gudanar da bututun ta hanyar amfani da hanyar 'run()'.

** LangChain: Tsari don Tsare-tsaren AI mai ƙarfi da juriya ***

LangChain wani tsari ne wanda ke ba da damar gina tsarin AI mai daidaitawa da juriya. Tare da LangChain, masu haɓakawa na iya ƙirƙirar tsarin AI waɗanda suke da sauƙin kiyayewa, daidaitawa, da juriya ga gazawa.

Ga misali na tsarin AI mai sauƙi wanda aka gina tare da LangChain:
`` Python
shigo da langchain kamar yadda lc

# Ƙayyade samfurin
model = lc.Model(
    suna = "my_model",
    nau'in = "mai canzawa",
    hanya = "/hanyar/zuwa/model"
)

# Ƙayyade faɗakarwa
m = lc. da sauri (
    suna = "my_prompt",
    text="Wannan samfurin faɗakarwa ne"
)

# Ƙirƙiri tsarin AI
ai_system = lc.AISystem(
    suna = "my_ai_system",
    model = model,
    gaggawa = gaggawa
)

# Gudanar da tsarin AI
ai_system.run()

A cikin wannan misali, mun ayyana samfuri, gaggawa, sannan mu ƙirƙiri tsarin AI wanda ke haɗa waɗannan abubuwan. Ana aiwatar da tsarin AI ta amfani da hanyar 'run()'.

** Kirkirar Manyan Bututun LLM tare da DSPy da LangChain ***

Don tsara manyan bututun LLM tare da DSPy da LangChain, za mu iya amfani da hanya mai zuwa:

Ƙayyade bututun bayanai ta amfani da DSPy wanda ke haɗa hanyoyin bayanai da yawa, masu sarrafawa, da kuma nutsewa.
Ƙayyade tsarin AI ta amfani da LangChain wanda ke haɗa nau'i-nau'i masu yawa, abubuwan da suka faru, da kuma fitarwa.
Yi amfani da DSPy don sarrafa bututun bayanai da LangChain don sarrafa tsarin AI.
Yi amfani da mai tsarawa kamar Apache Airflow ko Kubernetes don tsara tsarin aiwatar da bututun bayanai da tsarin AI.

Anan akwai misalin rubutun kaɗe-kaɗe mai sauƙi wanda ke amfani da DSPy da LangChain: `` Python shigo da dspy a matsayin dsp shigo da langchain kamar yadda lc shigo da apache_airflow kamar yadda af

Ƙayyade bututun bayanai

data_pipeline = dsp.Pipeline( name="my_data_pipeline", Source = [dsp.Source (suna = "my_data_source", nau'in = "csv", hanya = "/hanya/to/data.csv")], processors = [dsp.Processor(suna = "my_data_processor", nau'in = "pandas", func=lambda x: x.dropna())], sinks = [dsp.Sink (suna = "my_data_sink", nau'in = "csv", hanya = "/hanya/to/output.csv")] )

Ƙayyade tsarin AI

ai_system = lc.AISystem( suna = "my_ai_system", model = lc.Model (suna = "my_model", nau'in = "mai canzawa", hanya = "/ hanya / zuwa / samfurin"), gaggawa = lc.Prompt(suna = "my_prompt", rubutu = "Wannan samfurin faɗakarwa") )

Jadawalin aiwatar da bututun bayanai da tsarin AI

af.schedule_task( task_id = "aikina", dag_id = "my_dag", bututu = bututun bayanai, system=ai_tsarin )

A cikin wannan misali, mun ayyana bututun bayanai da tsarin AI ta amfani da DSPy da LangChain, sannan mu tsara aiwatar da waɗannan abubuwan da aka haɗa ta amfani da Apache Airflow.

** Manyan Dokoki 5-10 da Akafi Amfani da su/Snippets**

Anan akwai manyan 5-10 da aka fi amfani da umarni/snippets don DSPy da LangChain:

1. `dsp.Pipeline (suna = "my_pipeline", kafofin = [source], masu sarrafawa = [processor], sinks = [sink])"
2. `lc.Model (suna = "my_model", nau'in = "mai canzawa", hanya = "/ hanya / zuwa / samfurin")'
3. `lc.Prompt(suna = "my_prompt", rubutu = "Wannan samfurin faɗakarwa")'
4. `dsp.Source (suna = "my_data_source", nau'in = "csv", hanya = "/hanya/to/data.csv")`
5. `dsp.Processor(name = "my_data_processor", irin = "pandas", func=lambda x: x.dropna())`
6. `dsp.Sink (suna = "my_data_sink", nau'in = "csv", hanya = "/hanya/to/output.csv")'
7. `lc.AISystem(name = "my_ai_system", model=model, m=sauri)`
8. `af.schedule_task(aiki_id = "my_task", dag_id = "my_dag", bututun = bututu, tsarin = tsarin)`
9. `dsp.run_bututu(bututu)`
10. `lc.run_system(tsarin)`

**Kallon gani**

Anan ga hangen nesa na tsarin DSPy da LangChain ta amfani da Chart.js:
Json
{
  "type": "bar",
  "data": {
    "labels": ["DSPy", "LangChain"],
    "datasets": [
      {
        "label": "Features",
        "bayanai": [10, 15],
        "BackgroundLauni": [
          "rgba (255, 99, 132, 0.2)",
          "rgba (54, 162, 235, 0.2)"
        ],
        "Launi Border": [
          "rgba (255, 99, 132, 1)",
          "rgba (54, 162, 235, 1)"
        ],
        "BorderWidth": 1
      }
    ]
  },
  "zabuka": {}
}

Wannan hangen nesa yana nuna adadin fasali a cikin DSPy da LangChain.

Kammalawa

A ƙarshe, DSPy da LangChain su ne manyan tsare-tsare masu ƙarfi guda biyu waɗanda ke ba da damar gina tsarin haɓakawa da haɓaka tsarin software a cikin shekarun AI. Ta amfani da waɗannan ka'idoji, masu haɓakawa na iya ƙirƙirar manyan bututun LLM waɗanda ke da sauƙin kiyayewa, daidaitawa, da juriya ga gazawa. Tare da taimakon ƙungiyar makaɗa da kayan aiki na atomatik irin su Apache Airflow da Kubernetes, masu haɓakawa za su iya tsara aiwatar da aiwatar da waɗannan bututun da tsarin, yana sauƙaƙa sarrafa tsarin AI mai rikitarwa.

Aikin Gaba

Aikin gaba ya hada da:

Haɓaka ƙarin fasali don DSPy da LangChain.
Haɓaka aiki da haɓakawa na DSPy da LangChain.
Ƙirƙirar ƙarin kayan aikin ƙididdiga da kayan aiki na atomatik don DSPy da LangChain.
Ƙirƙirar ƙarin abubuwan gani da dashboards don DSPy da LangChain.

Nazari

Takardun DSPy: https://dspy.readthedocs.io/en/latest/
Takardun LangChain: https://langchain.readthedocs.io/en/latest/
Takardun ruwa na Apache: https://airflow.apache.org/docs/apache-airflow/stable/index.html
Takardun Kubernetes: https://kubernetes.io/docs/home/

Ƙirƙirar Manyan Bututun LLM tare da DSPy da LangChain don Ingantattun DevOps