Abaabulka Tubbooyinka LLM-weyn ee leh DSPy iyo LangChain ee DevOps la horumariyay | B4N1

Abaabulka Tubbooyinka LLM-weyn ee leh DSPy iyo LangChain ee DevOps la horumariyay

3 months ago · b4n1 Read time: 5 min

** Abaabulka Tubbooyinka Miisaanka Weyn ee LLM oo wata DSPy iyo LangChain ee DevOps La Hagaajiyay ***

hordhac

Imaanshaha Moodooyinka Luuqadaha Waaweyn (LLMs) waxa ay wax ka beddeleen dhinaca Habaynta Luuqadda Dabiiciga ah (NLP) iyo Sirdoonka Artificial (AI). Si kastaba ha ahaatee, sida kakanaanta iyo miisaanka nidaamyada ku salaysan LLM, sidoo kale baahida loo qabo qalabaynta adag oo hufan iyo qalabaynta. Maqaalkan, waxaan u dhexgeli doonaa adduunka DSPy iyo LangChain, oo ah laba qaab oo awood leh oo awood u siinaya dhisidda nidaamyada software ee adkeysiga iyo miisaanka leh ee da'da AI.

Caqabadda Tubbooyinka Miisaanka Weyn ee LLM

Tuubooyinka LLM ee baaxadda weyn waxay ku lug leeyihiin qaybo badan, oo ay ku jiraan diyaarinta xogta, tababarka moodeelka, iyo ka-fiirsashada. Dhuumahani waxay noqon karaan kuwo adag, oo leh ku tiirsanaan badan iyo isdhexgalka qaybaha kala duwan. Marka miisaanka dhuumahani kordho, sidoo kale khatarta khaladaadka, dib u dhaca, iyo caqabadaha kheyraadka.

** Geli DSPy iyo LangChain ***

DSPy iyo LangChain waa laba qaab-dhismeed ujeedadoodu tahay in la fududeeyo habaynta iyo automation-ka ee dhuumaha waaweyn ee LLM. DSPy waa qaab-dhismeedka Python-ku-salaysan kaas oo bixiya hab fudud oo hufan oo lagu dhiso laguna maamulo dhuumaha xogta adag, halka LangChain uu yahay qaab-dhismeed awood u siinaya dhisidda nidaamyada AI ee la qiyaasi karo iyo kuwa adkeysi leh.

** DSPy: Qaab-dhismeedka Python-ku-salaysan ee Tubooyinka Xogta ***

DSPy waa qaab-dhismeedka Python-ku-salaysan kaas oo bixiya hab fudud oo hufan oo lagu dhiso laguna maamulo dhuumaha xogta adag. Iyadoo la adeegsanayo DSPy, horumariyayaashu waxay abuuri karaan dhuumaha xogta kuwaas oo la miisaami karo, adkaysi leh, oo si sahlan loo ilaalin karo.

Waa kan tusaale dhuumaha xogta fudud ee lagu dhisay DSPy: `` Python soo dejinta dspy sida dsp

xogta_source = dsp.Isha( name="my_data_source", nooca = "csv", wadada = "/dariiqa/to/data.csv" )

Qeex processor-ka xogta

data_processor = dsp.Processor( name="my_data_processor", nooca = "pandas", func=lambda x: x.dropna() )

Qeex weelka xogta

data_sink = dsp.Sink( name="my_data_sink", nooca = "csv", wadada = "/dariiqa/to/output.csv" )

Samee dhuumaha xogta

dhuumaha = dsp. name="my_pipeline", ilo=[xog_source], Processor=[data_processor], saxanka =[xogta_sink] )

Daadi dhuumaha

pipeline.run() `` Tusaalahan, waxaan ku qeexeynaa isha xogta, processor-ka xogta, iyo godka xogta, ka dibna waxaan abuurnaa dhuumaha xogta ee isku xira qaybahan. Dhuumaha waxaa lagu socodsiiyaa iyadoo la isticmaalayo habka 'run()'.

** LangChain: Qaab-dhismeedka Nidaamyada AI La-Mayn karo iyo Adkeysiga

LangChain waa qaab-dhismeed awood u siinaya dhisidda nidaamyada AI ee la miisaami karo iyo kuwa adkeysi leh. Iyadoo LangChain, horumariyayaashu waxay abuuri karaan nidaamyada AI kuwaas oo si sahlan loo ilaalin karo, la qiyaasi karo, oo u adkeysanaya guuldarada.

Waa kuwan tusaale nidaam fudud oo AI ah oo lagu dhisay LangChain: `` Python soo dejinta langchain sida lc

Qeex qaab

model = lc.Model ( name="my_model", nooca = "transformer", dariiqa = "/dariiqa/ to/model" )

Qeex degdeg

degdeg = lc. degdeg name="my_prompt", text="Tani waa muunad degdeg ah" )

Samee nidaam AI

ai_system = lc.AISystem name="my_ai_system", model=qaab, degdeg = degdeg ah )

Ku socodsii nidaamka AI

ai_system.run() `` Tusaalahan, waxaan ku qeexaynaa moodal, degdeg ah, ka dibna waxaan abuurnaa nidaam AI oo isku xira qaybahan. Nidaamka AI ayaa markaa la isticmaalayaa habka 'run()'.

** Abaabulka Tubbooyinka Miisaanka Weyn ee LLM oo wata DSPy iyo LangChain ***

Si loo abaabulo dhuumaha waaweyn ee LLM DSPy iyo LangChain, waxaan isticmaali karnaa habka soo socda:

  1. Qeex dhuunta xogta adoo isticmaalaya DSPy oo isku xidha ilo xogeedyo badan, processor-yaasha, iyo saxarada.
  2. Qeex nidaamka AI adoo isticmaalaya LangChain kaas oo isku xira noocyo badan, dardargelin, iyo wax soo saar.
  3. Isticmaal DSPy si aad u maamusho dhuumaha xogta iyo LangChain si aad u maamusho nidaamka AI.
  4. Isticmaal jadwal sida Apache Airflow ama Kubernetes si aad u qorsheysato fulinta dhuumaha xogta iyo nidaamka AI.

Waa kuwan tusaale qoraalka fudud ee orchesterka isticmaala DSPy iyo LangChain: `` Python soo dejinta dspy sida dsp soo dejinta langchain sida lc soo deji apache_hawo socodka sida af

Qeex dhuunta xogta

data_pipeline = dsp.pipeline ( name="my_data_pipeline", Source=[dsp.Source(name="my_data_source", nooca = "csv", path="/path/to/data.csv")], Processor=[dsp.Processor(name="my_data_processor", nooca = "pandas", func=lambda x: x.dropna())], sinks=[dsp.Sink (name = "my_data_sink", nooca = "csv", path="/path/to/output.csv")] )

Qeex nidaamka AI

ai_system = lc.AISystem name="my_ai_system", model = lc.Model (magaca = "my_model", nooca = "transformer", dariiqa = "/ dariiqa / to / model"), Quick=lc.Prompt(name="my_prompt", text="Tani waa muunad degdeg ah") )

Jadwal u samee fulinta dhuumaha xogta iyo nidaamka AI

af.jadwalka_hawl task_id = " my_task", dag_id="my_dag", dhuumaha=xarunta_tuubada, system=ai_system ) `` Tusaalahan, waxaan ku qeexeynaa dhuumaha xogta iyo nidaamka AI iyadoo la adeegsanayo DSPy iyo LangChain, ka dibna jadwal u samee fulinta qaybahan adoo isticmaalaya Apache Airflow.

** 5-10ka ugu sarreeya amarada/Qodobka ugu Isticmaalka badan**

Waa kuwan 5-10-ka amarrada ugu badan ee la isticmaalo ee DSPy iyo LangChain:

  1. dsp.Pipeline(name="my_pipeline",source=[source],processor=[processor], sinks=[kuso])
  2. `lc.Model
  3. `lc.Prompt(name="my_prompt", text="Tani waa muunad degdeg ah")
  4. `dsp.source
  5. dsp.Processor(name="my_data_processor", nooca = "pandas", func=lambda x: x.dropna())
  6. `dsp.Sink (name = "my_data_sink", nooca = "csv", dariiqa = "/dariiqa/to/output.csv")
  7. lc.AISystem(name="my_ai_system",model=model, degdeg= degdeg ah)
  8. `af.schedule_task
  9. dsp.run_pipeline(tuubada)
  10. `lc.run_system(nidaamka)'

Muuqaalka

Waa kan muuqaalka qaab-dhismeedka DSPy iyo LangChain iyadoo la adeegsanayo Chart.js: json { "type": "bar", "xogta": { "labels": ["DSPy", "LangChain"], "datasets": [ { "label": "Features", "xogta": [10, 15], "backgroundColor": [ "rgba(255, 99, 132, 0.2)", "rgba (54, 162, 235, 0.2)" ], "BorderColor": [ "rgba(255, 99, 132, 1)", "rgba (54, 162, 235, 1)" ], "BorderWidth": 1 } ] }, "ikhtiyaarada": {} } Aragtidaani waxay muujinaysaa tirada sifooyinka ku jira DSPy iyo LangChain.

** Gabagabo**

Gebogebadii, DSPy iyo LangChain waa laba qaab-dhismeed awood leh oo awood u leh dhisidda nidaamyada software-ka-adkeysiga iyo miisaanka leh ee da'da AI. Isticmaalka qaababkan, horumariyayaashu waxay abuuri karaan dhuumo waaweyn oo LLM ah kuwaas oo ay fududahay in la ilaaliyo, la miisaami karo, iyo adkaysiga guuldarrada. Iyada oo la kaashanayo qalabaynta iyo qalabaynta sida Apache Airflow iyo Kubernetes, horumariyayaashu waxay jadwal u samayn karaan fulinta dhuumaha iyo nidaamyadan, taas oo sahlaysa in la maareeyo nidaamyada AI ee adag.

Shaqada mustaqbalka

Shaqada mustaqbalka waxaa ka mid ah:

  1. Horumarinta sifooyin badan DSPy iyo LangChain.
  2. Horumarinta waxqabadka iyo miisaanka DSPy iyo LangChain.
  3. Horumarinta agabka orchesterka iyo qalabaynta ee DSPy iyo LangChain.
  4. Abuuritaanka muuqaallo badan iyo dashboards DSPy iyo LangChain.

Tixraac

  1. Dukumentiyada DSPy: https://dspy.readthedocs.io/en/latest/
  2. Warqadaha LangChain: https://langchain.readthedocs.io/en/latest/
  3. Dukumentiyada socodka hawada Apache: https://airflow.apache.org/docs/apache-airflow/stable/index.html
  4. Kubernetes dukumeenti: https://kubernetes.io/docs/home/

Have Questions?

Let's Talk

Contact me on LinkedIn