_j5UddlmZddlmZmZddlmZddlmZddl m Z m Z ddl m Z ddlmZmZmZejeZidid id id id id idididididididididididiiidZeeeeeffed<edeeefdeeeffdZedeedeefdZdeeefeezdeeefeezfdZGd d!ZGd"d#eZGd$d%eZGd&d'eZed()d*eded+fd,Zd-ed.edefd/Z d(S)0) lru_cache)Anyoverload) constants)InferenceProviderMapping) MimeBytesRequestParameters)ChatCompletionInputMessage)build_hf_headers get_tokenloggingcerebrascohereclarifai deepinfrazfal-aiz fireworks-aigroqz hf-inference hyperbolicnebiusnscalenvidiaovhcloud replicate sambanovascalewaytogether) wavespeedzzai-org!HARDCODED_MODEL_INFERENCE_MAPPINGobjreturncdSNrs i/var/www/FlaskApp/flask-venv/lib/python3.11/site-packages/huggingface_hub/inference/_providers/_common.py filter_noner%/s8;cdSr!r"r#s r$r%r%1s.1cr&cNt|trQi}|D]8\}}|t|ttfrt |}|||<9|St|tr d|DSt dt |)Nchg|]/}t|ttfrt|n|0Sr") isinstancedictlistr%).0vs r$ zfilter_none..As5RRR*Qt "="=D A1RRRr&zExpected dict or list, got )r*r+itemsr,r% ValueErrortype)rcleanedkr.s r$r%r%5s#t"$IIKK  DAqy!dD\** #NNGAJJ#tSRRcRRRR >499>> ? ??r&ceZdZdZdedededdfdZddd ed eeefd ed edzd edzdeeefdzdefdZ d de ezdedzdefdZ d edzdefdZ d edzde fdZd eeefdeeefdzdedzdeeeffdZd ed edeeeffdZd ededefdZd edefdZded edefdZd ed ede dedzfdZd ed ede dedzdedzf dZdS)!TaskProviderHelperz.Base class for task-specific provider helpers.providerbase_urltaskrNc0||_||_||_dSr!)r7r9r8)selfr7r8r9s r$__init__zTaskProviderHelper.__init__Is     r&) extra_payloadinputs parametersheadersmodelapi_keyr=c||}||}|||}|||j}||||} | t | t|pi} |||||} | | td| | td| || | } t||j |j| | | S)z Prepare the request to be sent to the provider. Each step (api_key, model, headers, url, payload) can be customized in subclasses. )provider_mapping_infoNz8Both payload and data cannot be set in the same request.z2Either payload or data must be set in the request.)urlr9rAjsondatar@) _prepare_api_key_prepare_mapping_info_prepare_headers _prepare_url provider_id_prepare_payload_as_dictrecursive_merger%_prepare_payload_as_bytesr1_normalize_headersr r9) r;r>r?r@rArBr=rDrEpayloadrGnormalized_headerss r$prepare_requestz"TaskProviderHelper.prepare_requestNs7 ''00!% : :5 A A''99)>)JKK// Zo/pp  %g{=;NB/O/OPPG--fjBWYfgg  4#3WXX X ?t|QRR R"44WgtLL '3&     r&responserequest_paramsc|S)z Return the response in the expected format. Override this method in subclasses for customized response handling.r")r;rTrUs r$ get_responsezTaskProviderHelper.get_responses r&cZ|t}|td|jd|S)zZReturn the API key to use for the request. Usually not overwritten in subclasses.Nz)You must provide an api_key to work with z$ API or log in with `hf auth login`.)r r1r7r;rBs r$rHz#TaskProviderHelper._prepare_api_keys@ ?kkG ?oDMooo r&c |td|jdt|ji|rt|j|Sd}t |D]}|j|jkr|}n|td|d|jd|j|jkr+td|d|jd|jd|jd |jd kr&td|d |jd |jd kr&td |d|jd|S)zbReturn the mapped model ID to use for the request. Usually not overwritten in subclasses.Nz+Please provide an HF model ID supported by .zModel z is not supported by provider z is not supported for task z and provider z. Supported task: stagingz! is in staging mode for provider z. Meant for test purposes only.errorz,Our latest automated health check on model 'z' for provider 'z<' did not complete successfully. Inference call might fail.) r1r7rget!_fetch_inference_provider_mappingr9statusloggerwarning)r;rAprovider_mappingmappings r$rIz(TaskProviderHelper._prepare_mapping_infos =[4=[[[\\ \ - 0 0 C C G G N N K4T]CEJ J8??  G4=00#* 1  #[e[[4=[[[\\ \  DI - -<<<49<z9TaskProviderHelper._normalize_headers..s,hhhZS%V[Vgciikk5VgVgVgr&z content-typeNzapplication/json)r0r^ mime_type)r;r@rQrGrRs r$rPz%TaskProviderHelper._normalize_headerssiih7==??hhh  ! !. 1 1 9DN$>59^">22$5G">2!!r&c*it||S)zwReturn the headers to use for the request. Override this method in subclasses for customized headers. )token)r )r;r@rBs r$rJz#TaskProviderHelper._prepare_headerss >"111=W==r& mapped_modelc||}|||}|dd|dS)zVReturn the URL to use for the request. Usually not overwritten in subclasses./)_prepare_base_url_prepare_routerstriplstrip)r;rBrnr8routes r$rKzTaskProviderHelper._prepare_urlsX))'22##L':://#&&<<c):):<<r?rDs r$rMz+TaskProviderHelper._prepare_payload_as_dicts tr&cdS)zReturn the body to use for the request, as bytes. Override this method in subclasses for customized body data. Only one of `_prepare_payload_as_dict` and `_prepare_payload_as_bytes` should return a value. Nr")r;r>r?rDr=s r$rOz,TaskProviderHelper._prepare_payload_as_bytess tr&r!)__name__ __module__ __qualname____doc__strr<rr+r rSbytesrWrHrrIrrPrJrKrqrrrMrOr"r&r$r6r6Fs88!!!3!4!!!!043 3 3 3 cN 3  3 Tz 3 t3 CH~,3  3 3 3 3 p48  $, *D0      d s    # 3:# :R# # # # J "CH~ "04S#X0E "MVY]M] " c3h " " " ">>s>tCH~>>>>=C=s=s==== ! ! ! ! ! !3'+D\     8  d{  T       r&r6c peZdZdZdedeffd ZdededefdZd eee zd ed e ded zfd Z xZ S)BaseConversationalTaskz Base class for conversational (chat completion) tasks. The schema follows the OpenAI API format defined here: https://platform.openai.com/docs/api-reference/chat r7r8cPt||ddS)Nconversationalr7r8r9superr<r;r7r8 __class__s r$r<zBaseConversationalTask.__init__s) (XDTUUUUUr&rnrBrcdS)Nz/v1/chat/completionsr"rs r$rrz%BaseConversationalTask._prepare_routes%%r&r>r?rDNc:td|i|d|jiS)NmessagesrAr%rLrs r$rMz/BaseConversationalTask._prepare_payload_as_dicts* Ji*igG\Ghiijjjr&) rrrrrr<rrr,r+r rrM __classcell__rs@r$rr s VVVVVVVV&3&&&&&&kT667kk 8 k  kkkkkkkkr&rcHeZdZdZfdZdedefdZdedzdefdZxZ S) AutoRouterConversationalTaska Auto-router for conversational tasks. We let the Hugging Face router select the best provider for the model, based on availability and user preferences. This is a special case since the selection is done server-side (avoid 1 API call to fetch provider mapping). cNtdddS)Nautozhttps://router.huggingface.co)r7r8r)r;rs r$r<z%AutoRouterConversationalTask.__init__&s' &3RSSSSSr&rBrcX|dstd|jS)rwrxz>Cannot select auto-router when using non-Hugging Face API key.)ryr1r8rYs r$rqz.AutoRouterConversationalTask._prepare_base_url)s1 !!%(( !]^^ ^= r&rANcL|tdtd||ddS)z In auto-router, we don't need to fetch provider mapping info. We just return a dummy mapping info with provider_id set to the HF model ID. NzPlease provide an HF model ID.rliver)r7 hf_model_id providerIdr`r9)r1r)r;rAs r$rIz2AutoRouterConversationalTask._prepare_mapping_info3s> ==>> >'!     r&) rrrrr<rrqrrIrrs@r$rrsTTTTT!!!!!! 3: :R        r&rc ^eZdZdZdedeffd ZdededefdZd ed ed e ded zfd Z xZ S)BaseTextGenerationTaskz Base class for text-generation (completion) tasks. The schema follows the OpenAI API format defined here: https://platform.openai.com/docs/api-reference/completions r7r8cPt||ddS)Nztext-generationrrrs r$r<zBaseTextGenerationTask.__init__Js) (XDUVVVVVr&rnrBrcdS)Nz/v1/completionsr"rs r$rrz%BaseTextGenerationTask._prepare_routeMs  r&r>r?rDNc:td|i|d|jiS)NpromptrArrs r$rMz/BaseTextGenerationTask._prepare_payload_as_dictPs*Hfg gGEZEfgghhhr&) rrrrrr<rrrr+rrMrrs@r$rrDs WWWWWWWW!3!!!!!!ii'+iD\i iiiiiiiir&rN)maxsizerArcddlm}||dg}|j}|t d||S)z; Fetch provider mappings for a model from the Hub. r)HfApiinferenceProviderMapping)expandNz$No provider mapping found for model )huggingface_hub.hf_apir model_infoinference_provider_mappingr1)rArrzrcs r$r_r_Vsd -,,,,, 577  e-G,H  I ID6GGGHHH r&dict1dict2cLifd|DS)Nci|]R\}}||vrFt|tr+t|trt||n|SSr")r*r+rN)r-rhrirs r$rjz#recursive_merge..gsr   U u E#J!=!= *UTXBYBY !sU333   r&)r0)rrs` r$rNrNdsJ       $kkmm     r&)! functoolsrtypingrrhuggingface_hubrrr!huggingface_hub.inference._commonrr :huggingface_hub.inference._generated.types.chat_completionr huggingface_hub.utilsr r r get_loggerrrarr+r__annotations__r%r,r6rrrr_rNr"r&r$rs %%%%%%;;;;;;JJJJJJJJaaaaaaFFFFFFFFFF  H % % UU bUUU bUBU BUBU "!U" b#U$ b%U& b'U()U*+U,-U./U01U25UUU!4T#7O2O-P(P#Q< ;T#s(^;S#X;;; ; 1T#Y149111 1@T#s(^d3i/@DcNT#Y4N@@@@"@@@@@@@@Fkkkkk/kkk*# # # # # #9# # # Liiiii/iii$ 4 S T:T5U     4        r&