_j!ddlmZmZmZmZddlmZmZmZm Z ddl Z ddl Z ddl mZddlZGddeeZGddeeZdS) ) Embeddings DocumentsEmbeddingFunctionSpace)ListDictAnyOptionalNvalidate_config_schemac:eZdZdZ ddeededefdZd ed efd Z e d efd Z d e fd Z d ee fdZe deeefd dfdZd eeeffdZdeeefdeeefd dfdZe deeefd dfdZdS)HuggingFaceEmbeddingFunctionz This class is used to get embeddings for a list of texts using the HuggingFace API. It requires an API key and a model name. The default model name is "sentence-transformers/all-MiniLM-L6-v2". N&sentence-transformers/all-MiniLM-L6-v2CHROMA_HUGGINGFACE_API_KEYapi_key model_nameapi_key_env_varc ddl}n#t$rtdwxYw|tjdt t jdd|_n||_|pt j|j|_ |j std|jd||_ d||_ | |_ |j jd d |j idS) a Initialize the HuggingFaceEmbeddingFunction. Args: api_key_env_var (str, optional): Environment variable name that contains your API key for the HuggingFace API. Defaults to "CHROMA_HUGGINGFACE_API_KEY". model_name (str, optional): The name of the model to use for text embeddings. Defaults to "sentence-transformers/all-MiniLM-L6-v2". rNUThe httpx python package is not installed. Please install it with `pip install httpx`Direct api_key configuration will not be persisted. Please use environment variables via api_key_env_var for persistent storage.HUGGINGFACE_API_KEYzThe z! environment variable is not set.zAhttps://api-inference.huggingface.co/pipeline/feature-extraction/ AuthorizationBearer )httpx ImportError ValueErrorwarningswarnDeprecationWarningosgetenvrrr_api_urlClient_sessionheadersupdate)selfrrrrs ~/var/www/FlaskApp/flask-venv/lib/python3.11/site-packages/chromadb/utils/embedding_functions/huggingface_embedding_function.py__init__z%HuggingFaceEmbeddingFunction.__init__s*  LLLL   g     M_"    9* + + 7#8D #2D A")D,@"A"A | Nt+NNN %h\fhh    $$o7O7O7O%PQQQQQ!inputreturnc|j|j|ddid}d|DS)a Get the embeddings for a list of texts. Args: input (Documents): A list of texts to get embeddings for. Returns: Embeddings: The embeddings for the texts. Example: >>> hugging_face = HuggingFaceEmbeddingFunction(api_key_env_var="CHROMA_HUGGINGFACE_API_KEY") >>> texts = ["Hello, world!", "How are you?"] >>> embeddings = hugging_face(texts) wait_for_modelT)inputsoptionsjsoncNg|]"}tj|tj#S)dtypenparrayfloat32.0 embeddings r( z9HuggingFaceEmbeddingFunction.__call__..R)PPP)"*555PPPr$postr"r2r'r+responses r(__call__z%HuggingFaceEmbeddingFunction.__call__<s[ =%% M!/?.FGG&   $&&  QPxPPPPr?cdS)N huggingfacerGr?r(namez!HuggingFaceEmbeddingFunction.nameTs}r?cdSNcosinerGr's r( default_spacez*HuggingFaceEmbeddingFunction.default_spaceXxr?c gdSN)rKl2iprGrLs r(supported_spacesz-HuggingFaceEmbeddingFunction.supported_spaces[%%%%r?configEmbeddingFunction[Documents]c|d}|d}|| Jdt||S)NrrFzThis code should not be reachedrr)getr)rUrrs r(build_from_configz.HuggingFaceEmbeddingFunction.build_from_config^s[ **%677ZZ --  "j&8 ;; ; ; ;++     r?c |j|jdS)NrXrXrLs r( get_configz'HuggingFaceEmbeddingFunction.get_configjs#'#7tWWWr? old_config new_configc,d|vrtddS)NrzSThe model name cannot be changed after the embedding function has been initialized.)rr'r]r^s r(validate_config_updatez3HuggingFaceEmbeddingFunction.validate_config_updatems, : % %e  & %r?c&t|ddS) Validate the configuration using the JSON schema. Args: config: Configuration to validate Raises: ValidationError: If the configuration does not match the schema rFNr rUs r(validate_configz,HuggingFaceEmbeddingFunction.validate_configus v}55555r?)Nrr)__name__ __module__ __qualname____doc__r strr)rrrD staticmethodrHrrMrrSrr rZr\rarerGr?r(rr s"&B; +R+R#+R+R +R+R+R+RZQiQJQQQQ0#\u&$u+&&&&  $sCx.  5S    \  XDcNXXXXsCx.6:38n  6S#X 64 6 6 6\ 6 6 6r?rcDeZdZdZ ddedeedeefdZdedefd Z e defd Z de fd Z dee fd Ze d eeefddfdZdeeeffdZdeeefdeeefddfdZe d eeefddfdZdS)HuggingFaceEmbeddingServerz This class is used to get embeddings for a list of texts using the HuggingFace Embedding server (https://github.com/huggingface/text-embeddings-inference). The embedding model is configured in the server. Nurlrrc ddl}n#t$rtdwxYw|tjdt ||_||_tj dd|_|j!|ptj |j|_ n||_ ||_ | |_ |j +|j jdd|j idSdS)aa Initialize the HuggingFaceEmbeddingServer. Args: url (str): The URL of the HuggingFace Embedding Server. api_key (Optional[str]): The API key for the HuggingFace Embedding Server. api_key_env_var (str, optional): Environment variable name that contains your API key for the HuggingFace API. rNrrrrr)rrrrrrrnrr r!rr"r#r$r%r&)r'rnrrrs r(r)z#HuggingFaceEmbeddingServer.__init__s  LLLL   g     M_"    . 9* + + 7#8D   +"Ebi0D&E&EDLL"DL   < # M ! ( (/;ST\;S;S)T U U U U U $ #r*r+r,c|j|jd|i}d|DS)a Get the embeddings for a list of texts. Args: input (Documents): A list of texts to get embeddings for. Returns: Embeddings: The embeddings for the texts. Example: >>> hugging_face = HuggingFaceEmbeddingServer(url="http://localhost:8080/embed") >>> texts = ["Hello, world!", "How are you?"] >>> embeddings = hugging_face(texts) r/r1cNg|]"}tj|tj#Sr4r6r:s r(r=z7HuggingFaceEmbeddingServer.__call__..r>r?r@rBs r(rDz#HuggingFaceEmbeddingServer.__call__sH =%%dm8U:K%LLQQSSQPxPPPPr?cdS)Nhuggingface_serverrGrGr?r(rHzHuggingFaceEmbeddingServer.names##r?cdSrJrGrLs r(rMz(HuggingFaceEmbeddingServer.default_spacerNr?c gdSrPrGrLs r(rSz+HuggingFaceEmbeddingServer.supported_spacesrTr?rUrVc|d}|d}|tdt||S)Nrnrz3URL must be provided for HuggingFaceEmbeddingServerrnr)rYrrm)rUrnrs r(rZz,HuggingFaceEmbeddingServer.build_from_configsLjj **%677 ;RSS S)c?SSSSr?c |j|jdS)NrwrwrLs r(r\z%HuggingFaceEmbeddingServer.get_configsxD4HIIIr?r]r^cRd|vr |d|jkrtddSdS)NrnzLThe URL cannot be changed after the embedding function has been initialized.)rnrr`s r(raz1HuggingFaceEmbeddingServer.validate_config_updatesC J  :e#4#@#@^   #@#@r?c&t|ddS)rcrsNr rds r(rez*HuggingFaceEmbeddingServer.validate_configs v';<<<<rsNNNNNNNNNNNN,,,,,,,,,,,, MMMMMMw6w6w6w6w6#4Y#?w6w6w6tr=r=r=r=r=!29!=r=r=r=r=r=r?