hh ^ddlmZmZmZmZddlmZddlmZddl m Z GddeZ dS))CallableDictIteratorOptional)Document) BaseLoader)TensorflowDatasetsc reZdZdZ d dededeedeeege ffdZ d e e fd Z dS) TensorflowDatasetLoaderaLoad from `TensorFlow Dataset`. Attributes: dataset_name: the name of the dataset to load split_name: the name of the split to load. load_max_docs: a limit to the number of loaded documents. Defaults to 100. sample_to_document_function: a function that converts a dataset sample into a Document Example: .. code-block:: python from langchain_community.document_loaders import TensorflowDatasetLoader def mlqaen_example_to_document(example: dict) -> Document: return Document( page_content=decode_to_str(example["context"]), metadata={ "id": decode_to_str(example["id"]), "title": decode_to_str(example["title"]), "question": decode_to_str(example["question"]), "answer": decode_to_str(example["answers"]["text"][0]), }, ) tsds_client = TensorflowDatasetLoader( dataset_name="mlqa/en", split_name="test", load_max_docs=100, sample_to_document_function=mlqaen_example_to_document, ) dN dataset_name split_name load_max_docssample_to_document_functionc||_||_||_ ||_ t |j|j|j|j|_dS)a{Initialize the TensorflowDatasetLoader. Args: dataset_name: the name of the dataset to load split_name: the name of the split to load. load_max_docs: a limit to the number of loaded documents. Defaults to 100. sample_to_document_function: a function that converts a dataset sample into a Document. )r rrrN)r rrrr _tfds_client)selfr rrrs u/var/www/FlaskApp/flask-venv/lib/python3.11/site-packages/langchain_community/document_loaders/tensorflow_datasets.py__init__z TensorflowDatasetLoader.__init__,sc ".)*6 ' ( O.*,(,(H    returnc#HK|jEd{VdS)N)r lazy_load)rs rrz!TensorflowDatasetLoader.lazy_loadLs3$..00000000000r)r N) __name__ __module__ __qualname____doc__strrintrrrrrrrrr r s  L(+LP      }  &.hvx7G.H%I     @18H-111111rr N) typingrrrrlangchain_core.documentsr)langchain_community.document_loaders.baser1langchain_community.utilities.tensorflow_datasetsr r r rrr%s555555555555------@@@@@@PPPPPPD1D1D1D1D1jD1D1D1D1D1r