Datasets¶
Creating Datasets¶
Returns a |
|
Creates an empty dataset |
|
Creates a new dataset by copying the |
|
Ingests a dataset from a source and stores it as a structured dataset to destination. |
|
Download and ingest a kaggle dataset and store it as a structured dataset to destination. |
|
Convert pandas dataframe to a Deep Lake Dataset. |
|
Converts Hugging Face datasets to Deep Lake format. |
Loading Datasets¶
Loads an existing dataset |
Deleting and Renaming Datasets¶
Deletes a dataset at a given path. |
|
Renames dataset at |
Copying Datasets¶
Copies dataset at |
|
Copies dataset at |
Dataset Operations¶
Prints a summary of the dataset. |
|
Append samples to mutliple tensors at once. |
|
Appends multiple rows of samples to mutliple tensors at once. |
|
Returns a sliced |
|
Copies this dataset or dataset view to |
|
Deletes the entire dataset from the cache layers (if any) and the underlying storage. |
|
Renames the dataset to path. |
|
Visualizes the dataset in the Jupyter notebook. |
|
Removes a sample from all the tensors of the dataset. |
|
Rewrites the underlying chunks to make their sizes optimal. |
|
Necessary operation after writes if caches are being used. |
|
|
|
Estimates the size in bytes of the dataset. |
Dataset Visualization¶
Visualizes the dataset in the Jupyter notebook. |
Dataset Credentials¶
Adds a new creds key to the dataset. |
|
Populates the creds key added in add_creds_key with the given creds. |
|
Replaces the old creds key with the new creds key. |
|
Changes the management status of the creds key. |
|
Returns the list of creds keys added to the dataset. |
Dataset Properties¶
All tensors belonging to this group, including those within sub groups. |
|
All sub groups in this group |
|
Returns the length of the smallest tensor. |
|
Returns True if dataset is in read-only mode and False otherwise. |
|
Returns the information about the dataset. |
Dataset Version Control¶
Stores a snapshot of the current state of the dataset. |
|
Returns/displays the differences between commits/branches. |
|
Checks out to a specific commit_id or branch. |
|
Merges the target_id into the current dataset. |
|
Displays the details of all the past commits. |
|
Resets the uncommitted changes present in the branch. |
|
Get details of a particular commit. |
|
The lasted committed commit id of the dataset. |
|
The current branch of the dataset |
|
The commit_id of the next commit that will be made to the dataset. |
|
Returns True if currently at head node and uncommitted changes are present. |
|
Lists all the commits leading to the current dataset state. |
|
Lists all the branches of the dataset. |
Dataset Views¶
A dataset view is a subset of a dataset that points to specific samples (indices) in an existing dataset. Dataset views
can be created by indexing a dataset, filtering a dataset with Dataset.filter()
or querying a dataset with Dataset.query()
.
Filtering is done with user-defined functions or simplified expressions whereas query can perform SQL-like queries with our
Tensor Query Language. See the full TQL spec here.
Dataset views can only be saved when a dataset has been committed and has no changes on the HEAD node, in order to preserve data lineage and prevent the underlying data from changing after the query or filter conditions have been evaluated.
Example
>>> import deeplake
>>> # load dataset
>>> ds = deeplake.load("hub://activeloop/mnist-train")
>>> # filter dataset
>>> zeros = ds.filter("labels == 0")
>>> # save view
>>> zeros.save_view(id="zeros")
>>> # load_view
>>> zeros = ds.load_view(id="zeros")
>>> len(zeros)
5923
Returns a sliced |
|
Filters the dataset in accordance of filter function |
|
Saves a dataset view as a virtual dataset (VDS) |
|
Returns the dataset view corresponding to |
|
Loads the view and returns the |
|
Deletes the view with given view id. |
|
Returns list of views stored in this Dataset. |
|
Returns |