Curate MNIST

# !pip install 'lamindb[jupyter]' torch torchvision lightning
!lamin init --storage ./lamin-mlops

Show code cell output

Hide code cell output

! using anonymous user (to identify, call: lamin login)

→ initialized lamindb: anonymous/lamin-mlops

import lamindb as ln
from pathlib import Path

ln.track()

Show code cell output

Hide code cell output

→ connected lamindb: anonymous/lamin-mlops

→ created Transform('aPe9IFU50L8A0000'), started new Run('FQgQzAqM...') at 2025-07-14 06:39:50 UTC

→ notebook imports: lamindb==1.8.0 torchvision==0.22.1

• recommendation: to identify the notebook across renames, pass the uid: ln.track("aPe9IFU50L8A")

Download the MNIST dataset and save it in LaminDB to keep track of the training data that is associated with our model.

from torchvision.datasets import MNIST
from torchvision.transforms import ToTensor

dataset = MNIST(Path.cwd() / "download_mnist", download=True, transform=ToTensor())

Show code cell output

Hide code cell output

  0%|          | 0.00/9.91M [00:00<?, ?B/s]

100%|██████████| 9.91M/9.91M [00:00<00:00, 120MB/s]

  0%|          | 0.00/28.9k [00:00<?, ?B/s]

100%|██████████| 28.9k/28.9k [00:00<00:00, 5.83MB/s]

  0%|          | 0.00/1.65M [00:00<?, ?B/s]

100%|██████████| 1.65M/1.65M [00:00<00:00, 21.5MB/s]

  0%|          | 0.00/4.54k [00:00<?, ?B/s]

100%|██████████| 4.54k/4.54k [00:00<00:00, 10.1MB/s]

# no need for the zipped files
!rm -r download_mnist/MNIST/raw/*.gz
!ls -r download_mnist/MNIST/raw

train-labels-idx1-ubyte  t10k-labels-idx1-ubyte
train-images-idx3-ubyte  t10k-images-idx3-ubyte

training_data_artifact = ln.Artifact(
    "download_mnist/",
    key="testdata/mnist",
    type="dataset",
).save()
training_data_artifact

Show code cell output

Hide code cell output

! `type` will be removed soon, please use `kind`

! calling anonymously, will miss private instances

Artifact(uid='9YvhPw3lKenhXIk20000', is_latest=True, key='testdata/mnist', suffix='', kind='dataset', size=54950048, hash='amFx_vXqnUtJr0kmxxWK2Q', n_files=4, branch_id=1, space_id=1, storage_id=1, run_id=1, created_by_id=1, created_at=2025-07-14 06:39:56 UTC)

After saving the MNIST training dataset in LaminDB, one can see the dataset showing up in LaminHub:

ln.finish()

Show code cell output

Hide code cell output

→ finished Run('FQgQzAqM') after 7s at 2025-07-14 06:39:57 UTC