Using local data paths with zea

Most zea examples use Hugging Face links for convenience, but you can also work with local datasets by configuring a users.yaml file that points to your data root. This notebook shows how to set up local paths and load data from your own storage.

 

%%capture
%pip install zea

config_picmus_rf = "hf://zeahub/configs/config_picmus_rf.yaml"

Setting up your users.yaml

Many codebases and projects are littered with hardcoded absolute paths, which can make it difficult to share code or run it on different machines. To avoid this, zea makes use of a users.yaml file to define local data paths. The idea is that users can specify a local data root, and zea will use this to resolve paths dynamically, relative to the user’s data root.

Create a users.yaml file in your project directory. This file tells zea where your local data is stored. Example content:

data_root: /home/your_username/data

Replace /home/your_username/data with the actual path to your data directory. You can add an output key too, for where zea writes results, but it is optional and can be left out entirely.

Tip: You can auto-generate this file by running:

zea datapaths

and following the prompts. It asks where your data lives, and writes the answers to users.yaml for you.

So that the rest of this notebook runs anywhere, the next cell writes such a file for you, pointing at a folder it creates alongside this notebook. If you already have a users.yaml here, it is left alone and used as is.

from pathlib import Path

import yaml

users_yaml = Path("users.yaml")
data_root = Path("zea-data").resolve()

if users_yaml.exists():
    print(f"Using the {users_yaml} that is already here.")
else:
    # zea only warns about paths that do not exist, so create the folder as well.
    data_root.mkdir(parents=True, exist_ok=True)
    users_yaml.write_text(yaml.safe_dump({"data_root": str(data_root)}), encoding="utf-8")
    print(f"Wrote {users_yaml}, pointing at {data_root}")

Wrote users.yaml, pointing at /content/zea-data

Using Local Data Paths

Once your users.yaml is set up, you can load data from your local data root. Here’s a minimal example:

from zea import set_data_paths

user = set_data_paths("users.yaml")

data_root = user.data_root
username = user.username

print(f"🔔 Hi {username}! You are using data from {data_root}")

zea: Using backend 'jax'
🔔 Hi root! You are using data from /content/zea-data

Resolving Dataset Paths

With a data root in place you no longer need absolute paths in your code or configs. zea resolves paths with format_data_path, which is also what a zea config uses for its data.path entry:

• a relative path is resolved against your data_root,

• an absolute path is used as is,

• a Hugging Face path (hf://...) is passed through untouched.

from zea.datapaths import format_data_path

# Relative paths are resolved against the user's data_root
print(format_data_path("camus/val/patient0401_4CH_half_sequence.hdf5", user))

# Absolute paths and Hugging Face paths are returned unchanged, no user needed
print(format_data_path("/mnt/other_disk/patient0401_4CH_half_sequence.hdf5"))
print(format_data_path("hf://zeahub/camus-sample/val/patient0401_4CH_half_sequence.hdf5"))

/content/zea-data/camus/val/patient0401_4CH_half_sequence.hdf5
/mnt/other_disk/patient0401_4CH_half_sequence.hdf5
hf://zeahub/camus-sample/val/patient0401_4CH_half_sequence.hdf5

Advanced Data Path Configuration

In the above example, we use the most simple configuration in users.yaml, with just a data_root key. However, there are many more advanced options you can configure using users.yaml. For example, you can specify multiple data roots, for different users and machines. Additionally, you can define a path for local and remote data (if you use for instance a remote storage). Let’s have a look at a more advanced example.

Example: Complex users.yaml Layout

For collaborative projects or when working across multiple machines and operating systems, you can use a more structured users.yaml file. Here is an example:

alice:
  workstation1:
    system: linux
    data_root:
      local: /mnt/data/alice
      remote: /mnt/remote/alice
    output: /mnt/data/alice/output
  laptop:
    system: windows
    data_root: D:/data/alice
    output: D:/data/alice/output
bob:
  server:
    system: linux
    data_root:
      local: /mnt/data/bob
      remote: /mnt/remote/bob
  system: linux
  data_root: /mnt/data/bob
  output: /mnt/data/bob/output
# Default fallback if no user/machine matches
data_root: /mnt/shared/data
output: /mnt/shared/output

• Each user can have different machines, each with their own system and data_root.

• data_root can be a string or a dictionary with local and remote keys. Pick between the two with set_data_paths(..., local=True/False).

• A machine specific data_root takes precedence over a user specific one, which in turn takes precedence over the userless data_root at the bottom.

• If nothing matches, zea warns and falls back to a default path for your operating system, so you know your users.yaml still needs an entry.

# Example: Select remote data root (if defined in users.yaml)
user_remote = set_data_paths("users.yaml", local=False)
print("Remote data root:", user_remote.data_root)
user_local = set_data_paths("users.yaml", local=True)
print("Local data root:", user_local.data_root)

Remote data root: /content/zea-data
Local data root: /content/zea-data

Full Environment Setup with setup

For convenience, zea provides a setup function that configures everything in one step: config, data paths, and device (GPU/CPU).

• This will prompt for missing user profiles if needed, set up data paths, and initialize the device.

• Use this in your main scripts for reproducible and portable setups.

from zea.internal.setup_zea import setup

# config_path: path to your config YAML file
# user_config: path to your users.yaml file
config = setup(config_path=config_picmus_rf, user_config="users.yaml")

data_root = config.data.user.data_root
device = config.device

Summary

• Use users.yaml to manage local/remote data roots for different users and systems.

• Use set_data_paths to resolve your data root dynamically.

• Use format_data_path to turn a relative dataset path into an absolute one.

• For advanced setups, structure users.yaml with users, hostnames, and local/remote keys.

• Use setup for a one-liner to initialize config, data paths, and device.